Camera setting support device and method for supporting setting of camera

Abstract

To support setting of a plurality of cameras.SOLUTION: A camera setting support device acquires information of the number of cameras, information of an area, information of the moving route of a mobile body, and a taken image taken by a camera, extracts a mobile body image as an image of a mobile body on a camera-by-camera basis; and estimates the position of a camera on the area and outputs the position on the basis of the imaging time information of the mobile body image.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to a camera installation support device and a method for supporting camera installation.

Background Art

[0002] Patent Document 1 discloses a control device that detects a moving object from images acquired by a plurality of infrastructure cameras capable of acquiring images of a peripheral environment including the moving object, estimates the detection accuracy of the detected moving object, and estimates the state of an infrastructure camera network constructed by the plurality of infrastructure cameras based on the estimated detection accuracy.

[0003] Further, Patent Document 2 discloses a camera calibration method for calibrating a camera from a plurality of images captured by a plurality of cameras installed on a vehicle in a peripheral area of the vehicle including calibration indexes that are a plurality of intersections where at least two first parallel lines and two second parallel lines perpendicular to the two parallel lines intersect, which are provided in advance on the road surface where the vehicle is installed. The camera calibration method performs perspective transformation on each of the plurality of images based on first camera parameters stored in advance to generate an aerial view image, recognizes a plurality of intersections from the generated aerial view image, and calculates second camera parameters capable of generating an aerial view image without deviation based on the recognized plurality of intersections and information on the positions and postures of the plurality of cameras.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Conventionally, when installing a plurality of cameras in a store, warehouse, etc. to monitor, for example, shelves, products, inventory, shortages, etc., the user who installs the cameras selects candidates for the installation positions of the cameras from a map of the installation locations where the cameras are to be installed, etc., and manually associates the installation positions with the camera units in advance, or adjusts the installation positions or installation angles of the cameras while adjusting the installation positions or installation angles of the cameras capable of imaging the monitoring targets. This is quite laborious and costly. Also, the more cameras the user has, the more laborious it becomes, such as replacing cameras due to malfunctions or adjusting the installation positions of cameras due to changes in the layout of the shelves being monitored. Therefore, in the installation of a plurality of cameras, there has been a demand for a technology that identifies cameras that require adjustment of their installation positions and simplifies and reduces the cost of initial installation and post-installation management.

[0006] The present disclosure has been devised in view of the above-described conventional circumstances, and an object thereof is to provide a camera installation support device and a camera installation support method for supporting the installation of a plurality of cameras.

Means for Solving the Problems

[0007] The present disclosure provides a camera installation support device including: an acquisition unit that acquires the number of camera information, information regarding an area where the plurality of cameras are installed, movement route information of a moving object imaged by the plurality of cameras, and captured images captured by each of the plurality of cameras; and an estimation unit that extracts a moving object image in which the moving object is imaged from among the captured images, and estimates and outputs the positions of the plurality of cameras on the area based on the imaging time information at which the moving object image is captured.

[0008] The present disclosure also provides a method for assisting in the installation of cameras, performed by at least one computer capable of communicating with a plurality of cameras, the method including: obtaining the number information of the cameras, information regarding the area where the plurality of cameras are installed, movement path information of a moving object imaged by the plurality of cameras, and captured images captured by each of the plurality of cameras; extracting, for each camera, a moving object image in which the moving object is imaged from among the captured images; and estimating and outputting the positions of the plurality of cameras on the area based on the imaging time information at which the moving object image was captured.

Advantages of the Invention

[0009] According to the present disclosure, the installation of a plurality of cameras can be assisted.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Best Mode for Carrying Out the Invention

[0011] Hereinafter, with appropriate reference to the drawings, embodiments specifically disclosing the configuration and operation of the camera installation support device and the camera installation support method according to the present disclosure will be described in detail. However, a more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and duplicate descriptions of substantially the same configuration may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and it is not intended to limit the subject matter described in the claims thereby.

[0012] With reference to FIG. 1, a use case example of the camera installation support system 100 according to the embodiment will be described. FIG. 1 is a diagram showing an example of a use case of the camera installation support system 100 according to the embodiment. In FIG. 1, only the lens LS of the camera C11 is labeled, and the labeling of the lenses of other cameras is omitted.

[0013] In the example shown in FIG. 1, a plurality of cameras C11, C12, C13, C14, C21, C22, C23, C24, C31, C32, C33, C34 are installed on the merchandise shelves SH11, SH12, SH13, SH14, SH21, SH22, SH23, SH24, SH31, SH32, SH33, SH34 on which products are displayed, respectively. Each of the plurality of cameras C11 to C14, C21 to C24, C31 to C34 images a merchandise shelf (for example, in the case of the camera C11, the merchandise shelf SH21) installed opposite to the merchandise shelf on which the camera itself is installed. Further, the camera C41 is a camera attached to the ceiling and images the merchandise shelf SH41.

[0014] The terminal device P1 acquires captured images captured by each of a plurality of cameras C11 to C14, C21 to C24, C31 to C34, and C41. As camera installation support for the user, the terminal device P1 estimates the installation positions of the respective cameras based on the captured images captured by the respective cameras, and analyzes (determines) whether each camera can capture a product shelf that is the imaging target of each, and outputs the result.

[0015] Here, as an example, a case where a plurality of cameras C11,... are installed indoors will be described, but it goes without saying that they may be installed outdoors.

[0016] Referring to FIG. 2, an internal configuration example of the camera installation support system 100 according to the embodiment will be described. FIG. 2 is a diagram showing an internal configuration example of the camera installation support system 100 according to the embodiment.

[0017] The camera installation support system 100 includes at least one terminal device P1 and each of a plurality of cameras C11,.... The camera installation support system 100 estimates the installation positions of the current plurality of cameras C11,... and determines whether imaging of the imaging target is possible by each of the plurality of cameras C11,... capturing a moving body and the terminal device P1 performing image analysis processing on the captured images. The camera installation support system 100 executes support for making the installation of each of the current plurality of cameras C11,... suitable for imaging the imaging target based on the calculated installation positions of each of the plurality of cameras C11,....

[0018] The terminal device P1 is communicably connected to each of a plurality of cameras C11,.... The terminal device P1 is realized by, for example, a Personal Computer (hereinafter referred to as "PC"), a notebook PC, a tablet terminal, or the like. The terminal device P1 includes a communication unit 10, a processor 11, a memory 12, an input unit 13, and a monitor 14.

[0019] The communication unit 10 is connected so as to be capable of wired or wireless communication with each of a plurality of cameras C11, …. The communication unit 10 acquires the captured images transmitted from each of the plurality of cameras C11, … and outputs them to the processor 11. Here, the wireless communication network is provided in accordance with a wireless communication standard such as, for example, wireless LAN (Local Area NetWork), wireless WAN (Wide Area NetWork), 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), or Wi-Fi (registered trademark).

[0020] The processor 11 is configured using, for example, a Central Processing Unit (CPU) or a Field Programmable Gate Array (FPGA) and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 refers to the programs and data held in the memory 12 and executes those programs to execute the functions of the processor 11.

[0021] The memory 12 has, for example, a Random Access Memory (RAM) as a work memory used when executing each process of the processor 11, and a Read Only Memory (ROM) that stores programs and data defining the operation of the processor 11. Data or information generated or acquired by the processor 11 is temporarily stored in the RAM. A program defining the operation of the processor 11 is written in the ROM. The memory 12 records information such as the number of cameras set by the user, the models of the cameras, a map indicating the area where the cameras are installed, the movement path of the moving body, and information regarding the imaging target imaged by the cameras.

[0022] The input unit 13 is a user interface capable of receiving input operations by the user, and is realized by, for example, a keyboard, a mouse, a touch panel, etc. The input unit 13 converts the content of the received input operation into an electrical signal and outputs it to the processor 11. When the input unit 13 is realized by a touch panel, the input unit 13 may be integrally configured with the monitor 14.

[0023] The monitor 14 is configured using, for example, a Liquid Crystal Display (LCD) or an organic Electroluminescence (EL). The monitor 14 displays the analysis result screens SC0, SC11, SC12, SC2 (see FIGS. 8 to 11) output from the processor 11.

[0024] Each of the plurality of cameras C11,... is a so-called surveillance camera and is installed to image (monitor) an imaging target. Note that the plurality of cameras C11,... may each have a different camera type and model, and may be any camera such as an infrared camera, a camera capable of panning, tilting, or zooming, or a wide-angle camera. The plurality of cameras C11,... are arbitrarily installed by the user and execute imaging of the merchandise shelf that is the imaging target.

[0025] Each of the plurality of cameras C11,... is configured to include at least a lens LS and an image sensor (not shown). Each of the plurality of cameras C11,... is connected to be capable of wired or wireless communication with the terminal device P1, and associates imaging time information indicating the time when the imaging image was taken and identification information capable of identifying its own camera with the imaging image, and transmits it to the terminal device P1.

[0026] Next, with reference to FIGS. 3 and 4 respectively, an example of the operation procedure of the camera installation support system 100 according to the embodiment will be described. FIG. 3 is a sequence showing an example of the operation procedure of the camera installation support system 100 according to the embodiment. FIG. 4 is a flowchart showing an example of the analysis process of the terminal device P1 in the embodiment. Note that in FIGS. 3 and 4, for the sake of easy understanding of the explanation, an example of the case where the camera installation support system 100 is operated in the use case example shown in FIG. 1 will be described, but it goes without saying that it is not limited to this.

[0027] A plurality of cameras C11,... are installed by the user at arbitrary positions within the area shown in the map MP described later (St11).

[0028] The processor 11 receives an input operation by the user via the input unit 13, and acquires information regarding the area where the plurality of cameras C11,... are installed and information regarding the plurality of installed cameras C11,... (St12). Note that the processor 11 may receive an input operation by the user and further acquire additional information described later.

[0029] Specifically, as information regarding the area, the processor 11 acquires a map MP of the area (see FIG. 5), a movement route RT of the moving body on the map MP (see FIG. 5), information regarding the imaging target imaged by the plurality of cameras C11,... and the like. Further, as information regarding the plurality of cameras C11,..., the processor 11 acquires information such as the number of cameras, the model of the cameras, the number of Light Emitting Diodes (hereinafter referred to as "LEDs") provided in the cameras, the emission color, or the emission pattern.

[0030] Note that the various types of information input in step St12 are not limited to the examples described above. For example, when there are a plurality of imaging targets, the processor 11, as information regarding the imaging targets, may acquire information that enables each imaging target to be identifiable, such as an image of the imaging target, a QR code (registered trademark) assigned to the imaging target, information regarding an object arranged in front of, behind, or in the vicinity of the imaging target, information indicating the boundary of each imaging target, etc. Note that "in front of the imaging target" indicates the direction approaching the cameras C11,.... "Behind the imaging target" indicates the direction moving away from the cameras C11,....

[0031] Also, the movement path RT of the mobile body is set such that all the cameras C11,... can image the mobile body moving on the movement path RT at least once. The information of the movement path RT may be acquired by receiving an input operation of the movement path RT with respect to the map MP, or may be acquired by receiving an input operation of the position (coordinate) information of a plurality of passing points passed through by the movement path RT.

[0032] The processor 11 receives an input operation by the user via the input unit 13 and registers information regarding the mobile body (St13).

[0033] The information regarding the mobile body is information that enables the mobile body to be detected from the captured image, and includes at least one of information such as an image of the mobile body, mark information regarding at least one mark assigned to the mobile body (for example, information regarding a QR code, a color target, a marker, etc. attached to or possessed by the mobile body). When the mobile body is an object that automatically travels, the information regarding the mobile body may further include information such as the movement speed of the mobile body.

[0034] In addition, when a square mark such as a QR code or a sticker is attached to the moving body in order for the processor 11 to estimate the installation angle of each of the plurality of cameras C11, ..., the processor 11 may further accept registration of the size of this mark, the image of the mark, etc. Note that the mark does not necessarily have to be attached to the moving body itself. For example, when the moving body is a person, the mark may be attached to an object that moves with this person (e.g., a shopping cart, etc.).

[0035] The plurality of cameras C11, ... start imaging the moving body and the imaging target based on a direct input operation by the user to the terminal device P1 or the self-camera (St14). After the imaging by the plurality of cameras C11, ... is started, the moving body moves along the moving path RT. After the movement of the moving body ends, the plurality of cameras C11, ... end the imaging based on a direct input operation by the user to the terminal device P1 or the self-camera (St15), and associate imaging time information indicating the imaging time of the captured imaging image with the identification information capable of identifying the self-camera, and transmit it to the terminal device P1 in association with the imaging image (St16).

[0036] The processor 11 executes image analysis processing on the imaging images transmitted from each of the plurality of cameras C11, ..., and executes detection and identification processing of the moving body shown in the imaging images (St17). Based on the imaging image in which the moving body is imaged, the processor 11 executes analysis processing for estimating the installation position or installation angle of the plurality of cameras C11, ... installed on the map MP, and analyzing whether the imaging target can be imaged by the plurality of cameras C11, ... (St18).

[0037] The processor 11 identifies the model of each of the plurality of cameras C11, ... based on the information on the model of the camera input in advance and the characteristics of the captured imaging image for each camera (St18A). Based on the identified model information, the processor 11 executes image correction on the imaging images transmitted from each of the plurality of cameras C11, ... so as to be suitable for imaging the moving body and the imaging target (St18B).

[0038] Note that here, the processor 11 has a color target, a QR code, etc. as an example of a mark that enables the moving object to detect itself. When a color target, a QR code, etc. are detected from the captured image, correction information for adjusting the imaging parameters of each camera to imaging parameters more suitable for imaging the moving object and the imaging target may be generated based on the color target, the QR code, etc. shown in the captured image. The correction information is output together with the analysis result.

[0039] The processor 11 performs recognition processing of the imaging target shown in the captured image for each camera using the captured image after image correction (St18C). The processor 11 recognizes the imaging target based on a pre-registered image of the imaging target or identification information that can identify the imaging target. Specifically, the processor 11 executes recognition processing based on the image of the imaging target, or when a mark (for example, a QR code, a sticker, etc.) is attached to the imaging target, recognizes the mark, or when there is an object near the imaging target, recognizes this object.

[0040] Based on the recognition result, the processor 11 determines whether all imaging targets can be imaged by the plurality of cameras C11, …, and when it is determined that an imaging target cannot be imaged, generates support information for assisting in adjusting the installation position or installation angle of the camera to enable imaging of the imaging target (St18D).

[0041] Specifically, when a square mark (for example, a QR code, a sticker, etc.) is attached, the processor 11 estimates the distance, installation position, or installation angle of the camera with respect to the imaging target based on the size (number of pixels) of the mark shown in the captured image, the position where the mark is shown, and the ratio of the lengths of the four sides of the mark. Based on the estimated installation position or installation angle of the camera, the processor 11 calculates the installation position or installation angle of the camera that enables imaging of the entire imaging target and generates support information (support information for camera installation adjustment). Note that the support information may include information such as the zoom ratio of the camera more suitable for imaging the imaging target.

[0042] Further, the processor 11 re - executes the identification process of the moving object based on the captured image after image correction, and extracts the captured images of the moving object (hereinafter referred to as "moving object images") for each camera (St18E). The number of moving object images extracted here may be plural for one camera. The processor 11 arranges the extracted moving object images in chronological order based on the imaging time information of the captured images (St18E).

[0043] The processor 11 identifies the imaging order of the cameras that captured the moving object based on the imaging time information of the arranged moving object images. The processor 11 calculates the distance between two different cameras that are consecutive in the imaging order, which is the distance in the direction along the moving direction of the moving object, based on the difference (time difference) of the imaging time information of the moving object images captured by each camera (St18F).

[0044] For example, when the processor 11 estimates that the moving object is imaged in the order of camera C11, camera C21, camera C12, and camera C22 based on the moving object images arranged in chronological order, the processor 11 calculates the distances between camera C11 and camera C21, between camera C21 and camera C12, and between camera C12 and camera C22 in the direction along the moving direction of the moving object, respectively. When the viewing angles of two consecutive cameras in the imaging order of the cameras partially overlap and the moving speed and actual moving path (moving direction) of the moving object can be calculated with higher accuracy, the distance between the two cameras can be calculated with higher accuracy.

[0045] The processor 11 performs a grouping process of grouping the cameras among the plurality of cameras C11,... whose imaging directions of the moving object are similar based on the direction (imaging direction) of the moving object shown in the moving object image and the imaging time information of the moving object image (St18G). Note that one group may be generated including at least one camera.

[0046] For example, with the direction along the movement path RT as the front, the processor 11 analyzes from which direction among the right side, left side, or upper side of the moving object the imaging was performed, and groups the cameras that imaged the moving object from the right side, the cameras that imaged the moving object from the left side, and the cameras that imaged the moving object from the upper side, respectively. For example, in the example shown in FIG. 8, the processor 11 groups the cameras C11 to C14 into a group that images the moving object from the left side, the cameras C21 to C24 and C31 to C34 into a group that images the moving object from the right side, and the camera C41 into a group that images the moving object from the upper side.

[0047] The processor 11 extracts again the moving object images captured by the cameras belonging to the same group and arranges them in chronological order, and determines whether there is a timing when the moving object is not imaged by the cameras belonging to the same group for a predetermined time or more based on the imaging time information of the arranged moving object images. When the processor 11 determines that there is a timing when the moving object is not imaged by the cameras imaging from the same direction for a predetermined time or more, it further groups the cameras that imaged the moving object before this timing and the cameras that imaged the moving object after this timing.

[0048] Note that the processor 11 extracts again the moving object images captured by the cameras belonging to the same group and arranges them in chronological order, determines whether there is an overlapping area in two consecutive moving object images in chronological order, and when it determines that there is no overlapping area, it may execute a grouping process of further dividing the groups between the cameras that captured these two moving object images.

[0049] For example, in the example shown in FIG. 8, although the cameras C21 to C24 and C31 to C34 belong to the same group that images the moving body from the right side as described above, the processor 11 determines that there is a timing when the moving body is not imaged for a predetermined time or more between the camera C24 and the camera C34. The processor 11 further groups the group that images the moving body from the right side into two groups: a group including the cameras C21 to C24 that imaged the moving body before this timing, and the cameras C31 to C34 that imaged the moving body after this timing.

[0050] The processor 11 matches the map MP of the target area with each group, and maps each of the plurality of cameras C11,... on the map MP (St18H).

[0051] Based on the additional information acquired in step St12 or input after step St18H, the processor 11 adjusts the positions of the mapped plurality of cameras C11,... (St18I). Note that the process of step St18I is not essential and may be omitted when there is no additional information.

[0052] The processor 11 determines whether the mapping of all the cameras C11,... on the map MP of the target area is completed (St18J). When the processor 11 determines that the mapping of all the cameras C11,... on the map MP of the target area is completed (St18J, YES), it ends the analysis process of the installed cameras (step St18).

[0053] On the other hand, when the processor 11 determines that the mapping of all the cameras C11,... on the map MP of the target area is not completed (St18J, NO), it generates a reinstallation command that requests the reinstallation of the cameras that cannot be mapped (that is, are not mapped), and outputs it to the monitor 14 (St18K). Note that the reinstallation command may be output together with the analysis result screens SC1, SC11, SC12, SC2 (see FIGS. 8 to 11) described later.

[0054] The processor 11 generates analysis result screens SC1, SC11, SC12, SC2 (see FIGS. 8 to 11) that map the positions of a plurality of cameras C11, … on the map MP, and outputs them to the monitor 14 (St19). Note that when there is support information regarding the installation of the cameras, the processor 11 may generate analysis result screens SC11, SC2 (see FIGS. 9 and 11) that include the support information, and output them to the monitor 14. Further, the processor 11 may generate an analysis result screen SC2 (see FIG. 11) in which the viewing angles of a plurality of cameras C11, … are superimposed on the map MP, and output it to the monitor 14.

[0055] As described above, in the camera installation support system 100 according to the embodiment, in the installation of a plurality of cameras C11, …, without inputting the installation positions or installation angles of the respective cameras C11, … previously installed on the area, based on the movement path RT of the moving body and the moving body image, the positional relationship (i.e., installation position) of each of the plurality of cameras C11, … with respect to the moving body that has moved along the movement path RT, the installation angles, etc. of the plurality of cameras C11, … with respect to the moving body that has moved along the movement path RT can be estimated. Further, the camera installation support system 100 can determine whether or not an imaging target can be imaged by the plurality of cameras C11, … based on the estimated installation positions or installation angles of the respective cameras C11, … and the imaging images captured by the respective cameras C11, ….

[0056] Therefore, the camera installation support system 100 outputs analysis result screens SC11, SC2 (see FIGS. 9 and 11) including support information for supporting the installation positions or installation angles of the plurality of cameras C11, … for imaging the imaging target, based on the estimated installation positions or installation angles of the cameras C11, … and the determination result as to whether or not the imaging target can be imaged by the plurality of cameras C11, …, thereby supporting the user in installing the cameras.

[0057] Next, with reference to FIG. 5, the map MP and the movement path RT of the moving body will be described. FIG. 5 is a diagram showing an example of the map MP and the movement path RT.

[0058] The map MP shows a target area including an area where a plurality of cameras C11,... are installed, imaging targets (in the example shown in FIG. 5, merchandise shelves SH11~SH14, SH21~SH24, SH31~SH34, SH41) of the plurality of cameras C11,..., and the movement path RT of the mobile body. Note that the map MP may be data showing the target area in 2D or data showing the target area in 3D.

[0059] The information on the movement path RT of the mobile body may be acquired by receiving an input operation of the movement path RT with respect to the map MP displayed on the monitor 14, or may be acquired by receiving an input operation of a plurality of different position (coordinate) information.

[0060] Here, with reference to FIGS. 6 and 7 respectively, an example of additional information will be described. FIG. 6 is a diagram for explaining an identification example of an imaging target. FIG. 7 is a diagram for explaining an estimation example of the installation positions of cameras C11A, C12A. Note that FIGS. 6 and 7 each illustrate only a part of the map MP shown in FIG. 5 for easier understanding of the explanation.

[0061] First, with reference to FIG. 6, an example will be described in which there is a mark in front of, behind, or near the merchandise shelf SH11, which is the imaging target, that can identify the merchandise shelf SH11 itself and specify the position of the merchandise shelf SH11, and this mark is used as additional information.

[0062] In the example shown in FIG. 6, the processor 11 acquires additional information that there is a pillar OB11 (mark) to the left of the merchandise shelf SH11 on the paper surface as seen from the camera C21A and a QR code OB12 at the right end of the merchandise shelf SH11. The camera C21A has a viewing angle Ar21A, images the merchandise shelf SH11, and transmits the captured image to the terminal device P1. In such a case, the processor 11 can analyze where the left and right ends of the merchandise shelf SH11 are by recognizing the pillar OB11 and the QR code OB12 provided on the merchandise shelf SH11 from the captured image captured by the camera C21A.

[0063] That is, the processor 11 can estimate the position of the camera C21A with higher accuracy by detecting an index existing in front of, behind, or near the imaging target, such as the pillar OB11 or the QR code OB12. Further, when another camera (camera C11) is included in the viewing angle Ar21A of the camera C21A, the processor 11 can estimate the installation position or installation angle of the camera C11 with higher accuracy based on the position of the camera C11 with respect to the imaging target, the pillar OB11, or the QR code OB12.

[0064] In addition, when marks are provided for each imaging target (that is, each commodity shelf) for the purpose of identifying each imaging target, different marks are provided for imaging targets that are physically adjacent or belong to the same group and are imaged by adjacent cameras. Thereby, the processor 11 can identify any imaging target in which two or more imaging targets imaged by one camera or two or more imaging targets imaged by a plurality of cameras belonging to the same group and having at least partially overlapping viewing angles are arranged on the map MP.

[0065] Next, with reference to FIG. 7, an example will be described in which marks for identifying the cameras C21B and C22B and specifying the positions of the cameras C21B and C22B are provided on the cameras C21B and C22B, and this mark is used as additional information. The marks shown in FIG. 7 are, as an example, the number of lit LEDs provided in the cameras C21B and C22B. Note that the marks are not limited to the above-described example, and may be an LED lighting pattern, an LED lighting color, or a combination of LED lighting colors, etc., or the appearance of the cameras C21B and C22B, marks provided on the cameras C21B and C22B (for example, QR codes, stickers, etc.).

[0066] In the example shown in FIG. 7, the processor 11 acquires additional information indicating that the LED Lt1 included in the camera C21B and the LEDs Lt2A and Lt2B included in the camera C22B are lit. The camera C11A has a viewing angle Ar11A, images the merchandise shelves SH21 and SH22, and transmits the captured image to the terminal device P1. The camera C12A has a viewing angle Ar11A, images the merchandise shelf SH22, and transmits the captured image to the terminal device P1.

[0067] In such a case, the processor 11 identifies the camera C21B including the LED Lt1 and the cameras C22B including the LEDs Lt2A and Lt2B by detecting the LED Lt1 and the LEDs Lt2A and Lt2B respectively from the captured image captured by the camera C11A. The processor 11 identifies the camera C22B including the LEDs Lt2A and Lt2B by detecting the LEDs Lt2A and Lt2B respectively from the captured image captured by the camera C12A.

[0068] Further, when a plurality of cameras C21B and C22B are reflected in one captured image, the processor 11 may estimate the installation position or installation angle of the cameras C21B and C22B based on the positions and distances of the cameras C21B and C22B reflected in the captured image. When numbers, names, etc. that can identify each of a plurality of cameras C11,... have been input in advance, the processor 11 may specify the numbers, names, etc. of the cameras C21B and C22B based on the number of lit LEDs. When it is determined that the same camera C22B has been imaged by different cameras C11A and C12A, the processor 11 may estimate the installation position or installation angle of the cameras C11A and C12A based on the position of the camera C22B reflected in the captured image. Note that the cameras C21B and C22B or the LEDs Lt1, Lt2A, and Lt2B mentioned here may be replaced with the landmarks described in FIG. 6.

[0069] Next, with reference to FIG. 8, an example of grouping of a plurality of cameras C11, … and an analysis result screen SC0 will be described. Note that the analysis result screen SC0 shown in FIG. 8 shows an example that does not include support information and does not illustrate the viewing angles of the respective cameras in order to facilitate the explanation of the grouping example.

[0070] The analysis result screen SC0 is generated by mapping a plurality of cameras C11, … grouped on the map MP. In the analysis result screen SC0 shown in FIG. 8, cameras C11 to C14 are grouped into a first group GP1, cameras C21 to C24 are grouped into a second group GP2, cameras C31 to C34 are grouped into a third group GP3, and camera C41 is grouped into a fourth group GP4. Dashed lines are superimposed so as to surround the cameras belonging to each group on the map MP.

[0071] Here, the first group GP1 includes cameras C11 to C14 that image a moving body with the direction along the movement path RT as the front from the left side. The second group GP2 includes cameras C21 to C24 that image a moving body with the direction along the movement path RT as the front from the right side. The third group GP3 includes cameras C31 to C34 that image a moving body with the direction along the movement path RT as the front from the right side. The fourth group GP4 includes camera C41 that images a moving body with the direction along the movement path RT as the front from the upper side.

[0072] Note that the processor 11 in the present embodiment will be described with an example of performing grouping of a plurality of cameras C11, … based on the imaging time information when the camera images a moving body and the direction from which the moving body with the direction along the movement path RT as the front is imaged, i.e., from the right side, left side, or upper side. However, the grouping method is not limited to this.

[0073] Next, with reference to FIGS. 9 and 10, an example of support information for assisting in adjusting the installation angle of a camera will be described. FIG. 9 is a diagram showing an example of an analysis result screen SC11. FIG. 10 is a diagram showing an example of an analysis result screen SC12. Note that in FIGS. 9 and 10, the illustration of the dashed lines showing the grouping result shown in FIG. 8 is omitted in order to make the support information easier to view.

[0074] In the example shown in FIG. 9, when the processor 11 determines that the entire merchandise shelf SH24 is not shown in the captured image captured by the camera C14, the processor 11 estimates the installation position or installation angle at which the camera C14 can capture the entire merchandise shelf SH24. Here, the processor 11 generates a message Msg1 as support information for instructing adjustment of the installation direction of the camera C14 so as to face the right direction with respect to the current angle of view of the camera C14. The analysis result screen SC11 is generated to include a message Msg1 "Camera #14 Direction Adjustment "← Left"" for instructing adjustment of the installation direction (orientation) of the camera C14 as support information.

[0075] Note that the processor 11 may record the support information generated as an analysis result, the grouping result of the plurality of cameras C11,..., and the installation position or installation angle of each of the plurality of cameras C11,....

[0076] The user adjusts the installation direction of the camera C14 based on the message Msg1 on the analysis result screen SC11. After the user adjusts the installation angle of the camera C14, the user causes the plurality of cameras C11,... to capture the moving body again. The terminal device P1 re-analyzes the grouping of the plurality of cameras C11,... and the installation position or installation angle of each of the plurality of cameras C11,... based on the captured image captured again by the plurality of cameras C11,..., generates an analysis result screen SC12, and outputs it to the monitor 14.

[0077] In the example shown in FIG. 10, when the processor 11 determines that the entire merchandise shelf SH24 is shown in the captured image captured again by the camera C14, the processor 11 determines that the support indicated by the support information obtained by the previous analysis process is completed, and generates a message Msg2 "Camera #14 Direction Adjustment "OK"" indicating that the installation angle of the camera C14 has been improved. The processor 11 generates an analysis result screen SC12 including the message Msg2 and outputs it to the monitor 14.

[0078] As a result, the terminal device P1 in the embodiment can visualize information for adjusting the installation position or installation angle of the camera as information for assisting the installation of the camera and present it to the user. Further, the terminal device P1 can record the analysis result, and by comparing the analysis results, determine whether the installation position or installation angle of the camera has been improved, and visualize and present to the user information indicating whether the installation position or installation angle of the camera has been improved.

[0079] Next, with reference to FIG. 11, an example of support information for proposing the addition of a camera will be described. FIG. 11 is a diagram showing an example of the analysis result screen SC2. Note that, for ease of explanation, FIG. 11 shows only a part of the map MP shown in FIG. 5, and in order to make the support information easier to view, the dashed line illustration showing the grouping result shown in FIG. 8 is omitted.

[0080] In the example shown in FIG. 11, the camera C21C has a viewing angle Ar21C and images the merchandise shelf SH11C. The camera C22C has a viewing angle Ar22C and images the merchandise shelf SH12C. The processor 11 determines, based on the captured images captured by each of the adjacent cameras C21C and C22C, whether the viewing angle Ar21C and the viewing angle Ar22C overlap in the imaging of the merchandise shelf SH12C, or whether the number of cameras for imaging the entire merchandise shelf SH12C is sufficient or insufficient. If the processor 11 determines that the entire merchandise shelves SH11C and SH12C cannot be imaged even if the installation positions or installation angles of the cameras C21C and C22C are adjusted, it generates a message Msg3 as support information for proposing the addition of a new camera C51. The processor 11 generates an analysis result screen SC2 including the message Msg3 "Proposal for Adding a Camera" for proposing the addition of a new camera C51 and outputs it to the monitor 14.

[0081] As a result, the terminal device P1 in the embodiment can propose adding cameras as information for assisting in installing cameras. Here, an example of proposing adding cameras has been described. However, if the entire viewing angle of a predetermined camera overlaps with the viewing angles of other cameras, the terminal device P1 may propose reducing the number of cameras.

[0082] Also, as a result, the terminal device P1 in the embodiment can achieve optimization of the number of installed cameras and imaging areas.

[0083] Next, with reference to FIG. 12, a method of controlling an actual camera and notifying assistance information will be described. FIG. 12 is a diagram showing a control example of the camera C21D based on the analysis result.

[0084] In the example shown in FIG. 12, the camera C21D is a camera capable of performing lighting / extinguishing control of the LED Lt3 based on a control command transmitted from the terminal device P1. The camera C21D has a viewing angle Ar21D and images the merchandise shelf SH11.

[0085] When the processor 11 determines based on the captured image captured by the camera C21D that the installation angle of the camera C21D needs to be adjusted, the processor 11 generates a control command requesting lighting of the LED Lt3 to notify the user that the installation angle of the camera C21D needs to be adjusted, and transmits it to the camera C21D. The camera C21D lights the LED Lt3 based on the control command transmitted from the terminal device P1.

[0086] Note that when the number of cameras for which assistance information is generated is plural, the processor 11 may light them with different numbers of lit LEDs, lighting patterns, or lighting colors for each camera. Further, the processor 11 may generate and output an analysis result screen further including information on the number of lit LEDs, lighting patterns, or lighting colors of the LEDs of each camera. As a result, the processor 11 can visualize the cameras that need to have their installation positions or installation angles adjusted, and assist the user in installing the cameras.

[0087] This makes it easier for the user to find cameras on-site that require adjustment of their installation positions or angles, even when a large number of cameras are installed. Also, even when adjustment of the installation positions or angles is performed by remote instruction, the user can easily find which of the installed cameras requires adjustment of their installation positions or angles.

[0088] (Additional Note) The above description of each embodiment discloses the following techniques.

[0089] (Technology 1) an acquisition unit (communication unit 10) that acquires information on the number of cameras C11, ..., information on the area in which the multiple cameras C11, ... are installed, movement path information (movement path RT) of a moving object captured by the multiple cameras C11, ..., and a captured image captured by each of the multiple cameras C11, ...; an estimation unit (processor 11) that extracts moving body images in which the moving body is captured from the captured images for each of the cameras C11, ... and estimates the positions of the cameras C11, ... in the area based on image capture time information when the moving body images were captured; Equipped with Camera installation support device (terminal device P1). With this configuration, the camera installation support device (terminal device P1) can support the user in installing cameras by estimating and outputting the respective positional relationships (i.e., installation positions) of the multiple cameras C11, ... relative to the moving body that has moved along the movement path RT, and the installation angles of the multiple cameras C11, ... relative to the moving body that has moved along the movement path RT, based on the moving body's movement path RT and the moving body image, without the need to input the installation position or installation angle of each camera C11, ... installed in the area in advance.

[0090] (Technology 2) The acquisition unit (communication unit 10) acquires information regarding the imaging target (in this disclosure, product shelves SH11 to SH14, SH21 to SH24, SH31 to SH34, SH41) imaged by the plurality of cameras C11, ..., Based on the positions of the plurality of cameras C11,... that are estimated, information regarding the imaging target, and the captured images captured by the plurality of cameras C11,..., when it is determined that the imaging target cannot be imaged by the plurality of cameras C11,..., an analysis unit (processor 11) that generates and outputs support information (for example, messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11) regarding the installation position or installation angle of the cameras C11,... for imaging the imaging target. The camera installation support device (terminal device P1) according to (Technology 1). With this configuration, the camera installation support device (terminal device P1) can determine whether an imaging target imaged by a plurality of cameras C11,... can be imaged based on the estimated installation position or installation angle of each camera C11,... and the captured images captured by each camera C11,.... Therefore, the camera installation support device (terminal device P1) can support the installation of cameras by the user by outputting support information (for example, messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11) regarding the installation position or installation angle of the plurality of cameras C11,... for imaging the imaging target based on the estimated installation position or installation angle of the cameras C11,... and the determination result as to whether the imaging target imaged by the plurality of cameras C11,... can be imaged.

[0091] (Technology 3) The information regarding the area includes a map MP of the area. The analysis unit (processor 11) outputs a mapping image (for example, the analysis result screen SC0 shown in FIG. 8) in which the plurality of cameras C11,... are mapped on the map MP and the support information (for example, messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11) based on the positions of the plurality of cameras C11,... that are estimated. The camera installation support device (terminal device P1) according to (Technology 2). With this configuration, the camera installation support device (terminal device P1) generates a mapping image (for example, the analysis result screen SC0 shown in FIG. 8) that maps each of the cameras C11, ... on the map MP at a position or angle corresponding to the estimated installation position or installation angle of the cameras C11, ..., thereby visualizing the current installation positions or installation angles of the plurality of cameras C11, ... for the user. Further, the camera installation support device (terminal device P1) outputs the mapping image and the support information, that is, outputs the analysis result screens SC11, SC12, SC2 (see FIGS. 9 to 11), thereby visualizing for the user the cameras that need to have their installation positions or installation angles adjusted and supporting the user in installing the cameras.

[0092] (Technology 4) The estimation unit (processor 11) groups the plurality of cameras C11, ... based on the orientation of the moving body shown in the moving body image (that is, the posture of the moving body shown in the captured image or the imaging angle). The analysis unit (processor 11) outputs a mapping image (for example, the analysis result screen SC0 shown in FIG. 8) that maps the plurality of cameras C11, ... on the map based on the estimated positions of the plurality of cameras C11, ... and the result of the grouping, and the support information (for example, the messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11). The camera installation support device (terminal device P1) according to (Technology 3). With this configuration, the camera installation support device (terminal device P1) can visualize the cameras that have captured images from similar imaging angles with respect to the moving body moving along the movement path RT, and supports the user in installing and managing the cameras C11, ....

[0093] (Technology 5) The estimation unit (processor 11) groups the plurality of cameras C11, ... based on the orientation of the moving body shown in the moving body image (that is, the posture of the moving body shown in the captured image or the imaging angle) and the imaging time information of the moving body image. The camera installation support device (terminal device P1) according to (Technology 4). With this configuration, the camera installation support device (terminal device P1) can visualize cameras that image a moving object moving along the movement path RT from similar imaging angles, and supports the installation and management of cameras C11, … by the user.

[0094] (Technology 6) Based on the imaging time information of the moving object images for each of the cameras C11, …, the estimation unit (processor 11) estimates the distance between the plurality of cameras C11, …. The camera installation support device (terminal device P1) according to any one of (Technology 1) to (Technology 5). With this configuration, the camera installation support device (terminal device P1) can estimate the installation positions of adjacent cameras C11, … with higher accuracy.

[0095] (Technology 7) The moving object has landmark information (for example, a color target, a QR code, etc.) that can detect the moving object from the captured image. The adjustment unit (processor 11) further includes: calibrating the imaging parameters of the plurality of cameras C11, … based on the color target appearing in the captured images captured by the plurality of cameras C11, …. The analysis unit (processor 11) outputs the imaging parameters calibrated by the adjustment unit for each of the cameras C11, …. The camera installation support device (terminal device P1) according to any one of (Technology 2) to (Technology 6). With this configuration, even when the models of the plurality of cameras C11, … are different, or when the positions of lighting, etc. for the plurality of cameras C11, … are different, when imaging an imaging target from the set position or installation angle of each camera C11, …, more suitable imaging parameters can be obtained. As a result, the user can obtain imaging parameters suitable for imaging the imaging target for each camera, and thus can set the imaging parameters of each camera C11, … more efficiently.

[0096] (Technology 8) The adjustment unit (processor 11) identifies the models of the plurality of cameras C11,... based on the captured images captured by the plurality of cameras C11,.... The camera installation support device (terminal device P1) described in (Technology 7). With this configuration, even when the camera installation support device (terminal device P1) is not of the models of the plurality of cameras C11,... and there is no setting of the installation position or installation angle of each camera C11,... and the model information of each camera C11,... in advance, the models of each camera C11,... can be specified, and based on the characteristics of the specified models of each camera C11,..., imaging parameters suitable for imaging the imaging target can be obtained.

[0097] (Technology 9) At least one camera C21 among the plurality of cameras C11,... has a light emitting unit (LEDLt1, see FIG. 8) that emits light in a predetermined light emission pattern. When the analysis unit (processor 11) detects the predetermined light emission pattern of the light emitting unit (LEDLt1) based on the captured image, the analysis unit estimates the relative position between the camera C21 including the light emitting unit (LEDLt1) and the camera C11 from which the captured image was captured. The camera installation support device (terminal device P1) according to any one of (Technology 2) to (Technology 8). With this configuration, the camera installation support device (terminal device P1) can identify which camera among the plurality of cameras C11,... the camera captured in the captured image is, and can estimate the installation position or installation angle between the camera from which the captured image was captured and the identified camera with higher accuracy.

[0098] (Technology 10) The plurality of cameras C11,... have a light emitting unit (for example, LEDLt3, see FIG. 12) that emits light. When the analysis unit (processor 11) determines that the imaging target cannot be imaged by the plurality of cameras C11, …, it determines a target camera (for example, camera C21B, see FIG. 12) among the plurality of cameras C11, … that requires adjustment of the installation position or the installation angle, and causes the light emitting unit (LEDLt3) of the target camera (camera C21B) to emit light. The camera installation support device (terminal device P1) according to any one of (Technique 2) to (Technique 9). With this configuration, even when the number of cameras C11, … is large, the camera installation support device (terminal device P1) supports the installation of cameras by the user by visualizing which camera requires adjustment of the installation position or the installation angle. Based on the lighting of the LED (light emitting unit), the user can more easily find and adjust the camera that requires adjustment of the installation position or the installation angle at the site.

[0099] (Technique 11) A camera installation support method performed by at least one computer (terminal device P1) capable of communicating with a plurality of cameras C11, …, acquiring the number information of the cameras C11, …, information regarding the area where the plurality of cameras C11, … are installed, movement path information (movement path RT) of a moving body imaged by the plurality of cameras C11, …, and captured images captured by each of the plurality of cameras C11, …, extracting moving body images in which the moving body is imaged for each of the cameras C11, … from the captured images, estimating and outputting the positions of the plurality of cameras C11, … on the area based on the imaging time information when the moving body images are captured. Camera installation support method. With this configuration, the computer (terminal device P1) can assist the user in installing the cameras by estimating and outputting the positional relationship (i.e., installation position) of each of the plurality of cameras C11, … with respect to the moving object that has moved along the movement path RT and the installation angles of the plurality of cameras C11, … with respect to the moving object that has moved along the movement path RT, based on the movement path RT of the moving object and the moving object image, without inputting the installation positions or installation angles of the cameras C11, … installed in advance on the area.

[0100] As described above, various embodiments have been described with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various modification examples, correction examples, substitution examples, addition examples, deletion examples, and equivalent examples within the scope described in the claims, and it is understood that those also belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the invention, the components in the above-described various embodiments may be arbitrarily combined.

Industrial Applicability

[0101] The present disclosure is useful as a presentation of a camera installation support device and a camera installation support method for assisting the installation of a plurality of cameras.

Explanation of Reference Numerals

[0102] 10 Communication unit 11 Processor 12 Memory 13 Input unit 14 Monitor 100 Camera installation support system B1 Authentication device B1A Biometric information acquisition device C11, C12, C13, C14, C21, C22, C23, C24, C31, C32, C33, C34, C41, C51 Cameras Lt1, Lt2A, Lt2B, Lt3 LEDs MP Map Msg1, Msg2, Msg3 Messages OB11 Column OB12 QR Code P1 Terminal Device RT Movement Route SC0, SC1, SC2, SC11, SC12 Analysis Result Screen SH11, SH12, SH13, SH14, SH21, SH22, SH23, SH24, SH31, SH32, SH33, SH34, SH41 Merchandise Shelves

Claims

1. An acquisition unit that acquires the number of cameras, information regarding an area where the plurality of cameras are installed, movement route information of a moving object imaged by the plurality of cameras, and captured images captured by each of the plurality of cameras; An estimation unit that extracts a moving object image in which the moving object is imaged from among the captured images, and estimates and outputs the positions of the plurality of cameras on the area based on the imaging time information at which the moving object image was captured; A camera installation support device comprising: A camera installation support device.

2. The acquisition unit acquires information regarding an imaging target imaged by the plurality of cameras, and, based on the estimated positions of the plurality of cameras, the information regarding the imaging target, and the captured images captured by the plurality of cameras, when it is determined that the imaging target cannot be imaged by the plurality of cameras, an analysis unit that generates and outputs support information regarding the installation position or installation angle of the camera for imaging the imaging target; The camera installation support device according to claim 1.

3. The information regarding the area includes a map of the area, and the analysis unit outputs a mapping image in which the plurality of cameras are mapped on the map and the support information based on the estimated positions of the plurality of cameras. The camera installation support device according to claim 2.

4. The estimation unit groups the plurality of cameras based on the orientation of the moving object shown in the moving object image, and the analysis unit outputs a mapping image in which the plurality of cameras are mapped on the map and the support information based on the estimated positions of the plurality of cameras and the result of the grouping. The camera installation support device according to claim 3.

5. The estimation unit groups the plurality of cameras based on the orientation of the moving object shown in the moving object image and the imaging time information of the moving object image. The camera installation support device according to claim 4.

6. The estimation unit estimates the distance between the plurality of cameras based on the imaging time information of the moving object image for each camera. The camera installation support device according to claim 1.

7. The moving object has landmark information capable of detecting the moving object from the captured image. An adjustment unit that calibrates the imaging parameters of the plurality of cameras based on the landmark information shown in the captured images captured by the plurality of cameras. The analysis unit outputs the imaging parameters calibrated by the adjustment unit for each of the cameras. The camera installation support device according to claim 2.

8. The adjustment unit identifies the models of the plurality of cameras based on the captured images captured by the plurality of cameras. The camera installation support device according to claim 7.

9. At least one of the plurality of cameras has a light emitting unit that emits light in a predetermined light emission pattern. When the analysis unit detects the predetermined light emission pattern of the light emitting unit based on the captured image, the analysis unit estimates the relative position between the camera including the light emitting unit and the camera that captured the captured image. The camera installation support device according to claim 2.

10. The plurality of cameras have light emitting units that emit light. When the analysis unit determines that the imaging target cannot be imaged by the plurality of cameras, the analysis unit determines a target camera that needs to adjust the installation position or the installation angle among the plurality of cameras, and causes the light emitting unit of the target camera to emit light. The camera installation support device according to claim 2.

11. A camera installation support method performed by at least one computer capable of communicating with a plurality of cameras, acquiring the number information of the cameras, information regarding the area where the plurality of cameras are installed, movement path information of a moving body imaged by the plurality of cameras, and captured images captured by each of the plurality of cameras; extracting, for each of the cameras, a moving body image in which the moving body is imaged from the captured images; estimating and outputting the positions of the plurality of cameras on the area based on the imaging time information at which the moving body image was captured. Camera installation support method.

Images