Camera installation assistance device and camera installation assistance method

Abstract

This camera installation assistance device: acquires information on the number of cameras, information on an area, mobile object movement path information, and an image captured by each camera; extracts for each camera a mobile object image in which the mobile object is captured; and estimates, then outputs, the position of the camera in the area on the basis of information on the time when the mobile object image was captured.

Description

Camera installation support device and camera installation support method

[0001] The present disclosure relates to a camera installation support device and a camera installation support method.

[0002] Patent Document 1 discloses a control device that detects a moving object from images acquired by multiple infrastructure cameras installed so as to be able to acquire images of the surrounding environment including the moving object, estimates the detection accuracy of the detected moving object, and, based on the estimated detection accuracy, estimates the state of an infrastructure camera network constructed by the multiple infrastructure cameras.

[0003] Furthermore, Patent Document 2 discloses a camera calibration method for calibrating a camera from a plurality of images captured by a plurality of cameras installed on a vehicle of an area surrounding the vehicle, the area including calibration markers that are provided in advance on a road surface on which the vehicle is placed and that are a plurality of intersections where at least two first parallel lines and two second parallel lines perpendicular to the two parallel lines intersect. The camera calibration method generates an overhead image by performing viewpoint conversion on each of the plurality of images based on first camera parameters that are stored in advance, recognizes a plurality of intersections from the generated overhead image, and calculates second camera parameters that enable the generation of a precise overhead image based on the recognized plurality of intersections and information on the positions and attitudes of the plurality of cameras.

[0004] JP 2022-129103 A JP 2014-64224 A

[0005] Conventionally, when installing multiple cameras in a store or warehouse to monitor, for example, shelves, products, inventory, or missing items, the user installing the cameras had to select candidate camera installation locations from a map of the installation location, manually link the installation locations to the individual cameras in advance, and adjust the camera installation locations or installation angles to enable capturing images of the monitored objects, which was time-consuming and cost-intensive. Furthermore, the more cameras a user installed, the more time and effort it required, such as replacing cameras due to malfunctions or adjusting camera installation positions due to changes in the layout of the shelves to be monitored. Therefore, there has been a demand for technology that can identify cameras that need to be adjusted in installation locations when installing multiple cameras, thereby reducing the labor and cost of initial installation and post-installation management.

[0006] The present disclosure has been devised in view of the above-described conventional circumstances, and aims to provide a camera installation support device and a camera installation support method that support the installation of multiple cameras.

[0007] The present disclosure provides a camera installation support device including an acquisition unit that acquires information on the number of cameras, information on the area in which the multiple cameras are installed, movement path information of moving objects captured by the multiple cameras, and captured images captured by each of the multiple cameras, and an estimation unit that extracts moving object images of the moving object from the captured images for each camera, and estimates and outputs the positions of the multiple cameras in the area based on image capture time information when the moving object images were captured.

[0008] The present disclosure also provides a camera installation support method performed by at least one computer capable of communicating with a plurality of cameras, which acquires information on the number of cameras, information on the area in which the plurality of cameras are installed, information on the movement path of moving objects captured by the plurality of cameras, and captured images captured by each of the plurality of cameras, extracts moving object images of the moving objects from the captured images for each camera, and estimates and outputs the positions of the plurality of cameras in the area based on image capture time information when the moving object images were captured.

[0009] According to the present disclosure, it is possible to assist in the installation of multiple cameras.

[0010] FIG. 1 is a diagram showing an example of a use case of a camera installation support system according to an embodiment. FIG. 2 is a diagram showing an example of the internal structure of a camera installation support system according to an embodiment. FIG. 3 is a sequence showing an example of an operation procedure of a camera installation support system according to an embodiment. FIG. 4 is a flowchart showing an example of analysis processing of a terminal device in an embodiment. FIG. 5 is a diagram showing an example of advance setting information. FIG. 6 is a diagram explaining an example of identification of an imaging target. FIG. 7 is a diagram explaining an example of estimation of the installation position of a camera. FIG. 8 is a diagram showing an example of an analysis result screen. FIG. 9 is a diagram showing an example of an analysis result screen. FIG. 10 is a diagram showing an example of an analysis result screen. FIG. 11 is a diagram showing an example of analysis result screen. FIG. 12 is a diagram showing an example of camera control based on the analysis result.

[0011] Hereinafter, with reference to the drawings as appropriate, embodiments that specifically disclose the configuration and operation of a camera installation support device and a camera installation support method according to the present disclosure will be described in detail. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters or redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter recited in the claims.

[0012] An example use case of the camera installation support system 100 according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example use case of the camera installation support system 100 according to the embodiment. Note that in Fig. 1, only the lens LS of the camera C11 is given a reference numeral, and the reference numerals of the lenses of the other cameras are omitted.

[0013] 1, multiple cameras C11, C12, C13, C14, C21, C22, C23, C24, C31, C32, C33, and C34 are installed on product shelves SH11, SH12, SH13, SH14, SH21, SH22, SH23, SH24, SH31, SH32, SH33, and SH34, respectively, where products are displayed. Each of the multiple cameras C11 to C14, C21 to C24, and C31 to C34 captures an image of a product shelf installed opposite the product shelf on which the camera is installed (for example, product shelf SH21 in the case of camera C11). Camera C41 is a camera attached to the ceiling and captures an image of product shelf SH41.

[0014] Terminal device P1 acquires images captured by each of multiple cameras C11 to C14, C21 to C24, C31 to C34, and C41. To assist the user in installing the cameras, terminal device P1 estimates the installation position of each camera based on the images captured by each camera, and analyzes (determines) and outputs whether each camera can capture an image of the product shelf that is its target.

[0015] Here, as an example, the multiple cameras C11, . . . are installed indoors, but it goes without saying that they may also be installed outdoors.

[0016] An example of the internal configuration of the camera installation support system 100 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the internal configuration of the camera installation support system 100 according to the embodiment.

[0017] The camera installation support system 100 includes at least one terminal device P1 and a plurality of cameras C11, .... The camera installation support system 100 has each of the plurality of cameras C11, ... capture an image of a moving object, and performs image analysis processing on the captured image captured by the terminal device P1, thereby estimating the current installation positions of the plurality of cameras C11, ... and determining whether or not the imaging target can be imaged. Based on the calculated installation positions of each of the plurality of cameras C11, ..., the camera installation support system 100 provides support for adjusting the current installation of each of the plurality of cameras C11, ... to be more suitable for imaging the imaging target.

[0018] The terminal device P1 is connected to each of the multiple cameras C11, etc. so as to be able to communicate data with them. The terminal device P1 is realized, for example, by a personal computer (hereinafter referred to as "PC"), a notebook PC, a tablet terminal, etc. The terminal device P1 includes a communication unit 10, a processor 11, a memory 12, an input unit 13, and a monitor 14.

[0019] The communication unit 10 is connected to each of the multiple cameras C11, ... via wired or wireless communication. The communication unit 10 acquires captured images transmitted from each of the multiple cameras C11, ... and outputs them to the processor 11. Note that the wireless communication network referred to here is provided in accordance with a wireless communication standard such as a wireless LAN (Local Area Network), a wireless WAN (Wide Area Network), 4G (Fourth Generation Mobile Communication System), 5G (Fifth Generation Mobile Communication System), or Wi-Fi (registered trademark).

[0020] The processor 11 is configured using, for example, a Central Processing Unit (CPU) or a Field Programmable Gate Array (FPGA), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 references the programs and data stored in the memory 12 and executes the programs to perform its functions.

[0021] The memory 12 includes, for example, a random access memory (RAM) as a work memory used when the processor 11 executes each process, and a read-only memory (ROM) that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operation of the processor 11. The memory 12 records information such as the number of cameras set by the user, the camera models, a map showing the areas where the cameras are installed, the movement route of the moving object, and information about the subject to be imaged by the cameras.

[0022] The input unit 13 is a user interface that can accept input operations by a user, and is realized by, for example, a keyboard, a mouse, a touch panel, etc. The input unit 13 converts the content of the accepted input operation into an electrical signal and outputs it to the processor 11. When the input unit 13 is realized by a touch panel, the input unit 13 may be configured integrally with the monitor 14.

[0023] The monitor 14 is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL) display, and displays analysis result screens SC0, SC11, SC12, and SC2 (see FIGS. 8 to 11) output from the processor 11.

[0024] Each of the multiple cameras C11, ... is a so-called surveillance camera, and is installed to capture (monitor) an image capture target. The multiple cameras C11, ... may be of different types and models, and may be any camera, such as an infrared camera, a camera that can pan, tilt, or zoom, or a wide-angle camera. The multiple cameras C11, ... are installed arbitrarily by the user, and capture images of the product shelves that are the image capture target.

[0025] Each of the cameras C11, ... includes at least a lens LS and an image sensor (not shown). Each of the cameras C11, ... is connected to the terminal device P1 via wired or wireless communication, and transmits to the terminal device P1 image capture time information indicating the time the captured image was captured and identification information for identifying the camera.

[0026] Next, an example of an operation procedure of the camera installation support system 100 according to the embodiment will be described with reference to Fig. 3 and Fig. 4 . Fig. 3 is a sequence diagram showing an example of an operation procedure of the camera installation support system 100 according to the embodiment. Fig. 4 is a flowchart showing an example of an analysis process of the terminal device P1 according to the embodiment. Note that, for ease of understanding, Figs. 3 and 4 describe an example in which the camera installation support system 100 is operated in the example use case shown in Fig. 1 , but it goes without saying that the present invention is not limited to this.

[0027] The plurality of cameras C11, . . . are installed by the user at arbitrary positions within an area shown on a map MP, which will be described later (St11).

[0028] The processor 11 accepts an input operation by the user via the input unit 13, and acquires information about the area in which the multiple cameras C11, ... are installed and information about the multiple installed cameras C11, ... (St12). Note that the processor 11 may accept an input operation by the user and further acquire additional information, which will be described later.

[0029] Specifically, the processor 11 acquires, as information about the area, a map MP of the area (see FIG. 5), a movement route RT of a moving object on the map MP (see FIG. 5), information about the object captured by the multiple cameras C11, etc. Furthermore, the processor 11 acquires, as information about the multiple cameras C11, etc., information about the number of cameras, the model of the cameras, the number of light-emitting diodes (hereinafter referred to as "LEDs") provided in the cameras, the light emission color, the light emission pattern, etc.

[0030] Note that the various information input in step St12 is not limited to the examples described above. For example, when there are multiple imaging targets, the processor 11 may acquire, as information about the imaging targets, information that enables each imaging target to be identified, such as an image of the imaging target, a QR code (registered trademark, the same applies below) assigned to the imaging target, information about objects located in front of, behind, or near the imaging target, information indicating the boundaries of each imaging target, etc. Note that "in front of the imaging target" refers to a direction approaching the camera C11, etc. "Behind the imaging target" refers to a direction moving away from the camera C11, etc.

[0031] Furthermore, the movement route RT of the moving object is set so that all of the cameras C11, ... can capture an image of the moving object moving on the movement route RT at least once. Information about the movement route RT may be acquired by accepting an input operation of the movement route RT on the map MP, or by accepting an input operation of position (coordinate) information of a plurality of passing points passed on the movement route RT.

[0032] The processor 11 accepts an input operation by the user via the input unit 13 and registers information about the moving object (St13).

[0033] The information about the moving body is information that enables the moving body to be detected from a captured image, and includes, for example, at least one of information such as an image of the moving body, landmark information about at least one landmark given to the moving body (for example, information about a QR code, color target, marker, etc. attached to or carried by the moving body), etc. If the moving body is an object that travels automatically, the information about the moving body may further include information about the moving speed of the moving body, etc.

[0034] Furthermore, in order to estimate the installation angles of each of the multiple cameras C11, ..., if a square mark such as a QR code or a sticker is attached to the moving object, the processor 11 may further accept registration of the size of the mark, an image of the mark, etc. Note that the mark does not have to be attached to the moving object itself. For example, if the moving object is a person, the mark may be attached to an object (e.g., a shopping cart) that moves with the person.

[0035] The cameras C11,... begin capturing images of the moving object and the object to be captured based on a user's direct input operation to the terminal device P1 or their own cameras (St14). After the cameras C11,... begin capturing images of the moving object, the moving object moves along the movement route RT. After the cameras C11,... have finished moving, the cameras C11,... end capturing images based on a user's direct input operation to the terminal device P1 or their own cameras (St15), and associate the captured images with capturing time information indicating the capture time and identification information that can identify the own cameras, and transmit these to the terminal device P1 (St16).

[0036] The processor 11 performs an image analysis process on the captured images transmitted from each of the cameras C11, ..., and performs a process of detecting and identifying a moving object appearing in the captured images (St17). The processor 11 performs an analysis process of estimating the installation positions or installation angles of the cameras C11, ... installed on the map MP, based on the captured images of the moving object, and analyzing whether the cameras C11, ... can capture an image of the object to be captured (St18).

[0037] The processor 11 identifies the model of each of the cameras C11, ... based on information about the camera model that was input in advance and the characteristics of the captured image obtained for each camera (St18A). The processor 11 performs image correction on the captured images transmitted from each of the cameras C11, ... based on the identified model information so that the captured images are suitable for capturing images of moving objects and imaging targets (St18B).

[0038] In this case, if the moving body has a color target, a QR code, or the like as an example of a mark that enables the moving body itself to be detected, and the processor 11 detects the color target, the QR code, or the like from the captured image, the processor 11 may generate correction information for adjusting the imaging parameters of each camera to imaging parameters that are more suitable for imaging the moving body and the imaging target, based on the color target, the QR code, or the like that appears in the captured image. The correction information is output together with the analysis result.

[0039] The processor 11 uses the corrected captured image to perform a recognition process for the target captured in the captured image for each camera (St18C). The processor 11 recognizes the target based on a pre-registered image of the target or identification information that can identify the target. Specifically, the processor 11 performs a recognition process based on the image of the target, recognizes a mark (e.g., a QR code, a sticker, etc.) if the target is provided with a mark, or recognizes an object if there is one near the target.

[0040] Based on the recognition results, the processor 11 determines whether it is possible to capture images of all the objects to be imaged using the multiple cameras C11, ..., and if it determines that it is not possible to capture the objects to be imaged, it generates support information to assist in adjusting the installation position or installation angle of the cameras so that the objects to be imaged can be captured (St18D).

[0041] Specifically, when a square mark (e.g., a QR code, a sticker, etc.) is attached, the processor 11 estimates the distance, installation position, or installation angle of the camera relative to the imaging target based on the size (number of pixels) of the mark shown in the captured image, the position where the mark appears, and the ratio of the lengths of the four sides of the mark. Based on the estimated camera installation position or installation angle, the processor 11 calculates the camera installation position or installation angle that enables the entire imaging target to be captured, and generates support information (support information for adjusting the camera installation). Note that the support information may include information such as a camera zoom magnification that is more suitable for imaging the imaging target.

[0042] The processor 11 then performs the moving object identification process again based on the corrected captured images, and extracts captured images of the moving object (hereinafter referred to as "moving object images") for each camera (St18E). A single camera may extract multiple moving object images. The processor 11 then sorts the extracted moving object images in chronological order based on the capture time information of the captured images (St18E).

[0043] The processor 11 identifies the imaging order of the cameras that captured the moving object based on the imaging time information of the aligned moving object images. The processor 11 calculates the distance between two consecutive cameras in the imaging order, which is the distance in the direction along the moving object's movement, based on the difference (time difference) between the imaging time information of the moving object images captured by each camera (St18F).

[0044] For example, if the processor 11 estimates, based on images of the moving object arranged in chronological order, that the moving object has been imaged in the order of camera C11, camera C21, camera C12, and camera C22, the processor 11 calculates the distance between camera C11 and camera C21 in the direction along the moving direction of the moving object, the distance between camera C21 and camera C12, and the distance between camera C12 and camera C22. If the angles of view of two consecutive cameras in the camera image capturing order partially overlap and the moving speed and actual moving path (moving direction) of the moving object can be calculated with higher accuracy, the processor 11 can calculate the distance between the two cameras with higher accuracy.

[0045] The processor 11 executes a grouping process to group cameras among the cameras C11, ... that have similar imaging directions of the moving object into one group (for example, a first group GP1, a second group GP2, a third group GP3, and a fourth group GP4 shown in FIG. 8 ) based on the orientation (imaging direction) of the moving object captured in the moving object image and the imaging time information of the moving object image (St18G). Note that it is sufficient that one group includes at least one camera.

[0046] For example, the processor 11 analyzes whether the moving object was imaged from the right, left, or above, with the direction along the travel path RT being the front, and groups the cameras into those that imaged the moving object from the right, the left, and the above. For example, in the example shown in Figure 8, the processor 11 groups cameras C11 to C14 into a group that images the moving object from the left, cameras C21 to C24 and C31 to C34 into a group that images the moving object from the right, and camera C41 into a group that images the moving object from above.

[0047] The processor 11 re-extracts images of the moving object captured by the cameras belonging to the same group, sorts them in chronological order, and determines whether there is a time when the moving object is not captured by any of the cameras belonging to the same group for a predetermined period of time or longer, based on the image capture time information of the sorted moving object images. If the processor 11 determines that there is a time when the moving object is not captured by any of the cameras capturing images from the same direction for a predetermined period of time or longer, the processor 11 further groups the cameras into cameras that captured images of the moving object before this time and cameras that captured images of the moving object after this time.

[0048] In addition, the processor 11 may re-extract moving body images captured by cameras belonging to the same group, arrange them in chronological order, determine whether there are any overlapping areas between two consecutive moving body images in chronological order, and if it determines that there are no overlapping areas, perform a grouping process to further divide the groups between the cameras that captured these two moving body images.

[0049] 8, cameras C21 to C24 and C31 to C34 belong to the same group that captures images of a moving object from the right side as described above, but processor 11 determines that there is a time when the moving object is not captured for a predetermined period of time between cameras C24 and C34. Processor 11 further divides the group that captures images of the moving object from the right side into two groups: a group including cameras C21 to C24 that captured images of the moving object before this time, and cameras C31 to C34 that captured images of the moving object after this time.

[0050] The processor 11 matches each group with the map MP of the target area, and maps each of the multiple cameras C11, . . . on the map MP (St18H).

[0051] The processor 11 adjusts the positions of the mapped cameras C11, ... based on the additional information acquired in step St12 or input after step St18H (St18I). Note that the processing of step St18I is not essential and may be omitted if there is no additional information.

[0052] The processor 11 determines whether mapping of all the cameras C11, ... onto the map MP of the target area is complete (St18J). If the processor 11 determines that mapping of all the cameras C11, ... onto the map MP of the target area is complete (St18J, YES), the processor 11 ends the analysis process of the installed cameras (step St18).

[0053] On the other hand, if the processor 11 determines that mapping of all cameras C11, ... onto the map MP of the target area has not been completed (St18J, NO), it generates a reinstallation command requesting reinstallation of cameras that cannot be mapped (i.e., have not been mapped), and outputs the command to the monitor 14 (St18K). Note that the reinstallation command may be output together with the analysis result screens SC1, SC11, SC12, and SC2 (see FIGS. 8 to 11) described below.

[0054] The processor 11 generates analysis result screens SC1, SC11, SC12, and SC2 (see FIGS. 8 to 11 ) in which the positions of the multiple cameras C11, ... are mapped on the map MP, and outputs them to the monitor 14 (St19). Note that, if there is support information regarding camera installation, the processor 11 may generate analysis result screens SC11 and SC2 (see FIGS. 9 and 11 ) including the support information, and output them to the monitor 14. The processor 11 may also generate an analysis result screen SC2 (see FIG. 11 ) in which the angles of view of the multiple cameras C11, ... are superimposed on the map MP, and output them to the monitor 14.

[0055] As described above, the camera installation support system 100 according to the embodiment can estimate the respective positional relationships (i.e., installation positions) of the multiple cameras C11,... with respect to the moving body that has moved along the movement path RT, the installation angles of the multiple cameras C11,... with respect to the moving body that has moved along the movement path RT, etc., when installing the multiple cameras C11,..., without inputting the installation positions or installation angles of each camera C11,... installed in an area in advance, based on the movement path RT of the moving body and images of the moving body. Furthermore, the camera installation support system 100 can determine whether the imaging target is being captured by the multiple cameras C11,..., based on the estimated installation positions or installation angles of each camera C11,... and the captured images captured by each camera C11,...

[0056] Therefore, the camera installation support system 100 can support the user in installing the cameras by outputting analysis result screens SC11, SC2 (see Figures 9 and 11) that include support information for supporting the installation positions or installation angles of the multiple cameras C11, ... for capturing images of the target, based on the estimated installation positions or installation angles of the cameras C11, ... and the determination result of whether the target can be captured by the multiple cameras C11, ....

[0057] Next, the map MP and the moving route RT of the moving object will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the map MP and the moving route RT.

[0058] The map MP shows a target area including an area where the multiple cameras C11 are installed, image capture targets (product shelves SH11 to SH14, SH21 to SH24, SH31 to SH34, SH41 in the example shown in FIG. 5 ) and areas of the multiple cameras C11, and a movement route RT of the mobile object. Note that the map MP may be data showing the target area in 2D or may be data showing the target area in 3D.

[0059] Information on the moving body's movement route RT may be obtained by accepting an input operation of the movement route RT on the map MP displayed on the monitor 14, or by accepting an input operation of multiple different position (coordinate) information.

[0060] An example of the additional information will now be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a diagram illustrating an example of identifying an image capture target. Fig. 7 is a diagram illustrating an example of estimating the installation positions of cameras C11A and C12A. Note that Fig. 6 and Fig. 7 each illustrate only a portion of the map MP shown in Fig. 5 for ease of understanding.

[0061] First, referring to Figure 6, an example will be described in which there is a mark in front of, behind, or near the product shelf SH11 that is the subject of imaging, which makes it possible to identify the product shelf SH11 itself and to specify the position of the product shelf SH11, and this mark is used as additional information.

[0062] 6 , the processor 11 acquires additional information that a pillar OB11 (landmark) is located to the left of the product shelf SH11 as viewed from the camera C21A, and that a QR code OB12 is located at the right end of the product shelf SH11. The camera C21A has an angle of view Ar21A, captures an image of the product shelf SH11, and transmits the captured image to the terminal device P1. In this case, the processor 11 can analyze the locations of the left and right ends of the product shelf SH11 by recognizing the pillar OB11 and the QR code OB12 provided on the product shelf SH11 from the image captured by the camera C21A.

[0063] That is, the processor 11 can estimate the position of the camera C21A with higher accuracy by detecting an index that is in front of, behind, or near the image capture target, such as the pillar OB11 or the QR code OB12. Furthermore, when the angle of view Ar21A of the camera C21A includes another camera (camera C11), the processor 11 can estimate the installation position or installation angle of the camera C11 with higher accuracy based on the position of the camera C11 relative to the image capture target, the pillar OB11, or the QR code OB12.

[0064] When providing a marker for each imaging target (i.e., each product shelf) in order to identify each imaging target, different markers are provided for imaging targets that are physically adjacent to each other or that belong to the same group and are imaged by adjacent cameras. This allows the processor 11 to identify which of two or more imaging targets located on the map MP is an imaging target when the imaging targets are imaged by a single camera, or when the imaging targets are imaged by multiple cameras that belong to the same group and have at least partially overlapping angles of view.

[0065] Next, referring to Figure 7, an example will be described in which cameras C21B and C22B have marks that enable cameras C21B and C22B to be identified and their positions to be specified, and these marks are used as additional information. The marks shown in Figure 7 are, for example, the number of lit LEDs provided on cameras C21B and C22B. Note that the marks are not limited to the above-described example, and may be LED lighting patterns, LED lighting colors, or a combination of LED lighting colors, or the appearance of cameras C21B and C22B, or marks (e.g., QR codes, stickers, etc.) attached to cameras C21B and C22B.

[0066] 7 , the processor 11 acquires additional information that the LED Lt1 provided on the camera C21B and the LEDs Lt2A and Lt2B provided on the camera C22B are lit. The camera C11A has an angle of view Ar11A, captures images of the product shelves SH21 and SH22, and transmits the captured images to the terminal device P1. The camera C12A has an angle of view Ar11A, captures images of the product shelf SH22, and transmits the captured images to the terminal device P1.

[0067] In such a case, the processor 11 distinguishes between the camera C21B equipped with the LED Lt1 and the camera C22B equipped with the LEDs Lt2A and Lt2B by detecting the LED Lt1 and the LEDs Lt2A and Lt2B from the captured image captured by the camera C11A. The processor 11 distinguishes between the camera C22B equipped with the LEDs Lt2A and Lt2B by detecting the LEDs Lt2A and Lt2B from the captured image captured by the camera C12A.

[0068] Furthermore, when multiple cameras C21B and C22B appear in a single captured image, the processor 11 may estimate the installation positions or installation angles of the cameras C21B and C22B based on the positions and distances of the cameras C21B and C22B in the captured image. When numbers, names, etc. that can identify each of the multiple cameras C11, ... are input in advance, the processor 11 may identify the numbers, names, etc. of the cameras C21B and C22B based on the number of lit LEDs. When the processor 11 determines that the same camera C22B is being captured by multiple different cameras C11A and C12A, the processor 11 may estimate the installation positions or installation angles of the cameras C11A and C12A based on the position of the camera C22B in the captured image. The cameras C21B and C22B or the LEDs Lt1, Lt2A, and Lt2B referred to here may be replaced with the landmarks described in FIG. 6 .

[0069] Next, a grouping example of a plurality of cameras C11, ... and an analysis result screen SC0 will be described with reference to Fig. 8. Note that the analysis result screen SC0 shown in Fig. 8 does not include support information and does not show the angle of view of each camera, in order to make the grouping example easier to understand.

[0070] The analysis result screen SC0 is generated by mapping a plurality of grouped cameras C11, ... on a map MP. In the analysis result screen SC0 shown in Fig. 8, cameras C11 to C14 are grouped into a first group GP1, cameras C21 to C24 into a second group GP2, cameras C31 to C34 into a third group GP3, and camera C41 into a fourth group GP4, and dashed lines are superimposed on the map MP to surround the cameras belonging to each group.

[0071] Here, the first group GP1 includes cameras C11 to C14 that imaged moving objects from the left side with the direction along the travel route RT as their front. The second group GP2 includes cameras C21 to C24 that imaged moving objects from the right side with the direction along the travel route RT as their front. The third group GP3 includes cameras C31 to C34 that imaged moving objects from the right side with the direction along the travel route RT as their front. The fourth group GP4 includes camera C41 that imaged moving objects from above with the direction along the travel route RT as their front.

[0072] In this embodiment, the processor 11 will be described as performing grouping of multiple cameras C11, ... based on the image capture time information at which the camera captured an image of the moving object and whether the image of the moving object, with its front facing the direction along the travel route RT, was captured from the right, left, or top side, but the grouping method is not limited to this.

[0073] Next, an example of support information for assisting in adjusting the installation angle of the camera will be described with reference to Figures 9 and 10. Figure 9 is a diagram showing an example of an analysis result screen SC11. Figure 10 is a diagram showing an example of an analysis result screen SC12. Note that in Figures 9 and 10, the dashed lines indicating the grouping results shown in Figure 8 are omitted to make the support information easier to see.

[0074] 9 , when the processor 11 determines that the entire product shelf SH24 is not captured in the image captured by the camera C14, the processor 11 estimates an installation position or installation angle at which the camera C14 can capture the entire product shelf SH24. Here, the processor 11 generates a message Msg1 as support information instructing the adjustment of the installation direction of the camera C14 so that the camera C14 faces in the right direction relative to the current angle of view of the camera C14. The analysis result screen SC11 is generated by including, as support information, a message Msg1 "Camera #14 Orientation Adjustment '← Left'" instructing the adjustment of the installation direction (orientation) of the camera C14.

[0075] The processor 11 may record the support information generated as a result of the analysis, the grouping results of the multiple cameras C11, . . . , and the installation positions or installation angles of the multiple cameras C11, .

[0076] The user adjusts the installation direction of the camera C14 based on the message Msg1 on the analysis result screen SC11. After adjusting the installation angle of the camera C14, the user causes the multiple cameras C11 to capture images of the moving object again. Based on the captured images captured again by the multiple cameras C11, the terminal device P1 reanalyzes the grouping of the multiple cameras C11 and the installation positions or angles of each of the multiple cameras C11, and generates an analysis result screen SC12 and outputs it to the monitor 14.

[0077] 10 , when the processor 11 determines that the entire product shelf SH24 is captured in the image captured again by the camera C14, it determines that the support indicated by the support information obtained by the previous analysis process has been completed, and generates a message Msg2 "Camera #14 Orientation Adjustment 'OK'" to indicate that the installation angle of the camera C14 has been improved. The processor 11 generates an analysis result screen SC12 including the message Msg2 and outputs it to the monitor 14.

[0078] As a result, the terminal device P1 in the embodiment can visualize and present to the user information for adjusting the installation position or installation angle of the camera as information to support camera installation. Furthermore, by recording the analysis results, the terminal device P1 can determine whether the installation position or installation angle of the camera has been improved by comparing the analysis results, and visualize and present to the user information indicating whether the installation position or installation angle of the camera has been improved.

[0079] Next, an example of support information proposing the installation of additional cameras will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of an analysis result screen SC2. Note that Fig. 11 illustrates only a portion of the map MP shown in Fig. 5 for ease of understanding, and omits the dashed lines indicating the grouping results shown in Fig. 8 to make the support information easier to see.

[0080] 11 , camera C21C has an angle of view Ar21C and captures an image of product shelf SH11C. Camera C22C has an angle of view Ar22C and captures an image of product shelf SH12C. Based on the captured images captured by adjacent cameras C21C and C22C, processor 11 determines whether the angles of view Ar21C and Ar22C overlap when capturing the image of product shelf SH12C, or whether the number of cameras capturing the entire product shelf SH12C is sufficient or insufficient. If processor 11 determines that the entire product shelves SH11C and SH12C cannot be captured even after adjusting the installation positions or angles of cameras C21C and C22C, it generates message Msg3 as support information suggesting the addition of a new camera C51. The processor 11 generates an analysis result screen SC2 including a message Msg3 “Camera Addition Proposal” proposing the addition of a new camera C51, and outputs the screen to the monitor 14.

[0081] As a result, the terminal device P1 in the embodiment can propose the installation of additional cameras as information to support the installation of cameras. Note that, although an example of proposing the installation of additional cameras has been described here, the terminal device P1 may also propose the removal of cameras when the entire angle of view of a specific camera overlaps with the angle of view of another camera.

[0082] This also allows the terminal device P1 in the embodiment to optimize the number of cameras installed and the imaging area.

[0083] Next, a method for notifying support information by controlling an actual camera will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of control of camera C 21D based on the analysis results.

[0084] 12, the camera C21D is a camera that can control turning on / off of the LED Lt3 based on a control command transmitted from the terminal device P1. The camera C21D has an angle of view Ar21D and captures an image of the product shelf SH11.

[0085] When the processor 11 determines that the installation angle of the camera C21D needs to be adjusted based on the captured image taken by the camera C21D, the processor 11 generates a control command requesting that the LED Lt3 be turned on to notify the user that the installation angle of the camera C21D needs to be adjusted, and transmits the control command to the camera C21D. The camera C21D turns on the LED Lt3 based on the control command transmitted from the terminal device P1.

[0086] If assistance information is generated for multiple cameras, the processor 11 may illuminate the LEDs in different numbers, patterns, or colors for each camera. The processor 11 may also generate and output an analysis result screen that further includes information on the number, patterns, or colors of LEDs illuminated for each camera. This allows the processor 11 to visualize cameras that require adjustment of their installation positions or angles, thereby assisting the user in installing the cameras.

[0087] This makes it easier for users to find cameras on-site that require adjustment of their installation positions or angles, even when a large number of cameras are installed. Also, even when adjustment of the installation positions or angles is performed by remote instruction, it makes it easier for users to find which of the installed cameras requires adjustment of their installation positions or angles.

[0088] (Additional Notes) The above description of each embodiment discloses the following techniques.

[0089] (Technology 1) A camera installation support device (terminal device P1) comprising: an acquisition unit (communication unit 10) that acquires information on the number of cameras C11, ..., information on the area in which the multiple cameras C11, ... are installed, movement path information (movement path RT) of moving objects captured by the multiple cameras C11, ..., and captured images captured by each of the multiple cameras C11, ...; and an estimation unit (processor 11) that extracts moving object images captured by the moving objects from the captured images for each of the cameras C11, ..., and estimates the positions of the multiple cameras C11, ... in the area based on image capture time information when the moving object images were captured.

[0090] With this configuration, the camera installation support device (terminal device P1) can assist the user in installing cameras by estimating and outputting the positional relationship (i.e., installation position) of each of the multiple cameras C11, ... relative to the moving body that has moved on the movement path RT, the installation angle of the multiple cameras C11, ... relative to the moving body that has moved on the movement path RT, etc., based on the moving body's movement path RT and the moving body image, without the need to input the installation position or installation angle of each camera C11, ... installed in the area in advance.

[0091] (Technology 2) The acquisition unit (communication unit 10) acquires information about an imaging target (in the present disclosure, product shelves SH11 to SH14, SH21 to SH24, SH31 to SH34, SH41) imaged by the plurality of cameras C11, ..., and when it is determined that the imaging target has not been imaged by the plurality of cameras C11, ... based on the estimated positions of the plurality of cameras C11, ..., information about the imaging target, and the captured images captured by the plurality of cameras C11, ..., an analysis unit (processor 11) generates and outputs support information (for example, messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11) about the installation position or installation angle of the cameras C11, ... for imaging the imaging target. (Technology 1) The camera installation support device (terminal device P1) described above is further provided with:

[0092] With this configuration, the camera installation support device (terminal device P1) can determine whether or not the imaging target imaged by the multiple cameras C11, ... is being captured, based on the estimated installation position or installation angle of each camera C11, ... and the captured image captured by each camera C11, .... Therefore, the camera installation support device (terminal device P1) can support the user in installing the cameras by outputting support information (e.g., messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11 ) that supports the installation position or installation angle of the multiple cameras C11, ... for capturing the imaging target, based on the estimated installation position or installation angle of the cameras C11, ... and the determination result of whether or not the imaging target imaged by the multiple cameras C11, ... is being captured.

[0093] (Technology 3) The information about the area includes a map MP of the area, and the analysis unit (processor 11) outputs a mapping image (for example, an analysis result screen SC0 shown in FIG. 8) in which the multiple cameras C11, ... are mapped on the map MP based on the estimated positions of the multiple cameras C11, ..., and the support information (for example, messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11). This is the camera installation support device (terminal device P1) described in (Technology 2).

[0094] With this configuration, the camera installation support device (terminal device P1) can generate a mapping image (for example, the analysis result screen SC0 shown in FIG. 8 ) in which each camera C11, ... is mapped on the map MP at a position or angle corresponding to the estimated installation position or installation angle of the camera C11, ..., thereby visualizing to the user the current installation positions or installation angles of the multiple cameras C11, .... Furthermore, the camera installation support device (terminal device P1) outputs the mapping image and support information, that is, by outputting the analysis result screens SC11, SC12, SC2 (see FIGS. 9 to 11 ), it can visualize to the user cameras that require adjustment of their installation position or installation angle, thereby supporting the user in installing the cameras.

[0095] (Technology 4) The camera installation support device (terminal device P1) described in (Technology 3) is characterized in that the estimation unit (processor 11) groups the multiple cameras C11, ... based on the orientation of the moving body captured in the moving body image (i.e., the posture or imaging angle of the moving body captured in the captured image), and the analysis unit (processor 11) outputs a mapping image (e.g., analysis result screen SC0 shown in FIG. 8) in which the multiple cameras C11, ... are mapped on the map based on the estimated positions of the multiple cameras C11, ... and the grouping result, and the support information (e.g., messages Msg1, Msg2, Msg3, etc. shown in FIGS. 9 to 11).

[0096] With this configuration, the camera installation support device (terminal device P1) can visualize cameras that capture images from similar imaging angles of a moving object moving along the travel route RT, thereby supporting the user in installing and managing cameras C11, ...

[0097] (Technology 5) The camera installation support device (terminal device P1) described in (Technology 4) is configured such that the estimation unit (processor 11) groups the plurality of cameras C11, ... based on the orientation of the moving body shown in the moving body image (i.e., the posture or imaging angle of the moving body shown in the captured image) and imaging time information of the moving body image.

[0098] With this configuration, the camera installation support device (terminal device P1) can visualize cameras that capture images from similar imaging angles of a moving object moving along the travel route RT, thereby supporting the user in installing and managing cameras C11, ...

[0099] (Technology 6) The camera installation support device (terminal device P1) described in any one of (Technology 1) to (Technology 5), wherein the estimation unit (processor 11) estimates the distance between the plurality of cameras C11, ... based on image capture time information of the moving body image for each of the cameras C11, ....

[0100] With this configuration, the camera installation support device (terminal device P1) can estimate the installation positions of adjacent cameras C11, . . . with higher accuracy.

[0101] (Technology 7) The camera installation support device (terminal device P1) described in any one of (Technology 2) to (Technology 6) is further provided with an adjustment unit (processor 11) that calibrates imaging parameters of the plurality of cameras C11, ... based on the color target that appears in the image captured by the plurality of cameras C11, ..., and the analysis unit (processor 11) outputs the imaging parameters calibrated by the adjustment unit for each of the cameras C11, ....

[0102] With this configuration, the camera installation support device (terminal device P1) can acquire imaging parameters that are more suitable for capturing an image of an imaging target from the set position or installation angle of each camera C11, even if the camera is not a model of multiple cameras C11, ... or if the positions of lighting for the multiple cameras C11, ... are different. This allows the user to acquire imaging parameters that are more suitable for capturing an image of an imaging target for each camera, allowing the user to more efficiently set the imaging parameters for each camera C11, ...

[0103] (Technology 8) The camera installation support device (terminal device P1) described in (Technology 7), wherein the adjustment unit (processor 11) identifies models of the plurality of cameras C11, ... based on the captured images captured by the plurality of cameras C11, ....

[0104] With this configuration, the camera installation support device (terminal device P1) can identify the model of each camera C11, ... even if the model is not one of multiple cameras C11, ... and the installation position or installation angle of each camera C11, ... and model information of each camera C11, ... have not been set in advance, and can obtain imaging parameters that are more suitable for imaging the subject based on the characteristics of the identified model of each camera C11, ...

[0105] (Technology 9) At least one camera C21 of the plurality of cameras C11, ... has a light-emitting unit (LEDLt1, see FIG. 8) that emits light in a predetermined light-emitting pattern, and when the analysis unit (processor 11) detects the predetermined light-emitting pattern of the light-emitting unit (LEDLt1) based on the captured image, it estimates a relative position between the camera C21 equipped with the light-emitting unit (LEDLt1) and the camera C11 that captured the captured image. (Technology 2) - (Technology 8) The camera installation support device (terminal device P1) described in any one of (Technology 2) to (Technology 8).

[0106] With this configuration, the camera installation support device (terminal device P1) can identify which of the multiple cameras C11, ... the camera appearing in the captured image is, and can estimate with higher accuracy the installation position or installation angle between the camera that captured the captured image and the identified camera.

[0107] (Technology 10) The plurality of cameras C11, ... have light-emitting units that emit light (e.g., LEDLt3, see FIG. 12), and when the analysis unit (processor 11) determines that the plurality of cameras C11, ... are not able to capture the image of the target, it determines a target camera (e.g., camera C21B, see FIG. 12) among the plurality of cameras C11, ... that needs to have its installation position or installation angle adjusted, and causes the light-emitting unit (LEDLt3) of the target camera (camera C21B) to emit light. (Technology 2) - (Technology 9) The camera installation support device (terminal device P1) described in any one of (Technology 9).

[0108] With this configuration, the camera installation support device (terminal device P1) supports the user in installing the cameras by visualizing which cameras require adjustment of their installation position or installation angle, even when there are a large number of cameras C11, etc. Based on the lighting of the LEDs (light-emitting elements), the user can more easily find and adjust cameras on-site that require adjustment of their installation position or installation angle.

[0109] (Technology 11) A camera installation support method performed by at least one computer (terminal device P1) capable of communicating with a plurality of cameras C11, ..., the camera installation support method comprising: acquiring information on the number of cameras C11, ..., information on the area in which the plurality of cameras C11, ... are installed, movement path information (movement path RT) of moving objects captured by the plurality of cameras C11, ..., and captured images captured by each of the plurality of cameras C11, ...; extracting, for each camera C11, ..., moving object images captured by the moving objects from the captured images; and estimating and outputting the positions of the plurality of cameras C11, ... in the area based on image capture time information when the moving object images were captured.

[0110] With this configuration, the computer (terminal device P1) can assist the user in installing cameras by estimating and outputting the positional relationship (i.e., installation position) of each of the multiple cameras C11, ... relative to the moving body that has moved on the movement path RT, the installation angle of the multiple cameras C11, ... relative to the moving body that has moved on the movement path RT, etc., based on the moving body's movement path RT and the moving body image, without the need to input the installation position or installation angle of each camera C11, ... installed in the area in advance.

[0111] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0112] The present disclosure is useful as a camera installation support device and a camera installation support method that support the installation of multiple cameras.

[0113] 10 Communication unit 11 Processor 12 Memory 13 Input unit 14 Monitor 100 Camera installation support system B1 Authentication device B1A Biometric information acquisition device C11, C12, C13, C14, C21, C22, C23, C24, C31, C32, C33, C34, C41, C51 Camera Lt1, Lt2A, Lt2B, Lt3 LED MP Map Msg1, Msg2, Msg3 Message OB11 Pillar OB12 QR code P1 Terminal device RT Movement route SC0, SC1, SC2, SC11, SC12 Analysis result screen SH11, SH12, SH13, SH14, SH21, SH22, SH23, SH24, SH31, SH32, SH33, SH34, SH41 Product shelves

Claims

1. A camera installation support device comprising: an acquisition unit that acquires information on the number of cameras, information on the area in which the multiple cameras are installed, movement path information of moving objects captured by the multiple cameras, and captured images captured by each of the multiple cameras; and an estimation unit that extracts moving object images of the moving object from the captured images for each camera, and estimates and outputs the positions of the multiple cameras in the area based on imaging time information when the moving object images were captured.

2. The camera installation support device of claim 1, further comprising an analysis unit that acquires information about the target imaged by the multiple cameras, and when it is determined that the target imaged by the multiple cameras has not been captured based on the estimated positions of the multiple cameras, information about the target imaged, and the images captured by the multiple cameras, generates and outputs support information about the installation positions or installation angles of the cameras for capturing images of the target imaged.

3. The camera installation support device of claim 2, wherein the information relating to the area includes a map of the area, and the analysis unit outputs a mapping image in which the multiple cameras are mapped onto the map based on the estimated positions of the multiple cameras, and the support information.

4. The camera installation support device described in claim 3, wherein the estimation unit groups the multiple cameras based on the orientation of the moving body captured in the moving body image, and the analysis unit outputs a mapping image in which the multiple cameras are mapped onto the map and the support information based on the estimated positions of the multiple cameras and the result of the grouping.

5. The camera installation support device according to claim 4, wherein the estimation unit groups the multiple cameras based on the orientation of the moving body captured in the moving body image and image capture time information of the moving body image.

6. The camera installation support device according to claim 1, wherein the estimation unit estimates the distance between the plurality of cameras based on image capture time information of the moving object image for each of the cameras.

7. The camera installation support device described in claim 2, further comprising: an adjustment unit that calibrates imaging parameters of the multiple cameras based on the landmark information that appears in the captured images captured by the multiple cameras; and an analysis unit that outputs the imaging parameters calibrated by the adjustment unit for each camera, wherein the moving object has landmark information that allows the moving object to be detected from the captured images.

8. The camera installation support device according to claim 7, wherein the adjustment unit identifies the models of the plurality of cameras based on the images captured by the plurality of cameras.

9. The camera installation support device of claim 2, wherein at least one of the plurality of cameras has a light-emitting unit that emits light in a predetermined light pattern, and when the analysis unit detects the predetermined light-emitting pattern of the light-emitting unit based on the captured image, it estimates the relative position between the camera equipped with the light-emitting unit and a camera that captured the captured image.

10. The camera installation support device described in claim 2, wherein the multiple cameras have light-emitting units that emit light, and when the analysis unit determines that the multiple cameras are not able to capture the image of the target, it determines which of the multiple cameras needs to have its installation position or installation angle adjusted, and causes the light-emitting unit of the target camera to emit light.

11. A camera installation support method performed by at least one computer capable of communicating with a plurality of cameras, comprising: acquiring information on the number of cameras, information on the area in which the plurality of cameras are installed, movement path information of moving objects captured by the plurality of cameras, and captured images captured by each of the plurality of cameras; extracting moving object images of the moving objects from the captured images for each camera; and estimating and outputting positions of the plurality of cameras in the area based on image capture time information when the moving object images were captured.

Images