Camera position estimation device, method, and program

Abstract

To provide a camera position estimation device, a method, and a program capable of efficiently grasping the position of a camera installed in facilities.SOLUTION: First acquisition means of a camera position estimation device acquires map information showing a map of the inside of facilities. Second acquisition means acquires a first position of a moving body on a map shown by map information according to the movement of the moving body within the facilities. Third acquisition means acquires multiple images including a moving body traveling within facilities imaged by a camera. First estimation means estimates the position of a camera on a map shown by map information such that each of first positions of a moving body acquired according to the movement of the moving body matches a second position of a moving body on an image including the moving body in the first positions from among the acquired multiple images.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] An embodiment of the present invention relates to a camera position estimation device, method, and program. [Background technology]

[0002] It is known that various services are provided by utilizing multiple cameras installed (placed) in facilities such as buildings, factories, commercial facilities, etc. Specifically, for example, when multiple cameras are installed in multiple areas in a facility, a service can be provided that detects the intrusion of people and the degree of congestion in the area based on images of the area captured by each of the multiple cameras.

[0003] Incidentally, in order to provide the above-mentioned services, it is necessary to know (information about) the positions of each of the multiple cameras installed within the facility.

[0004] In this case, it is conceivable to manually check and register the positions of each of the multiple cameras installed in the facility. However, for example, if there are a large number of cameras installed in the facility, such a task would require a lot of manpower and time.

[0005] For this reason, there is a need to efficiently grasp the positions of cameras installed within facilities. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2014-216813 A Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, an object of the present invention is to provide a camera position estimation device, method, and program that can efficiently grasp the positions of cameras installed within a facility. [Means for solving the problem]

[0008] According to an embodiment, there is provided a camera position estimation device that estimates a position of a camera installed in a facility. The camera position estimation device includes a first acquisition means, a second acquisition means, a third acquisition means, and a first estimation means. The first acquisition means acquires map information showing a map of the facility. The second acquisition means acquires a first position of the moving object on the map shown by the map information in response to the movement of the moving object in the facility. The third acquisition means A plurality of images including a moving object moving within the facility are acquired by the camera. The first estimation means estimates a position of the camera on a map indicated by the map information such that each of first positions of the moving object acquired in response to the movement of the moving object matches a second position of the moving object on an image including the moving object at the first position among the plurality of acquired images. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration including a camera position estimation device according to an embodiment. [Diagram 2] FIG. 1 is a diagram showing an example of cameras installed within a facility. [Diagram 3] FIG. 2 is a diagram showing an example of the functional configuration of a camera position estimation device according to the embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a hardware configuration of a camera position estimation apparatus according to the embodiment. [Diagram 5] 10 is a flowchart showing an example of a processing procedure of a map information acquisition unit according to the embodiment. [Figure 6] 10 is a flowchart showing an example of a processing procedure of an on-map position acquisition unit according to the embodiment. [Figure 7]10 is a flowchart showing an example of a processing procedure of a moving object detection unit according to the embodiment. [Figure 8] 10 is a flowchart illustrating an example of a processing procedure of a camera position estimation unit according to the embodiment. [Figure 9] 10 is a flowchart showing an example of a processing procedure of an imaging range estimation unit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an example of a system configuration including a camera position estimation device 1 according to this embodiment.

[0011] The camera position estimation device 1 is connected to each of n (n is an integer equal to or greater than 2) cameras 2-1 to 2-n, for example, via a network 4a. Each of the n cameras 2-1 to 2-n is installed in a facility such as a building, a factory, or a commercial facility. Although it is assumed that the camera position estimation device 1 is connected to each of the n cameras 2-1 to 2-n by wires, it may also be connected to each of the n cameras 2-1 to 2-n by wireless.

[0012] Here, for example, assume that a plurality of cameras 2-1 to 2-n are installed in a facility 100 having a plurality of areas A1 to A5 (for example, conference rooms, corridors, etc.) as shown in Fig. 2. Note that Fig. 2 shows an example in which six cameras 2-1 to 2-6 are installed in the facility 100.

[0013] 2, a service can be provided that detects the intrusion of people and the degree of congestion in each area A1-A5 based on the images of the areas A1-A5 captured by the cameras 2-1-2-6. Alternatively, a service can be provided that detects (analyzes) the amount of traffic (people flow) in each area A1-A5 and abnormal behavior of people by tracking people detected in each area A1-A5.

[0014] Furthermore, the multiple cameras 2-1 to 2-n according to this embodiment may be used to provide services other than those described above. For example, when multiple cameras 2-1 to 2-n are installed in multiple areas in a facility where products are sold, a service can be provided that analyzes the purchasing behavior of people moving through the areas based on images of the areas captured by each of the multiple cameras.

[0015] Furthermore, it is also possible to provide a service that uses the cameras 2-1 to 2-n to analyze the flow of workers moving between predetermined areas in a facility (e.g., a factory, etc.), or the location and operating status of industrial robots and service robots (security robots, cleaning robots, transport robots, guide robots, etc.) Such a service can improve the productivity of workers and the operating efficiency of industrial robots.

[0016] In the above service, in order to identify the area within the facility in the image used for analysis, it is necessary to know in advance the positions of the cameras 2-1 to 2-n within the facility.

[0017] The positions of the cameras 2-1 to 2-n are generally determined in advance based on design drawings of the facility and manually registered (digitized) in the system. However, it is time-consuming to install the cameras 2-1 to 2-n while checking the positions determined in advance.

[0018] Furthermore, when the cameras 2-1 to 2-n are installed at possible locations within the facility, rather than at predetermined locations, the locations at which the cameras 2-1 to 2-n are installed must be manually surveyed, confirmed, and registered. The same applies when cameras already installed for a different purpose are repurposed (used) as the cameras 2-1 to 2-n.

[0019] The position of a camera that has already been installed for another purpose may be known from the facility's design drawings, etc. However, if the position of the camera known from the facility's design drawings, etc. is the floor or room number on which the camera is installed, the accuracy of the camera's position may be low, and the above-mentioned service may not be realized.

[0020] In this way, when grasping the positions of the cameras 2-1 to 2-n, generally, a large number of steps (effort, cost, time, etc.) are required for manual confirmation and registration work.

[0021] Therefore, as shown in FIG. 1, the camera position estimation device 1 according to this embodiment is communicably connected to a moving object 3 that autonomously moves within a facility in which cameras 2-1 to 2-n are installed, for example, via a network 4b. Examples of the moving object 3 include a security robot, a cleaning robot, a transport robot, and a guide robot. It is assumed that the camera position estimation device 1 is wirelessly connected to the moving object 3, but it may also be connected to the moving object 3 by wire. In addition, the camera position estimation device 1 may be connected to multiple moving objects 3, but the following description assumes that it is connected to one moving object 3.

[0022] Here, the mobile body 3 in this embodiment acquires surrounding environment information using sensors such as a camera, infrared radar, or millimeter wave radar. The surrounding environment information is information such as the presence or absence of objects (e.g., walls and obstacles) around the mobile body, and the distance between the mobile body and the objects. Based on the above-mentioned environment information, the mobile body 3 generates information (hereinafter referred to as map information) showing a plan view (map) of the inside of the facility including walls forming the facility, navigable paths within the facility, and obstacles located within the facility.

[0023] In addition, the moving object 3 moves while grasping its own position on the map shown by the map information based on the surrounding environment information acquired by the above sensor. In the following description, the position of the moving object 3 on the map shown by the above map information is referred to as the "position on the map" of the moving object 3.

[0024] The camera position estimation device 1 according to this embodiment acquires images from each of the cameras 2-1 to 2-n via a network 4a, and acquires map information and (position information indicating) the position on the map of the mobile object 3 from a mobile object 3 within the facility via a network 4b. The camera position estimation device 1 automatically estimates the position of each of the cameras 2-1 to 2-n based on the acquired information (the images from each of the cameras 2-1 to 2-n, the map information from the mobile object 3, and the position on the map of the mobile object 3).

[0025] 3 shows an example of the functional configuration of the camera position estimation device 1 according to this embodiment. The camera position estimation device 1 includes a moving object information collection unit 10, a moving object detection unit 20, a camera position estimation unit 30, and an imaging range estimation unit .

[0026] The mobile object information collection unit 10 acquires (collects) and stores various information from the mobile object 3. The mobile object information collection unit 10 includes a map information acquisition unit 11, a map information storage unit 12, a map position acquisition unit 13, and a map position storage unit 14.

[0027] The map information acquisition unit 11 acquires map information generated by the mobile object 3. The map information acquisition unit 11 stores the acquired map information in the map information storage unit 12.

[0028] The map position acquisition unit 13 acquires the map position of the moving object 3 in accordance with the movement of the moving object 3 within the facility. The map position acquisition unit 13 stores the map position of the moving object 3 in the map position storage unit 14 together with the time when the moving object 3 was at the map position.

[0029] The moving object detection unit 20 detects a moving object 3 on a plurality of images (frames) captured by each of the cameras 2-1 to 2-n, and acquires and stores the position of the moving object 3 on the image (the position of the moving object 3 on the image). The moving object detection unit 20 includes an image acquisition unit 21, a detection unit 22, and an image position storage unit 23.

[0030] The image acquisition unit 21 acquires (collects) a plurality of images captured by each of the cameras 2-1 to 2-n for each of the cameras 2-1 to 2-n. The detection unit 22 detects the moving object 3 included in the image acquired by the image acquisition unit 21. In addition, the detection unit 22 detects the position of the moving object 3 on the image (the position of the moving object 3 on the image). The detection unit 22 stores the position of the moving object 3 on the image in the image position storage unit 23 in association with information specifying the image including the moving object 3 (hereinafter referred to as image identification information). The image identification information includes camera identification information for specifying the camera that captured the image, the time when the image was captured (image number set for the image), and the like.

[0031] The camera position estimation unit 30 estimates and stores the position of each of the cameras 2-1 to 2-n based on map information, and the position on the map and the position on the image of the moving object 3. The camera position estimation unit 30 includes a camera position estimation processing unit 31 and a camera position storage unit 32.

[0032] The camera position estimation processing unit 31 estimates the position of each of the cameras 2-1 to 2-n on the map indicated by the map information stored in the map information storage unit 12 included in the mobile object information collecting unit 10, based on the map position of the mobile object 3 stored in the map position storage unit 14 included in the mobile object information collecting unit 10 and the image position of the mobile object 3 stored in the image position storage unit 23 included in the mobile object detection unit 20. In this case, the position of each of the cameras 2-1 to 2-n is estimated such that each of the map positions of the mobile object 3 (i.e., the map positions of the mobile object 3 acquired by the mobile object information collecting unit 10 in response to the movement of the mobile object 3) matches with the image position of the mobile object 3 (i.e., the image position of the mobile object 3 on the image including the mobile object 3 at the map position among the multiple images acquired by the mobile object detection unit 20). The camera position estimation processing unit 31 performs such estimation processing for each of the multiple cameras 2-1 to 2-n.

[0033] The camera position estimation processing unit 31 estimates the angle (installation angle) of each of the cameras 2-1 to 2-n in addition to the position of each of the cameras 2-1 to 2-n. The camera position estimation processing unit 31 stores the estimated position and angle of each of the cameras 2-1 to 2-n in the camera position storage unit 32.

[0034] The imaging range estimating unit 40 estimates the imaging range of each of the cameras 2-1 to 2-n. The imaging range estimating unit 40 includes an imaging range estimation processing unit 41 and an imaging range storage unit .

[0035] The imaging range estimation processing unit 41 estimates the imaging range of each of the cameras 2-1 to 2-n based on map information stored in the map information storage unit 12 included in the mobile object information collection unit 10, pre-stored angle-of-view information indicating the angles of view of the cameras 2-1 to 2-n, and the positions and angles of each of the cameras 2-1 to 2-n stored in the camera position storage unit 32 included in the camera position estimation unit 30. The imaging range estimation processing unit 41 stores in the imaging range storage unit 42 the imaging range information indicating the estimated imaging range of each of the cameras 2-1 to 2-n.

[0036] 4 shows an example of a hardware configuration of the camera position estimation device 1 according to this embodiment. The camera position estimation device 1 includes, for example, a CPU 5, a non-volatile memory 6, a RAM 7, and a communication device 8.

[0037] The CPU 5 is a processor for controlling the operation of various components in the camera position estimation device 1. The CPU 5 may be a single processor or may be configured with multiple processors. The CPU 5 executes various programs loaded from the non-volatile memory 6 to the RAM 7. These programs include an operating system (OS) and various application programs.

[0038] The non-volatile memory 6 is a storage medium used as an auxiliary storage device. The RAM 7 is a storage medium used as a main storage device. Although only the non-volatile memory 6 and the RAM 7 are shown in Fig. 4, the camera position estimation device 1 may include other storage devices such as a hard disk drive (HDD) and a solid state drive (SSD).

[0039] The communication device 8 is a device configured to perform wired communication or wireless communication. As described above, the camera position estimation device 1 according to this embodiment is assumed to be connected to the cameras 2-1 to 2-n via wires (cables), but may be connected to perform wireless communication. Also, the camera position estimation device 1 is assumed to be connected to the moving object 3 to perform wireless communication, but may be connected to the moving object 3 via wires (cables).

[0040] In this embodiment, some or all of the map information acquisition unit 11, the map position acquisition unit 13, the image acquisition unit 21, the detection unit 22, the camera position estimation processing unit 31, and the imaging range estimation processing unit 41 shown in Fig. 3 are realized by causing the CPU 5 to execute a predetermined program, that is, by software. This program may be stored in a computer-readable storage medium and distributed, or may be downloaded to the camera position estimation device 1 via a network. Some or all of the map information acquisition unit 11, the map position acquisition unit 13, the image acquisition unit 21, the detection unit 22, the camera position estimation processing unit 31, and the imaging range estimation processing unit 41 may be realized by dedicated hardware or the like, or may be realized by a combination of software and hardware.

[0041] In addition, in this embodiment, the map information storage unit 12, the map position storage unit 14, the image position storage unit 23, the camera position storage unit 32, and the imaging range storage unit 42 shown in Figure 3 are realized, for example, by a non-volatile memory 6 or other storage device.

[0042] Next, an example of a procedure of a map information acquisition process performed by the map information acquisition unit 11 according to this embodiment will be described with reference to a flowchart of FIG.

[0043] Here, the map information acquisition unit 11 acquires map information generated by the moving object 3, and stores the acquired map information in the map information storage unit 12. However, the map information generated by the moving object 3 is updated based on surrounding environment information acquired when the moving object 3 moves to a new area. Furthermore, the placement of obstacles in a facility may change over time, and the map information may be updated in response to such changes in the placement of obstacles.

[0044] For this reason, the map information acquisition unit 11 detects whether the map information has been updated by the mobile object 3 (step S11). The process of step S11 is executed based on the difference between the map information held by the mobile object 3 and the map information already stored in the map information storage unit 12. Note that the map information acquisition unit 11 may detect that the map information has been updated when a notification that the map information has been updated is received from the mobile object 3.

[0045] Next, the map information acquisition unit 11 determines whether or not the map information has been updated based on the detection result in step S11 (step S12).

[0046] If it is determined that the map information has not been updated (NO in step S12), the process returns to step S11 and is repeated.

[0047] On the other hand, if it is determined that the map information has been updated (YES in step S12), the map information acquisition unit 11 acquires the updated map information from the mobile object 3 (step S13).

[0048] The map information acquisition unit 11 stores the map information acquired in step S13 in the map information storage unit 12 (step S14). After that, the process returns to step S11, and the subsequent processes are repeated. This allows the map information acquisition unit 11 to periodically acquire map information reflecting the current state inside the facility (including changes in the placement of obstacles as described above) and store the map information in the map information storage unit 12. Note that the map information acquisition unit 11 may periodically acquire map information regardless of whether the map information has been updated.

[0049] Next, an example of a procedure of a map position acquisition process performed by the map position acquisition unit 13 according to this embodiment will be described with reference to a flowchart in FIG.

[0050] Here, the map position acquisition unit 13 acquires the self-position on the map grasped by the moving object 3 (the map position of the moving object 3), and stores the map position of the moving object 3 in the map position storage unit 14. However, since the moving object 3 moves within the facility, the map position of the moving object 3 needs to be acquired in accordance with the movement.

[0051] For this reason, the map position acquisition unit 13 detects whether the map position of the mobile object 3 has been updated (step S21). The process of step S21 is executed based on the difference between the self-position on the map grasped by the mobile object 3 (the map position of the mobile object 3) and the map position of the mobile object 3 already stored in the map position storage unit 14 (movement of the mobile object 3). Note that the map position acquisition unit 13 may detect that the map position has been updated when it receives a notification from the mobile object 3 that the map position has been updated.

[0052] Next, the map position acquisition unit 13 determines whether or not the map information has been updated based on the detection result in step S21 (step S22).

[0053] If it is determined that the position on the map of the moving object 3 has not been updated (that is, the moving object 3 has not moved) (NO in step S22), the process returns to step S21 and is repeated.

[0054] When it is determined that the on-map position of the moving object 3 has been updated (i.e., the moving object 3 has moved) (YES in step S22), the on-map position acquisition unit 13 acquires the on-map position from the moving object 3 (step S23). At this time, the on-map position acquisition unit 13 acquires the on-map position of the moving object 3 together with the time when the on-map position of the moving object 3 was updated (the time when the moving object 3 was at that on-map position).

[0055] Then, the map position acquisition unit 13 associates the map position of the mobile object 3 with the time and stores them in the map position storage unit 14 (step S24). After that, the process returns to step S21 periodically, and the subsequent processes are repeated. As a result, every time the map position of the mobile object 3 is updated (i.e., in response to the movement of the mobile object 3), the map position of the mobile object 3 is stored in the map position storage unit 14. Note that the map position acquisition unit 13 may periodically acquire the map position regardless of whether or not there is an update to the map position.

[0056] Next, an example of the procedure of the moving object detection process by the moving object detection unit 20 according to this embodiment will be described with reference to the flowchart in Fig. 7. Note that the moving object detection unit 20 performs the process in Fig. 7 in parallel for each of the cameras 2-1 to 2-n. Alternatively, the moving object detection unit 20 may perform the process in Fig. 7 in a predetermined order for each of the cameras 2-1 to 2-n. In the following description, an example in which the process is performed for the camera 2-i, which is one of the cameras 2-1 to 2-n, will be described.

[0057] The image acquisition unit 21 of the moving object detection unit 20 acquires an image captured by the camera 2-i (step S31). The detection unit 22 analyzes the image acquired in step S31 and detects the presence or absence of a moving object 3 in the image (step S32).

[0058] Next, the image acquisition unit 21 determines whether or not the moving object 3 is included in the image acquired in step S31 (that is, whether or not the moving object 3 is present in the image) based on the detection result in step S32 (step S33).

[0059] If it is determined that the moving object 3 is not included in the image (NO in step S33), the process returns to step S31 and the process is repeated.

[0060] If it is determined that the moving object 3 is included in the image (YES in step S33), the detection unit 22 detects the position of the moving object 3 on the image (step S34). The position of the moving object 3 on the image is a two-dimensional coordinate value with a predetermined position on the image as the origin. The detection unit 22 acquires the two-dimensional coordinate value of a predetermined feature point (e.g., the upper end, the lower end, or the center part) of the moving object 3 on the image as the position of the moving object 3 on the image.

[0061] The detection unit 22 stores the on-image position of the moving object 3 detected in step S34 in the on-image position storage unit 23 in association with the image identification information of the image including the moving object 3 (step S35). Thereafter, the process returns to step S31, and the subsequent processes are repeated for the images captured by the camera 2-i. Note that the process shown in FIG. 7 is assumed to be performed for all images captured by the camera 2-i, but may be performed for some of the images. In this case, for example, the process shown in FIG. 7 may be performed for one of the images for every predetermined number of images (i.e., the process is performed once every few frames).

[0062] Next, an example of the procedure of the camera position estimation process by the camera position estimation unit 30 according to this embodiment will be described with reference to the flowchart in Fig. 8. Note that the camera position estimation unit 30 performs the process in Fig. 8 in parallel for each of the cameras 2-1 to 2-n. Alternatively, the camera position estimation unit 30 may perform the process in Fig. 8 in a predetermined order for each of the cameras 2-1 to 2-n. In the following description, an example will be described in which the process is performed for camera 2-i, which is one of the cameras 2-1 to 2-n.

[0063] The camera position estimation processing unit 31 of the camera position estimation unit 30 acquires the number of images including the moving object 3 captured by the camera 2-i (step S41). Specifically, the camera position estimation processing unit 31 acquires the number of on-image positions of the moving object 3 captured by the camera 2-i, which are stored in the on-image position storage unit 23 included in the moving object detection unit 20.

[0064] Next, the camera position estimation processing unit 31 determines whether or not the number of images acquired in step S41 is equal to or greater than a predetermined number (step S42).

[0065] If the number of images including the moving body 3 is less than the predetermined number (NO in step S42), the camera position estimation processing unit 31 returns to step S41 and repeats the process until the number of images including the moving body 3 (i.e., the number of image positions of the moving body 3 captured by the camera 2-i) becomes equal to or greater than the predetermined number.

[0066] If the number of images including the moving object 3 is equal to or greater than a predetermined number (YES in step S42), the camera position estimation processing unit 31 acquires, for each of the images equal to or greater than the predetermined number, the image identification information stored in the on-image position storage unit 23 and the on-image position of the moving object 3 corresponding to the image identification information (step S43).

[0067] The camera position estimation processing unit 31 also acquires the map information stored in the map information storage unit 12 included in the mobile object information collection unit 10, and the map position of the mobile object 3 stored in the map position storage unit 14 (step S44). At this time, the camera position estimation processing unit 31 acquires the map position of the mobile object 3 based on the image identification information corresponding to each of the image positions of the mobile object 3 acquired in step S43. Specifically, the camera position estimation processing unit 31 refers to the time at which the image included in the image identification information corresponding to each of the image positions of the mobile object 3 was captured, and acquires the map position of the mobile object 3 at that time (corresponding to that time).

[0068] In addition, since the map position of the moving object 3 is acquired according to the movement of the moving object 3, if the moving object 3 is temporarily stopped, the map position of the moving object 3 during the stopped period is not stored in the map position storage unit 14. If the moving object 3 is imaged within the above period, in step S44, the map position of the moving object 3 at the time when the image included in the image identification information corresponding to the image position of the moving object 3 is captured may not be stored in the map position storage unit 14. In this case, the camera position estimation processing unit 31 discards the image identification information and the image position of the moving object 3 corresponding to the image identification information stored in the image position storage unit 23 acquired in step S43 as unused data (step S45). As a result of discarding the image identification information (image identification information and the image position of the moving object 3 corresponding to the image identification information) in step S45, there may be no or insufficient number of image positions of the moving object 3 to be processed. Therefore, the camera position estimation processing unit 31 further determines whether the number of image identification information acquired in step S43 and the positions of the moving object 3 on the image corresponding to the image identification information are equal to or greater than a predetermined number (step S46). If the number of image positions of the moving object 3 is less than the predetermined number (NO in step S46), the process returns to step S41 and is repeated.

[0069] If the number of positions on the image of the moving object 3 is less than a predetermined number (YES in step S46), the camera position estimation processing unit 31 acquires the currently estimated position and angle of the camera 2-i stored in the camera position storage unit 32 (step S47). Hereinafter, the currently estimated position and angle of the camera 2-i are referred to as the "estimated position" and "estimated angle" of the camera 2-i. In the initial state (when the process of FIG. 8 is executed for the first time), the camera position storage unit 32 stores the estimated position and estimated angle of the camera 2-i that have been arbitrarily set.

[0070] The estimated position of the camera 2-i stored in the initial state camera position storage unit 32 is assumed to be any position on the map indicated by the map information. Alternatively, the estimated position of the camera 2-i stored in the initial state camera position storage unit 32 may be any position (for example, along a wall or pillar) that is estimated to be possible to install the camera 2-i in the facility based on the map indicated by the map information.

[0071] Next, assuming that the camera 2-i is installed at the estimated position and the estimated angle, the camera position estimation processing unit 31 calculates (estimates) the position of the moving object 3 on the image captured by the camera 2-i (hereinafter referred to as the estimated image position) based on the map position of the moving object 3 (step S48). That is, the estimated image position is the position of the moving object 3 on the image captured (to be captured) by the camera 2-i installed at the estimated position and the estimated angle when the moving object 3 is present at the map position acquired in step S44. The camera position estimation processing unit 31 calculates the estimated image position of the moving object 3 for each map position acquired in step S44.

[0072] The camera position estimation processing unit 31 calculates, for each map position acquired in step S44, a difference between each of the estimated image positions of the moving object 3 calculated in step S48 and the image position of the moving object 3 acquired in step S43 (step S49). That is, for each map position acquired in step S44, a difference (so-called reprojection error) is calculated between the position of the moving object 3 on the image when the moving object 3 at that map position is captured by the camera 2-i installed at the estimated position and at the estimated angle (estimated image position) and the position of the moving object 3 on the image when the moving object 3 at that map position is actually captured by the camera 2-i (image position).

[0073] This difference occurs because the estimated position and estimated angle are different from the actual position and angle of the camera 2-i. Therefore, by adjusting (optimizing) the estimated position and estimated angle so as to reduce this difference, it is possible to obtain the position and angle of the camera 2-i that captured the image of the moving object 3 located at that position on the map.

[0074] The camera position estimation processing unit 31 estimates (calculates) the position and angle of the camera 2-i so as to reduce the difference (error) between the estimated image position of the moving body 3 for each map position of the moving body 3 and the image position (i.e., so that the estimated image position matches the image position of the moving body 3 actually captured by the camera 2-i) (step S50). In this case, the camera position estimation processing unit 31 executes a process of optimizing the estimated position and estimated angle of the camera 2-i by, for example, a bundle adjustment method, and estimates the optimized estimated position and estimated angle as the position and angle of the camera 2-i.

[0075] The camera position estimation processing unit 31 stores (updates) the estimated position and angle of the camera 2-i in the camera position storage unit 32 (step S51).

[0076] The camera position estimation processing unit 31 calculates the estimated image position of the moving object 3 again using the position and angle of the camera 2-i stored in step S51 as the estimated position and estimated angle. Then, it is determined whether or not the total value of the difference between the estimated image position and the image position for each map position acquired in step S44 is equal to or smaller than a threshold value (step S52).

[0077] If the sum of the differences between the estimated on-image position and the on-image position of the moving object 3 is not equal to or less than the threshold value (NO in step S52), the accuracy of the position and angle of the camera 2-i estimated in step S50 may be low (there is a large difference between the estimated position and estimated angle and the actual position and angle of the camera 2-i). Therefore, the process returns to step S41, and steps S41 to S52 are repeated using another image captured by the camera 2-i. If the sum of the differences between the estimated on-image position and the on-image position is equal to or less than the threshold value (YES in step S52), the process ends.

[0078] The position of the camera 2-i estimated by the camera position estimation process shown in FIG. 8 is, for example, a position in three-dimensional space, but the position information finally stored in the camera position storage unit 32 after completing the process of FIG. 8 and the process of FIG. 9 described below may be a position on a two-dimensional plane or a position in three-dimensional space.

[0079] In addition, the camera position estimation processing unit 31 may omit the processing of steps S41 and S42, and only determine whether the number of image identification information (i.e., image identification information that has a corresponding moving body 3 position on the map) after discarding image identification information that does not have a corresponding moving body 3 position on the map in step S46 is equal to or greater than a predetermined number.

[0080] Furthermore, in step S44, if the map position of the moving object 3 at the time when the image included in the image identification information corresponding to the image position of the moving object 3 was captured is not stored in the map position storage unit 14, the camera position estimation processing unit 31 may acquire the map position of the moving object 3 at the time closest to (immediately before) the time when the image included in the image identification information corresponding to the image position of the moving object 3 was captured, from among the map positions of the moving object 3 updated before the time when the image included in the image identification information corresponding to the image position of the moving object 3 was captured. In this case, the processes of steps S45 and S46 can be omitted.

[0081] Next, an example of the procedure of the imaging range estimation process by the imaging range estimation unit 40 according to this embodiment will be described with reference to the flowchart in Fig. 9. Note that the imaging range estimation unit 40 performs the process in Fig. 9 in parallel for each of the cameras 2-1 to 2-n. Alternatively, the imaging range estimation unit 40 may perform the process in Fig. 9 in a predetermined order for each of the cameras 2-1 to 2-n. In the following description, an example will be described in which the process is performed for the camera 2-i, which is one of the cameras 2-1 to 2-n.

[0082] First, the imaging range estimation processing unit 41 of the imaging range estimation unit 40 detects whether the camera position estimation process shown in Fig. 8 for the camera 2-i has been completed (step S61). The imaging range estimation processing unit 41 may detect that the camera position estimation process has been completed when it receives a notification from the camera position estimation unit 30 that the process for the camera 2-i has been completed. Alternatively, the imaging range estimation processing unit 41 may detect whether the camera position estimation process has been completed by periodically checking whether the position and angle of the camera 2-i in the camera position storage unit 32 have been updated.

[0083] Next, the imaging range estimation processing unit 41 determines whether or not the camera position estimation process has ended based on the detection result in step S61 (step S62).

[0084] If it is determined that the camera position estimation process has not ended (NO in step S62), the process returns to step S61 and repeats the process.

[0085] If it is determined that the camera position estimation process has been completed (YES in step S62), the imaging range estimation processing unit 41 obtains the position and angle of the camera 2-i estimated by the camera position estimation processing unit 31 from the camera position storage unit 32 (step S63).

[0086] The imaging range estimation processing unit 41 also acquires map information from the map information storage unit 12 (step S64). Furthermore, the imaging range estimation processing unit 41 acquires angle-of-view information indicating the angle of view of the camera 2-i (step S65).

[0087] The angle of view information includes the angle of view and distortion parameters of the camera 2-i, and is included in the camera parameters defined as the specifications of the camera 2-i. The camera parameters are input in advance, for example, when the camera 2-i is installed by a user, and are stored in a storage unit (not shown) in the camera position estimation device 1. Alternatively, the camera parameters may be transmitted from the camera 2-i when the camera position estimation device 1 is connected to the camera 2-i, and may be stored in a storage unit (not shown) in the camera position estimation device 1. The imaging range estimation processing unit 41 refers to the camera parameters in the storage unit to acquire the angle of view information of the camera 2-i.

[0088] The imaging range estimation processing unit 41 estimates the area (imaging range) imaged by the camera 2-i installed at the position and angle based on the map information, the angle of view information of the camera 2-i, and the position and angle of the camera 2-i estimated by the camera position estimation unit 30 (step S66).

[0089] Then, the imaging range estimation processing unit 41 generates imaging range information in which the imaging range of the camera 2-i is mapped on a map indicated by the map information (step S67). The imaging range estimation processing unit 41 stores the generated imaging range information in the imaging range storage unit 42 (step S68). According to such imaging range information, for example, it is possible to display a map in which a color is added to an area indicating the imaging range of the camera 2-i, and it is possible to visually grasp the imaging range.

[0090] In this embodiment, the angles of the cameras 2-1 to 2-n are estimated in addition to the positions, but if the angles of the cameras 2-1 to 2-n are known (fixed values), for example, the system may estimate only the positions of the cameras 2-1 to 2-n. Also, if it is not necessary to estimate the imaging ranges of the cameras 2-1 to 2-n, the system may not include the imaging range estimation unit 40 (i.e., the process shown in FIG. 9 is not executed).

[0091] Furthermore, although the present embodiment has been described as generating imaging range information, for example, the camera position estimation processing unit 31 may generate camera position information in which the estimated position (and angle) of the camera 2-i is mapped on a map indicated by map information. According to such camera position information, for example, it is possible to display the camera position (and angle) of each of the cameras 2-1 to 2-n estimated by the camera position estimation processing unit 31 on a map, and it is possible to visually grasp the position of each of the cameras 2-1 to 2-n.

[0092] Further, in the above description, a case has been described in which a plurality of cameras 2-1 to 2-n are installed within the facility, but the number of cameras may be one or more.

[0093] As described above, the camera position estimation device 1 in this embodiment acquires map information showing a map of the facility, acquires the map position (first position) of the moving object 3 on the map shown by the map information according to the movement of the moving object 3 in the facility, acquires a plurality of images including the moving object 3 moving in the facility captured by a camera, and estimates the position of the camera on the map shown by the map information such that each of the map positions of the moving object 3 acquired according to the movement of the moving object 3 matches with the image position (second position) of the moving object 3 on an image including the moving object 3 at the map position among the acquired images. Note that when a plurality of cameras 2-1 to 2-n are installed in the facility, the position of the camera is estimated for each of the cameras 2-1 to 2-n.

[0094] With this configuration, the positions of the cameras installed in the facility can be efficiently grasped (estimated). When a large number of cameras are installed in the facility, when the cameras are installed in a remote facility, or when the cameras are installed in a place where workers cannot enter, a significant amount of work is required to grasp the positions of the cameras. According to this embodiment, even in the above cases, the amount of work can be reduced and the positions of the cameras can be efficiently grasped.

[0095] In this embodiment, the configuration of "estimating the position of the camera so that each of the map positions of the moving body 3 is consistent with the image position of the moving body 3" can be realized, for example, by assuming that the camera is installed at an estimated position (third position) on the map indicated by map information, calculating the estimated image position (fourth position) of the moving body 3 on the image captured by the camera based on the map position of the moving body 3, calculating for each map position the difference between the image position of the moving body 3 on the image that includes the moving body 3 at that map position and the calculated estimated image position, and estimating the position of the camera (fifth position) such that the calculated difference for each map position is reduced.

[0096] The camera position where each of the map positions of the moving object 3 matches the image position of the moving object 3 may be estimated by a method other than the above. For example, the camera position where each of the map positions matches the image position may be selected from a plurality of candidates for the camera position on the map shown by the map information (hereinafter, referred to as candidate positions). In this case, the candidate camera position is set to a position where it is estimated that a camera can be installed in the facility (for example, along a wall or a pillar). Then, the estimated image position of the moving object 3 for each of the candidate camera positions is calculated, and the candidate camera position where the difference between the estimated image position and the image position is the smallest (i.e., the map positions and the image position of the moving object 3 match most closely) may be estimated as the position of the camera.

[0097] In this embodiment, the estimation of the camera position where each of the positions of the moving object 3 on the map matches the position on the image has been described, but it is also possible to estimate the camera position where the moving path of the moving object 3 on the map matches the moving trajectory of the moving object 3 on the captured image. Even in this case, similarly to the above embodiment, assuming that the camera is installed at an estimated position on the map indicated by the map information, the estimated moving trajectory of the moving object 3 on the image captured by the camera may be calculated based on the moving path of the moving object 3 on the map, the difference between the moving trajectory of the moving object 3 on the image including the moving object 3 on the moving path and the calculated estimated moving trajectory of the moving object 3 on the image may be calculated, and the position of the camera may be estimated so that the difference is reduced.

[0098] In this embodiment, the case where there is one moving object 3 in the facility has been described, but there may be a plurality of moving objects 3 in the facility. In this case, it is necessary to obtain the map position for each of the plurality of moving objects 3 and detect the position on the image (position on the image) for each of the plurality of moving objects 3. The case where there are a plurality of moving objects 3 in the facility will be described below.

[0099] The map information acquisition unit 11 included in the mobile object information collection unit 10 shown in FIG. 3 acquires map information from each of the multiple mobile objects 3. The acquired map information may differ for each of the multiple mobile objects 3. In this case, the map information acquisition unit 11 may perform a process of integrating the multiple map information acquired from each of the mobile objects 3 whose map information has been updated. When the multiple mobile objects 3 are moving on the same route within a facility, the map information from the mobile object that updated the map information the latest may be stored in the map information storage unit 12 (i.e., the newest map information among the map information held by each of the multiple mobile objects 3 is stored). In addition, the map position acquisition unit 13 acquires and stores the map position of each of the multiple mobile objects 3.

[0100] The detection unit 22 included in the moving body detection unit 20 identifies which of the multiple moving bodies 3 the moving body 3 detected from the image is, and acquires information for identifying the moving body 3 (hereinafter, referred to as moving body identification information). The moving body identification information may be acquired based on the feature amount of each moving body 3 extracted by image processing, for example, but may be acquired by other methods. Specifically, each moving body 3 may be identified by reading a code such as a QR code (registered trademark) attached to the moving body 3. Also, each moving body 3 may be identified by using a signal transmitted from the moving body 3 (a signal such as Bluetooth (registered trademark) or Wi-fi (registered trademark)). In this case, the moving body 3 may be identified (inferred) based on the strength of the signal transmitted from the moving body 3, for example. Furthermore, it is also possible to identify the moving body 3 by analyzing the unique sound or the like emitted by each moving body 3.

[0101] The detection unit 22 stores the on-image position of the moving object 3 together with the acquired moving object identification information in the on-image position storage unit 23 in association with the image identification information. Note that, although the detection unit 22 has been described as acquiring the moving object identification information here, the moving object identification information may be acquired by another functional unit, such as a specification unit, different from the detection unit 22.

[0102] By storing the moving object identification information together with the position on the image, the camera position estimation processing unit 31 included in the camera position estimation unit 30 can estimate the position of the camera using the position on the image for each moving object 3.

[0103] The camera position estimation device 1 may be connected to a management system (or a control system) that manages the movements of one or more moving bodies 3. In this case, the map information acquisition unit 11 may acquire map information generated by one or more moving bodies 3 from the management system. Alternatively, the map information acquisition unit 11 may acquire map information generated by the management system (integrating map information from multiple moving bodies 3). Furthermore, the map position acquisition unit 13 may acquire the map positions of one or more moving bodies 3 from the management system.

[0104] According to at least one of the embodiments described above, it is possible to provide a camera position estimation device, method, and program capable of efficiently grasping the positions of cameras installed within a facility.

[0105] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0106] 1...camera position estimation device, 2-1 to 2-n...cameras, 3...moving object, 4a, 4b...network, 5...CPU, 6...non-volatile memory, 7...RAM, 8...communication device, 10...moving object information collection unit, 11...map information acquisition unit, 12...map information storage unit, 13...map position acquisition unit, 14...map position storage unit, 20...moving object detection unit, 21...image acquisition unit, 22...detection unit, 23...image position storage unit, 30...camera position estimation unit, 31...camera position estimation processing unit, 32...camera position storage unit, 40...imaging range estimation unit, 41...imaging range estimation processing unit, 42...imaging range storage unit, 100...facility.

Claims

1. A camera position estimation device that estimates the position of a camera installed in a facility, A first acquisition means for acquiring map information indicating a map of the facility; a second acquisition means for acquiring a first position of the moving object on a map indicated by the map information in response to a movement of the moving object within the facility; a third acquisition means for acquiring a plurality of images including a moving object moving within the facility, the images being captured by the camera; a first estimation means for estimating a position of the camera on a map indicated by the map information such that each of first positions of the moving object acquired in response to the movement of the moving object matches a second position of the moving object on an image including the moving object at the first position among the acquired multiple images; A camera position estimation device comprising:

2. 2. The camera position estimation device of claim 1, wherein the first estimation means calculates a fourth position of the moving object on an image captured by the camera based on a first position of the moving object, assuming that the camera is installed at a third position on a map indicated by the map information, calculates a difference between a second position of the moving object on the image including the moving object at the first position and the calculated fourth position for each of the first positions, and estimates a fifth position of the camera at which the calculated difference for each of the first positions decreases.

3. The camera position estimation device according to claim 2 , wherein the first estimation means estimates the fifth position of the camera by bundle adjustment.

4. 2. The camera position estimation device of claim 1, wherein the first estimation means calculates a fourth position of the moving object on an image captured by the camera based on a first position of the moving object, assuming that the camera is installed at a first angle at a third position on a map indicated by the map information, calculates a difference between a second position of the moving object on the image including the moving object at the first position and the calculated fourth position for each of the first positions, and estimates a fifth position and second angle of the camera at which the calculated difference for each of the first positions decreases.

5. 5. The camera position estimation device according to claim 4, further comprising a second estimation means for estimating an imaging range of the camera based on the map information, pre-stored angle-of-view information indicating the angle of view of the camera, and the estimated fifth position and second angle of the camera.

6. The camera position estimation device according to claim 5 , wherein the second estimation means generates imaging range information in which the imaging range of the camera is mapped on a map indicated by the map information.

7. The camera position estimation device according to claim 1 , wherein the first acquisition means acquires the map information from the moving object or a management system that manages the moving object.

8. The camera position estimation device according to claim 1 , wherein the second acquisition means acquires the first position of the moving object from the moving object or a management system that manages the moving object.

9. 9. The camera position estimation device according to claim 1, wherein, when a plurality of cameras are installed within the facility, the position of each of the cameras is estimated for each of the cameras.

10. A method executed by a computer of a camera position estimation device for estimating a position of a camera installed in a facility, comprising: The computer includes: acquiring map information showing a map of the facility; acquiring a first position of the moving object on a map indicated by the map information in response to a movement of the moving object within the facility; acquiring a plurality of images including a moving object moving within the facility captured by the camera; estimating a position of the camera on a map indicated by the map information such that each first position of the moving object acquired in response to the movement of the moving object matches a second position of the moving object on an image including the moving object at the first position among the acquired images; A method comprising the steps of:

11. A program executed by a computer of a camera position estimation device that estimates the position of a camera installed in a facility, The computer includes: acquiring map information showing a map of the facility; acquiring a first position of the moving object on a map indicated by the map information in response to a movement of the moving object within the facility; acquiring a plurality of images including a moving object moving within the facility captured by the camera; estimating a position of the camera on a map indicated by the map information such that each first position of the moving object acquired in response to the movement of the moving object matches a second position of the moving object on an image including the moving object at the first position among the acquired images; A program for executing the above.

Images