Estimation device, estimation method, and estimation program

Abstract

To provide an estimation apparatus which estimates coordinates of an object accurately.SOLUTION: An acquisition unit 121 acquires images from cameras. A detection unit 122 detects an object from the images acquired by the acquisition unit 121. A first estimation unit 123 estimates, for each camera, coordinates of the object detected by the detection unit 122. A second estimation unit 124 estimates coordinates of an intersection point of lines that connect the cameras and the object detected by the detection unit 122. A distance calculation unit 125 calculates distances between the coordinates of the object and the coordinates of the intersection point of the lines that connect the cameras and the object, on the basis of the coordinates of the object estimated by the first estimation unit 123 and the coordinates of the intersection point estimated by the second estimation unit 124. A selection unit 126 selects a pair of distances that makes the minimum sum out of the distances calculated by the distance calculation unit 125. A coordinate estimation unit 127 estimates coordinates of the object on the basis of the pair of distances that makes the minimum sum selected by the selection unit 126.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an estimation device, an estimation method, and an estimation program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there exists a technique for estimating the position of an object present in a room from images taken by multiple cameras for purposes such as air conditioning control and entrance / exit management. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-002105 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technologies, there are cases where the accurate position of an object cannot be determined due to incorrect linking. For example, when multiple objects are located close to each other, the coordinates of an object estimated by one camera may be linked to the coordinates of another object estimated by another camera. Thus, the problem that the present invention aims to solve is the inability to determine the accurate position of an object due to incorrect linking. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems and achieve the object, the present invention includes an acquisition unit, a detection unit, a first estimation unit, a second estimation unit, a distance calculation unit, a selection unit, and an estimation unit. The acquisition unit acquires images from cameras. The detection unit detects objects from the images acquired by the acquisition unit. The first estimation unit estimates the coordinates of the objects detected by the detection unit for each camera. The second estimation unit estimates the coordinates of the intersections of lines connecting the objects detected by the detection unit to each camera. The distance calculation unit calculates the distance between the coordinates of the objects and the coordinates of the intersections of lines connecting the objects to each camera based on the coordinates of the objects estimated by the first estimation unit and the coordinates of the intersections estimated by the second estimation unit. The selection unit selects a pair of distances with the smallest sum of the distances from the distances calculated by the distance calculation unit. The coordinate estimation unit estimates the coordinates of the objects based on the pair of distances with the smallest sum selected by the selection unit.

[0006] In the estimation device, the detection unit may detect a human head as the object.

[0007] In the above estimation device, the detection unit may detect an accessory worn on a human head as the target object.

[0008] In the above estimation device, the selection unit may use the Hungarian method to select a pair of distances that minimizes the sum of the distances. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an example of a functional configuration diagram of an estimation device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of an estimation process performed by the estimation device according to the embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of an estimation process performed by the estimation device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0010] An example of an estimation device according to an embodiment of the present invention will be described below with reference to the drawings, in the order of the configuration of the estimation device, estimation processing, flowchart, and computer hardware configuration.

[0011] [Configuration of the estimation device] First, the configuration of the estimation device 100 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the estimation device 100 according to the embodiment. As shown in Fig. 1, the estimation device 100 includes a communication unit 110, a control unit 120, and a storage unit 130. Note that these units may be distributed and held by multiple devices. The processing of these units will be described below.

[0012] The communication unit 110 is realized by a NIC (Network Interface Card) or the like, and enables communication between an external device and the control unit 120 via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 110 enables communication between the external device and the control unit 120.

[0013] The storage unit 130 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The information stored in the storage unit 130 includes a camera ID, information about the installation location of the camera, information about the space, and other information related to estimating the position of the target object. Note that the above is an example, and the information stored in the storage unit 130 is not limited to the above.

[0014] The control unit 120 is realized using a CPU (Central Processing Unit), an NP (Network Processor), an FPGA (Field Programmable Gate Array), or the like, and executes a processing program stored in a memory. As shown in Fig. 1, the control unit 120 has an acquisition unit 121, a detection unit 122, a first estimation unit 123, a second estimation unit 124, a distance calculation unit 125, a selection unit 126, and a coordinate estimation unit 127. Each unit of the control unit 120 will be described below.

[0015] The acquisition unit 121 acquires image data from a camera. For example, the acquisition unit 121 acquires still image data from a camera. Also, for example, the acquisition unit 121 acquires moving image data from a camera.

[0016] The detection unit 122 detects an object from the image acquired by the acquisition unit 121. For example, the detection unit 122 detects a human head as the object from the image acquired by the acquisition unit 121. The reason why the human head is used as the object here is that it is less affected by individual differences among parts of the human body and can be detected relatively stably. When detecting the human head, the detection unit 122 may detect the position of the neck or shoulders as auxiliary information. Furthermore, for example, the detection unit 122 may detect an accessory worn on the human head as the object. For example, the detection unit 122 detects eyeglasses as the object.

[0017] The first estimation unit 123 estimates the coordinates of the object detected by the detection unit 122 for each camera. For example, the first estimation unit 123 estimates the coordinates of the object by calculating the distance between each camera and the human head detected by the detection unit 122 using a trigonometric function. In other words, the first estimation unit 123 estimates the coordinates of the object by using the angle of the object with respect to the camera and the distance from the camera to the object.

[0018] The second estimation unit 124 estimates the coordinates of the intersection of lines connecting each camera to the object detected by the detection unit 122. For example, the second estimation unit 124 estimates the coordinates of the intersection of a line connecting camera 1 to a human head, which is the object detected by the detection unit 122, and a line connecting camera 2 to the human head, which is the object detected by the detection unit 122.

[0019] The distance calculation unit 125 calculates the distance between the coordinates of the object and the coordinates of the intersection of the lines connecting the object and each camera, based on the coordinates of the object estimated by the first estimation unit 123 and the coordinates of the intersection estimated by the second estimation unit 124. For example, the distance calculation unit 125 calculates the distance between two points by using the coordinates of the human head estimated by the first estimation unit 123 and the coordinates of the intersection estimated by the second estimation unit 124.

[0020] The selection unit 126 selects a pair of distances whose sum is the smallest among the distances calculated by the distance calculation unit 125. For example, the selection unit 126 selects a pair of distances whose sum is the smallest among the distances calculated by the distance calculation unit 125 using the Hungarian algorithm.

[0021] The coordinate estimation unit 127 estimates the position of the object based on the pair of distances with the smallest sum of the distances selected by the selection unit 126. For example, the coordinate estimation unit 127 estimates the coordinates of the object detected by each camera in a correctly linked state, based on the information on the pair of distances with the smallest sum of the distances selected by the selection unit 126.

[0022] [Outline of estimation process] Next, an overview of the estimation process performed by the estimation device 100 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the estimation process performed by the estimation device according to the embodiment. First, (1) is a diagram showing, on a two-dimensional plane, the positions of people estimated by the first estimation unit 123 from images captured by cameras 1 and 2 that are installed perpendicular to each other. As shown in (1), the position of the person estimated from the image captured by camera 1 is different from the position of the person estimated from the image captured by camera 2.

[0023] Next, (2) is a diagram showing, in addition to (1), the coordinates of the intersections of the lines connecting each camera and each detected person estimated by the second estimation unit 124. Then, (3) is a diagram showing the distances between the intersections of the lines connecting each camera and each detected person calculated by the distance calculation unit 125 and the person positions estimated by each camera. Here, in (3), the distance calculation unit 125 calculates four types of distances, and two combinations are possible for linking the person positions estimated by camera 1 and camera 2. At this time, the selection unit 126 selects, from the four types of distances, the combination that minimizes the sum of the distances for each combination. Then, the coordinate estimation unit 127 estimates the position of the person from the combination selected by the selection unit 126 (4).

[0024] [flowchart] Next, the estimation process performed by the estimation device 100 having the above-described configuration will be described with reference to the flowchart of Fig. 3. Fig. 3 is a flowchart showing an example of the estimation process performed by the estimation device according to the embodiment.

[0025] First, the acquisition unit 121 acquires image data (step S11). For example, the acquisition unit 121 acquires moving image data captured by a camera. Alternatively, for example, the acquisition unit 121 acquires still image data captured by a camera.

[0026] Next, the detection unit 122 detects an object from the image acquired by the acquisition unit 121 (step S12). For example, the detection unit 122 detects a human head as the object from the image acquired by the acquisition unit 121.

[0027] Then, the first estimation unit 123 estimates the coordinates of the object detected by the detection unit 122 for each camera (step S13). For example, the first estimation unit 123 estimates the coordinates of a human head, which is the object detected by the detection unit 122, by using trigonometric functions.

[0028] Then, the second estimation unit 124 estimates the coordinates of the intersection of the line connecting the object detected by the detection unit 122 and each camera (step 14). For example, the second estimation unit 124 estimates the coordinates of the intersection of the line connecting the human head, which is the object detected by the detection unit 122, and camera 1, and the line connecting the human head, which is the object detected by the detection unit 122, and camera 2.

[0029] The distance calculation unit 125 calculates the distance between the coordinates of the object and the coordinates of the intersection of the lines connecting the object and each camera, based on the coordinates of the object estimated by the first estimation unit 123 and the coordinates of the intersection estimated by the second estimation unit 124 (step 15). For example, the distance calculation unit 125 calculates the distance between the two points by using the coordinates of the human head estimated by the first estimation unit 123 and the coordinates of the intersection estimated by the second estimation unit 124.

[0030] The selection unit 126 selects a pair of distances whose sum is the smallest among the distances calculated by the distance calculation unit 125 (step S16). For example, the selection unit 126 uses the Hungarian algorithm to select a pair of distances whose sum is the smallest.

[0031] The coordinate estimation unit 127 estimates the coordinates of the object based on the pair of distances selected by the selection unit 126, which has the smallest sum of the distances (step S17). For example, the coordinate estimation unit 127 estimates the position of the object by using the pair of distances selected by the selection unit 126 as a correct association.

[0032] [effect] As described above, the estimation device 100 according to the embodiment includes the acquisition unit 121, the detection unit 122, the first estimation unit 123, the second estimation unit 124, the distance calculation unit 125, the selection unit 126, and the coordinate estimation unit 127. The acquisition unit 121 acquires images from the cameras. The detection unit 122 detects objects from the images acquired by the acquisition unit 121. The first estimation unit 123 estimates the coordinates of the objects detected by the detection unit 122 for each camera. The second estimation unit 124 estimates the coordinates of the intersections of the lines connecting the objects detected by the detection unit 122 to each camera. The distance calculation unit 125 calculates the distance between the coordinates of the objects and the coordinates of the intersections of the lines connecting the objects and each camera, based on the coordinates of the objects estimated by the first estimation unit 123 and the coordinates of the intersections estimated by the second estimation unit 124. The selection unit 126 selects a pair of distances whose sum is the smallest among the distances calculated by the distance calculation unit 125. The coordinate estimation unit 127 estimates the coordinates of the object based on the pair of distances whose sum is the smallest selected by the selection unit 126.

[0033] As a result, the estimation device 100 according to the embodiment uses the coordinates of the object estimated for each camera and the coordinates of the intersections of the lines connecting each camera and each object to link the objects, thereby enabling correct linking and estimating the coordinates of the object with high accuracy. In other words, the estimation device 100 according to the embodiment can correctly link the coordinates of each object detected by each camera and accurately estimate the coordinates of the object.

[0034] Furthermore, in the estimation device 100 according to the embodiment, the detection unit 122 detects a human head as the target object. As a result, the estimation device 100 according to the embodiment can stably detect a person present in space by using a human head, which has little individual variability, as the target object.

[0035] Furthermore, in the estimation device 100 according to the embodiment, the detection unit 122 detects an accessory worn on a human head as a target object. As a result, the estimation device 100 according to the embodiment can stably detect a human head by detecting an accessory worn on a human head and using the detected accessory as auxiliary information.

[0036] Furthermore, in the estimation device 100 according to the embodiment, the selection unit 126 selects a pair of distances with the smallest sum of the distances using the Hungarian algorithm, which allows the estimation device 100 according to the embodiment to accurately link the objects estimated from each camera in a short processing time.

[0037] [Hardware configuration] Fig. 4 is a diagram illustrating an example of a hardware configuration. As shown in Fig. 4, the estimation device 100 includes a communication device 100a, a hard disk drive (HDD) 100b, a memory 100c, and a processor 100d. The components shown in Fig. 4 are connected to each other via a bus or the like.

[0038] The communication device 100a is a network interface card or the like, and communicates with other devices. The HDD 100b stores programs and DBs that operate the functions shown in FIG.

[0039] The processor 100d reads out a program that executes the same processes as the respective processing units shown in Fig. 1 from the HDD 100b etc. and loads it into the memory 100c, thereby operating a process that executes the respective functions described in Fig. 1 etc. For example, taking the monitoring device 50 as an example, this process executes the same functions as the respective processing units possessed by the monitoring device 50. Specifically, the processor 100d reads out a program that has the same functions as the receiving unit 53a and the output control unit 53b etc. from the HDD 100b etc. Then, the processor 100d executes a process that executes the same processes as the receiving unit 53a and the output control unit 53b etc.

[0040] In this way, the estimation device 100 operates as an estimation device that executes various estimation methods by reading and executing a program. The estimation device 100 can also realize functions similar to those of the above-described embodiments by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the estimation device 100. For example, the above-described embodiments may also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0041] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer.

[0042] [others] Although one example of an embodiment of the present invention has been described above, the present invention is not limited to the above example. That is, a person skilled in the art can implement various modifications in accordance with conventionally known knowledge without departing from the gist of the present invention. As long as such modifications still include the estimation device of the present invention, they are of course included in the scope of the present invention. [Explanation of symbols]

[0043] 100 Estimator 110 Communications Department 120 control section 121 Acquisition Department 122 Detector 123 First Estimation Department 124 Second Estimation Department 125 Distance calculation unit 126 Selection Section 127 Coordinate estimation unit 130 Memory Department

Claims

1. an acquisition unit that acquires an image from a camera; a detection unit that detects an object from the image acquired by the acquisition unit; a first estimation unit that estimates the coordinates of the object detected by the detection unit for each camera; a second estimation unit that estimates coordinates of intersections between lines connecting the object detected by the detection unit and each camera; a distance calculation unit that calculates a distance between the coordinates of the object and the coordinates of the intersections of lines connecting the object and each camera, based on the coordinates of the object estimated by the first estimation unit and the coordinates of the intersections estimated by the second estimation unit; a selection unit that selects a pair of distances whose sum of distances is smallest from among the distances calculated by the distance calculation unit; a coordinate estimation unit that estimates the coordinates of the object based on the pair of distances selected by the selection unit, the sum of which is the smallest; An estimation device comprising:

2. The estimation device according to claim 1 , wherein the detection unit detects a human head as the target object.

3. The estimation device according to claim 1 , wherein the detection unit detects an accessory worn on a human head as the target object.

4. 2. The estimation device according to claim 1, wherein the selection unit uses the Hungarian method to select a pair of distances whose sum of distances is smallest.

5. 1. A computer-implemented estimation method comprising: an acquisition step of acquiring an image from a camera; a detection step of detecting an object from the image acquired in the acquisition step; a first estimation step of estimating, for each camera, the coordinates of the object detected by the detection step; a second estimation step of estimating coordinates of intersections between lines connecting the object detected in the detection step and each camera; a distance calculation step of calculating a distance between the coordinates of the object and the coordinates of the intersections of lines connecting the object and each camera, based on the coordinates of the object estimated in the first estimation step and the coordinates of the intersections estimated in the second estimation step; a selection step of selecting a pair of distances whose sum of distances is smallest from the distances calculated in the distance calculation step; a coordinate estimation step of estimating the coordinates of the object based on the pair of distances selected by the selection step, the sum of which is the smallest; An estimation method comprising:

6. an acquisition step to acquire images from the camera; a detection step of detecting an object from the image acquired by the acquisition step; a first estimation step of estimating, for each camera, the coordinates of the object detected by the detection step; a second estimation step of estimating coordinates of intersections between lines connecting the object detected by the detection step and each camera; a distance calculation step of calculating a distance between the coordinates of the object and the coordinates of the intersections of lines connecting the object and each camera, based on the coordinates of the object estimated by the first estimation step and the coordinates of the intersections estimated by the second estimation step; a selection step of selecting a pair of distances whose sum of distances is smallest from among the distances calculated by the distance calculation step; a coordinate estimation step of estimating the coordinates of the object based on the pair of distances selected by the selection step, the sum of which is the smallest; An estimation program for causing a computer to execute the above.

Images