Target information creation system, target information creation method, and recording medium

Abstract

The present invention provides a target information creation system, a target information creation method, and a recording medium which are capable of suppressing duplicate counting of objects when generating target information from images captured by a plurality of cameras.　This target information creation system includes: an acquisition means that acquires images captured by first and second cameras that capture, from different viewpoints, images of a place where the target information is created; an extraction means that extracts an image area in which an object is captured in the plurality of acquired images and identifies the position and movement of the object; a determination means that determines whether or not the object captured in an image area of one camera is the same as an object captured in an image area of the other camera, on the basis of one or more among the position, the movement direction, and the speed of the object in the image areas; and a creation means that uses the determination results and creates the target information of the object located at the place where the target information is created.

Description

Target information creation system, target information creation method, and recording medium

[0001] The present invention relates to a target information creation system, a target information creation method, and a recording medium.

[0002] Attempts have been made to detect objects based on data obtained from roadside cameras, LiDAR (Light Detection and Ranging), and the like, and further provide the direction and speed of their movement to self-driving vehicles. Patent Document 1 discloses an image processing device that can accurately detect and track multiple types of objects that differ in size, movement speed, etc. Specifically, the image processing device of Patent Document 1 determines the type of object to be detected for each processing area on a frame image, resizes the image of the processing area, and then detects the object.

[0003] It is also being considered to configure this as target information indicating the type of object, its direction of movement, and its speed, and provide it to self-driving vehicles.

[0004] Japanese Patent Application Laid-Open No. 2015-2401

[0005] When generating target information from camera images, multiple cameras are sometimes used because there is a limit to the number of objects that can be captured within the field of view (angle of view) of a single camera.However, when multiple cameras are used, there is a problem that objects captured in the images of each camera may be recognized as completely different objects and counted multiple times, or objects may be missed due to inappropriate allocation of the object detection range.

[0006] An object of the present disclosure is to provide a target information creation system, a target information creation method, and a recording medium that can accurately generate target information from images captured by the above-mentioned multiple cameras.

[0007] According to a first aspect, there is provided a target information creation system comprising: an acquisition means for acquiring images taken by a first and a second camera, each of which captures a location for which target information is to be created from different viewpoints; an extraction means for extracting an image area in which an object is captured from the plurality of acquired images and identifying the position and movement of the object; a determination means for determining whether an object captured in the image area of ​​one camera is the same as an object captured in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and a creation means for creating target information for an object located at the location for which target information is to be created using the determination result.

[0008] According to a second aspect, there is provided a target information creation method which acquires images of a location for which target information is to be created using first and second cameras, each of which captures an image from a different viewpoint; extracts an image area in which an object is captured from the acquired images; identifies the position and movement of the object; and determines whether the object captured in the image area of ​​one camera is the same as the object captured in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and uses the determination result to create target information for an object located at the location for which target information is to be created.

[0009] According to a third aspect, there is provided a recording medium storing a program for causing a computer to execute the following processes: acquiring images taken by first and second cameras, each of which takes an image of a location where target information is to be created from different viewpoints; extracting image areas in which an object is captured from the acquired images and identifying the position and movement of the object; determining whether an object captured in the image area of ​​one camera is the same as an object captured in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and creating target information for an object located at the location where target information is to be created using the determination result.

[0010] According to the present disclosure, it is possible to provide a target information creation system, a target information creation method, and a recording medium that can accurately generate target information from images captured by multiple cameras.

[0011] FIG. 1 is a diagram showing one configuration of the present disclosure. FIG. 2 is a flowchart showing the operation of the present disclosure. FIG. 3 is a diagram for explaining the operation of the present disclosure. FIG. 4 is a diagram for explaining the operation of the present disclosure. FIG. 5 is a diagram showing an example of target information created by the present disclosure. FIG. 1 is a diagram showing one configuration of the present disclosure. FIG. 2 is a diagram for explaining the operation of the present disclosure. FIG. 3 is a flowchart showing the operation of the present disclosure. FIG. 4 is a diagram showing an example of a target information map created by the present disclosure. FIG. 5 is a diagram showing another configuration of the present disclosure. FIG. 6 is a diagram showing another configuration of the present disclosure. FIG. 7 is a diagram showing the configuration of a computer constituting a target information creation system of the present disclosure.

[0012] First, an overview of one embodiment of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. The reference numerals in the drawings attached to this overview are attached to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated form. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of main signals (data) and do not exclude bidirectionality. A program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and, if necessary, a display device. Furthermore, this computer device is configured to be able to communicate with internal or external devices (including computers) via the communication interface, whether wired or wireless. Although ports or interfaces are present at the input / output connection points of each block in the drawings, they are not shown.

[0013] 1, the present disclosure can be realized in a target information creation system 10 including an acquisition unit 11, an extraction unit 12, a determination unit 13, and a creation unit 14. More specifically, the acquisition unit 11 acquires images captured by first and second cameras C1 and C2, which capture images of a location for which target information is to be created, from different viewpoints.

[0014] The extraction means 12 extracts image areas in which an object appears from the multiple acquired images, and further identifies the position and movement of the object. The determination means 13 determines whether an object appearing in an image area of ​​one camera is the same as an object appearing in an image area of ​​another camera based on one or more of the object's position, movement direction, and speed in the image area. Note that the "movement" of an object can be represented by the movement direction, speed, or a vector representing a combination of these, or a movement trajectory over a certain period of time in the past.

[0015] The creating means 14 uses the determination result to create target information of an object located at a location for which the target information is to be created.

[0016] The target object information creation system 10 configured as described above operates as follows: First, the target object information creation system 10 acquires images of the location for which target object information is to be created, taken by the first and second cameras C1 and C2, which take images from different viewpoints (step S01 in FIG. 2).

[0017] Next, the target object information creation system 10 extracts image areas in which objects appear from the acquired multiple images (step S02 in FIG. 2). For example, the target object information creation system 10 extracts areas in which vehicles CAR1 to CAR5, truck TRUCK1, and pedestrian PE1 appear as objects from the image in FIG. 3.

[0018] Next, the target object information creation system 10 determines whether an object captured in the image area of ​​one camera is the same as an object captured in the image area of ​​another camera based on one or more of the position, movement direction, and speed of the object in the image area. For example, if the positions, movement direction, and speed of vehicles CAR1 to CAR5 in Fig. 3 correspond to the positions, movement direction, and speed of vehicles CAR1 to CAR5 in Fig. 4, the target object information creation system 10 determines that the two are the same object.

[0019] Finally, the target information creation system 10 uses the determination result to create target information for the object located at the location for which the target information is to be created. Specifically, the target information creation system 10 selects one of the overlapping vehicles CAR1 to CAR5 from the image areas extracted in FIGS. 3 and 4 , and excludes the other from the target information creation target. FIG. 5 is a diagram showing an example of target information created by the target information creation system 10. In the example of FIG. 5 , target information indicating the type, position, speed, direction, etc. of vehicles CAR1 to CAR5, truck TRUCK1, and pedestrian PE1 is created. It can be seen that CAR1 to CAR5, whose positions, moving directions, and speeds as determined from the images in FIGS. 3 and 4 correspond, are determined to be the same object, and non-overlapping target information is created.

[0020] By using the target object information creation system 10 that operates as described above, it is possible to prevent duplicate counting of objects when generating target object information from images captured by multiple cameras. Furthermore, the method disclosed herein can also prevent omissions due to multiple deletion of objects that appear in multiple images.

[0021] In the above description, an example has been given in which images are acquired from two cameras, the first and second cameras C1 and C2, but there is no limit to the number of cameras that can capture images for which target information is to be created. Even if three or more cameras are installed, the target information creation system 10 determines whether or not an object captured in each image area in these images is the same as an object captured in the image area of ​​another camera, and eliminates any duplication.

[0022] [First embodiment] Next, a first embodiment will be described in which a function for creating a map using target information is added to the target information creation system 10. Fig. 6 is a diagram showing one configuration of the present disclosure. Referring to Fig. 6, a configuration in which cameras C1 and C2 and the target information creation system 100 are connected is shown.

[0023] Cameras C1 and C2 are installed so that they can capture images of traffic participants, such as vehicles and pedestrians, traveling on roads that intersect at the intersection. Camera C1 is installed, for example, on a signal pole on one road at the intersection. Meanwhile, camera C2 is installed on a signal pole on the other road that intersects at the intersection. By arranging cameras C1 and C2 in this way, it is possible to reduce blind spots when capturing images of traffic participants traveling at the intersection.

[0024] The target information creation system 100 includes an acquisition unit 101 , an extraction unit 102 , a determination unit 103 , a creation unit 104 , a target map creation unit 105 , and a map information storage unit 106 .

[0025] The acquisition unit 101 acquires images captured by the first and second cameras C1 and C2 and sends them to the extraction unit 102.

[0026] The extraction unit 102 extracts an image area in which an object appears in the image sent from the acquisition unit 101 and identifies the type and location of the object. Furthermore, the extraction unit 102 tracks the same object from multiple temporally consecutive images sent from the acquisition unit 101 and identifies the object's movement (movement direction and speed). This identified information serves as basic data for creating target information. For example, the extraction unit 102 extracts an image area in which a vehicle or the like appears in the image shown in FIG. 7. FIGS. 3 and 4 show an image area extracted by the extraction unit 102. This object detection mechanism can use various techniques known as object recognition technology. For example, the extraction unit 102 can employ a method of detecting an object in an image using an inference model created in advance by machine learning or the like. Furthermore, the position and movement of the object may be identified from the image using the results of prior calibration, or may be identified using distance obtained by LiDAR or the like. The extraction unit 102 sends the extracted image area and the object's type, location, and movement to the determination unit 103. The speed of an object may be measured using the measurement results of a measuring device such as a speed gun.

[0027] The determination unit 103 determines whether an object captured in the image area of ​​one camera is identical to an object captured in the image area of ​​another camera based on the position and movement of the object in the image area, and sends the result to the creation unit 104. For example, if the position and movement of vehicle CAR4 in FIG. 3 match the position and movement of vehicle CAR4 in FIG. 4, the determination unit 103 determines that the two image areas contain the same object. On the other hand, if there is no object in FIG. 4 whose position and movement correspond to those of truck TRUCK1 and pedestrian PE1 in FIG. 3, the determination unit 103 determines that truck TRUCK1 is a unique object captured in FIG. 3. Note that it is desirable to determine identity here based on the estimated accuracy of the object's position and movement, rather than requiring strict agreement in all aspects of position, movement direction, and speed. If the degree of agreement is too high, substantially identical objects may be recognized as different objects.

[0028] The creation unit 104 uses the results of the determination by the determination unit 103 to organize data indicating the type, position, and movement of the extracted object to create target information. Fig. 5 shows an example of target information created by the creation unit 104. In the example of Fig. 5, information such as reliability, type (target type), position (latitude and longitude), speed, and direction is added to an object detected in an image at a certain time. Note that reliability is information indicating the accuracy (likelihood) of the object type when the extraction unit 102 detects the object.

[0029] The target map creating unit 105 plots the target information on a map read from the map information storage unit 106, and creates a target-added map showing information about objects on the map. This target map creating unit 105 corresponds to a map creating means that plots the target information on a map of the location where the target information is to be created, and creates a target-added map.

[0030] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 8 is a flowchart showing the operation of the present disclosure. Referring to Fig. 8, first, the target information creation system 100 acquires images captured by the cameras C1 and C2 (step S001).

[0031] Next, the target information creation system 100 extracts areas in which objects are captured from the acquired images, and identifies the type, position, and movement of the objects captured therein (step S002).

[0032] Next, the target information creation system 100 determines whether the object captured in the image area of ​​one camera is the same as the object captured in the image area of ​​the other camera based on the position and movement of the object in the image area, and sends the result to the creation unit 104 (step S003).

[0033] Next, the target information creation system 100 uses the determination result to organize the information extracted in step S002 and create target information (step S004).

[0034] Various methods can be considered as rules for selecting overlapping target information when creating the target information.

[0035] (1) Distance Priority: Target information created from an image area captured by a camera that is closer to the object is retained. This is because a camera that is closer to the object is considered to have higher accuracy. (2) Size Priority: Target information created from an image area that is larger on the screen is retained. This is because a larger size on the screen is considered to be able to capture an object in greater detail. (3) Reliability Priority: Target information created from an image area with a higher reliability (likelihood of object recognition) measured by the extraction unit 102 is retained. This is because a higher reliability is considered to be able to capture the characteristics of an object more accurately. (4) Sharpness Priority: Target information created from an image area with higher clarity is retained. This is because target information created from an image area with higher clarity is considered to be able to obtain more accurate information. Note that sharpness can be calculated using the number of pixels in the image area or a predetermined evaluation formula. (5) Image Area Near the Center of the Image: Target information created from an image area near the center of the image is retained. An image at the center of the image has less distortion, while image periphery has more image distortion due to lens distortion. Also, the center of the image tends to be brighter (higher contrast between light and dark) (the periphery tends to be darker, a condition known as vignetting). For this reason, by leaving target information near the center of the image, it is possible to obtain information with fewer errors.

[0036] The target information selection rules (1) to (5) above may be selected and applied as one of them, but may also be changed depending on the purpose of using the target information, the weather on the day, the type of object of interest, etc. The target information selection rules (1) to (5) above may also be combined. For example, for overlapping image areas, the score for each target information selection rule may be calculated, and the one with the higher total score may be selected.

[0037] Finally, the target information creation system 100 plots the target information on the map read from the map information storage unit 106, and creates a target-attached map showing information about objects on the map (step S005).

[0038] Fig. 9 is a diagram showing an example of a target information map created by the present disclosure. In the example of Fig. 9, markers are plotted to indicate the positions of objects passing through intersections at each time point. When any marker in Fig. 9 is selected, the type, position, and movement of the object are displayed as shown in Fig. 9. Using such a map makes it easier to understand the flow of objects and traffic volume at each intersection.

[0039] As described above, according to this embodiment, it is possible to generate consistent target information from images captured by a plurality of cameras.

[0040] In the above embodiment, the movement of an object is described as being determined by specifying its movement direction and speed. However, the movement trajectory of the object may also be used as the movement of the object. The movement trajectory of the object can be obtained by tracking the same object in images taken over a certain period of time in the past and determining its coordinates. In this case, the movement direction and speed of the object included in the target information may be determined from the movement trajectory, or the movement trajectory itself may be used as information indicating the movement of the object.

[0041] [Second Embodiment] In the first embodiment described above, the target object information creation system 100 has been described as using the selection rule for overlapping target information to sort out overlapping targets. However, the target object information creation system 100 may be configured to predict the movement of each object and sort out overlapping targets based on the results. FIG. 10 is a diagram showing another configuration of the present disclosure. The difference in configuration and operation from the first embodiment shown in FIG. 6 is that an estimation unit 1041 is provided in the creation unit 104a of the target object information creation system 100a. Since the other configurations are the same as those of the first embodiment, the following description will focus on the differences.

[0042] The estimation unit 1041 in the creation unit 104a estimates the time the object stays on the screen. This time may be determined, for example, using the change pattern of the traffic light aspect at the intersection in addition to the movement of the object. For example, the estimated time for vehicles CAR1 to CAR3 in Fig. 3 is the time from when the traffic light in the direction of travel is red to when the traffic light in the direction of travel turns green, when the vehicle starts moving, and when it disappears from the screen. The time for vehicle CAR4 in Fig. 3 is moving because the traffic light in the direction of travel is green, and when it disappears from the screen at its current speed, the estimated time for vehicle CAR4 is the time from when the traffic light in the direction of travel is green to when it disappears from the screen.

[0043] The creation unit 104a of this embodiment creates target information using the stay time of each object on the screen in addition to the result of determination by the determination unit 103. Specifically, the creation unit 104a selects overlapping data and creates target information using the stay time priority rule (6) below in addition to the selection rule for overlapping target information described above.

[0044] (6) Stay Time Priority: Target information created from an image area with a longer stay time on the image is retained. This is because it is believed that target information with consistency in terms of continuity, etc. can be created for an object with a longer stay time on the image.

[0045] In this embodiment, similarly to the first embodiment, it is possible to combine a plurality of selection rules for target information. In this case, the scores for the selection rules for each of the target information are calculated for the overlapping image areas, and the image with the highest total score is selected.

[0046] As described above, according to this embodiment, it is possible to create target information that is consistent in terms of continuity and the like.

[0047] [Third Embodiment] In the first embodiment described above, the target information creation system 100 selects one of the overlapping target information pieces using a selection rule and organizes the overlapping data. However, it is also possible to create target information using both of the overlapping target information pieces. Figure 11 is a diagram showing another configuration of the present disclosure. The difference in configuration and operation from the first embodiment shown in Figure 6 is that the creation unit 104b of the target information creation system 100b creates target information using both of the overlapping target information pieces. Since the other configurations are the same as those of the first embodiment, the following description will focus on the differences.

[0048] The creation unit 104b in this embodiment creates target data using both of the overlapping data. For example, the creation unit 104b creates target information by taking the average values ​​of the position, speed, and movement direction of the overlapping data. Although the approximate positions and movements of the overlapping data match and can be considered to be the same object, the positions and movements may differ. In this case, the creation unit 104b creates target information by taking the average value of the overlapping data. In addition to taking the average value, if there are three or more images obtained from the camera, methods such as taking the mode or median may also be considered. Therefore, the creation unit 104b corresponds to a creation means that creates target information using information obtained from each of the image areas in which the same object is captured.

[0049] According to this embodiment, it becomes possible to create target information in which the differences in overlapping data are smoothed out.

[0050] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and further modifications, substitutions, and adjustments can be made without departing from the basic technical concept of the present disclosure. For example, the network configurations, element configurations, and data representation formats shown in the drawings are examples intended to aid in understanding the present disclosure, and are not limited to the configurations shown in these drawings.

[0051] For example, in the above-described embodiments, the target object information creation systems 100, 100a, and 100b are described as being equipped with the map information storage unit 106. However, the map information storage unit 106 may be located on a network. In this case, the target object information creation systems 100, 100a, and 100b access the map information storage unit 106 on the network via a communication function to acquire map information.

[0052] In addition, in the above-described embodiments, the cameras C1 and C2 are connected to the target object information creation systems 100, 100a, and 100b, but an image storage device or the like may be disposed between the cameras C1 and C2 and the target object information creation system 100, etc. In this case, the target object information creation systems 100, 100a, and 100b acquire images captured by the cameras C1 and C2 via the image storage device.

[0053] In addition, in the above-described embodiments, the cameras C1 and C2 are described as cameras installed so as to be able to capture images of traffic participants such as vehicles and pedestrians passing through the intersection. However, the cameras C1 and C2 may be cameras mounted on vehicles passing through the intersection. In this case, the target information creation systems 100, 100a, and 100b are provided with a communication function with vehicles passing through the intersection and acquire images from the vehicles. It is preferable that the images acquired from the vehicles are not moving and are in a fixed position. For example, it is conceivable to acquire images from vehicles stopped at the stop line of the intersection.

[0054] (Hardware Configuration) In each embodiment of the present disclosure, each component of each device represents a functional unit block. Some or all of the components of each device are realized by an arbitrary combination of an information processing device 900 and a program, for example, as shown in FIG. 12 . FIG. 12 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration: - CPU (Central Processing Unit) 901 - ROM (Read Only Memory) 902 - RAM (Random Access Memory) 903 - Program 904 loaded into RAM 903 - Storage device 905 that stores the program 904 - Drive device 907 that reads and writes to a recording medium 906 - Communication interface 908 that connects to a communication network 909 - Input / output interface 910 that inputs and outputs data - Bus 911 that connects each component

[0055] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in FIG. 12 executes an object extraction program and an identity determination program, and performs an update process for each calculation parameter stored in the RAM 903, the storage device 905, etc. The program 904 that realizes the function of each component of each device is stored in the storage device 905 or the ROM 902 in advance, for example, and is read by the CPU 901 as needed. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in advance on the recording medium 906, and the drive device 907 may read the program and supply it to the CPU 901.

[0056] There are various variations in the method of realizing each device. For example, each device may be realized by any combination of a separate information processing device 900 and a program for each component. Furthermore, multiple components included in each device may be realized by any combination of a single information processing device 900 and a program. That is, each unit (processing means, function) of the target information creation system shown in the first to third embodiments can be realized by a computer program that causes a processor installed in the device to execute each of the above-mentioned processes using its hardware.

[0057] In addition, some or all of the components of each device may be realized by other general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus.

[0058] Some or all of the components of each device may be realized by a combination of the above-mentioned circuits and programs.

[0059] When some or all of the components of each device are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each device is connected via a communication network.

[0060] It should be noted that the above-described embodiments are preferred embodiments of the present disclosure, and the scope of the present disclosure is not limited to only the above-described embodiments. In other words, those skilled in the art can modify or substitute the above-described embodiments to construct various modified forms without departing from the gist of the present disclosure.

[0061] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0062] [Supplementary Note 1] A target information creation system comprising: an acquisition means for acquiring images captured by first and second cameras, each capturing an image of a location for which target information is to be created, from different viewpoints; an extraction means for extracting image areas in which an object is captured from the acquired images and identifying the position and movement of the object; a determination means for determining whether an object captured in an image area of ​​one camera is the same as an object captured in an image area of ​​the other camera based on one or more of the object's position, movement direction, and speed in the image area; and a creation means for creating target information for an object located at the location for which target information is to be created, using the determination result. [Supplementary Note 2] The creation means of the target information creation system may be configured to create target information using information obtained from an image area that is closer to the camera than the image areas in which the same object is captured. [Supplementary Note 3] The creation means of the target information creation system may be configured to create target information using information obtained from an image area that is larger on the screen than the image areas in which the same object is captured. [Supplementary Note 4] The creation means of the target information creation system may be configured to create target information using information obtained from an image area in which the same object is clearly visible, among image areas in which the same object is captured. [Supplementary Note 5] The creation means of the target information creation system may be configured to create target information using information obtained from an image area in which the same object is captured and has a high reliability when the object is recognized, among image areas in which the same object is captured. [Supplementary Note 6] The creation means of the target information creation system may be configured to create target information using information obtained from an image area in which the same object is captured and that is close to the center of the image. [Supplementary Note 7] The target information creation system may further include an estimation means for estimating the object's on-screen stay time, and the creation means may be configured to create target information using information obtained from an image area in which the same object is captured and that has a longer on-screen stay time, among image areas in which the same object is captured.[Supplementary Note 8] The creation means of the above-mentioned target object information creation system may be configured to create target object information from an image area in which the same object is clearly captured among image areas in which the same object is captured. [Supplementary Note 9] The above-mentioned target object information creation system may further be configured to include a map creation means for plotting the target object information on a map of a location where the target object information is to be created, to create a map with target object information. [Supplementary Note 10] In the above-mentioned target object information creation system, at least one of the first and second cameras may be a camera of a vehicle passing through the location where the target object information is to be created. [Supplementary Note 11] A method for creating target information, comprising: acquiring images taken by a first and a second camera, each of which photographs a location for which target information is to be created, from different viewpoints; extracting image areas in which an object appears in the acquired images, and identifying the position and movement of the object; determining whether the object shown in the image area of ​​one camera is the same as the object shown in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and creating target information for an object located at the location for which target information is to be created using the determination result. [Supplementary Note 10] A recording medium storing a program that causes a computer to execute the following steps: acquiring images taken by first and second cameras that capture a location for which target information is to be created from different viewpoints; extracting image areas in which an object is captured from the acquired images and identifying the position and movement of the object; determining whether an object captured in the image area of ​​one camera is the same as an object captured in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and creating target information for an object located at the location for which target information is to be created using the determination result. Furthermore, the forms of Supplementary Notes 11 to 12 above can be expanded into the forms of Supplementary Notes 2 to 10, similar to Supplementary Note 1.

[0063] The disclosures of the above-cited patent documents are incorporated herein by reference and may be used as the basis or part of this disclosure, as necessary. Modifications and adjustments of the embodiments and examples are possible within the scope of this disclosure (including the claims), and further based on its basic technical concept. Furthermore, various combinations and selections (including partial deletions) of various disclosed elements (including elements of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible within the scope of this disclosure. In other words, this disclosure naturally includes various modifications and alterations that would be possible by a person skilled in the art in accordance with the entire disclosure, including the claims, and the technical concept. In particular, with regard to the numerical ranges described herein, any numerical value or subrange within that range should be construed as specifically described, even if not otherwise specified. Furthermore, the disclosures of the above-cited documents, when used in part or in whole in combination with the disclosures herein as part of this disclosure, in accordance with the spirit of this disclosure, are also deemed to be included in the disclosures of this application.

[0064] 10, 100, 100a, 100b Target information creation system 11 Acquisition means 12 Extraction means 13 Determination means 14 Creation means 101 Acquisition unit 102 Extraction unit 103 Determination unit 104, 104a, 104b Creation unit 105 Target map creation unit 106 Map information storage unit 1041 Estimation unit 900 Information processing device 901 CPU (Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage device 906 Recording medium 907 Drive device 908 Communication interface 909 Communication network 910 Input / output interface 911 Bus C1, C2 Camera CAR1 to CAR4 Vehicle TRUCK1 Truck

Claims

1. A target information creation system comprising: an acquisition means for acquiring images taken by a first and a second camera, each of which captures a location for which target information is to be created from different viewpoints; an extraction means for extracting image areas in which objects appear in the acquired images and identifying the position and movement of the objects; a determination means for determining whether an object captured in the image area of ​​one camera is the same as an object captured in the image area of ​​the other camera based on one or more of the position, direction of movement, and speed of the object in the image area; and a creation means for creating target information for an object located in the location for which target information is to be created using the determination result.

2. The target information creation system of claim 1, wherein the creation means creates target information using information obtained from an image area that is closer to the camera than other image areas that show the same object.

3. A target object information creation system according to claim 1 or 2, wherein the creation means creates target object information using information obtained from an image area that is larger on the screen than the other image areas in which the same object appears.

4. A target object information creation system according to any one of claims 1 to 3, wherein the creation means creates target object information using information obtained from an image area in which the same object is clearly captured, among image areas in which the same object is captured.

5. A target object information creation system according to any one of claims 1 to 4, wherein the creation means creates target object information using information obtained from an image area that has a high reliability when recognizing the object, among image areas in which the same object appears.

6. A target object information creation system according to any one of claims 1 to 5, wherein the creation means creates target object information using information obtained from an image area close to the center of the image among image areas in which the same object appears.

7. A target information creation system according to any one of claims 1 to 6, further comprising an estimation means for estimating the time that the object stays on the screen, and wherein the creation means creates target information using information obtained from an image area in which the object stays on the screen for a longer time than other image areas in which the same object appears.

8. A target information creation system according to any one of claims 1 to 7, wherein said creation means creates target information using information obtained from each of the image areas in which the same object appears.

9. A target object information creation system according to any one of claims 1 to 8, further comprising a map creation means for creating a map with target object information by plotting the target object information on a map of the location where the target object information is to be created.

10. A target object information creation system according to any one of claims 1 to 9, wherein at least one of the first and second cameras is a camera mounted on a vehicle passing through the location where the target object information is created.

11. A method for creating target information, comprising: acquiring images taken by a first and a second camera, each of which photographs a location for which target information is to be created, from different viewpoints; extracting image areas in which objects appear in the acquired images, and identifying the position and movement of the objects; determining whether the object shown in the image area of ​​one camera is the same as the object shown in the image area of ​​the other camera based on one or more of the object's position, direction of movement, and speed in the image area; and using the result of the determination, creating target information for the object located in the location for which target information is to be created.

12. A recording medium storing a program that causes a computer to execute the following steps: acquiring images taken by a first and second camera, each taking an image of a location for which target information is to be created from different viewpoints; extracting image areas in which objects appear in the acquired images and identifying the position and movement of the objects; determining whether an object shown in the image area of ​​one camera is the same as an object shown in the image area of ​​the other camera based on one or more of the object's position, direction of movement, and speed in the image area; and creating target information for an object located in the location for which target information is to be created using the results of the determination.

Images