Information processing apparatus, information processing system using the same, information processing method, and information processing program

Abstract

To provide an information processing apparatus capable of complementing and making redundant information by a plurality of sensors by preventing control due to erroneous information.SOLUTION: The information processing apparatus is provided with: a distance measurement data acquisition unit for acquiring a measurement distance to a distance measurement target present within a detection range of a distance measurement unit; an image data acquisition unit for acquiring image data of an image pickup range overlapping a detection range from an image pickup unit; an image analysis unit for analyzing features of a target image object included in the image data and targeted for analysis; an object size calculation unit for setting a distance measurement target corresponding to the target image object as an estimation distance measurement object and calculating an object size based on the measurement distance to the estimation distance measurement object and an image size of the target image measurement object obtained from the image data; a feature match determination unit for determining whether or not the object size matches the features of the target image object; and a coincidence probability calculation unit for, if it is determined to match, determining the estimation distance measurement object as a target distance measurement object and calculating a coincidence probability in which the target image object coincides with the target distance measurement object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system using the same, an information processing method, and an information processing program. [Background technology]

[0002] Autonomous driving assistance systems and autonomous driving systems are being considered that automatically control driving position, speed, etc. to drive the vehicle to its destination. These systems improve the accuracy of the information they obtain by acquiring information about the vehicle's surroundings using multiple sensors with different measurement principles, such as cameras, millimeter wave radar, and LiDAR (Light Detection and Ranging), and by having these sensors complement each other or by acquiring elements with the same role and putting them on standby (so-called redundancy).

[0003] For example, Patent Document 1 discloses an information processing system that performs sensor fusion using a millimeter-wave radar and a camera. According to this technology, the millimeter-wave radar detects the distance and direction of multiple targets, and the camera captures images so that the imaging range overlaps with the detection range of the millimeter-wave radar. Then, the system calculates a match probability indicating the possibility that the imaged target and the distance-measured target match based on area information from the imaged targets and area information from the distance-measured targets, thereby associating the correct distance-measured target with the imaged target. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7154470 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technology, even if there are other objects in front of or around an object and the distance is measured by mistake using the object in front or around it as a ranging target, there is a risk that they will be determined to be the same target if there is a large overlap between the area of ​​the erroneously obtained ranging target and the area of ​​the captured target. In such a case, not only cannot the information provided by a plurality of sensors be complemented or made redundant, but there is also the problem that vehicle control, etc. may be erroneously performed based on erroneous information.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an information processing device capable of preventing control due to erroneous information and complementing and redundancy of information from multiple sensors, as well as an information processing system, an information processing method, and an information processing program using the same. [Means for solving the problem]

[0007] The information processing device according to the present disclosure includes a ranging data acquisition unit that acquires a measured distance to a ranging target that is present within the detection range of the ranging unit detected by the ranging unit; an image data acquisition unit that acquires image data from the imaging unit of an imaging range that is set so that at least a portion of the imaging range overlaps with the detection range; an image analysis unit that analyzes the characteristics of a target imaging target that is included in the image data and is an imaging target set as an analysis target; a target size calculation unit that sets a ranging target estimated to correspond to the target imaging target as an estimated ranging target and calculates the target size based on the measured distance to the estimated ranging target acquired by the ranging data acquisition unit and the image size of the target imaging target obtained from the image data; a feature match determination unit that determines whether the target size matches the features of the target imaging target analyzed by the image analysis unit; and a match probability calculation unit that determines the estimated ranging target as a target ranging target corresponding to the target imaging target when the feature match determination unit determines that there is a match, and calculates a match probability that the target imaging target and the target ranging target match based on area information of the target imaging target and area information of the target ranging target.

[0008] The information processing system of the present disclosure includes a ranging unit that detects a measurement distance to a ranging target located within a detection range, an imaging unit that captures image data of an imaging range that is set to overlap at least partially with the detection range of the ranging unit, an information processing device of the present disclosure, and a vehicle control unit that controls the vehicle based on the data transmitted from the information processing device.

[0009] The information processing method according to the present disclosure includes a ranging data acquisition step of acquiring a measured distance to a ranging target present within a detection range of the ranging unit detected by the ranging unit; an image data acquisition step of acquiring from the imaging unit image data within an imaging range set to overlap at least a portion of the detection range; an image analysis step of analyzing the characteristics of a target imaging target which is included in the image data and is an imaging target set as an analysis target; a target size calculation step of setting a ranging target estimated to correspond to the target imaging target as an estimated ranging target and calculating the target size based on the measured distance to the estimated ranging target acquired in the ranging data acquisition step and the image size of the target imaging target obtained from the image data; a feature match determination step of determining whether the target size matches the features of the target imaging target analyzed in the image analysis step; and, if it is determined that there is a match in the feature match determination step, determining the estimated ranging target as a target ranging target corresponding to the target imaging target and calculating the match probability that the target imaging target and the target ranging target match based on area information of the target imaging target and area information of the target ranging target.

[0010] The information processing program according to the present disclosure causes a computer to execute a ranging data acquisition step of acquiring a measured distance to a ranging target present within a detection range of the ranging unit detected by the ranging unit; an image data acquisition step of acquiring from the imaging unit image data within an imaging range set to overlap at least a portion of the detection range; an image analysis step of analyzing the characteristics of a target imaging target which is included in the image data and is an imaging target set as an analysis target; a target size calculation step of setting a ranging target estimated to correspond to the target imaging target as an estimated ranging target and calculating the target size based on the measured distance to the estimated ranging target acquired in the ranging data acquisition step and the image size of the target imaging target obtained from the image data; a feature match determination step of determining whether the target size matches the features of the target imaging target analyzed in the image analysis step; and, if it is determined that there is a match in the feature match determination step, determining the estimated ranging target as a target ranging target corresponding to the target imaging target and calculating the match probability that the target imaging target and the target ranging target match based on area information of the target imaging target and area information of the target ranging target. Effect of the Invention

[0011] According to the present disclosure, control based on erroneous information can be prevented, information from multiple sensors can be complemented and made redundant, and control accuracy can be improved. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic block diagram showing a configuration of an information processing system according to a first embodiment. [Diagram 2] 3 is a schematic diagram showing an example of image data according to the first embodiment. FIG. [Diagram 3] 4 is a schematic diagram showing an example of a target area of ​​an image capture target in an image capture range and a provisional area onto which a range measurement target is provisionally projected in accordance with the first embodiment. FIG. [Figure 4] 4 is a schematic diagram showing an example of a target area of ​​an image capture target in an image capture range and a provisional area onto which a range measurement target is provisionally projected in accordance with the first embodiment. FIG. [Diagram 5]4 is a schematic diagram showing an example of a target area of ​​an image capture target in an image capture range and a provisional area onto which a range measurement target is provisionally projected in accordance with the first embodiment. FIG. [Figure 6] 4 is an example of a feature data table according to the first embodiment. [Figure 7] 2 is a schematic diagram showing the positional relationship between an image capture target and a distance measurement target according to the first embodiment. FIG. [Figure 8] 4 is a schematic block diagram showing the configuration of a match probability calculation unit according to the first embodiment. FIG. [Figure 9] 10 is a flowchart showing a processing routine executed by an information processing device according to the second embodiment. [Figure 10] 11 is a flowchart showing an example of a processing routine executed by the information processing device according to the second embodiment. [Figure 11] 11 is a schematic configuration diagram showing an example of a processing circuit that realizes each function of an information processing device according to a second embodiment. FIG. [Figure 12] FIG. 11 is a schematic block diagram showing a configuration of an information processing system according to a third embodiment. [Figure 13] FIG. 11 is a schematic diagram showing an example of image data according to the third embodiment. [Figure 14] 13 is a schematic diagram showing an example of a target area of ​​an image capture target in an image capture range and a provisional area onto which a range measurement target is provisionally projected in accordance with the third embodiment. FIG. [Figure 15] FIG. 11 is an explanatory diagram for explaining road information according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following description of the embodiments will be given with reference to the accompanying drawings, in which the same contents and corresponding parts are designated by the same reference numerals and detailed description thereof will be omitted.

[0014] Embodiment 1 1 is a schematic block diagram showing a configuration of an information processing system 1000 according to embodiment 1. The information processing system 1000 includes an information processing device 100, an imaging unit 11, a distance measuring unit 12, and a vehicle control unit 13. The information processing device 100 is mounted on a host vehicle 40 such as an automobile, a train, or the like (not shown).

[0015] The information processing device 100 includes a distance measurement data acquisition unit 102 that acquires the measured distance to a distance measurement target present within the detection range of the distance measurement unit 12 detected by the distance measurement unit 12, an image data acquisition unit 101 that acquires image data of an imaging range set to overlap at least a part of the detection range from the imaging unit 11, an image analysis unit 103 that analyzes the characteristics of a target imaging target that is included in the image data and is an imaging target set as an analysis target, and a distance measurement target that is estimated to correspond to the target imaging target is set as an estimated distance measurement target, and the estimated distance measurement target acquired by the distance measurement data acquisition unit 102 is analyzed. and the image size of the target object obtained from the image data; a feature match determination unit 105 which determines whether the target size matches the features of the target object analyzed by the image analysis unit 103; and a match probability calculation unit 106 which, when the feature match determination unit 105 determines that the estimated distance measurement target is a target distance measurement target corresponding to the target image measurement target, calculates a match probability that the target image measurement target and the target distance measurement target match based on area information of the target image measurement target and area information of the target distance measurement target.

[0016] The distance measuring unit 12 is capable of measuring the distance to a target in a non-contact manner, such as a millimeter wave radar, a laser measuring instrument, or an ultrasonic sensor. The distance measuring unit 12 measures the distance from a distance measurement reference to a distance measurement target, which is a target that exists within the detection range, and provides the measured distance to the distance measurement data acquisition unit 102. The distance measuring unit 12 may detect data indicating the direction and coordinates of the distance measurement target and send it to the distance measurement data acquisition unit 102.

[0017] The imaging unit 11 is, for example, a digital camera that converts the optical signal of the subject entering through the lens into an electrical signal by an image sensor. The image data acquisition unit 101 acquires image data captured so as to overlap with the detection range of the distance measurement unit 12 from the imaging unit 11. The image analysis unit 103 then determines a target imaged target among the imaged targets included in the image data, and analyzes the characteristics of the target imaged target. The characteristics are, for example, the type of the target, or the type of the target plus the target's attitude, target direction, etc.

[0018] The image analysis unit 103 analyzes the type of surrounding object, such as a person 2, a vehicle 4 such as an automobile or bicycle, a guardrail 1, road facilities such as a fence, a tree 7, etc., by matching it with pattern data previously stored in the storage device 81, for example. It also analyzes the attitude and direction of the target captured object.

[0019] The image analysis unit 103 may analyze characteristics such as the type, attitude, and direction of the target object based on image data captured by multiple cameras from different positions. The image analysis unit 103 may predict the movement using a convolutional neural network (CNN) technique. Then, the captured target area and position in the image data are obtained.

[0020] The target size calculation unit 104 determines a ranging target that is estimated to correspond to the target imaged target as an estimated ranging target, and calculates the target size from the measured distance to the estimated ranging target acquired by the ranging data acquisition unit 102 and the image size of the target imaged target obtained from the image data acquired by the image data acquisition unit 101.

[0021] Fig. 2 is a schematic diagram showing an example of image data, in which the image data acquisition unit 101 or the image analysis unit 103, which acquires image data captured by the imaging unit 11, superimposes the area in which the captured target is recognized as a bounding box. In Fig. 2, a guardrail 1, a person 2, a utility pole 3, a vehicle 4, and a road 5 can be recognized, and a target area 2a of the person 2 and a target area 4a of the vehicle 4 processed by the image data acquisition unit 101 or the image analysis unit 103 are displayed.

[0022] FIG. 3 is a schematic diagram showing an example of a target area of ​​an imaged target in the image capturing range shown in FIG. 2 and a provisional area onto which a range-finding target is provisionally projected. The distance measurement data acquisition unit 102 calculates the area in which the distance measurement target is projected onto the image data using position information such as the measured distance, direction, and coordinates to the distance measurement target measured by the distance measurement unit 12, and displays it on the screen as a provisionally calculated provisional area.

[0023] For example, in a sensor such as a millimeter wave radar, if there is an object with a high reflection intensity in the vicinity, the object with a low reflection intensity may be absorbed. In such a case, the target may not be detected or multiple targets may be detected as a composite.

[0024] For example, in the example of Fig. 3, a person 2 with low reflection intensity is not detected, but a guardrail 1 and a utility pole 3 with high reflection intensity that are present around the person 2 are detected, and provisional areas 1b and 3b are displayed. In the example of Fig. 4, a provisional area 6b is displayed based on a target (here called a composite target 6) in which the person 2 with low reflection intensity is composited with the guardrail 1 with high reflection intensity that is present in the vicinity. In the example of Fig. 5, a provisional area 6b is displayed based on a composite target 6 in which the person 2 with low reflection intensity is composited with the guardrail 1 and utility pole 3 with high reflection intensity that are present in the vicinity. In these examples, since there are no objects around the vehicle 4 that would interfere with the measurement, a provisional area 4b is obtained that almost overlaps with the target area 4a of the vehicle 4. Meanwhile, there are objects with high reflection intensity such as a utility pole 3 and a guardrail 1 around the person 2, and the reflection intensity of the person 2 is low, making him or her difficult to detect.

[0025] Even in such a case, the provisional area 1b and the provisional area 6b overlap with the target area 2a in the image data recognized as the person 2, so that the coincidence probability P of the provisional area 1b or the provisional area 6b and the target area 2a may be calculated to be high. If the coincidence probability P is calculated to be high, the imaged target and the distance-measured target may be erroneously linked, and an erroneous distance-measured distance may be used.

[0026] Therefore, the image analysis unit 103 first regards the person 2 set as the analysis target among the captured objects included in the image data as the target object, and analyzes the characteristics of the corresponding target captured object. The image analysis unit 103 refers to threshold setting map data in which the relationship between the features of the target and the first and second thresholds is preset, and calculates the first and second thresholds corresponding to the features of the target captured target.

[0027] The threshold setting map data stores data such as a first threshold and a second threshold for size for each of types such as person 2, animal, car, bicycle, guardrail 1, fence, tree 7, etc., orientation such as front, side, etc., and posture such as standing state, crouching state, hand raised state, etc. The first threshold and the second threshold can be, for example, a lower limit and an upper limit of size. When the range between the first threshold and the second threshold is large, calculation may be performed based on statistical data, for example, so that the first threshold is the average value -2σ (standard deviation) and the second threshold is the average value +2σ. When specification data is available, an error value ΔS that takes into account measurement accuracy may be used to calculate the specification value -ΔS as the first threshold and the specification value +ΔS as the second threshold. The first and second thresholds may be determined by function calculation using variables. Furthermore, the first threshold value and the second threshold value may be changed according to the type and the orientation, or the type and the posture, for example. Also, the image data of the person 2 may be further analyzed to determine whether it is an adult or a child, and the first threshold value and the second threshold value may be changed.

[0028] The threshold setting map data is not limited to a table and may be a function formula or the like, but may be prepared and stored in a feature data table such as that shown in Fig. 6. In the example of Fig. 6, the lower limit of the size of height, width, etc. based on the feature that the orientation of the automobile, person, and guardrail is the front or side is set as the first threshold, and the upper limit is set as the second threshold.

[0029] For example, if the target image target in the image data of FIG. 2 is person 2, its features are analyzed to be type person 2, orientation facing forward, and posture standing up, and from FIG. 6, the first height threshold is calculated to be 0.5 m and the second height threshold is calculated to be 3 m.

[0030] Next, a target object to be measured that corresponds to the target object to be photographed is searched for, and the measured distance is calculated. In the example of Fig. 2, since the person 2 overlaps with the guardrail 1 in front, the distance measurement unit 12 or the distance measurement data acquisition unit 102 may determine that the guardrail 1 is the target to be measured. Therefore, the target size calculation unit 104, the feature match determination unit 105, and the match probability calculation unit 106 are used to search for the correct target to be measured. The functions of these units will be described below.

[0031] The target size calculation unit 104 identifies a ranging target that is estimated to correspond to the target imaged target as an estimated ranging target, and calculates the target size based on the measured distance to the estimated ranging target acquired by the ranging data acquisition unit 102 and the image size of the target imaged target obtained from the image data. Here, the measured distance acquired by the distance measurement data acquisition unit 102 is corrected to the measured distance from the imaging reference point 111 of the imaging unit 11. This correction may be performed by the distance measurement data acquisition unit 102, or may be performed by the image analysis unit 103 or the target size calculation unit 104.

[0032] An example of calculating the target size will be described with reference to FIG. When image data of a guardrail 1 and a person 2 standing behind the guardrail 1 is acquired by the imaging unit 11, the positional relationship between them is, for example, as shown in Fig. 7. The projection surface 112 is located near an imaging reference point 111 such as the center of the camera lens and is small, but is shown enlarged and schematic for the purpose of explanation.

[0033] Generally, an object is rendered as pixels on the intersection of a straight line connecting the imaging reference point 111 and the target with the projection plane 112. Therefore, for example, the height of an image of a guardrail 1 is a length D1 connecting the intersection of a dashed line E1t connecting the top end of the guardrail 1 and the imaging reference point 111 with a dashed line E1b connecting the bottom end of the guardrail 1 and the imaging reference point 111 with the projection plane 112. Similarly, the height of an image of a person 2 is a length D2 connecting the intersection of a solid line E2t connecting the top end of the person 2 and the imaging reference point 111 with a solid line E2b connecting the bottom end of the person 2 and the imaging reference point 111 with the projection plane 112. Comparing the actual height R1 of the guardrail 1 and the height R2 of the person 2, the person 2 is taller, but on the screen the guardrail 1 appears taller. This is because the size of an object projected on the projection surface 112 becomes smaller in proportion to its distance from the imaging reference point 111.

[0034] Therefore, in order to determine whether or not the distance measurement target and the imaged target coincide in the area on the screen, it is necessary to make corrections using information on the distance from the image capture reference point 111 to the object. Therefore, for example, the distance measurement data acquisition unit 102 corrects the distance data to the multiple distance measurement targets detected by the distance measurement unit 12 to the distance from the imaging reference point 111, and the image analysis unit 103 displays the calculated provisional area on the screen. Then, the target size calculation unit 104 estimates and calculates the actual size of the distance measurement target based on the provisional area projected on the image data.

[0035] For example, the actual height R2 of the imaged object, that is, person 2, can be calculated using the height D2 of person 2 on the projection surface 112, the distance F from the image capturing reference point 111 to the projection surface 112 (equivalent to, for example, the focal length of the camera), and the distance L2 from the image capturing reference point 111 to person 2, as shown in the following formula (1).

[0036]

number

[0037]

number

[0038] However, if the distance L2 to the person 2 is mistakenly measured as the distance L1 to the guardrail 1 in front (corresponding to the case where the provisional area 1b based on the guardrail 1 shown in Figure 3 is used), the incorrect height R2' of the person 2 can be calculated using the following formula (3).

[0039]

number

[0040] Here, for example, when D2=0.04, F=0.05, and L1=0.5 are input, the target size calculation unit 104 derives the target size R2'=0.4 based on the formula (3). In this case, the feature match determination unit 105 determines that the features do not match because the target size does not exceed the first threshold value of 0.5 for the height of the person 2 in the feature data table shown in FIG. For example, if D2=0.05, F=0.05, and L1=0.5 are input, and the target size is R2'=0.5, which is equal to the first threshold value of the height of person 2 of 0.5, the image analysis unit 103 may analyze whether person 2 is an adult or a child and make a determination again. In other words, the first threshold value and the second threshold value are changed in stages to be more sophisticated, thereby increasing reliability. If the image analysis results in the analysis showing that the features are of an adult, it is determined that the features do not match.

[0041] For example, when D2=0.04, L2=2, and F=0.05 are input, the target size becomes R2=1.6, which is within the range of the first threshold value of 0.5 and the second threshold value of 3 for the height of person 2, so the feature match determination unit 105 determines that the features match. Furthermore, in order to increase the reliability of the match determination, the image analysis unit 103 may perform an advanced analysis of the features and increase the first and second threshold values ​​to determine whether the features match. If the feature match determination unit 105 does not determine that there is a match, the target size is calculated using the measured distance to the estimated distance measurement target that is estimated to correspond to the target image capture target, and the feature match determination is repeated.

[0042] Then, if the feature match determination unit 105 determines that there is a match, the estimated distance measurement target is set as the target distance measurement target corresponding to the target imaging target, and the match probability calculation unit 106 calculates the match probability P that the target imaging target and the target distance measurement target match based on the area information of the target imaging target and the area information of the target distance measurement target. For example, if the target area of ​​the target to be imaged is 2a and the provisional area onto which the distance measurement target is provisionally projected is 2b, the match probability P is calculated by the following formula (4) or formula (5).

[0043]

number

[0044]

number

[0045] That is, using formula (4) or formula (5), the size of overlap between the target imaged object and the provisional area is calculated, and a matching probability P is calculated, which increases as the overlap increases. 2a and 2b in formula (4) or formula (5) may be calculated using the area of ​​each region. The height or width of each region may also be used. The match probability P may also be calculated using other formulas for calculating the degree of overlap.

[0046] Furthermore, when the image analysis unit 103 analyzes the features as, for example, person 2, the target size calculation unit 104 calculates the target size, and the features do not match in the feature match determination unit 105, the match probability P is not calculated in the match probability calculation unit 106, and may instead be set to, for example, 0, indicating a mismatch.

[0047] As shown in FIG. 8, the match probability calculation unit 106 includes a probability calculation unit 1061 that calculates the match probability P described above, a determination unit 1062 that determines whether or not the target distance measurement target and the target captured object match based on the match probability P, and a combination unit 1063 that combines the target distance measurement target that is determined to match with the target captured object.

[0048] The determination unit 1062 determines that the target distance measurement target and the target captured object match when the matching probability P is equal to or greater than a tolerance value. The target distance measurement target to be combined with the target captured object by the combination unit 1063 may be the target distance measurement target at the time when the matching probability P becomes equal to or greater than a tolerance value, or the feature match determination unit 105 may calculate the matching probability P from all data whose features match, and combine the target distance measurement target with the maximum matching probability P with the target captured object.

[0049] In the case of using the information for vehicle driving, if the coincidence probability P is equal to or greater than a permissible value, the target object to be measured is set as a control target for vehicle driving. The target object to be measured that is determined by the determination unit 1062 as an object to be measured is combined with the target imaged object by the combination unit 1063, and the distance measurement information of the combined target object to be measured is transmitted to the vehicle control unit 13.

[0050] In this manner, the present invention includes a distance measurement data acquisition unit 102 that acquires the measured distance to a distance measurement target present within the detection range of the distance measurement unit 12 detected by the distance measurement unit 12, an image data acquisition unit 101 that acquires image data from the imaging unit 11 within an imaging range set so as to overlap at least a part of the detection range, an image analysis unit 103 that analyzes the characteristics of a target imaging target that is included in the image data and is an imaging target set as an analysis target, and a target analysis unit 104 that sets a distance measurement target estimated to correspond to the target imaging target as an estimated distance measurement target, and calculates the target size based on the measured distance to the estimated distance measurement target acquired by the distance measurement data acquisition unit 102 and the image size of the target imaging target acquired from the image data. By providing a size calculation unit 104, a feature match determination unit 105 which determines whether the target size matches the features of the target captured target analyzed by the image analysis unit 103, and a match probability calculation unit 106 which, when the feature match determination unit 105 determines that the estimated target is a target captured target corresponding to the target captured target, and calculates a match probability that the target captured target and the target captured target match based on the area information of the target captured target and the area information of the target captured target, it is possible to prevent a mistaken determination that the captured target and the target corresponding to the range-finding target match because of a large overlap between the target area and the provisional area, even though the captured target and the target corresponding to the range-finding target are different. In other words, control due to erroneous information can be prevented, and information from multiple sensors can be complemented and made redundant, thereby improving the accuracy of control.

[0051] Furthermore, by referring to threshold setting map data in which the relationship between the characteristics of the imaged target and the first and second thresholds based on the characteristics is preset, the first and second thresholds corresponding to the characteristics of the target imaged target are calculated, and it is determined that the target size calculated based on the image size of the target imaged target and the measured distance to the estimated distance-measurement target is equal to or larger than the first threshold and equal to or smaller than the second threshold, it is possible to prevent the target imaged target from being linked to an erroneous distance-measurement target. By calculating the first and second thresholds with a high degree of accuracy, it is possible to eliminate erroneous distance-measurement targets.

[0052] Furthermore, if the type of target, or at least one of the target attitude and the target direction in addition to the type of target, is used as the feature of the target, errors in distance measurement can be eliminated with high accuracy. If the height, width, and area are used as the size of the target, errors in distance measurement can be eliminated with high accuracy.

[0053] Furthermore, if the feature match determination unit 105 determines that there is a clear mismatch, the match probability P at which the target image target and the range-measurement target match can be set to a value representing a mismatch, thereby making it possible to reduce the number of calculations in the match probability calculation unit 106. When the coincidence probability P is equal to or greater than a permissible value, the data of the target imaged object and the target distance measured object are combined, thereby preventing processing based on erroneous information and enabling information from multiple sensors to be complemented and made redundant. When the information is used for vehicle driving, vehicle control due to erroneous information can be prevented, and information from multiple sensors can be complemented and made redundant, thereby improving the accuracy of vehicle control.

[0054] In the above description, image data of an imaging range including at least one of a plurality of detected distance measurement targets is acquired. However, distance measurement data including a plurality of distance measurement targets may be acquired in accordance with the imaging range.

[0055] Furthermore, the image data captured by the imaging unit 11 and the distance measurement data measured by the distance measurement unit 12 may be weighted and added or multiplied. This may be set appropriately depending on the detection position accuracy of the imaging camera, distance measurement sensor, etc.

[0056] In addition, if there are objects that exist in the vicinity of the target image target and are frequently detected as composite targets, and it is possible to perform a feature match determination and calculate the match probability P even if these are treated as composite targets, they may be stored as objects that can be used as composite targets based on past data.

[0057] Embodiment 2 A processing method using the information processing device 100 will be described. FIG. 9 is a flowchart showing a processing routine executed by the information processing device 100 according to the second embodiment. First, the distance measurement unit 12 obtains the measured distance to the target within the detection range of the distance measurement unit 12 (step S101). Then, image data of the imaging range set so as to overlap at least a part of the detection range is obtained from the imaging unit 11 (step S102). Then, the characteristics of the target imaging target, which is included in the image data and is the imaging target set as the analysis target, are analyzed (step S103). Next, the target estimated to correspond to the target imaging target is set as the estimated target, and the target size is calculated based on the measured distance to the estimated target acquired in step S101 and the image size of the target imaging target obtained from the image data (step S104). Then, it is determined whether the target size matches the characteristics of the target imaging target analyzed in step S103 (step S105). If it is determined in step S105 that they match, the estimated distance measurement target is determined to be a target distance measurement target corresponding to the target imaging target, and a match probability that the target imaging target and the target distance measurement target match is calculated based on the area information of the target imaging target and the area information of the target distance measurement target (step S106). Information processing can be achieved by repeating such steps.

[0058] FIG. 10 is a flowchart showing an example of a processing routine executed by the information processing device 100, and shows some of the processing in detail. The distance measurement data acquisition unit 102 acquires distance measurement data including the measured distance from the distance measurement unit 12 to the distance measurement target (step S201). The image data acquisition unit 101 acquires image data of an imaging range including at least one distance measurement target (step S202). Then, the image data is included in the image data, and the characteristics of the target image target that is the image target set as the analysis target are analyzed (step S203). Then, the distance measurement target estimated to correspond to the target image target is set as the estimated distance measurement target i (i is a natural number from 1 to n), and the target size i is calculated based on the measured distance i to the estimated distance measurement target i acquired and the image size of the target image target acquired from the image data (step S204). Then, it is determined whether the target size i exceeds a second threshold value related to the characteristics of the target image target (step S205). Here, if it is determined that the target size i exceeds the second threshold value related to the characteristics of the target image target (YES in step S205), the match probability P is set to 0 representing a mismatch (step S206), and the process proceeds to step S211. When it is determined that the target size i is less than the first threshold value related to the characteristics of the target captured object (YES in step S207), the match probability P is also set to 0 indicating no match (step S208), and the process proceeds to step S211.

[0059] If the target size i is equal to or larger than the first threshold value and equal to or smaller than the second threshold value related to the characteristics of the target imaged object (NO in step S207), a provisional area where the target is provisionally projected is calculated (step S209), and the target area of ​​the target imaged object and the provisional area are used to calculate the coincidence probability P where the target imaged object and the distance-measured object coincide (step S210). If the coincidence probability P is not an allowable value (NO in step S211), the process returns to step 204 where the i-th target size i is calculated as i=i+1. If the coincidence probability P is an allowable value (YES in step S211), the target imaged object and the distance-measured object are combined (step S212). If there is another target imaged object (YES in step S213), the process returns to step 203 where the characteristics of the target imaged object are analyzed. If there is no other target imaged object (NO in step S213), the process routine ends.

[0060] In FIG. 10, an example is shown in which an estimated distance measurement target is set as i (i is a natural number from 1 to n), the target size i is calculated successively from n=1, and it is determined whether it matches the target imaged target using the first threshold value and the second threshold value. However, the calculation time can be shortened by weighting the overlap of the coordinate positions of the target imaged target and the distance measurement target.

[0061] Here, each function of the information processing device 100 is realized by a processing circuit. Fig. 11 is a schematic configuration diagram showing an example of a processing circuit that realizes each function of the information processing device 100. The information processing device 100 has a processor 80, a storage device 81, a communication I / F (interface) 82, a CAN (Controller Area Network) I / F (interface) 83, etc. For example, a CPU (Central Processing Unit) is used as the processor 80. The storage device 81 transmits and receives data to and from the processor 80 and stores the data. Image data captured by the imaging unit 11 and distance measurement data measured by the distance measurement unit 12 are acquired via the communication I / F 82. The calculations and judgments of the image data acquisition unit 101, distance measurement data acquisition unit 102, image analysis unit 103, target size calculation unit 104, feature match determination unit 105, match probability calculation unit 106, probability calculation unit 1061 of the match probability calculation unit 106, judgment unit 1062, and combination unit 1063 are executed by the processor 80. Threshold setting map data, feature data table, calculation formulas, etc. are stored in the storage device 81. Commands of the information processing device 100 are transmitted to the vehicle control unit 13 via the CANI / F 83.

[0062] The processor 80 and the storage device 81 may be one shared device or a plurality of devices may be provided. The processor 80 may include, for example, a logic circuit using an ASIC (Application Specific Integrated Circuit), an IC (Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or the like, and various signal processing circuits. A plurality of processors 80, either of the same type or of different types, may be provided, so that each process is shared and executed by a plurality of arithmetic processing devices.

[0063] The multiple storage devices 81 may include, for example, a RAM (Random Access Memory) configured to enable data to be read from and written to the processor 80, a ROM (Read Only Memory) configured to enable data to be read from the processor 80, a hard disk (HDD), etc.

[0064] Each function of the information processing device 100 is realized by the processor 80 executing software or a program stored in the storage device 81 and cooperating with the hardware. Setting data to be set in the information processing device 100 may be stored in the storage device 81 as part of the software or program, or may be input by the user. A non-transitory recording medium 85 on which an information processing program 84 is recorded may be distributed and installed in the information processing device 100 (storage device 81).

[0065] In this manner, the measured distance to the distance measurement target present within the detection range of the distance measurement unit 12 detected by the distance measurement unit 12 is acquired, image data of the imaging range set so as to overlap at least a part of the detection range is acquired from the imaging unit 11, the characteristics of the target imaging target which is included in the image data and is the imaging target set as the analysis target is analyzed, the distance measurement target estimated to correspond to the target imaging target is set as the estimated distance measurement target, and the target size is calculated based on the measured distance to the estimated distance measurement target acquired and the image size of the target imaging target acquired from the image data. It is determined whether the target size matches the characteristics of the analyzed target imaged target, and if it is determined that the characteristics match, the estimated distance measurement target is determined to be the target distance measurement target corresponding to the target imaged target, and a match probability that the target imaged target and the target distance measurement target match is calculated based on the area information of the target imaged target and the area information of the target distance measurement target, thereby preventing a mistaken determination that the imaged target and the target corresponding to the distance measurement target match due to a large overlap between the target area and the provisional area even though the target corresponding to the imaged target and the distance measurement target are different. In other words, control due to erroneous information can be prevented, information from multiple sensors can be complemented and made redundant, and the accuracy of control can be improved.

[0066] Similar processing may be performed when there are no other targets in the vicinity of the target imaged target. However, when the target imaged target has no other objects in the vicinity, the image analysis unit 103 may be programmed to skip the processing of the target size calculation unit 104 and the feature match determination unit 105, since there is a high probability that the features match.

[0067] Embodiment 3 12 is a schematic block diagram showing a configuration of an information processing system 1000 according to embodiment 3. The information processing system 1000 includes an information processing device 100, an imaging unit 11, a distance measuring unit 12, a map storage unit 14, a position receiver 15, and a vehicle control unit 13. The information processing device 100 according to embodiment 3 differs from embodiment 1 in that it includes a map data acquisition unit 107 that acquires surrounding map data from the map storage unit 14, and a position information acquisition unit 108 that acquires a current position from the position receiver 15, but is otherwise similar to embodiment 1.

[0068] Here, road information is stored in the map storage unit 14, and the position receiver 15 receives position information of the vehicle 40 by, for example, a GPS (Global Positioning System) signal from a satellite. The map data acquisition unit 107 acquires map data from the map storage unit 14, and the position information acquisition unit 108 acquires the current position from the position receiver 15. Then, the image analysis unit 103 acquires target information from the map data of the map range corresponding to the detection range, using the map data acquired by the map data acquisition unit 107 and the current position acquired from the position information acquisition unit 108, and determines at least one of the first threshold value and the second threshold value based on the acquired target information.

[0069] For example, if it is known from the current position that the vehicle is traveling on a national highway, and the map data indicates that a guardrail 1 is installed along the national highway, that there is a sidewalk behind the guardrail 1, and that on the other side of the sidewalk there is a park with trees 7, the image analysis unit 103 recognizes, for example, the types of guardrail 1 and tree 7 as target information, and determines the first and second threshold values ​​for these.

[0070] FIG. 13 is a schematic diagram showing an example of image data. The image analysis unit 103 sets the guard rail 1 and the tree 7 included in the image data as analysis targets, and analyzes the features of these as target captured targets. The target size calculation unit 104 sets the distance measurement target estimated to correspond to the target captured target as an estimated distance measurement target, and calculates the target size based on the measured distance to the estimated distance measurement target acquired by the distance measurement data acquisition unit 102 and the image size of the target captured target obtained from the image data. The feature match determination unit 105 determines whether the target size matches the features of the target captured target analyzed by the image analysis unit 103. If the feature match determination unit 105 determines that there is a match, the match probability calculation unit 106 determines that the estimated distance measurement target is a target distance measurement target corresponding to the target captured target, and calculates the match probability P that the target captured target and the target distance measurement target match based on the area information of the target captured target and the area information of the target captured target shown in FIG. 14.

[0071] In this way, by acquiring target object information from map data of the map range and determining at least one of the first threshold value and the second threshold value based on the acquired target object information, the probability of a match in the feature matching determination unit 105 can be increased.

[0072] Furthermore, even if the information of a tree 7 acquired by the distance measurement data acquisition unit 102 is mistakenly recognized as a person 2, the characteristics are analyzed to calculate the target size, and it is determined from the target size whether the characteristics match, so that erroneous detection by sensor fusion can be prevented, and since the target information is acquired from map data, the accuracy can be further improved. In this way, it is possible to improve the accuracy of control, prevent control based on erroneous information, and complement and make redundant information from a plurality of sensors.

[0073] In addition, if it is confirmed that the characteristics match based on the target size, and it is determined from the map data that the tree 7 is sufficiently far away from the guardrail 1, the tree 7 can be excluded from the control targets for vehicle travel, and the processing of the match probability calculation unit 106 for the tree 7 can be skipped. In this way, the processing speed can be improved, control based on erroneous information can be prevented, and information from a plurality of sensors can be complemented and made redundant.

[0074] In addition, when the vehicle 4 traveling in front of the vehicle 40 is the target object, for example, when the road on which the vehicle 40 and the vehicle 4 travel is curved as shown in FIG. 15, if the width of the vehicle 4 is measured by the imaging unit 11 installed in front of the vehicle 40, the width of the vehicle 4 will be larger by the amount of inclination than when it is measured from directly behind. In this case, the width, which is the image size of the target object obtained from the image data, can be corrected based on the road information included in the map data. For example, curvature information such as the curvature radius R of the road on which the vehicle 40 and the vehicle 4 travel is obtained as road information, and the inclination degree of the vehicle 4, which is the target object, relative to the front-rear direction of the vehicle 40 is obtained using the distance from the vehicle 40 to the vehicle 4, and the width of the image size of the vehicle 4 is corrected based on the inclination degree. As the inclination degree increases, the width of the image size of the vehicle 4 is reduced. In this way, by supplementing the image data acquired by the image data acquisition unit 101 with map data, it is possible to further improve accuracy and analyze the characteristics of the target object and calculate the target size.

[0075] The calculation for correcting the image size using this map data may be performed by the image analysis unit 103 or the target size calculation unit 104. Alternatively, the image size may not be corrected, and the first threshold and the second threshold stored in the threshold setting map data in association with the target feature may be corrected. In this case, at least one of the first threshold and the second threshold may be corrected based on the inclination degree. As the inclination degree increases, at least one of the first threshold and the second threshold is increased. The device includes a map data acquisition unit 107 for acquiring surrounding map data and a position information acquisition unit 108 for acquiring the current position, and if the image size of the target captured target obtained from the image data or at least one of the first threshold and the second threshold is corrected based on road information included in the acquired map data of the surroundings of the current position, the probability of matching in the feature match determination unit 105 can be increased.

[0076] In the above-described first to third embodiments, when the target size calculated by the target size calculation unit 104 does not match the characteristics of the target object analyzed by the image analysis unit 103, the distance measurement is repeated by the distance measurement unit 12. However, when the target size does not match within the time limit even after the repeated processing, the distance measurement data may be acquired again from the distance measurement unit 12, or the process may proceed to a routine using a different measurement method without using the distance data acquired from the distance measurement unit 12. For example, the distance information may be obtained from the displacement (parallax) of corresponding positions by the principle of triangulation using a pair of image data captured in stereo by the imaging unit 11. If the information does not converge within the time limit, an alarm may be issued and the information may be released. If the information is to be used for driving the vehicle, the autonomous driving may be released.

[0077] Summary of the various aspects of the present application Various aspects of the present application are summarized below as appendices. (Appendix 1) a distance measurement data acquisition unit that acquires a measured distance to a distance measurement target that exists within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition unit that acquires image data of an imaging range that is set so as to at least partially overlap with the detection range from an imaging unit; an image analysis unit that analyzes features of a target captured object that is included in the image data and is set as an analysis target; a target size calculation unit that sets the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculates a target size based on the measured distance to the estimated distance measurement target acquired by the distance measurement data acquisition unit and an image size of the target image capture target acquired from the image data; a feature match determination unit that determines whether the target size matches the feature of the target object analyzed by the image analysis unit; a match probability calculation unit that, when it is determined by the feature match determination unit that there is a match, determines that the estimated distance measurement target is a target distance measurement target corresponding to the target imaging target, and calculates a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing device comprising:

[0078] (Appendix 2) The image analysis unit calculates the first threshold value and the second threshold value corresponding to the feature of the target captured object by referring to threshold setting map data in which a relationship between the feature of the target and a first threshold value and a second threshold value is preset; The information processing device according to claim 1, wherein the feature match determination unit determines that the target size matches when the target size is equal to or greater than the first threshold and equal to or less than the second threshold.

[0079] (Appendix 3) The information processing device according to claim 1 or 2, wherein the features include a type of target, or in addition to the type of target, at least one of an attitude of the target and an orientation of the target.

[0080] (Appendix 4) 4. The information processing device according to claim 1, wherein the target size calculation unit calculates at least one of a height, a width, and an area as the target size.

[0081] (Appendix 5) The information processing device according to any one of claims 1 to 4, wherein the match probability calculation unit sets a match probability between the target imaged target and the estimated distance measurement target to a value representing a mismatch when the feature match determination unit determines that they do not match.

[0082] (Appendix 6) The information processing device according to any one of claims 1 to 5, wherein the matching probability calculation unit is provided with a combining unit that combines data of the target imaging target and the target distance measurement target when the matching probability is equal to or greater than a tolerance value.

[0083] (Appendix 7) 7. The information processing device according to claim 1, wherein the match probability calculation unit determines that the match probability is equal to or greater than a tolerance value as the information to be used for vehicle driving.

[0084] (Appendix 8) The device includes a map data acquisition unit that acquires surrounding map data, and a location information acquisition unit that acquires a current location, The information processing device described in Appendix 2, wherein the image analysis unit uses the map data and the current location to acquire target information from the map data of a map range corresponding to the detection range, and determines at least one of the first threshold value and the second threshold value based on the acquired target information.

[0085] (Appendix 9) The device includes a map data acquisition unit that acquires surrounding map data, and a location information acquisition unit that acquires a current location, The information processing device according to claim 2, wherein the image analysis unit corrects the image size of the target captured object obtained from the image data or at least one of the first threshold value and the second threshold value based on road information included in the map data around the acquired current location.

[0086] (Appendix 10) The match probability calculation unit If the feature match determination unit determines that the features match, An information processing device according to any one of appendices 1 to 9, which identifies a provisional area indicating an area on an imaging surface of the estimated distance measurement target from the image data, and calculates a match probability indicating a possibility that the target imaging target and the target distance measurement target will match using a size of overlap between the provisional area and a target area in which the target imaging target is photographed.

[0087] (Appendix 11) a distance measuring unit for detecting a measured distance to a target within a detection range; an imaging unit that captures image data of an imaging range that is set so as to at least partially overlap the detection range of the distance measuring unit; An information processing device according to any one of Supplementary Notes 1 to 10; a vehicle control unit that controls a vehicle based on the data transmitted from the information processing device; An information processing system comprising:

[0088] (Appendix 12) a distance measurement data acquisition step of acquiring a measured distance to a distance measurement target present within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition step of acquiring, from an imaging unit, image data of an imaging range set so as to at least partially overlap the detection range; an image analysis step of analyzing features of a target captured object, which is included in the image data and is set as an analysis target; a target size calculation step of setting the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculating a target size based on the measured distance to the estimated distance measurement target acquired in the distance measurement data acquisition step and an image size of the target image capture target acquired from the image data; a feature matching determination step of determining whether the target size matches the feature of the target object analyzed in the image analysis step; a match probability calculation step of determining, when it is determined in the feature match determination step that there is a match, the estimated distance measurement target as a target distance measurement target corresponding to the target imaging target, and calculating a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing method comprising:

[0089] (Appendix 13) a distance measurement data acquisition step of acquiring a measured distance to a distance measurement target present within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition step of acquiring, from an imaging unit, image data of an imaging range set so as to at least partially overlap the detection range; an image analysis step of analyzing features of a target captured object, which is included in the image data and is set as an analysis target; a target size calculation step of setting the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculating a target size based on the measured distance to the estimated distance measurement target acquired in the distance measurement data acquisition step and an image size of the target image capture target acquired from the image data; a feature matching determination step of determining whether the target size matches the feature of the target object analyzed in the image analysis step; a match probability calculation step of determining, when it is determined in the feature match determination step that there is a match, the estimated distance measurement target as a target distance measurement target corresponding to the target imaging target, and calculating a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing program that causes a computer to execute the above.

[0090] In the present disclosure, various exemplary embodiments are described, but various features, aspects, and functions described in one or more embodiments are not limited to the application of a specific embodiment, and can be applied to the embodiments alone or in various combinations. Therefore, countless modifications not illustrated are expected within the scope of the technology disclosed in the present specification. For example, the modification, addition, or omission of at least one component, and the extraction and combination of at least one component with components of other embodiments are included. [Explanation of symbols]

[0091] 1 guardrail, 2 person, 3 utility pole, 4 vehicle, 5 road, 6 synthetic target, 7 tree, 11 imaging unit, 12 distance measurement unit, 13 vehicle control unit, 14 map storage unit, 15 position receiver, 80 processor, 81 storage device, 82 communication I / F, 83 CANI / F, 84 information processing program, 85 recording medium, 100 information processing device, 101 image data acquisition unit, 102 distance measurement data acquisition unit, 103 image analysis unit, 104 target size calculation unit, 105 feature match determination unit, 106 match probability calculation unit, 107 map data acquisition unit, 108 position information acquisition unit, 1000 information processing system

Claims

1. a distance measurement data acquisition unit that acquires a measured distance to a distance measurement target that exists within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition unit that acquires image data of an imaging range that is set so as to at least partially overlap with the detection range from an imaging unit; an image analysis unit that analyzes features of a target captured object that is included in the image data and is set as an analysis target; a target size calculation unit that sets the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculates a target size based on the measured distance to the estimated distance measurement target acquired by the distance measurement data acquisition unit and an image size of the target image capture target acquired from the image data; a feature match determination unit that determines whether the target size matches the feature of the target object analyzed by the image analysis unit; a match probability calculation unit that, when it is determined by the feature match determination unit that there is a match, determines that the estimated distance measurement target is a target distance measurement target corresponding to the target imaging target, and calculates a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing device comprising:

2. The image analysis unit calculates the first threshold value and the second threshold value corresponding to the feature of the target captured object by referring to threshold setting map data in which a relationship between the feature of the target and a first threshold value and a second threshold value is preset; The information processing apparatus according to claim 1 , wherein the feature match determination unit determines that the target size matches when the target size is equal to or larger than the first threshold value and equal to or smaller than the second threshold value.

3. The information processing apparatus according to claim 1 , wherein the features include at least one of a type of the target, or an attitude of the target and an orientation of the target in addition to the type of the target.

4. The information processing apparatus according to claim 1 , wherein the target size calculation unit calculates at least one of a height, a width, and an area as the target size.

5. The information processing device according to claim 1 , wherein the match probability calculation unit sets a match probability between the target image target and the estimated distance measurement target to a value representing a mismatch when the feature match determination unit determines that they do not match.

6. The information processing apparatus according to claim 1 , wherein the coincidence probability calculation unit includes a combining unit that combines data of the target imaging object and the target distance measurement object when the coincidence probability is equal to or greater than a tolerance value.

7. The information processing apparatus according to claim 6 , wherein the coincidence probability calculation unit uses the information as the information to be used for vehicle driving when the coincidence probability is equal to or greater than a tolerance value.

8. The device includes a map data acquisition unit that acquires surrounding map data, and a location information acquisition unit that acquires a current location, 3. The information processing device according to claim 2, wherein the image analysis unit uses the map data and the current location to acquire target information from the map data of a map range corresponding to the detection range, and determines at least one of the first threshold value and the second threshold value based on the acquired target information.

9. The device includes a map data acquisition unit that acquires surrounding map data, and a location information acquisition unit that acquires a current location, 3. The information processing device according to claim 2, wherein the image analysis unit corrects the image size of the target object obtained from the image data or at least one of the first threshold value and the second threshold value based on road information included in the acquired map data around the current location.

10. The match probability calculation unit If the feature match determination unit determines that the features match, 2. The information processing device according to claim 1, further comprising: a provisional area indicating an area on an imaging surface of the estimated distance measurement target is identified from the image data; and a matching probability indicating a possibility that the target imaging target and the target distance measurement target will match is calculated using a size of overlap between the provisional area and a target area in which the target imaging target is photographed.

11. a distance measuring unit for detecting a measured distance to a target within a detection range; an imaging unit that captures image data within an imaging range that is set so as to at least partially overlap the detection range of the distance measuring unit; An information processing device according to any one of claims 1 to 10; a vehicle control unit that controls a vehicle based on the data transmitted from the information processing device; An information processing system comprising:

12. a distance measurement data acquisition step of acquiring a measured distance to a distance measurement target present within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition step of acquiring, from an imaging unit, image data of an imaging range set so as to at least partially overlap the detection range; an image analysis step of analyzing features of a target captured object, which is included in the image data and is set as an analysis target; a target size calculation step of setting the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculating a target size based on the measured distance to the estimated distance measurement target acquired in the distance measurement data acquisition step and an image size of the target image capture target acquired from the image data; a feature matching determination step of determining whether the target size matches the feature of the target object analyzed in the image analysis step; a match probability calculation step of determining, when it is determined in the feature match determination step that there is a match, the estimated distance measurement target as a target distance measurement target corresponding to the target imaging target, and calculating a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing method comprising:

13. a distance measurement data acquisition step of acquiring a measured distance to a distance measurement target present within a detection range of the distance measurement unit detected by the distance measurement unit; an image data acquisition step of acquiring, from an imaging unit, image data of an imaging range set so as to at least partially overlap the detection range; an image analysis step of analyzing features of a target captured object, which is included in the image data and is set as an analysis target; a target size calculation step of setting the distance measurement target estimated to correspond to the target image capture target as an estimated distance measurement target, and calculating a target size based on the measured distance to the estimated distance measurement target acquired in the distance measurement data acquisition step and an image size of the target image capture target acquired from the image data; a feature matching determination step of determining whether the target size matches the feature of the target object analyzed in the image analysis step; a match probability calculation step of determining, when it is determined in the feature match determination step that there is a match, the estimated distance measurement target as a target distance measurement target corresponding to the target imaging target, and calculating a match probability that the target imaging target and the target distance measurement target match based on area information of the target imaging target and area information of the target distance measurement target; An information processing program that causes a computer to execute the above.

Images