Object identification device and object identification method

Abstract

To improve the performance of object identification by suppressing erroneous recognition regardless of whether a distance from a moving object to an object is near or far.SOLUTION: An object identification device includes: a distance calculation unit 22 for receiving external information including an object outside a vehicle acquired by an external information acquisition unit 1 mounted on the vehicle to obtain the distance from the vehicle to the object; an identification score calculation unit 23 for acquiring an identification score showing a confidence level at which an object included in the external information is of a given type by use of the external information; and an object identification unit 26 for identifying that the object is a type associated with the identification score in the case that the identification score exceeds a predetermined threshold 71. The threshold 71 differs depending on the distance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an object identification device and an object identification method for identifying the type of an object on an image.

Background Art

[0002] For road traffic safety support, technologies for identifying an object in front of the host vehicle and applying automatic braking have been made mandatory. As a method for identifying the type of the target object, a method using an identifier created by learning images of a large number of specific objects is generally used. The identifier learns the shape and texture features of an object obtained from the edges of the learning images, and at the time of inference, outputs a score that is the probability of being an identification target for the input image. Then, a threshold value is set in advance for the output score, and when the score exceeds the threshold value, it is generally determined as "identified" to avoid misidentification and make a more probable identification.

[0003] For example, Patent Document 1 describes that "in an area where the user's usage frequency is high (a specific area described later), such as the garage of a house, by pre-learning the environment (scenery, installations, etc.) around the garage, the occurrence frequency of false detections can be reduced without lowering the detection rate of people." and "The person detection unit may detect a person reflected in the surrounding image using a known image analysis method. In the first embodiment, the person detection unit detects an object as a person when an index value regarding the object reflected in the surrounding image is equal to or greater than a first threshold value." That is, in the pedestrian detection device disclosed in Patent Document 1, in a limited area such as a home parking lot, the target object is detected when a preset threshold value is exceeded by an index value (score) regarding the object.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In contrast, when performing identification for automatic braking while driving on ordinary roads or highways, it is necessary to identify objects from the immediate vicinity to the distant location within the vehicle's braking distance, rather than within a limited area. In this case, as objects become more distant, they become relatively smaller, and the resolution of the objects themselves in the image decreases. When the resolution decreases, the accuracy of extracting the object region from the image deteriorates, and the outline of the object itself becomes unclear, which tends to lower the identification score.

[0006] When setting object recognition thresholds, misidentification of objects, especially those near the vehicle, can lead to false braking. Therefore, it is desirable to set strict object recognition thresholds, but this presents the problem of reduced identification rates for distant objects.

[0007] This invention has been made in view of the above circumstances, and aims to suppress misidentification and improve object identification performance, regardless of whether the distance from the moving object to the object is near or far. [Means for solving the problem]

[0008] To solve the above problems, an object identification device according to one aspect of the present invention includes: a distance calculation unit that receives external information, including objects outside the vehicle, acquired by an external information acquisition unit mounted on the vehicle, and calculates the distance from the vehicle to the object; an identification score calculation unit that uses the external information to calculate an identification score indicating the confidence that the object included in the external information is of a predetermined type; and an object identification unit that identifies the object as being of the type associated with the identification score when the identification score exceeds a predetermined threshold. The threshold value varies depending on the distance. [Effects of the Invention]

[0009] According to at least one aspect of the present invention, by setting an appropriate object identification threshold that matches the actual conditions at different distances, it is possible to suppress misidentification and improve object identification performance, regardless of whether the distance from a moving object such as a vehicle to an object is near or far. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example configuration of an object identification device according to the first embodiment of the present invention. [Figure 2] This figure shows an example of distance-based thresholds for object identification in the first embodiment of the present invention. [Figure 3] This flowchart shows an example of the procedure for object identification processing by the object identification device according to the first embodiment of the present invention. [Figure 4] This figure shows an example of an image input to an object recognition device. [Figure 5] This figure shows an example of detecting an object region from an image input to an object recognition device according to the first embodiment of the present invention. [Figure 6] This figure shows an example of distance, score, threshold, and identification result at the time the object identification process in the first embodiment of the present invention is completed. [Figure 7] This figure shows distance-based thresholds, scores, and examples of identification results when the object identification process according to the first embodiment of the present invention is applied to multiple objects. [Figure 8] This figure shows examples of scores and identification results when using a fixed threshold regardless of distance, as in the conventional method. [Figure 9] This is a block diagram showing an example configuration of an object identification device according to a second embodiment of the present invention. [Figure 10] This figure shows an example of the configuration of a database of targets to be identified, which has been acquired in advance, according to a second embodiment of the present invention. [Figure 11] This figure shows an example of plotting the relationship between distance and score for each element of the database to be identified in the object identification device according to the second embodiment of the present invention. [Figure 12]In the second embodiment of the present invention, it is a diagram showing a threshold value obtained from a distance, a result of identification by the threshold value, and a result of filling in the correctness of the identification result for each element of the identification target database. [Figure 13] It is a block diagram showing a configuration example of an object identification device according to the third embodiment of the present invention. [Figure 14] It is a diagram showing an example of a distance-based threshold value, a score, and an identification result for each braking distance when the object identification process according to the third embodiment of the present invention is applied to a plurality of objects. [Figure 15] It is a diagram showing an example of the relationship between the distance to the target and the threshold value in a modified example of the third embodiment of the present invention. [Figure 16] It is a block diagram showing a configuration example of an object identification device according to the fourth embodiment of the present invention. [Figure 17] It is a block diagram showing a hardware configuration example of the object identification device according to the first to fourth embodiments of the present invention. [Embodiments for Carrying Out the Invention]

[0011] Hereinafter, examples of embodiments for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. In this specification and the accompanying drawings, the same reference numerals are given to the same components or components having substantially the same functions, and redundant descriptions are omitted.

[0012] <First Embodiment> First, the configuration of the object identification device according to the first embodiment of the present invention will be described with reference to FIG. 1.

[0013] [Configuration of Object Identification Device] FIG. 1 is a block diagram showing a configuration example of the object identification device according to the first embodiment. As shown in FIG. 1, the object identification device 2 includes an object region detection unit 21, an object distance calculation unit 22, an identification score calculation unit 23, a distance-based identification method storage unit 24, a distance-based identification method selection unit 25, and an object identification unit 26.

[0014] The object region detection unit 21 detects the region in which an object is found (object region) from the stereo image Im (left and right images) acquired from a stereo camera 1 (an example of an external information acquisition unit) having a pair of cameras mounted on a moving object such as a vehicle. This is a process of detecting an object region consisting of a group of pixels in an image, and various methods can be used. For example, when a stereo camera is used as the camera that acquires the image, the distance between pixels with similar features on the left and right images can be determined by parallax. The object region can be determined by grouping adjacent pixels (regions) that are close in distance on the image.

[0015] The object distance calculation unit 22 (an example of a distance calculation unit) calculates the distance from the stereo camera 1 to the object detected by the object region detection unit 21 (hereinafter referred to as "object distance"). When a stereo camera is used, the object distance can be determined from the disparity values ​​of the same object region in two images detected by the object region detection unit 21. In this embodiment, the calculated value of the distance from the stereo camera 1 to the object is used as the distance from a moving object such as a vehicle, but the mounting position of the stereo camera 1 on the moving object may be reflected in the calculated value to determine the distance from the moving object.

[0016] Generally, the distance to an object within the camera's field of view is measured using the principle of triangulation, based on stereo images captured by a stereo camera. The principle of triangulation calculates the distance from the camera to the object using the positional difference (parallax) between the images of the same object captured by the left and right cameras. Parallax is derived by identifying where the image of the object in one image corresponds to in the other image.

[0017] Various methods have been proposed for deriving disparity. For example, a classical method known as block matching searches for the image region with the lowest dissimilarity in one image for an image region consisting of multiple pixels in one image.

[0018] The identification score calculation unit 23 (an example of an identification score calculation unit) has a classifier 231, which performs identification processing on object regions detected by the object region detection unit 21 and calculates a score (sometimes referred to as the "identification score") that represents the likelihood that the object region is an object to be identified. This score can be said to be an indicator of the likelihood that the object region is an object to be identified. For example, the classifier 231 is a machine learning model that has learned the shape and texture features of objects obtained from the edges of training images, and during inference, it outputs a score that represents the likelihood that the object region of the input image is an object to be identified.

[0019] The distance-based identification method storage unit 24 is a storage means that stores threshold scores for determining "identification" based on the distance to the object. The distance-based thresholds stored in the distance-based identification method storage unit 24 will be explained with reference to Figure 2.

[0020] [Thresholds based on distance] Figure 2 shows an example of distance-based thresholds for object identification in this embodiment. In Figure 2, the horizontal axis represents the distance to the object (m), and the vertical axis represents the threshold. As is clear from Figure 2, the closer the distance from the stereo camera 1 to the object, the higher the threshold value, and the farther the distance to the object, the lower the threshold value. By setting such object identification thresholds, it is possible to suppress misidentification, where objects other than pedestrians near the vehicle are identified as pedestrians, and to identify even distant pedestrians with low scores as pedestrians. The effects of adopting these distance-based thresholds will be clarified in subsequent embodiments. Note that although Figure 2 shows an example of distance-based thresholds for pedestrian identification, the objects to be identified are not limited to this example. For example, various objects such as obstacles such as structures and small animals can be identified, and distance-based thresholds can be set for each object to be identified.

[0021] The distance-based identification method selection unit 25 obtains a threshold value corresponding to the distance to the object calculated by the object distance calculation unit 22 from the distance-based identification method storage unit 24.

[0022] The object identification unit 26 (an example of an object identification unit) compares the score calculated by the identification score calculation unit 23 with distance-specific thresholds obtained by the distance-specific identification method selection unit 25. If the score exceeds the threshold, the object of interest is identified as an object to be identified. The identification result Ob from the object identification unit 26 is output to the vehicle control device 3, which controls the operation of a moving body such as a vehicle on which the object identification device 2 is installed. For example, the vehicle control device 3 may be an electronic control unit (ECU) that controls the automatic braking of the moving body or an electronic control unit (ECU) that controls the steering of the moving body.

[0023] [Procedure for object identification processing] Next, the flow of object identification processing by object identification device 2 will be explained using Figure 3. Figure 3 is a flowchart showing an example of the object recognition process performed by the object recognition device 2. In this embodiment, an example of identifying pedestrians as the object to be identified is shown. A stereo camera is used as a means of acquiring images of the area around the vehicle, and it is assumed that the distance corresponding to the position of an object on the acquired stereo image Im can be obtained.

[0024] The object recognition device 2 starts the object recognition process when the recognition process execution timing (current time) arrives. First, in step S21, the object region detection unit 21 receives an image acquired by the image input unit (not shown) of the object recognition device 2.

[0025] Figure 4 shows an example of image 31 input to object recognition device 2. This image 31 can be considered as one of the left and right images output by stereo camera 1 and input to object recognition device 2. Image 31 shows a pedestrian and trees on the right side of the road, a bus stop on the left side of the road, and a crosswalk and two pedestrians in the distance ahead of the vehicle.

[0026] Next, in step S22, the object region detection unit 21 detects an object region from the image 31 acquired in step S21.

[0027] Figure 5 shows an example of object region detection from image 31 input to object recognition device 2. Here, object region detection unit 21 detects an object like a pedestrian indicated by reference numeral 41 as object No. 0, an object like a bus stop indicated by reference numeral 42 as object No. 1, an object like a tree indicated by reference numeral 43 as object No. 2, an object like a pedestrian indicated by reference numeral 44 as object No. 3, and an object like a pedestrian indicated by reference numeral 45 as object No. 4.

[0028] Next, in step S23, the object region detection unit 21 initializes the reference counter i to 0 (i=0) for sequentially identifying the multiple detected objects (object regions). The object (object region) of reference counter i is denoted as "object i".

[0029] Next, in step S24, the object distance calculation unit 22 acquires distance information for object i. The object distance calculation unit 22 acquires the distance of object i to be processed from the parallax of the stereo image Im obtained by the stereo camera 1. Assume that object No. 0, indicated by reference numeral 41 in Figure 5, is at a distance of "30m" from the stereo camera 1 (vehicle).

[0030] Next, in step S25, the identification score calculation unit 23 calculates the identification score of the object i to be processed. In this example, the degree to which an object is likely to be a pedestrian is indicated by an index from 0 to 1, with a value closer to 1 indicating a higher likelihood of it being a pedestrian. Here, for example, the score of object No. 0, shown by reference numeral 41, is assumed to be "0.98".

[0031] Next, in step S26, the distance-based identification method selection unit 25 obtains a threshold value corresponding to the distance of object i obtained in step S24 from the distance-based identification method storage unit 24 (Figure 2). Here, the threshold value corresponding to the distance of object No. 0, "20m", is set to "0.89".

[0032] Next, in step S27, the object identification unit 26 determines whether the score of object i exceeds the threshold. In the case of object No. 0, the score calculated in step S25 is "0.98", which exceeds the threshold of "0.89". If the score of object i exceeds the threshold (YES determination in step S27), the process proceeds to step S28; if the score of object i does not exceed the threshold (NO determination in step S27), the process proceeds to step S29.

[0033] In step S28, the object identification unit 26 determines that object i is a pedestrian. On the other hand, in step S29, the object identification unit 26 determines that object i is not a pedestrian. For example, if the score of object No. 2 shown by reference numeral 43 is "0.80" and the distance is "40m", the threshold corresponding to the distance "40m" is "0.83" (Figure 2), so the type of object i is determined to be not a pedestrian.

[0034] After processing in step S28 or S29, in step S30, the object region detection unit 21 increments the object reference counter i by 1 (i=i+1) and proceeds to the identification process for the next object i.

[0035] Next, in step S31, the object region detection unit 21 determines whether the reference counter i is equal to the number of object regions detected in step S22. If the reference counter i is not equal to the number of object regions detected in step S22 (a NO determination in step S31), the object region detection unit 21 returns to the process in step S24, stating that the identification process has not been completed for all objects.

[0036] On the other hand, if the reference counter i becomes equal to the number of object regions detected in step S22 (YES determination in step S31), the object region detection unit 21 considers that the identification process for all objects at the current time has been completed and moves on to processing at the next time.

[0037] Following the procedure described above, the score and distance of the object detected in the image acquired from stereo camera 1 are calculated, and object identification processing is performed.

[0038] Here, for the objects indicated by symbols 41, 42, 43, 44, and 45 in Figure 5, Figure 6 shows examples of the distance, score, threshold, and identification result at the time the object identification process in Figure 3 is completed.

[0039] Figure 6 shows an example of distance, score, threshold, and identification result at the time the object identification process according to this embodiment is completed. The table shown in Figure 6 has a field 51 for "Object No.", a field 52 for "Object Code", a field 53 for "Object Type", a field 54 for "Distance", a field 55 for "Score", a field 56 for "Threshold", a field 57 for "Identification Result", and a field 58 for "Correctness of Identification Result".

[0040] "Object No." is, for example, information that uniquely identifies a record within this table. An "object code" is information such as a code assigned to each object (object region) in an image, linked to an object number, in order to identify the object (object region) in the image. This object code is included to explain the object in the image in Figure 5, but it is not always necessary. "Object type" is information that represents the actual type of object in the image. "Distance" is information representing the distance to the object obtained in step S24. The "score" is information representing the score used to identify the object, calculated in step S25. The "threshold" is information representing a threshold value based on the distance to the object obtained in step S26. The "identification result" is information representing the identification result determined in step S27. "Correctness of Identification Result" is information indicating whether the identification result was correct or incorrect, based on a comparison of the actual "object type" in field 53 with the "identification result" in field 57, for explanatory purposes.

[0041] Here, we will explain each of the five objects shown in Figure 6, from object No. 0 (reference numeral 41) to object No. 4 (reference numeral 45).

[0042] The object No. 0, code 41, shown in the first row, is actually a "pedestrian" and is located at a distance of "20m". Because the contour of the object's region is clear, a high identification score of "0.98" is obtained. The threshold corresponding to "20m" is obtained from the distance-specific thresholds in Figure 2, which is "0.89". In step S27, the score exceeds the threshold, so the identification result is "pedestrian". Since the object, whose correct answer is a pedestrian, is identified as a "pedestrian", the identification result is "correct" (○ in Figure 6). This is an example where a high score is obtained as an identification result and the identification judgment based on the threshold is correct because the object is at close range, the object region can be accurately acquired, and the contour is clear.

[0043] The object No. 1, code 42, shown in the second row, is actually a "non-pedestrian" (bus stop), and although the object's outline is clear because it is at a distance of "30m", the score is high at "0.88" because bus stops are quite similar in shape to pedestrians. The threshold for a distance of "30m" is obtained from the distance-specific thresholds in Figure 2, which is "0.86". In step S27, the score exceeds the threshold, so the identification result is "pedestrian". Since the object, which is actually a non-pedestrian, is identified as a "pedestrian", the identification result is "incorrect" (× in Figure 6). This is an example where an object is at close range, its object area is accurately acquired, and its outline is clear, but because its shape is very similar to a pedestrian, a high score is obtained as an identification result, and the threshold judgment leads to misidentification as a pedestrian.

[0044] The object No. 2, code 43, shown in the third row, is actually a "non-pedestrian" (tree) and is at a distance of "40m," so the outline of the object region is somewhat clear. Because its shape is somewhat similar to a pedestrian, the score is a slightly high value of "0.80." When the threshold for a distance of "40m" is obtained from the distance-specific thresholds in Figure 2, it is "0.83," and in the processing of step S27, the score does not exceed the threshold, so the identification result is "non-pedestrian." Since the object, whose correct answer is non-pedestrian, is identified as "non-pedestrian," the identification result is "correct." This is an example where, although the object's shape is similar to a pedestrian because it is at close range, the object region can be accurately obtained, and the outline is clear, setting a high threshold for close range resulted in the reliability as a pedestrian not meeting the threshold, thus avoiding misidentification as a pedestrian.

[0045] The object No. 3, code 44, shown in the fourth row, is actually a "pedestrian" and is located at a distance of "70m," making the contour of the object's region less clear compared to nearby objects. In this example, the detected object region, indicated by code 44 in Figure 5, does not extract the entire area corresponding to the object, and the head portion of the object is excluded from the object region. When detecting distant objects, the low resolution of the object region can make it difficult to accurately detect the entire object region, as shown in this example. The classifier 231 in the identification score calculation unit 23 learns to capture features such as the contour of the pedestrian's head, so if part of the pedestrian's body is not included in the object region, the identification score for pedestrian decreases. For these reasons, the score is a relatively low value of "0.78." When the threshold for a distance of "70m" is obtained from the distance-specific thresholds in Figure 2, it is "0.74," and since the score exceeds the threshold in step S27, the identification result is "pedestrian." The system identifies the object, which is a pedestrian, as a "pedestrian," resulting in a "correct" identification. This is an example where, although the object is at a distance and its region cannot be accurately captured, resulting in a lower score, the system can still identify it as a pedestrian because the distance threshold is set low for distant objects.

[0046] The object No. 4, code 45, shown in the 5th row, is actually a "pedestrian" and is located at a distance of "80m," making its contour less clear compared to when it is nearby. In this example, the resolution of the object region is low and the contour of the object region is unclear, resulting in a low score of "0.75." The threshold for a distance of "80m" is obtained from the distance-specific thresholds in Figure 2, which is "0.71." Since the score exceeds the threshold in step S27, the identification result is "pedestrian." Because the object, whose correct answer is a pedestrian, is identified as a "pedestrian," the identification result is "correct." This is an example where, although the object is at a distance and its contour is unclear, resulting in a lower score, the distance-specific thresholds are set low for distant objects, allowing for correct identification as a pedestrian.

[0047] [Example of score and identification result according to this embodiment] Figure 7 illustrates an example of the score and identification result obtained by applying the object recognition processing by the object recognition device 2 according to this embodiment to the five objects mentioned above.

[0048] Figure 7 shows an example of distance-based thresholds, scores, and identification results when object identification processing by the object identification device 2 is applied to multiple objects. In Figure 7, the horizontal axis represents the distance to the object (m), and the vertical axis represents the threshold. In the figure, symbols 41, 42, 43, 44, and 45 represent the distance and score to the object regions indicated by the same symbols on image 31 in Figure 5. In Figure 7, the distance-based threshold 71 is a function that shows the threshold for the distance to the object (an example of a distance-based threshold function), and is the same as the distance-based threshold shown in Figure 2. Markers indicated by "▲" indicate the score of object regions where the correct answer is "pedestrian". Markers indicated by "●" indicate the score of object regions where the correct answer is "other than pedestrian".

[0049] A 100% correct state is achieved when, for all of the symbols 41, 44, and 45 whose correct value is "pedestrian," the score exceeds the distance-specific threshold of 71, and for symbols 42 and 43 whose correct value is "not a pedestrian," the score does not exceed the distance-specific threshold of 71. However, in the example in Figure 7, one marker, indicated by symbol 42, whose correct value is "not a pedestrian," is misidentified as a pedestrian. However, by adopting variable thresholds that are higher for short distances and lower for long distances, it is possible to correctly identify 80% of the object regions.

[0050] [Examples of scores and identification results when conventional technology is applied] For comparison with this embodiment, an example of the score and identification result when a constant threshold is used regardless of distance will be explained using Figure 8.

[0051] Figure 8 shows an example of the score and identification result when a constant (fixed) threshold value is used regardless of distance. In Figure 8, the horizontal axis represents the distance to the object (m), and the vertical axis represents the threshold value. In the figure, symbols 41, 42, 43, 44, and 45 represent the distance and score to the object regions indicated by the same symbols on image 31 in Figure 5. In Figure 8, as in Figure 7, the markers indicated by "▲" indicate the score of object regions where the correct value is "pedestrian". The markers indicated by "●" indicate the score of object regions where the correct value is "other than pedestrian". Here, we will explain three examples: when threshold 81 is set to a constant value of "0.9", when threshold 82 is set to a constant value of "0.79", and when threshold 83 is set to a constant value of "0.74".

[0052] If the threshold is set to a high value, such as threshold 81, then the score of code 41, which is a pedestrian, exceeds the threshold and is identified as a "pedestrian," making it a correct answer. Codes 42 and 43, which are not pedestrians, do not have scores that exceed the threshold and are identified as "not pedestrians," making these two also correct answers. On the other hand, codes 44 and 45, which are pedestrians at a distance, do not have scores that exceed the threshold and are identified as "not pedestrians," making them incorrect answers. Thus, setting a certain threshold to a high value works well for distinguishing between "pedestrians" and "non-pedestrians" in the vicinity of the vehicle. However, for pedestrians at a distance, where the score tends to be low even if they are pedestrians, a high threshold will cause them to be judged as not pedestrians, making identification impossible and thus not working well. As a result, the accuracy rate for the above five objects in the conventional technology is 60%, which is lower than in this embodiment.

[0053] Furthermore, if the threshold is set to a moderate value of "0.78," as in threshold 82, the score for code 41, which is correctly identified as a pedestrian, exceeds the threshold and is identified as a "pedestrian," thus being correct. Codes 42 and 43, which are not pedestrians, also have scores above the threshold and are therefore judged as "pedestrians," resulting in these two being misidentified. Additionally, codes 44 and 45, which are pedestrians at a distance, do not have scores above the threshold and are therefore identified as "not pedestrians," resulting in misidentification. Thus, setting a certain threshold to an intermediate value results in identification results that do not work well regardless of whether the object is at close or far distance. As a result, the accuracy rate for the 5 objects is 20%.

[0054] Furthermore, when the threshold is set to a relatively low value of "0.74," as in threshold 83, the scores for all of the correct objects, 41, 44, and 45 (which are pedestrians), exceed the threshold and are identified as "pedestrians." On the other hand, the scores for objects other than pedestrians, 42 and 43, also exceed the threshold and are misidentified as "pedestrians." Thus, setting a low threshold has the effect of correctly identifying object regions corresponding to pedestrians with low scores at a distance, but it also has the problem of identifying object regions other than pedestrians with high scores at close range as pedestrians. As a result, the accuracy rate for 5 objects is 60%.

[0055] To minimize misidentification and correctly identify pedestrian object regions where the score for long distances is low, it is effective to perform identification determination using thresholds that are set high for close distances and low for long distances, as shown in Figure 7. The above describes the configuration and operation of the object identification device 2 according to the first embodiment of the present invention.

[0056] As described above, the object identification device (object identification device 2) according to the first embodiment includes: a distance calculation unit (object distance calculation unit 22) that receives external information (e.g., stereo image Im) including an external object (object region on the image) acquired by an external information acquisition unit (e.g., stereo camera 1) mounted on the vehicle and calculates the distance from the vehicle to the object; an identification score calculation unit (identification score calculation unit 23) that uses the external information to calculate an identification score indicating the confidence that the object included in the external information is of a predetermined type; and an object identification unit (object identification unit 26) that identifies the object as being of the type associated with the identification score when the identification score exceeds a predetermined threshold. Here, the threshold value differs depending on the distance from the vehicle to the object (Figure 2).

[0057] According to this embodiment with the above configuration, an appropriate object identification threshold is set that matches the actual situation at different distances. As a result, misidentification can be suppressed and the performance (accuracy) of object identification can be improved, regardless of whether the distance from a moving object such as a vehicle to the object is near or far.

[0058] <Second Embodiment> Next, as a second embodiment of the present invention, an example will be described in which the object identification device 2 (Figure 1) according to the first embodiment is provided with a function to create distance-based threshold functions.

[0059] [Configuration of object identification device] Figure 9 is a block diagram showing an example configuration of an object identification device according to the second embodiment. The object identification device 2A shown in Figure 9 has a configuration that adds a distance-based identification method creation unit 91 and an identification target database (DB) 92 to the object identification device 2 according to the first embodiment shown in Figure 1.

[0060] The added component, the distance-based identification method creation unit 91, will be explained further. The distance-based identification method creation unit 91 is a means for calculating distance-based threshold functions to be stored in the distance-based identification method storage unit 24. The calculated distance-based threshold functions are stored in the distance-based identification method storage unit 24 as distance-based identification methods.

[0061] The distance-based identification method selection unit 25 inputs the distance to the object on the image calculated by the object distance calculation unit 22 into the distance-based identification method storage unit 24 to obtain a threshold value. Then, the object identification unit 26 compares the score calculated by the identification score calculation unit 23 with the threshold value obtained by the distance-based identification method selection unit 25 to identify the object on the image.

[0062] As described in the first embodiment, the distance-based threshold function works as follows: when the distance from the vehicle to an object is short, the contour of the object region on the input image is clearly detected, resulting in a high score for objects that are correctly identified as pedestrians, as well as high scores for objects other than pedestrians that have a similar shape to pedestrians. Therefore, the threshold is set high to suppress misidentification. On the other hand, at long distances, the score tends to be low due to reasons such as the contour of the object region becoming unclear and the detection area not properly extracting the object region. Therefore, the threshold is set low.

[0063] The purpose of setting distance-based thresholds is to ensure accurate identification of pedestrians at both short and long distances while suppressing misidentification. Therefore, the distance-based threshold function should be a downward-sloping function where the threshold decreases as the distance increases, and any function that appropriately achieves the objective can be used. Here, we will explain the relationship between distance and threshold using a linear function. We will explain using an example of distinguishing between pedestrians and other objects as the objects to be identified. To set appropriate thresholds for each distance, we will use data from the identification target DB92 that has been acquired in advance.

[0064] Figure 10 shows an example of the configuration of the identification target DB 92 that was acquired in advance. Figure 10 shows an example of the fields (items) of the identification target database 92. The identification target DB 92 has an "ID" field 101, an "Object Type" field 102, a "Distance" field 103, and a "Score" field 104.

[0065] "ID" is information that indicates an identifier used to identify the image (object) of the object region to be identified. If the correct answer is a pedestrian, it is a sequential number with the code p added to the beginning of the identifier. If the correct answer is something other than a pedestrian, it is a sequential number with the code n added to the beginning of the identifier. Overall, there are Np objects to be identified where the correct answer is a pedestrian, and Nn objects to be identified where the correct answer is something other than a pedestrian. "Object type" is information that represents the type of object being identified. "Distance" is information that represents the distance to the object being identified. The "score" is information that represents the score of the object being identified.

[0066] These data were obtained by actually performing object identification processing according to the flowchart shown in Figure 3, and identifying pedestrians and other objects at various distances, as shown in Figure 4 as an example. These data may be fixed values ​​or values ​​that are updated by performing object identification processing as needed. Furthermore, the identification target DB92 may be located outside the object identification device 2A and inside the vehicle, or data corresponding to the identification target DB92 may be acquired from outside the vehicle via a wide-area network (not shown).

[0067] Figure 11 shows an example of plotting the relationship between distance and score for each element of the identification target DB92 shown in Figure 10, in the object identification device 2A. For simplicity, the relationship between distance and score is shown as a score distribution every 10m. In the figure, the score distributions 111, 113, 115, 117, 11a, 11c, 11e, and 11g, shown as solid lines, are the score distributions for objects whose object type is "pedestrian". The score distributions 112, 114, 116, 118, 11b, 11d, 11f, and 11h, shown as dashed lines, are the score distributions for objects whose object type is "other than pedestrian".

[0068] When comparing the score distributions of two object types at the same distance, it is expected that pedestrians will have higher scores and non-pedestrians will have lower scores. Furthermore, as described in the first embodiment, as the distance to the target increases, the score tends to decrease relatively due to reasons such as the outline of the object region on the image becoming unclear and the detection region failing to properly extract the target region (object region).

[0069] For distance and score identification targets with score distributions 111 to 11h, setting a variable threshold according to distance, such as threshold 110, allows us to determine the threshold corresponding to the "distance" field 103 for each element of the identification target DB92 shown in Figure 10. The variable threshold, assuming linearity, can be defined by the following equation (1). Note that the coefficient a is a negative value in this embodiment.

[0070] Threshold = distance × a + b ····(1)

[0071] Figure 12 shows the results for each element of the DB92 to be identified shown in Figure 10, including the threshold value obtained from the distance, the result of identification based on the threshold value, and the correctness of the identification result. In the table shown in Figure 12, in addition to fields 101 to 104, there is a field 121 for "threshold value", a field 122 for "identification result", and a field 123 for "correctness of identification result".

[0072] A "threshold" is information that represents a threshold value corresponding to the distance to an object. The "identification result" is information that represents the result of identification based on the "score" and "threshold." "Correctness of Identification Result" is information indicating the correctness of the "Identification Result" by comparing the actual "Object Type" in Field 102 with the "Identification Result" in Field 122.

[0073] Once this is determined, the correct and incorrect classification rates when the threshold value 110 shown in Figure 11 is applied to this dataset can be calculated using equations (2) and (3) below. The correct classification rate is the rate at which an object type that is correctly identified as a pedestrian is correctly identified as a pedestrian. The incorrect classification rate is the rate at which an object type that is not a pedestrian is correctly identified as a pedestrian.

[0074] Correct identification rate = (Number of objects of the pedestrian type that are correctly identified) ÷ Np ···(2) Misidentification rate = (Number of cases where the identification result was incorrect among categories other than pedestrians) ÷ Nn ... (3)

[0075] In equation (1), by determining the coefficients a and b that result in the lowest misidentification rate while maintaining a certain level of accuracy, it is possible to determine a threshold that minimizes misidentification while ensuring accuracy using the identification target DB92 in Figure 10. By including as much information as possible about potential identification targets in the identification target DB92 in Figure 10, distance-based thresholds that match actual vehicle usage can be calculated. The above describes the configuration and operation of the object identification device 2A according to the second embodiment of the present invention.

[0076] Thus, in the object identification device (object identification device 2A) according to this embodiment, a threshold setting function (for example, threshold 110) is used as the distance-based threshold, which ensures a certain level of accuracy and minimizes misidentification when used with respect to an image database (identification target DB92) in which information on objects with known correct values ​​is stored by distance. The object identification unit (object identification unit 26) calculates the threshold by inputting the distance to the object into the threshold setting function.

[0077] Furthermore, while this embodiment describes an example in which the relationship between the distance to the target and the threshold is expressed using a linear function, it is not necessarily limited to this, and other relationships between the distance to the target and the threshold can also be used. For example, the relationship between the distance to the target and the threshold may be expressed by a nonlinear function that satisfies the threshold conditions in the above embodiment, instead of a linear function. Alternatively, the relationship between the distance to the target and the threshold may be defined in the form of table or map data.

[0078] <Third Embodiment> Next, as a third embodiment of the present invention, an example will be described in which the object identification device 2 (Figure 1) according to the first embodiment is equipped with a function to determine a threshold value by taking into account the braking distance in addition to the distance from a moving object such as a vehicle to the object.

[0079] [Configuration of object identification device] Figure 13 is a block diagram showing an example configuration of an object identification device according to the third embodiment. The object identification device 2B shown in Figure 13 is configured in a way that, compared to the object identification device 2 according to the first embodiment shown in Figure 1, a speed detection unit 131 and a road surface condition detection unit 132 are added, the distance-based identification method storage unit 24 is replaced with a braking distance / distance-based identification method storage unit 133, and the distance-based identification method selection unit 25 is replaced with a braking distance / distance-based identification method selection unit 134. In other words, the difference between this embodiment and the first embodiment is that the braking distance has a function of the distance to the target and a threshold value.

[0080] The primary application of the present invention is to mount it on a moving object such as a vehicle to identify an object that poses a risk of collision ahead and to apply the brakes automatically. In other words, the present invention is suitable for vehicle control for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD). The braking distance varies depending on the vehicle's speed and road conditions. What is important for activating the automatic brake is to reliably identify an object that is within the range of the braking distance that can be calculated according to the speed and road conditions at that time. Therefore, the braking distance is calculated based on the vehicle's speed and road conditions acquired by the speed detection unit 131 and the road condition detection unit 132. Then, distance-specific thresholds are calculated that minimize the misidentification rate while ensuring a correct identification rate within that braking distance range, and these are stored in the braking distance / distance-specific identification method storage unit 133.

[0081] The speed detection unit 131 can be configured, for example, using a vehicle speed sensor mounted on the vehicle. The speed detection unit 131 outputs the detected vehicle speed information to the braking distance / distance-based identification method selection unit 134.

[0082] The road surface condition detection unit 132 determines the road surface conditions from images of the area in front of the vehicle, for example, input from the stereo camera 1. For example, information on road surface conditions may include unevenness, bumps, wet conditions, and icy conditions. The road surface condition detection unit 132 outputs the detected road surface condition information to the braking distance / distance-based identification method selection unit 134.

[0083] The braking distance / distance-based identification method storage unit 133 stores distance-based threshold values ​​for different braking distances, as shown in Figure 14. The contents of Figure 14 will be described later.

[0084] The braking distance / distance-based identification method selection unit 134 calculates the vehicle's braking distance based on the vehicle's speed detected by the speed detection unit 131 and the road surface conditions detected by the road surface condition detection unit 132. The braking distance is also affected by the vehicle's weight and the performance and specifications of the automatic brakes, but this information can be prepared in advance and stored in the object identification device. For example, the braking distance can be calculated using a machine learning model (not shown) that has been trained to output the braking distance with the vehicle's speed and road surface conditions as input. The parameters of the machine learning model are pre-configured to reflect other information such as the vehicle's weight and the performance and specifications of the automatic brakes. The braking distance / distance-based identification method selection unit 134 uses this machine learning model to infer the braking distance from the vehicle's speed and road surface conditions. Then, the braking distance / distance-based identification method selection unit 134 obtains a threshold value corresponding to the inferred braking distance from the braking distance / distance-based identification method storage unit 133 and outputs it to the object identification unit 26.

[0085] Here, we will explain the method of identifying objects based on their distance from the target, using Figure 14 as a reference.

[0086] Figure 14 shows an example of distance-specific thresholds, scores, and identification results for each braking distance when the object identification process according to this embodiment is applied to multiple objects. In Figure 14, the horizontal axis represents the distance to the object (m), and the vertical axis represents the threshold. In the figure, threshold 141, shown as a solid line (linear function), is the distance-specific threshold for the target when the braking distance is "50m". Threshold 142, shown as a dashed line (linear function), is the distance-specific threshold for the target when the braking distance is "90m". Reference numerals 41, 42, 43, 44, and 45 indicate the distance and score to the object region shown with the same reference numerals on image 31 in Figure 5.

[0087] If the vehicle's speed is above a certain speed, and the braking distance is long, with a threshold of 90m (142), then a threshold with a larger negative slope is set so that the long-distance threshold becomes lower in order to identify pedestrians at a distance.

[0088] In contrast, when the vehicle's speed is low and the braking distance is 50m, with a threshold of 141, it is possible to prioritize identifying pedestrians closer than 50m and suppressing misidentification of non-pedestrians, rather than correctly identifying pedestrians further than 50m. Therefore, the negative slope of the threshold can be reduced.

[0089] As a result, if a threshold of 141 is adopted for a braking distance of "50m", the pedestrian object indicated by code 41, which is located more than 50m away, is correctly identified. Also, the score of the non-pedestrian bus stop indicated by code 42 at 30m is below the threshold, so it can be correctly identified as a non-pedestrian object. Similarly, the non-pedestrian tree indicated by code 43 at a distance of 40m also does not exceed the threshold in score, so it is correctly identified as a non-pedestrian object.

[0090] As described above, in this embodiment, distance-based thresholds (thresholds 141 and 142 in Figure 14) are set according to the distance to the object and the braking distance. According to this embodiment, in addition to the same effects as in the first embodiment, the following effects can be obtained. That is, according to this embodiment, by changing the distance-based thresholds according to the braking distance and narrowing the application range of the automatic brake, it is possible to perform object identification with higher accuracy within the required range.

[0091] Furthermore, to represent the relationship between the distance to the target and the threshold for each braking distance, you may prepare tables or map data that register the relationship between the distance to the target and the threshold for each braking distance.

[0092] Furthermore, in this embodiment, the threshold for object identification was set based on the distance to the object and the braking distance, but this is not the only example. For example, since the braking distance is greatly affected by the speed of the moving object, the threshold could be set according to the distance and speed.

[0093] [Modified example of the third embodiment] Furthermore, in the above embodiment, the distance-based threshold function (threshold setting function) was a linear function, but the distance-based threshold function may also be defined by a combination of a fixed threshold and a linear variable threshold. Below, an example of defining the distance-based threshold function by a combination of a fixed threshold and a linear variable threshold will be explained with reference to Figure 15.

[0094] Figure 15 shows an example of the relationship between the distance to the object and the threshold in a modified version of the third embodiment. The threshold 151 shown in the upper part of Figure 15 is an example of a method in which a fixed threshold is used when the distance to the object is short, and a linear variable threshold is used when the distance is long. This is useful when you want to strengthen the suppression of misidentification when the distance to the object is short. It is also effective for object identification when the braking distance is short.

[0095] The threshold value 152 shown in the middle of Figure 15 is an example of a method that uses a linear variable threshold when the distance to the object is short, and a fixed threshold when the distance is long. This is useful when you want to enhance the suppression of misidentification even when the distance to the object is long. It is also effective for object identification when the braking distance is long. Note that the long-distance values ​​of different distance-based threshold functions may be the same; for example, the fixed value of threshold value 152 at long distances in the middle of Figure 15 and the fixed value of threshold value 153 at long distances in the lower part of Figure 15 may be the same.

[0096] The threshold value 153 shown in the lower part of Figure 15 is an example of a method that uses a high fixed threshold for short distances, a linear variable threshold for medium distances, and a low fixed threshold for long distances. This is a combination of threshold values ​​141 and 142 and is effective when the braking distance is of a medium magnitude. Note that the short distances of different distance-specific threshold functions, i.e., the fixed value of threshold value 151 at short distances in the upper part of Figure 15 and the fixed value of threshold value 153 at short distances in the lower part of Figure 15 may be the same.

[0097] Furthermore, a distance to an object is considered a short distance when it is closer than the first distance, a distance is considered a long distance when it is farther than the second distance (first distance < second distance), and a distance is considered a moderate distance when it is between the first and second distances.

[0098] As described above, in the object identification devices (object identification devices 2 to 2B) according to the first to third embodiments, the distance threshold is set to be a larger value when the distance to the object is shorter than the second distance (close distance) than when the distance to the object is longer than the second distance (long distance). For example, the distance threshold used for object identification by the distance threshold function is a linear variable value (first to third embodiments) or a combination of a fixed value and a linear variable value (modified version of the third embodiment). The distance threshold may be set to different values ​​in stages from long distance to short distance. For example, the distance from the moving body to the object can be divided into three stages: long distance, medium distance, and short distance, and the distance threshold can be set to a small value for long distance, a medium value for medium distance, and a large value for short distance. The distance may also be divided into four or more stages instead of three. In this case, the load on determining the distance threshold (stepwise different threshold) stored in the distance identification method storage unit 24 can be reduced compared to the embodiments described above.

[0099] <Fourth Embodiment> Next, as a fourth embodiment of the present invention, an example will be described in which the object identification device 2A (Figure 9) according to the second embodiment is equipped with a function to update distance-based thresholds based on the object identification result.

[0100] Figure 16 is a block diagram showing an example configuration of an object identification device according to the fourth embodiment. The object identification device 2C shown in Figure 16 has a configuration that adds an identification condition / result output unit 161 and a distance-based identification result storage unit 162 to the object identification device 2A according to the second embodiment shown in Figure 9. In addition, the identification target DB92 of the object identification device 2A in Figure 9 is removed in the object identification device 2C.

[0101] The identification condition / result output unit 161 obtains the identification result and identification conditions from the object identification unit 26 and outputs them to the distance-based identification result storage unit 162. Here, the identification condition / result output unit 161 outputs the image of the object to be identified (object region) identified by the object identification unit 26, the distance to the object to be identified, the score, and the threshold value used for identification.

[0102] The distance-based identification result storage unit 162 stores the image of the object to be identified output by the identification condition / result output unit 161, the distance to the object to be identified, the score, and the threshold value used for identification.

[0103] The various information about the object to be identified stored in the distance-based identification result storage unit 162 can be used to confirm whether the object identification process is working properly. Furthermore, by assigning an object type to the individual information of the object to be identified stored, the distance-based identification method creation unit 91 can calculate an appropriate distance-based threshold based on the various information about the object to be identified (image of the object to be identified, distance to the object, score) stored in the distance-based identification result storage unit 162. The distance-based identification method creation unit 91 outputs the calculated distance-based threshold (for example, a distance-based threshold function) to the distance-based identification method storage unit 24 and updates the distance-based threshold stored in the distance-based identification method storage unit 24.

[0104] Here, the purpose of assigning object types is to assign the correct type to the identification target identified by the object identification unit 26. For this reason, the following methods can be used to assign object types.

[0105] (1) A person visually inspects the output image (corresponding to an image of the object region) to determine its type and assigns it accordingly. (2) The classifier (object classification unit) identifies and assigns a classification to the output image (corresponding to the image of the object region) that is to be identified. At this time, it is desirable to use a classifier whose recognition performance is equal to or higher than that of a human. For example, ResNet, described in Non-Patent Document 1 below, can handle high-dimensional features by using a deep network of 152 layers among convolutional neural networks.

[0106] (Non-patent document 1) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Deep residual learning for image recognition” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

[0107] The assignment of object types can be performed in an offline environment separate from the vehicle by storing information about the object to be identified, which is stored in the distance-based identification result storage unit 162, on an external storage medium or the like. Furthermore, if an identifier configured with a high-performance network, such as that described in Non-Patent Document 1, can be installed in the vehicle (for example, in an object identification device or other electronic control device), this can be achieved by reading the image of the object to be identified stored in the distance-based identification result storage unit 162 and performing the identification.

[0108] As described above, in this embodiment, for each individual vehicle, a dataset is created in which the correct object type is assigned to various information about the object to be identified, which is output by the identification condition / result output unit 161 and stored in the distance-based identification result storage unit 162. Distance-based thresholds can then be set for this data using the distance-based identification method creation unit 91 described in the second embodiment.

[0109] In other words, the object identification device according to this embodiment is configured to include an identification result storage unit (distance-based identification result storage unit 162) that stores an image of an object (object region) identified by an object identification unit (object identification unit 26), the distance to the object, and an identification score, and a function creation unit (distance-based identification method creation unit 91) that creates a threshold setting function (distance-based threshold function) from the information stored in the identification result storage unit.

[0110] The types of objects to be identified and the types of objects other than pedestrians that are easily misidentified vary depending on the region and environmental conditions in which they are used, the speed ranges at which they are most frequently used, etc. By using the object identification device according to this embodiment, it becomes possible to update variable thresholds that are suited to the driving environment of each individual vehicle as needed. For example, in the initial stages of operation of the system including the object identification device according to this embodiment described above, a general-purpose distance-based threshold is created and used. On the other hand, the actual usage of mobile objects differs from user to user, i.e., from vehicle to vehicle. According to the object identification device according to this embodiment, the distance-based threshold is updated according to the actual usage. The update frequency may be once, or it may be updated periodically or multiple times based on certain conditions.

[0111] The above is an example of an embodiment of the object identification device according to the present invention. In the above embodiment, an example was used in which pedestrians are identified as non-pedestrians, but the objects to be identified may include multiple objects, including pedestrians.

[0112] Furthermore, while we have described an example of using external information obtained from a stereo camera as a means of calculating the distance to an object in an image, the external information acquisition unit is not limited to a stereo camera. For example, a monocular camera and millimeter-wave radar may be used as the external information acquisition unit.

[0113] [Hardware configuration of object recognition device] Next, the configuration (hardware configuration) of the control systems of the object identification devices 2 to 2C according to the first to fourth embodiments described above will be explained with reference to Figure 17.

[0114] Figure 17 is a block diagram showing examples of hardware configurations of object identification devices 2 to 2C according to the first to fourth embodiments. The computer 170 shown in Figure 17 is hardware used as a so-called computer.

[0115] The computer 170 includes a CPU (Central Processing Unit) 171, ROM (Read Only Memory) 172, RAM (Random Access Memory) 173, non-volatile storage 176, and a network interface 177, all connected to bus B.

[0116] The CPU 171 reads the program code of the software that implements each function of the passenger conveyor control system 100 according to this embodiment from the ROM 172, loads it into the RAM 173, and executes it. Alternatively, the CPU 171 reads the program code directly from the ROM 172 and executes it as is. Note that the computer 170 may be equipped with a processing unit such as an MPU (Micro-Processing Unit) instead of the CPU 171. Variables and parameters that occur during the calculation processing by the CPU 171 are temporarily written to the RAM 173. The functions of each block of the object identification devices 2 to 2C are implemented by the CPU 171.

[0117] Non-volatile storage 176 can include, for example, HDDs (Hard Disk Drives), SSDs (Solid State Drives), flexible disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, and non-volatile memory cards. This non-volatile storage 176 stores the OS (Operating System), various parameters, and programs necessary for the computer 170 to function. The functions of the distance-based identification method storage unit 24, the identification target DB 92, the braking distance / distance-based identification method storage unit 133, and the braking distance / distance-based identification method selection unit 134 of the object identification devices 2-2C are realized by the non-volatile storage 176.

[0118] The program may also be stored in ROM 172. The program is stored in the form of computer-readable program code, and the CPU 171 sequentially executes operations according to the program code. In other words, ROM 172 or non-volatile storage 176 is used as an example of a computer-readable, non-transient recording medium that stores a program executed by the computer.

[0119] The network interface 177 consists of communication devices and the like that control communication between other devices. The communication functions of the object identification devices 2-2C are realized by the network interface 177.

[0120] The embodiments and various modifications described above are merely examples, and the present invention is not limited to these embodiments or modifications as long as the features of the invention are not impaired. Furthermore, the various embodiments and modifications described above are explained in detail and concretely in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those that include all the components described. Other embodiments that can be conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention.

[0121] Furthermore, some or all of the above configurations, functions, and processing units may be implemented in hardware, for example, by designing them as integrated circuits. Broadly defined processor devices such as FPGAs (Field Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits) may be used as hardware.

[0122] Furthermore, each component of the object identification device according to the above-described embodiment may be implemented on any hardware, as long as the respective hardware can send and receive information from each other via a network. Also, the processing performed by a certain processing unit may be realized by a single piece of hardware, or by distributed processing by multiple pieces of hardware.

[0123] Furthermore, in this specification, processing steps describing chronological processing include not only processing performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is executed in parallel or individually (for example, processing by objects). In addition, the processing order of processing steps describing chronological processing may be changed to the extent that it does not affect the processing result.

[0124] Furthermore, in the embodiments described above, the control lines and information lines indicated by arrows and solid lines are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In practice, it can be assumed that almost all components are interconnected. [Explanation of Symbols]

[0125] 1…Stereo camera, 2~2C…Object identification device, 22…Object distance calculation unit, 23…Identification score calculation unit, 26…Object identification unit

Claims

1. External information, including objects outside the vehicle, acquired by an external information acquisition unit mounted on the vehicle is input, and a distance calculation unit calculates the distance from the vehicle to the object. An identification score calculation unit that uses the external information to determine an identification score indicating the confidence that the object included in the external information is of a predetermined type, The system includes an object identification unit that identifies an object as belonging to a type associated with an identification score when the identification score exceeds a threshold value set in advance according to the distance to the object and the braking distance. Object identification device.

2. The threshold value is set to be larger when the distance to the object is shorter than the second distance (a first distance) than when the distance to the object is greater than the second distance. The object identification device according to claim 1.

3. As the threshold, a threshold setting function is used for an image database in which information on objects with known correct values ​​is stored by distance, such that the correct identification rate is above a certain value and the misidentification rate is minimized. The object identification unit calculates the threshold value by inputting the distance to the object into the threshold setting function. The object identification device according to claim 2.

4. The threshold setting function is defined by a combination of a fixed threshold and a linear variable threshold. The object identification device according to claim 3.

5. An identification result storage unit stores information such as the image of the object identified by the object identification unit, the distance to the object, and the identification score. The system comprises a function creation unit that creates the threshold setting function using the information stored in the identification result storage unit. The object identification device according to claim 3.

6. An object identification method using an object identification device to which external information, including objects outside the vehicle, acquired by an external information acquisition unit mounted on the vehicle is input, A process to determine the distance from the vehicle to the object based on the external information, A process to obtain an identification score indicating the confidence that the object included in the external information is of a predetermined type, using the external information; The process includes identifying an object as belonging to the type associated with the identification score when the identification score exceeds a threshold predetermined according to the distance to the object and the braking distance. Object identification method.

Images