Object recognition device and object recognition method

Abstract

To obtain an object recognition device that calculates the reliability of an object's size, in addition to its position and size. [Solution] The object recognition device includes an object detection unit that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; and an object size reliability estimation unit that uses at least one of the object's position, size, angle, reflectivity, identification result, and the positions of the sensor units stored in advance to calculate the reliability of the object's size.

Description

Technical Field

[0001] The present disclosure relates to an object recognition device and an object recognition method.

Background Art

[0002] In the technologies of autonomous driving and accident avoidance driving of automobiles, an object recognition device is provided in one or both of an autonomous driving vehicle and a roadside unit. The object recognition device measures the position of an object around the object recognition device, the size of the object, and the like. These measured values are used in a collision damage mitigation braking system that reduces damage when an autonomous driving vehicle collides with a forward object, an adaptive cruise control system in which an autonomous driving vehicle follows a forward vehicle, an automatic parking system in which an autonomous driving vehicle automatically parks in a parking space, and a negotiation system in which an autonomous driving vehicle gives priority to other vehicles and pedestrians at an intersection. That is, the object recognition device is for avoiding accidents during the driving of an autonomous driving vehicle and improving the comfort of driving of the autonomous driving vehicle, and is used in vehicle applications.

[0003] The object recognition device is equipped with a sensor such as LiDAR that captures three-dimensional data of the environment. Therefore, the object recognition device can acquire the position, size, and the like of an object around the object recognition device. A method for determining the minimum and / or maximum height of a surrounding object based on the three-dimensional data of the surrounding environment obtained by LiDAR has been disclosed (see, for example, Patent Document 1).

[0004] In the above-mentioned Patent Document 1, the position and size information of an object is obtained by removing ground-based data from three-dimensional environmental data obtained by LiDAR and then clustering it. The height of an object can be represented as a first height based on the range of sensor data associated with the object. The height of an object can also be represented as a second height based on the beam diffusion pattern of the sensor data and / or sensor data associated with additional objects. Therefore, the minimum and / or maximum height of an object can be determined in a robust manner. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Special Publication No. 2023-505059 [Overview of the project] [Problems that the invention aims to solve]

[0006] In the technique for estimating the height of an object described in Patent Document 1, for example, the reliability, which is the accuracy, of the estimated height of the object is not estimated. When the sensor unit that acquires three-dimensional data and the object from which the three-dimensional data is acquired are in close proximity, three-dimensional data is acquired from the entire object, making it possible to accurately estimate the size of the object. On the other hand, when the sensor unit and the object are far apart, it becomes difficult to acquire three-dimensional data from the entire object due to signal attenuation by air and a larger angle of incidence. Therefore, when the sensor unit and the object are far apart, the estimated size of the object will contain a larger error compared to the estimated size when the sensor unit and the object are in close proximity.

[0007] As described in Patent Document 1 above, if there is no information on the reliability of the estimated size (e.g., height) of the object, there is a problem that applications placed after the object recognition device cannot perform highly flexible processing using the estimated size of the object with high accuracy.

[0008] Therefore, the purpose of this disclosure is to provide an object recognition device and an object recognition method that calculate the reliability of the object size in addition to the position and size of the object. [Means for solving the problem]

[0009] The object recognition device of this disclosure includes an object detection unit that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; and an object size confidence estimation unit that uses at least one of the position, size, angle, reflectivity, identification result, and the positions of the sensor units stored in advance to calculate the confidence level of the object size.

[0010] The object recognition method of this disclosure includes: an object detection step that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; an object size confidence estimation step that uses at least one of the object's position, size, angle, reflectivity, identification result, and the position of a sensor unit stored in advance to calculate the confidence level of the object's size; and an update step that receives the object's position, size, and confidence level of the object's size as input and performs a filter processing that updates and outputs the confidence levels of the object's position, size, and size. [Effects of the Invention]

[0011] The object recognition device of this disclosure includes an object detection unit that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; and an object size reliability estimation unit that calculates the reliability of the object size using at least one of the object's position, size, angle, reflectivity, identification result, and the positions of the sensor units stored in advance. Therefore, in addition to the object's position and size, the reliability of the object size can be calculated. Since the reliability of the object size is calculated, subsequent applications can perform highly flexible processing using the estimated reliability of the object size.

[0012] The object recognition method of this disclosure includes: an object detection step that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; an object size confidence estimation step that uses at least one of the position, size, angle, reflectivity, identification result, and the position of a sensor unit stored in advance to calculate the confidence level of the object size; and an update step that receives the position, size, and confidence level of the object size as input and performs a filter processing that updates and outputs the position, size, and confidence level of the object size. As a result, the object size information can be estimated with high accuracy by filtering while considering the confidence level of the object size, thus enabling the estimation of the object size with higher accuracy. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram illustrating the schematic configuration of the object recognition device according to Embodiment 1. [Figure 2] This figure shows a schematic of the processing performed by the object recognition device according to Embodiment 1. [Figure 3] This is a block diagram illustrating the schematic configuration of another object recognition device according to Embodiment 1. [Figure 4] It is a block diagram showing an outline of the configuration of another object recognition device according to Embodiment 1. [Figure 5] It is a diagram showing an outline of the sensor unit of the object recognition device according to Embodiment 1. [Figure 6] It is a diagram showing an outline of another sensor unit of the object recognition device according to Embodiment 1. [Figure 7] It is a diagram showing an outline of another sensor unit of the object recognition device according to Embodiment 1. [Figure 8] It is a diagram showing the hardware configuration of the object recognition device according to Embodiment 1. ​​​​​​​​​​​​​​​​​​​​​​​​Figure 1 is a block diagram showing the schematic configuration of the object recognition device 10 according to Embodiment 1, also showing the external connection configuration of the object recognition device 10. Figure 2 is a diagram showing the schematic of the processing performed by the object recognition device 10. Figure 3 is a block diagram showing the schematic configuration of another object recognition device 10 according to Embodiment 1. Figure 4 is a block diagram showing the schematic configuration of yet another object recognition device 10 according to Embodiment 1. Figure 5 is a diagram showing the schematic of the sensor unit 50 of the object recognition device 10. Figure 6 is a diagram showing the schematic of another sensor unit 50 of the object recognition device 10. Figure 7 is a diagram showing the schematic of yet another sensor unit 50 of the object recognition device 10. Figure 8 is a diagram showing the hardware configuration of the object recognition device 10. Figure 9 is a diagram showing the object recognition method according to Embodiment 1. The object recognition device 10 is a device that uses three-dimensional data of the surrounding environment obtained from the sensor unit 50 to calculate the reliability of the object size in addition to the information on the position and size of the object. The details of the object recognition device 10 will be described below.

[0016] <Object recognition device 10> As shown in Figure 1, the object recognition device 10 comprises an object detection unit 11 and an object size reliability estimation unit 12. In the figure, the area enclosed by the dashed line is the object recognition device 10. The object recognition device 10 is connected to a sensor unit 50 and a vehicle control unit 20. The arrows shown in Figure 1 indicate the flow of signals input to and output from the object recognition device 10. The object detection unit 11 uses three-dimensional data input from one or more sensor units 50, which detect three-dimensional data of surrounding objects, to detect the position, size, angle, and reflectivity of an object, and to identify the object. The object size reliability estimation unit 12 calculates the reliability of the object size using at least one of the following: the object's position, size, angle, reflectivity, identification result, and the previously stored sensor unit positions. The object recognition device 10 outputs the object's position, size, and object size reliability to the vehicle control unit 20. Object size reliability refers to a reliable value with a certain range of dimensions (length, width, height) for the object's size information. The position of the sensor unit is pre-stored in memory (not shown) within the object recognition device 10, and the sensor unit position information is output from the memory to the object size reliability estimation unit 12.

[0017] In the configuration shown in FIG. 1, the sensor unit 50 is not included in the object recognition device 10, but the sensor unit 50 may be included in the object recognition device 10, and the object recognition device 10 integrated with the sensor unit 50 is also acceptable. The object recognition device 10 is mounted on a roadside unit or an autonomous vehicle. The configuration is not limited to mounting the object recognition device 10 on a roadside unit or an autonomous vehicle, and the object recognition device 10 may perform processing equivalent to that of the object recognition device 10 in the cloud without being mounted at these locations. The sensor unit 50 is mounted on a roadside unit or an autonomous vehicle. The vehicle control unit 20 is mounted on an autonomous vehicle. When the sensor unit 50, the object recognition device 10, and the vehicle control unit 20 are provided at different locations, signals are transmitted and received wirelessly. When the sensor unit 50, the object recognition device 10, and the vehicle control unit 20 are provided at the same location, signals are transmitted and received wirelessly or wired.

[0018] <Sensor unit ⑤〇, vehicle control unit ②〇> In the present embodiment, a plurality of sensor units 50 are provided. The plurality of sensor units 50 are the first sensor unit 30 and the second sensor unit 40. The plurality of sensor units 50 are not limited to two, and more may be provided. By providing a plurality of sensor units 50, objects can be detected from a plurality of viewpoints, so that three-dimensional data can be accurately acquired.

[0019] In the present embodiment, the first sensor unit 30 and the second sensor unit 40 are mounted on a roadside unit. The roadside unit is an attachment structure arranged adjacent to a road. As shown in FIG. 2, the roadside unit 60 is provided, for example, in a columnar shape. The first sensor unit 30 and the second sensor unit 40 may be mounted on one roadside unit 60, or may be mounted on each of two adjacent roadside units 60. In addition to the sensor unit 50, communication devices, display devices, etc. are also provided on the roadside unit 60.

[0020] The sensor unit 50 consists of a device that acquires three-dimensional data, such as a millimeter-wave radar, laser radar, or LiDAR. The sensor unit 50 mounted on the roadside unit 60 acquires three-dimensional data of the environment surrounding the roadside unit 60. Specifically, the sensor unit 50 acquires three-dimensional data from objects 61 and the road surface 62 around the roadside unit 60, as shown by the "x" marks in Figure 2(a). The shaded area in Figure 2(a) is the detection area 63 of the sensor unit 50. The acquired three-dimensional data is output as a three-dimensional data signal to the object detection unit 11 of the object recognition device 10, as shown in Figure 1.

[0021] The vehicle control unit 20 controls the vehicle's motion, such as yaw rate, steering angle, and speed, based on the position, size, and reliability of objects surrounding the autonomous vehicle, which are output from the object recognition device 10. Specifically, the vehicle control unit 20 controls the collision mitigation braking system, which reduces damage when the vehicle collides with an object in front, and the adaptive cruise control system, which follows the vehicle in front. In this way, the vehicle can be driven autonomously based on the position, size, and reliability of objects output by the object recognition device 10.

[0022] <Object detection unit 11> The object detection unit 11 is a processing unit that detects an object 61 from the three-dimensional data signal output by the sensor unit 50. The object detection unit 11 integrates the three-dimensional data obtained from the first sensor unit 30 and the second sensor unit 40 and removes the three-dimensional data of the road surface 62 included in the three-dimensional data signal. Figure 2(b) shows the three-dimensional data after the removal of the road surface 62, and the three-dimensional data after the three-dimensional data of the road surface 62 has been removed becomes the three-dimensional data of the object 61.

[0023] The object detection unit 11 detects objects by performing clustering of each object based on the three-dimensional data of the object 61 and calculating the object's position and size. Figure 2(c) shows the three-dimensional data of object 61 after object detection processing and the detection result of object 61. The detection result of object 61 is the part indicated by the diagonal lines, which is different from the detection area 63. The object detection unit 11 may also detect object 61 using a deep learning-based algorithm such as PointPillars. In addition to the object's position and size, the object detection unit 11 also detects the object's angle and reflectivity, and identifies the object. Object identification is the identification of what the object is, such as whether it is a car or a pedestrian.

[0024] <Comparative Example> Prior to describing the object size reliability estimation unit 12, which is the main part of this disclosure, a comparative example will be described using Figures 10, 11, and 12. Figure 10 is a block diagram showing the general configuration of the comparative object recognition device 100, and also shows the connection configuration to the outside of the object recognition device 100. Figures 11 and 12 are diagrams showing the general processing performed by the comparative object recognition device 100. As shown in Figure 10, the comparative object recognition device 100 does not have the object size reliability estimation unit 12.

[0025] Using Figures 11 and 12, the detection results of the object recognition device 100 for an object 61 that was adjacent to the sensor unit 50 and then moved away from the sensor unit 50 will be explained. In Figure 11, the three-dimensional data obtained from the object 61 by the sensor unit 50 and the object 61 are shown, and in Figure 12, the detection results of the object recognition device 100 for the object 61 are shown by diagonal lines different from the detection area 63. The three-dimensional data is shown by an "x" in the figures. The detection result in Figure 12(a) corresponds to Figure 11(a), the detection result in Figure 12(b) corresponds to Figure 11(b), and the detection result in Figure 12(c) corresponds to Figure 11(c). Figure 11(a) is the case when the object 61 is closest to the sensor unit 50, and Figures 11(b) and 11(c) are the cases when the object 61 moves away from the sensor unit 50.

[0026] When the object 61 in Figure 11(a) is adjacent to the sensor unit 50, three-dimensional data is acquired from the entire object 61, making it possible to accurately estimate the size of the object. On the other hand, as shown in Figures 11(b) and 11(c), when the sensor unit 50 and the object 61 are separated, it becomes difficult to acquire three-dimensional data from the entire object 61 due to signal attenuation by air and a larger angle of incidence. Therefore, when the sensor unit 50 and the object 61 are separated, the size estimation result will contain a larger error compared to the size estimation result when the sensor unit 50 and the object 61 are close together.

[0027] <Object size confidence estimation unit 12> The object size reliability estimation unit 12, which is the core part of this disclosure, will now be described. The object size reliability estimation unit 12 is a processing unit that calculates the reliability of the object size using at least one of the following: the object's position, the object's size, the object's angle, the object's reflectivity, the object's identification result, and the position of a sensor unit that has been stored in advance. The object identification result is the result of identifying the object as a car, a pedestrian, or the like.

[0028] By providing the object size confidence estimation unit 12, the object recognition device 10 can calculate the confidence level of the object size based on at least one of the following: the object's position, the object's size, the object's angle, the object's reflectivity, the object's identification result, and the position of the sensor unit, in addition to the object's position and size. This allows subsequent applications to perform highly flexible processing using the estimated confidence level of the object size.

[0029] An example of how the object size reliability is calculated in the object size reliability estimation unit 12 will be described. The object size reliability estimation unit 12 calculates the reliability by comparing it with map data. The object size reliability estimation unit 12 calculates the object size reliability using map data in which the relationship between the object size reliability and at least one of the following—the sensor unit position, the object position, the object size, the object angle, the object reflectivity, and the object identification result—is predetermined based on measured values. For example, the measured values ​​of the object's position are set on the vertical axis and the object's size on the horizontal axis, and the object size reliability is set at the intersection of the two. If there is a difference between the map data and the detected data, the object size reliability can be calculated by linear interpolation, for example. Alternatively, instead of linear interpolation, the resolution of the map data can be set finely, and a value close to the object size reliability can be used. With this configuration, the object size reliability estimation unit 12 can easily and quickly calculate the object size reliability.

[0030] The calculation of object size reliability is not limited to the method described above. The object size reliability estimation unit 12 may calculate object size reliability using map data in which the relationship between at least one of the following—the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectivity of the object, and the object identification result—and object size reliability is predetermined based on theoretical or simulated values. In this case, the object size reliability may be, for example, the value of the error 1σ of the length, width, and height, which are information about the size of the object. By using map data based on theoretical or simulated values, actual measurement is unnecessary, and the resolution of the map data can be easily and precisely set. The object size reliability estimation unit 12 may use map data created by one sensor unit 50, or it may use map data created by multiple sensor units 50.

[0031] <Updated part 13> An object recognition device 10a, which is different from the object recognition device 10 shown in Figure 1, will be explained using Figure 3. The object recognition device 10a further includes an update unit 13. The update unit 13 is a processing unit that receives the object's position, object's size, and object's size reliability from the object's size reliability estimation unit 12, and performs filtering processing to update and output the object's position, object's size, and object's size reliability. By providing the update unit 13, the object's size information can be estimated by filtering processing while considering the object's size reliability, thus enabling more accurate estimation of the object's size.

[0032] The filtering process in the update unit 13 uses, for example, a Kalman filter to which the object's position, size, and confidence level of the object's size are input. By setting the confidence level of the object's size as the observation noise for the object's size in the Kalman filter, it is possible to set appropriate observation noise and estimate the object's size with higher accuracy. Furthermore, by updating the confidence level of the object's size using the Kalman filter, it becomes possible to calculate a highly accurate confidence level of the object's size. The filtering process in the update unit 13 is not limited to a Kalman filter; other filters such as an αβ filter or a particle filter may also be used.

[0033] The update unit 13 adjusts the value of the observation noise for the object size in the filtering process according to the reliability of the object size. Since a lower reliability of the object size suggests a larger error in the object's position and size, the observation noise value is set higher for lower reliability. This configuration allows for more accurate estimation of the object's size.

[0034] When updating the position, size, and confidence level of an object, which were updated in the previous cycle, in the current cycle, it is necessary to determine the combination of the object information entered in the current cycle and the object information updated in the previous cycle. The object information updated in the previous cycle is predicted up to the time of the current cycle, and a tracking frame is set centered on the point where the object's position has transitioned. The objects in the current cycle that fall within this tracking frame become the candidate combinations. At this time, the setting of the tracking frame is changed taking into account the confidence level of the object size.

[0035] The update unit 13 adjusts the size of the tracking frame for the correlation process, which determines the correlation of an object with respect to the passage of time, according to the reliability of the object size. Since a lower reliability of the object size suggests a larger error in the object's position and size, setting a larger tracking frame for lower reliability of the object size makes it possible to determine more accurate combination candidates, thereby enabling a more accurate estimation of the object's size.

[0036] Another method for determining potential combinations is to use IoU (Intersection over Union). When predicting the object information updated in the previous cycle up to the time of the current cycle, and considering the transition of the object's position, if the IoU of the currently obtained object exceeds an arbitrarily specified threshold, it is considered a potential combination. The lower the confidence in the object's size, the lower the IoU threshold should be set. Since a lower confidence in the object's size suggests a larger error in both the object's position and size, it becomes possible to set more accurate combination candidates.

[0037] <Sensor unit 50> The configuration of the object recognition device 10b equipped with a sensor unit 50 will now be described. The object recognition device 10b further comprises one or more sensor units 50. The object recognition device 10b shown in Figure 4 comprises multiple sensor units 50. The multiple sensor units 50 are a first sensor unit 30 and a second sensor unit 40. The number of sensor units 50 is not limited to two, but may be more. By equipping the object recognition device 10b with multiple sensor units 50, objects can be detected from multiple viewpoints, and three-dimensional data can be acquired with high accuracy.

[0038] An example of the configuration of the sensor unit 50 in an object recognition device 10b equipped with a sensor unit 50 will be described. If the object recognition device 10b is equipped with one sensor unit, the one sensor unit 50 has multiple sensors of different types that acquire three-dimensional data, and if the object recognition device 10b is equipped with multiple sensor units, at least two sensor units have sensors that acquire three-dimensional data that are different from each other.

[0039] Figure 5 shows the sensor unit 50 when the object recognition device 10b has one sensor unit. The sensor unit 50 has multiple sensors 70 and 71 of different types. For example, sensor 70 is LiDAR and sensor 71 is RADAR. Figure 6 shows the sensor unit 50 when the object recognition device 10b has two sensor units. The first sensor unit 30 has sensor 70, and the second sensor unit 40 has a sensor 71 of a different type than sensor 70. For example, sensor 70 is LiDAR and sensor 71 is RADAR. Figure 6 shows the case where the object recognition device 10b has two sensor units 50, but the object recognition device 10b may have even more sensor units 50. In that case, the sensors in the additional sensor units 50 may be LiDAR, RADAR, or even different types of sensors.

[0040] By using different types of sensors in this way, the unique characteristics of each sensor can be leveraged to recognize objects with higher accuracy. For example, radar is suitable for detecting objects at long distances, but has limitations in detecting their size with high precision. On the other hand, LiDAR or cameras excel at detecting objects at close range with high precision. By combining sensors with these different characteristics, it is possible to recognize both distant and nearby objects, improving the overall accuracy of recognition. Furthermore, by using different types of sensors, even if the performance of some sensors deteriorates under specific environmental conditions (e.g., bad weather), stable object recognition can be maintained by having other sensors compensate. This improves the reliability and robustness of the system.

[0041] Next, another example of the configuration of the sensor unit 50 in the object recognition device 10b equipped with the sensor unit 50 will be described. The object recognition device 10b further comprises multiple sensor units, each of which has the same type of sensor for acquiring three-dimensional data. Figure 7 shows the sensor unit 50 when the object recognition device 10b is equipped with two sensor units. The first sensor unit 30 has a sensor 70, and the second sensor unit 40 also has a sensor 70. For example, the sensor 70 is a LiDAR. Figure 7 shows the case where the object recognition device 10b is equipped with two sensor units 50, but the object recognition device 10b may be equipped with even more sensor units 50. In that case, the sensors in the further provided sensor units 50 are the same sensors as those in the other sensor units 50.

[0042] By having multiple sensors of the same type and using multiple sensors of the same type, the reliability and accuracy of the data can be improved. By using multiple sensors of the same type, it becomes possible to compare, integrate, or complement the data obtained from each sensor, allowing the object detection unit 11 to obtain more accurate information than data obtained from a single sensor. Furthermore, by arranging multiple sensors of the same type at different positions or angles, it becomes possible to grasp the position and shape of objects in more detail, thereby improving the accuracy of object recognition.

[0043] The object recognition devices 10, 10a, and 10b consist of a processor 101 and a storage device 102, as shown in Figure 8 as an example of the hardware. Although not shown, the storage device comprises a volatile storage device such as random access memory and a non-volatile auxiliary storage device such as flash memory. Alternatively, a hard disk may be provided as an auxiliary storage device instead of flash memory. The processor 101 executes the program input from the storage device 102. In this case, the program is input from the auxiliary storage device to the processor 101 via the volatile storage device. The processor 101 may also output data such as calculation results to the volatile storage device of the storage device 102, or it may save the data to the auxiliary storage device via the volatile storage device.

[0044] <Object recognition method> The object recognition method is described below. As shown in Figure 9, the object recognition method comprises an object detection step (S101), an object size confidence estimation step (S102), and an update step (S103). In the object detection step, the object's position, size, angle, and reflectivity are detected and the object is identified using three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects. In the object size confidence estimation step, the confidence of the object size is calculated using at least one of the following: the object's position, size, angle, reflectivity, the object's identification result, and the positions of the sensor units that have been stored in advance. In the update step, the object's position, size, and confidence of the object size are input, and a filter process is performed to update and output the object's position, size, and confidence of the object size.

[0045] In this way, in addition to the object detection step, an object size confidence estimation step and an update step are provided. Therefore, the object size information can be estimated by filtering while considering the confidence level of the object size, thus enabling more accurate estimation of the object size.

[0046] In this embodiment, during the update step, the size of the tracking frame for the correlation process, which determines the correlation of objects over time, is adjusted according to the reliability of the object size. Since a lower reliability of the object size suggests a larger error in the object's position and size, setting a larger tracking frame for lower object size reliability makes it possible to determine more accurate combination candidates, thereby enabling a more accurate estimation of the object's size.

[0047] In this embodiment, during the update step, the value of the observation noise for the object size in the filtering process is adjusted according to the reliability of the object size. Since a lower reliability of the object size suggests a larger error in the object's position and size, the observation noise value is set higher for lower reliability of the object size. By processing in this way, the object size can be estimated with higher accuracy.

[0048] As described above, the object recognition device 10 according to Embodiment 1 includes an object detection unit 11 that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; and an object size reliability estimation unit 12 that calculates the reliability of the object size using at least one of the object's position, size, angle, reflectivity, identification result, and the positions of the sensor units stored in advance. Therefore, in addition to the object's position and size, the reliability of the object size can be calculated. Since the reliability of the object size is calculated, subsequent applications can perform highly flexible processing using the estimated reliability of the object size.

[0049] If the system is further equipped with an update unit 13 that receives the object's position, size, and confidence level of the object's size as input, and performs filtering processing to update and output the object's position, size, and confidence level of the object's size, then the object's size information can be estimated by filtering processing while taking into account the confidence level of the object's size, thereby enabling more accurate estimation of the object's size.

[0050] When the object size reliability estimation unit 12 calculates the reliability of an object size using map data in which the relationship between the object size reliability and at least one of the following—the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectivity of the object, and the identification result of the object—is predetermined based on measured values, the object size reliability estimation unit 12 can easily and quickly calculate the reliability of the object size.

[0051] When the object size reliability estimation unit 12 calculates the object size reliability using map data that has been pre-set based on theoretical or simulated values, and the relationship between at least one of the sensor unit position, object position, object size, object angle, object reflectivity, and object identification result and the object size reliability, using map data based on theoretical or simulated values ​​eliminates the need for actual measurements, making it easy to finely set the resolution of the map data.

[0052] When the update unit 13 adjusts the size of the tracking frame for the correlation process that determines the correlation of an object with respect to the passage of time, according to the reliability of the object size, the lower the reliability of the object size, the larger the error is considered to be in the position and size of the object. Therefore, by setting a larger tracking frame for lower reliability of the object size, it becomes possible to determine more accurate combination candidates, and thus estimate the size of the object with higher accuracy.

[0053] When the update unit 13 adjusts the value of the observation noise for the object size in the filtering process according to the reliability of the object size, the lower the reliability of the object size, the larger the error in the object's position and size is considered to be. Therefore, by setting a larger value for the observation noise as the reliability of the object size decreases, the object size can be estimated with higher accuracy.

[0054] The object recognition device 10b further comprises one or more sensor units, and when the object recognition device 10b comprises one sensor unit, the one sensor unit 50 has multiple sensors of different types that acquire three-dimensional data, and when the object recognition device 10b comprises multiple sensor units, at least two sensor units have sensors that acquire three-dimensional data that are different from each other, by using different types of sensors, the characteristics of each sensor can be utilized to recognize objects with higher accuracy.

[0055] If the object recognition device 10b further includes multiple sensor units, and each of these sensor units has the same type of sensor for acquiring three-dimensional data, then using multiple sensors of the same type can improve the reliability and accuracy of the data. By using multiple sensors of the same type, it becomes possible to compare, integrate, or complement the data obtained from each sensor, allowing the object detection unit 11 to obtain more accurate information than data obtained from a single sensor.

[0056] The object recognition method according to Embodiment 1 includes an object detection step that uses three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of an object, and to identify the object; an object size confidence estimation step that uses at least one of the position, size, angle, reflectivity, identification result, and the position of a sensor unit stored in advance to calculate the confidence level of the object size; and an update step that receives the position, size, and confidence level of the object size as input and performs a filter processing to update and output the position, size, and confidence level of the object size. As a result, the object size information can be estimated by filtering while considering the confidence level of the object size, thus enabling the estimation of the object size with higher accuracy.

[0057] In the update step, when adjusting the size of the tracking frame for the correlation process that determines the correlation of an object over time, if the reliability of the object size is low, it is assumed that the error in the object's position and size is larger. Therefore, by setting a larger tracking frame for lower reliability of the object size, it becomes possible to determine more accurate combination candidates, thus enabling a more accurate estimation of the object's size.

[0058] In the update step, when adjusting the value of the observation noise for object size in the filtering process according to the confidence level of the object size, a lower confidence level in the object size suggests a larger error in the object's position and size. Therefore, setting a larger value for the observation noise when the confidence level of the object size is low allows for a more accurate estimation of the object's size.

[0059] Furthermore, while this disclosure describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but can be applied individually or in various combinations to the embodiments. Accordingly, countless variations not illustrated are conceivable within the scope of the art disclosed in this specification. These include, for example, modifying, adding or omitting at least one component, or even extracting at least one component and combining it with components of other embodiments.

[0060] The various aspects of this disclosure are summarized below as an appendix. (Note 1) An object detection unit that detects the position, size, angle, and reflectivity of an object, and identifies the object, using the three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects. An object recognition device comprising: an object size reliability estimation unit that calculates the reliability of the object size using at least one of the following: the position of the object, the size of the object, the angle of the object, the reflectivity of the object, the identification result of the object, and the position of the sensor unit stored in advance. (Note 2) The object recognition device according to Appendix 1, further comprising an update unit that receives the position of the object, the size of the object, and the confidence level of the object size as input, and performs a filtering process that updates and outputs the position of the object, the size of the object, and the confidence level of the object size. (Note 3) The object recognition device according to Appendix 1 or 2, wherein the object size reliability estimation unit calculates the reliability of the object size using map data in which the relationship between the object size reliability and at least one of the following—the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectance intensity of the object, and the identification result of the object—is predetermined based on measured values. (Note 4) The object recognition device according to Appendix 1 or 2, wherein the object size reliability estimation unit calculates the reliability of the object size using map data in which the relationship between the object size reliability and at least one of the following: the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectance intensity of the object, and the identification result of the object is predetermined based on theoretical or simulated values. (Note 5) The object recognition device described in Appendix 2, wherein the update unit adjusts the size of the tracking frame for the correlation process that determines the correlation relationship of an object with respect to the passage of time, according to the reliability of the object size. (Note 6) The object recognition device according to Appendix 2 or 5, wherein the updating unit adjusts the value of the observation noise of the object size in the filtering process according to the reliability of the object size. (Note 7) The system further comprises one or more of the aforementioned sensor units, If a single sensor unit is provided, that single sensor unit has multiple sensors of different types that acquire the three-dimensional data. The object recognition device according to any one of the appendices 1 to 6, wherein, when comprising a plurality of the sensor units, at least two of the sensor units have sensors that acquire the three-dimensional data, and are different from each other. (Note 8) The sensor unit further comprises multiple such sensor units, The object recognition device according to any one of the appendices 1 to 6, wherein each of the multiple sensor units has the same type of sensor for acquiring the three-dimensional data. (Note 9) An object detection step that uses the three-dimensional data input from one or more sensor units to detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of the object, and to identify the object. An object size reliability estimation step, which calculates the reliability of the object size using at least one of the following: the position of the object, the size of the object, the angle of the object, the reflectivity of the object, the identification result of the object, and the previously stored position of the sensor unit; An object recognition method comprising: an update step in which the position of the object, the size of the object, and the confidence level of the object size are input, and a filter processing step is performed to update and output the position of the object, the size of the object, and the confidence level of the object size. (Note 10) The object recognition method according to Appendix 9, wherein in the update step, the size of the tracking frame for the correlation process that determines the correlation of an object with respect to the passage of time is adjusted according to the reliability of the object size. (Note 11) The object recognition method according to Appendix 9 or 10, wherein in the update step, the value of the observation noise of the object size in the filtering process is adjusted according to the reliability of the object size. [Explanation of symbols]

[0061] 10, 10a, 10b Object recognition device, 20 Vehicle control unit, 30 First sensor unit, 40 Second sensor unit, 50 Sensor unit, 60 Roadside unit, 61 Object, 62 Road surface, 63 Detection area, 70, 71 Sensor, 11 Object detection unit, 12 Object size reliability estimation unit, 13 Update unit, 100 Object recognition device, 101 Processor, 102 Memory device

Claims

1. An object detection unit that detects the position, size, angle, and reflectivity of an object, and identifies the object, using the three-dimensional data input from one or more sensor units that detect three-dimensional data of surrounding objects. An object recognition device comprising: an object size reliability estimation unit that calculates the reliability of the object size using at least one of the following: the position of the object, the size of the object, the angle of the object, the reflectivity of the object, the identification result of the object, and the position of the sensor unit stored in advance.

2. The object recognition device according to claim 1, further comprising an update unit that receives the position of the object, the size of the object, and the confidence level of the object size as input, and performs filter processing to update and output the confidence levels of the position of the object, the size of the object, and the confidence level of the object size.

3. The object recognition device according to claim 1 or 2, wherein the object size reliability estimation unit calculates the reliability of the object size using map data in which the relationship between the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectance intensity of the object, and the identification result of the object is predetermined based on measured values.

4. The object recognition device according to claim 1 or 2, wherein the object size reliability estimation unit calculates the reliability of the object size using map data in which the relationship between the object size reliability and at least one of the position of the sensor unit, the position of the object, the size of the object, the angle of the object, the reflectance intensity of the object, and the identification result of the object is predetermined based on theoretical or simulated values.

5. The object recognition device according to claim 2, wherein the update unit adjusts the size of the tracking frame for the correlation process that determines the correlation relationship of an object with respect to the passage of time, according to the reliability of the object size.

6. The object recognition device according to claim 2, wherein the updating unit adjusts the value of the observation noise of the object size in the filtering process according to the reliability of the object size.

7. The system further comprises one or more of the aforementioned sensor units, If a single sensor unit is provided, that single sensor unit has multiple sensors of different types that acquire the three-dimensional data. The object recognition device according to claim 1 or 2, wherein, in the case of having a plurality of the sensor units, at least two of the sensor units have sensors that acquire the three-dimensional data, which are different from each other.

8. The sensor unit further comprises multiple such sensor units, The object recognition device according to claim 1 or 2, wherein each of the multiple sensor units has the same type of sensor for acquiring the three-dimensional data.

9. An object detection step that uses the three-dimensional data input from one or more sensor units to detect three-dimensional data of surrounding objects to detect the position, size, angle, and reflectivity of the object, and to identify the object. An object size reliability estimation step, which calculates the reliability of the object size using at least one of the following: the position of the object, the size of the object, the angle of the object, the reflectivity of the object, the identification result of the object, and the previously stored position of the sensor unit; An object recognition method comprising: an update step in which the position of the object, the size of the object, and the confidence level of the object size are input, and a filter processing step is performed to update and output the position of the object, the size of the object, and the confidence level of the object size.

10. The object recognition method according to claim 9, wherein in the update step, the size of the tracking frame for the correlation process that determines the correlation of an object with respect to the passage of time is adjusted according to the reliability of the object size.

11. The object recognition method according to claim 9, wherein in the update step, the value of the observation noise of the object size in the filtering process is adjusted according to the reliability of the object size.

Images