Object detection device and driving assistance system

Abstract

To provide an object detection device capable of detecting a position and a shape of a target material with high accuracy.SOLUTION: An object detection device comprises: a data acquisition unit 20 that acquires individual point cloud data that were measured at the same time by individual sensors and generates individual clusters; a surface estimation unit 30 that, for each individual cluster, fits a first square to the individual cluster and sets individual cluster surfaces, which are two vertical surfaces containing two adjacent sides of the first square; a same object determination unit 30 that determines whether or not the individual cluster surfaces of different individual clusters are attributed to the same object; a cluster integration unit 50 that integrates individual clusters for which individual cluster surfaces determined to be attributed the same object are set into one and outputs it as an integrated cluster; and a shape deriving unit 40 that derives a second square matched to a shape of a horizontal surface of a detected object from information on the individual cluster surface that is set as the integrated cluster.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to an object detection device and a driving assistance system. [Background technology]

[0002] Technologies have been developed that use sensors mounted on vehicles or roadside sensors such as LiDAR (Light Detection and Ranging) or stereo cameras installed on the roadside to detect objects such as pedestrians on the sidewalk or vehicles and obstacles on the roadway, and use the detection results to assist vehicle driving. Information such as the position, orientation, and shape of objects on the sidewalk or roadway estimated from the sensor detection results is used to generate information to assist vehicle driving. As a method for estimating the position, orientation, and shape of an object, a method has been proposed in which point cloud data obtained from a sensor is used to apply a predetermined shape suitable for approximating the outer shape of the object (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-148514 Summary of the Invention [Problem to be solved by the invention]

[0004] In the object detection device shown in Patent Document 1, when the object to be detected is located directly in front of the sensor, or when the distance from the sensor to the object to be detected is large, a bias occurs in the point cloud data related to the object to be detected, and a shape different from the actual shape of the object to be detected is applied, resulting in the problem that the position and shape of the object to be detected cannot be detected with high accuracy.

[0005] The present application has been made to solve the above-mentioned problems, and has an object to provide an object detection device and a driving assistance system that detect the position and shape of an object to be detected with high accuracy. [Means for solving the problem]

[0006] The object detection device disclosed in the present application is an object detection device that detects an object to be detected using individual point cloud data including position information of a plurality of measurement points output from each of one or more individual sensors, and includes a data acquisition unit that acquires, from each individual sensor, individual point cloud data measured at the same time by each individual sensor, and generates individual clusters by clustering each individual point cloud data, and a data acquisition unit that fits a first quadrangle to each individual cluster, and sets two individual cluster planes, a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first quadrangle. The system is characterized by comprising a surface estimation unit, an identical object determination unit that determines whether individual cluster surfaces of different individual clusters are caused by the same object, a cluster integration unit that integrates each individual cluster in which individual cluster surfaces determined by the identical object determination unit to be caused by the same object are set and outputs the integrated cluster, and that outputs each individual cluster in which individual cluster surfaces not determined by the identical object determination unit to be caused by the same object are set as separate integrated clusters, and a shape derivation unit that derives a second quadrangle that matches the shape of the horizontal plane of the detected object from information on the individual cluster surfaces set in the integrated cluster. [Effects of the Invention]

[0007] The object detection device disclosed in the present application is an object detection device that detects an object to be detected using individual point cloud data including position information of a plurality of measurement points output from each of one or more individual sensors, and includes a data acquisition unit that acquires, from the individual sensors, individual point cloud data measured at the same time by each of the individual sensors, and generates individual clusters by clustering each of the individual point cloud data; a plane estimation unit that fits a first quadrangle to each individual cluster, and sets two individual cluster planes, a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first quadrangle; The device is equipped with an identical object determination unit that determines whether individual cluster faces of a cluster are caused by the same object, a cluster integration unit that integrates each individual cluster in which individual cluster faces determined by the identical object determination unit to be caused by the same object are set and outputs it as an integrated cluster, and outputs each individual cluster in which individual cluster faces not determined by the identical object determination unit to be caused by the same object are set as separate integrated clusters, and a shape derivation unit that derives a second quadrangle that matches the shape of the horizontal plane of the object to be detected from information about the individual cluster faces set in the integrated cluster, thereby making it possible to detect the position and shape of the object to be detected with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an object detection device in accordance with Embodiment 1. FIG. [Figure 2] FIG. 3 is a diagram showing the state of individual point cloud data before ground surface removal processing according to the first embodiment. [Figure 3] FIG. 4 is a diagram showing the state of individual point cloud data after ground removal processing according to the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating an example of clustering processing according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating another example of the clustering process according to the first embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of a surface estimation process according to the first embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a surface estimation process in a comparative example. [Figure 8] FIG. 10 is a diagram illustrating another example of the surface estimation process according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating another example of the surface estimation process in the comparative example. [Figure 10] FIG. 2 is a diagram showing an example of an individual cluster plane estimated by the plane estimation process according to the first embodiment. [Figure 11] FIG. 10 is a diagram showing another example of an individual cluster surface estimated by the surface estimation process according to the first embodiment. [Figure 12] 10 is a diagram for explaining condition 1 of the identical object determination process according to the first embodiment. FIG. [Figure 13] FIG. 10 is a diagram for explaining condition 2 of the identical object determination process according to the first embodiment. [Figure 14] FIG. 10 is a diagram for explaining condition 3 of the identical object determination process according to the first embodiment. [Figure 15] FIG. 10 is a diagram for explaining condition 4 of the identical object determination process according to the first embodiment. [Figure 16] FIG. 2 is a diagram showing an example of an integrated cluster generated by the reclustering process according to the first embodiment. [Figure 17] FIG. 10 is a diagram showing another example of an integrated cluster generated by the reclustering process of the first embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a second quadrangle derived by the quadrangle derivation process according to the first embodiment. [Figure 19] FIG. 10 is a diagram showing another example of a second quadrangle derived by the quadrangle derivation process according to the first embodiment. [Figure 20] 4A to 4C are diagrams for explaining the quadrangle correction process according to the first embodiment. [Figure 21] FIG. 10 is a block diagram showing the configuration of an object detection device according to a second embodiment. [Figure 22] FIG. 10 is a diagram illustrating an example of a horizontal plane estimation process according to the second embodiment. [Figure 23] FIG. 10 is a diagram illustrating an example of height determination processing in the second embodiment. [Figure 24] FIG. 11 is a block diagram showing the configuration of an object detection device according to a third embodiment. [Figure 25] FIG. 13 is a diagram illustrating the process of changing the size of a second rectangle to the size of a reference rectangle in the third embodiment. [Figure 26] FIG. 10 is a block diagram showing the configuration of a driving assistance system according to a fourth embodiment. [Figure 27] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an object detection device according to an embodiment. [Figure 28] FIG. 10 is a schematic diagram illustrating another example of the hardware configuration of the object detection device according to the embodiment. [Figure 29] FIG. 10 is a schematic diagram illustrating an example of a hardware configuration of a driving support information generating device according to a fourth embodiment. [Figure 30] FIG. 10 is a schematic diagram showing another example of the hardware configuration of the driving support information generating device according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an object detection device and a driving assistance system according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the same reference numerals in the various drawings indicate the same or corresponding parts.

[0010] Embodiment 1 FIG. 1 is a block diagram showing the configuration of an object detection device 100 according to a first embodiment. Sensor 10 includes one or more individual sensors, each of which measures the positions of multiple measurement points. In the example shown in FIG. 1, sensor 10 includes three individual sensors: first sensor 11, second sensor 12, and third sensor 13. However, sensor 10 may include any number of individual sensors as long as it includes one or more individual sensors. Object detection device 100 detects an object to be detected in a detection area based on information from each of the individual sensors that measure the positions of multiple measurement points. Object detection device 100 includes a data acquisition unit 20, a surface estimation unit 30, and an object detection unit 40. Object detection system 200 includes sensor 10 and object detection device 100.

[0011] The sensor 10 includes one or more individual sensors, each of which detects the position of a target object (such as a vehicle, a person, an animal, or a structure on the ground) by irradiating it with visible light, ultraviolet light, or infrared light. Each individual sensor may be, for example, a LiDAR, a radar, or a depth camera. The individual sensors may be installed anywhere, and may be mounted on a vehicle, may be mobile, or may be installed on a structure such as a streetlight or on a road. Each individual sensor outputs individual point cloud data including measurement data from multiple measurement points. In the example shown in FIG. 1 , the first sensor 11 outputs first point cloud data, the second sensor 12 outputs second point cloud data, and the third sensor 13 outputs third point cloud data. The individual point cloud data is typically output from each individual sensor at a frequency of several frames per second (fps) to several tens of fps, and is transmitted to the data acquisition unit 20 of the object detection device 100 via communication means. In the following description, it is assumed that each individual sensor is implemented as a LiDAR.

[0012] The data acquisition unit 20 acquires individual point cloud data including measurement data at a plurality of measurement points from each individual sensor included in the sensor 10. The data acquisition unit 20 has individual data acquisition units in the same number as the individual sensors, and in the example shown in Fig. 1, the data acquisition unit 20 has a first data acquisition unit 21, a second data acquisition unit 22, and a third data acquisition unit 23. The first data acquisition unit 21 acquires individual point cloud data from the first sensor 11, the second data acquisition unit 22 acquires individual point cloud data from the second sensor 12, and the third data acquisition unit 23 acquires individual point cloud data from the third sensor 13. The data acquisition unit 20 acquires individual point cloud data measured at the same time by each individual sensor included in the sensor 10, for example. When the individual sensors are LiDARs and the individual point cloud data output from the individual sensors is expressed in a coordinate system seen from the LiDAR (LiDAR coordinate system), each individual data acquisition unit of the data acquisition unit 20 converts the individual point cloud data expressed in the LiDAR coordinate system into individual point cloud data expressed in a world coordinate system in the real world. The world coordinate system is, for example, a coordinate system expressed by latitude, longitude, and altitude.

[0013] Furthermore, the data acquisition unit 20 performs ground removal processing and clustering processing on each acquired individual point cloud data, and outputs the individual point cloud data divided into each cluster to the surface estimation unit 30 as individual clusters. Here, the ground removal processing is processing to remove measurement data of measurement points indicating the position of the ground from the individual point cloud data, and extract only the measurement data of measurement points indicating the position of detected objects on the Earth's surface. The ground removal processing algorithm uses known techniques such as the RANSAC (Random Sample Consensus) algorithm. The clustering processing is processing to classify the individual point cloud data into clusters corresponding to each detected object. The clustering processing algorithm uses known techniques such as treating measurement data of measurement points within a certain interval as the same cluster.

[0014] Fig. 2 is a diagram showing the state of individual point cloud data before the ground removal processing in embodiment 1 is performed. Fig. 2 shows the state of individual point cloud data 321 acquired by first sensor 11, and individual point cloud data 321 includes measurement data of measurement points indicating the position of ground surface 300 and measurement data of measurement points indicating the position of vehicle 311, which is a detected object. Fig. 3 is a diagram showing the state of individual point cloud data after the ground removal processing in embodiment 1 is performed. By the ground removal processing, the measurement data of measurement points indicating the position of ground surface 300 is removed from individual point cloud data 321, and individual point cloud data 321 now includes only the measurement data of measurement points indicating the position of vehicle 311.

[0015] Fig. 4 is a diagram showing an example of clustering processing in the first embodiment. Fig. 4 shows an example when the sensor 10 is equipped with only the first sensor 11, and the clustering processing is performed on the individual point cloud data after the ground removal processing. The first data acquisition unit 21 acquires the individual point cloud data from the first sensor 11, performs the ground removal processing, and then performs the clustering processing. Through the clustering processing, the individual point cloud data is classified into two individual clusters: individual cluster 331a including individual point cloud data 321a corresponding to the vehicle 311, which is the first detected object, and individual cluster 331b including individual point cloud data 321b corresponding to the vehicle 312, which is the second detected object.

[0016] FIG. 5 is a diagram illustrating another example of the clustering process according to the first embodiment. FIG. 5 illustrates an example in which the sensor 10 includes a first sensor 11 and a second sensor 12. The data acquisition unit 20 includes a first data acquisition unit 21 and a second data acquisition unit 22. In FIG. 5, open circles represent individual point cloud data acquired by the first data acquisition unit 21 from the first sensor 11, and open triangles represent individual point cloud data acquired by the second data acquisition unit 22 from the second sensor 12. The first data acquisition unit 21 acquires the individual point cloud data from the first sensor 11, performs ground removal processing, and then performs clustering processing. Through the clustering process in the first data acquisition unit 21, the individual point cloud data is classified into two clusters: individual cluster 331a including individual point cloud data 321a corresponding to the vehicle 311, which is the first detected object, and individual cluster 331b including individual point cloud data 321b corresponding to the vehicle 312, which is the second detected object. The second data acquisition unit 22 acquires individual point cloud data from the second sensor 12, performs ground removal processing, and then performs clustering processing. Through the clustering processing in the second data acquisition unit 22, the individual point cloud data is classified into two clusters: an individual cluster 332a including individual point cloud data 322a corresponding to the vehicle 311, and an individual cluster 332b including individual point cloud data 322b corresponding to the vehicle 312. In other words, in the clustering processing in the data acquisition unit 20, even if the individual point cloud data corresponds to the same object, individual point cloud data acquired from different individual sensors is classified into different individual clusters.

[0017] The surface estimation unit 30 performs a surface estimation process to acquire individual clusters from each individual data acquisition unit included in the data acquisition unit 20, fit a first rectangle in a horizontal plane to each individual cluster, and estimate two individual cluster surfaces, which are a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first rectangle, that is, a first side and a second side. The surface estimation unit 30 has the same number of individual surface estimation units as the number of individual sensors, and in the example shown in FIG. 1 , the surface estimation unit 30 has a first surface estimation unit 31, a second surface estimation unit 32, and a third surface estimation unit 33. The first surface estimation unit 31 acquires individual clusters from the first data acquisition unit 21, the second surface estimation unit 32 acquires individual clusters from the second data acquisition unit 22, and the third surface estimation unit 33 acquires individual clusters from the third data acquisition unit 23.

[0018] Fig. 6 is a diagram illustrating an example of the surface estimation process according to the first embodiment. Fig. 6 illustrates the surface estimation process performed on the individual clusters by the first surface estimation unit 31 after the individual point cloud data is acquired by the first sensor 11 and classified by the first data acquisition unit 21 into individual clusters corresponding to the vehicle 311, which is the detected object. In Fig. 6, the individual point cloud data classified into the individual clusters is indicated by open circles. In the surface estimation process according to the first embodiment, a first quadrangle in a horizontal plane is fitted to the individual cluster, and the positions of two individual cluster surfaces, which are a vertical plane including the first side and a vertical plane including the second side, are estimated for two adjacent sides of the first quadrangle, a first side and a second side. 6 shows an example of a process in which the detected object is a vehicle 311, the shape of the detected object, vehicle 311, in a horizontal plane is approximated by a first rectangle, first rectangle 403 is fitted to individual point cloud data classified into individual clusters, and the positions of a vertical plane including a first side and a vertical plane including a second side are estimated for two adjacent sides of first rectangle 403. In the plane estimation process, first, as shown in the left diagram of FIG. 6, first rectangle 403 is fitted to vehicle 311 based on the positions of the individual point cloud data classified into individual clusters. Next, as shown in the center diagram of FIG. 6, the vertex of first rectangle 403 that is closest to first sensor 11, which is the individual sensor that output the individual point cloud data, is set as reference point 400, and the vertices of the first rectangle adjacent to reference point 400 are set as first vertex 401 and second vertex 402, respectively. Finally, as shown in the right diagram of Fig. 6, a vertical plane including a first side, which is a line segment connecting the reference point 400 and the first vertex 401, is defined as a first cluster plane 510, and a vertical plane including a second side, which is a line segment connecting the reference point 400 and the second vertex 402, is defined as a second cluster plane 520. The first plane estimation unit 31 outputs position information for each of the first cluster plane 510 and the second cluster plane 520, treating them as individual cluster planes. Here, a known technique such as an Oriented Bounding Box (OBB) may be used as a method for fitting a rectangle to the individual point cloud data classified into individual clusters, or the rectangle may be obtained from an approximate straight line for the individual point cloud data.

[0019] FIG. 7 is a diagram illustrating an example of a surface estimation process in a comparative example. In the surface estimation process in the comparative example, first, as shown in the left diagram of FIG. 7, individual point cloud data classified into individual clusters is acquired. Next, as shown in the middle diagram of FIG. 7, the observation point in the individual point cloud data that is closest to the first sensor 11, which is an individual sensor, is set as a reference point 400a, the observation point located on the rightmost side as seen from the first sensor 11, which is an individual sensor, is set as a first vertex 401a, and the observation point located on the leftmost side as seen from the first sensor 11, which is an individual sensor, is set as a second vertex 402a. Finally, as shown in the right diagram of FIG. 7, a vertical plane including a first side, which is a line segment connecting the reference point 400a and the first vertex 401a, is set as a first cluster surface 510a, and a vertical plane including a second side, which is a line segment connecting the reference point 400a and the second vertex 402a, is set as a second cluster surface 520a.

[0020] Next, the accuracy of the plane estimation process in the first embodiment and the plane estimation process in a comparative example will be described. FIG. 8 is a diagram showing another example of the plane estimation process in the first embodiment. Comparing FIG. 8 with FIG. 6, data for the lower left of vehicle 311 does not exist in the individual point cloud data in the left diagram of FIG. 8. However, in the plane estimation process in the first embodiment shown in FIG. 8, a first rectangle is fitted to the individual point cloud data classified into individual clusters, as shown in the left diagram of FIG. 8, and therefore a first rectangle 403 identical to the example shown on the left of FIG. 6 is fitted. As a result, as shown in the middle diagram of FIG. 8, a reference point 400, a first vertex 401, and a second vertex 402 are set at the same positions as in the example shown in FIG. 6, and as shown in the right diagram of FIG. 8, a first cluster plane 510 and a second cluster plane 520 are set at the same positions as in the example shown in FIG. 6.

[0021] FIG. 9 is a diagram illustrating another example of the plane estimation process in the comparative example. Comparing FIG. 9 with FIG. 7, the individual point cloud data in the left diagram of FIG. 9 does not contain data on the lower left of vehicle 311. As a result, in the plane estimation process in the comparative example illustrated in FIG. 9, as shown in the center diagram of FIG. 9, reference point 400b is set as the observation point closest to first sensor 11, which is an individual sensor. Next, first vertex 401b and second vertex 402b are set. Finally, as shown in the right diagram of FIG. 9, first cluster plane 510b and second cluster plane 520b, which are different from the example illustrated in FIG. 7, are set. As described above, in the plane estimation process according to the comparative example, if the observation point closest to the individual sensor is not near a corner of vehicle 311, which is the detected object, it is not possible to estimate first cluster planes and second cluster planes that are suitable for the shape of vehicle 311, which is the detected object. However, in the plane estimation process according to the first embodiment, it is possible to accurately estimate the first cluster plane and the second cluster plane.

[0022] Fig. 10 is a diagram showing an example of individual cluster surfaces estimated by the surface estimation process of embodiment 1. Fig. 10 shows examples of first and second cluster surfaces estimated by first surface estimation unit 31 using individual point cloud data acquired by first sensor 11 in the situation shown in Fig. 4. First surface estimation unit 31 estimates first cluster surface 511a and second cluster surface 521a from individual point cloud data classified into individual cluster 331a corresponding to vehicle 311, and estimates first cluster surface 511b and second cluster surface 521b from individual point cloud data classified into individual cluster 331b corresponding to vehicle 312.

[0023] Fig. 11 is a diagram showing another example of individual cluster surfaces estimated by the surface estimation process of embodiment 1. Fig. 11 shows examples of first and second cluster surfaces estimated by the first and second surface estimation units 31 and 32 using the individual point cloud data acquired by the first sensor 11 and the individual point cloud data acquired by the second sensor 12 in the situation shown in Fig. 5. The first surface estimation unit 31 estimates a first cluster surface 511a and a second cluster surface 521a from the individual point cloud data classified into the individual cluster 331a corresponding to the vehicle 311, and estimates a first cluster surface 511b and a second cluster surface 521b from the individual point cloud data classified into the individual cluster 331b corresponding to the vehicle 312. Furthermore, the second surface estimation unit 32 estimates a first cluster surface 512a and a second cluster surface 522a from the individual point cloud data classified into an individual cluster 332a corresponding to the vehicle 311, and estimates a first cluster surface 512b and a second cluster surface 522b from the individual point cloud data classified into an individual cluster 332b corresponding to the vehicle 312.

[0024] 1 includes a cluster determination unit 50 that determines whether or not individual cluster surfaces of different individual clusters, for each individual cluster surface estimated by each individual surface estimation unit of the surface estimation unit 30, represent the same object, performs reclustering, and outputs an integrated cluster, and a shape derivation unit 60 that derives a second quadrangle from the multiple individual cluster surfaces in the integrated cluster output by the cluster determination unit 50. The cluster determination unit 50 includes an identical object determination unit 51 that determines whether or not individual cluster surfaces of different individual clusters, for each individual cluster surface estimated by each individual surface estimation unit of the surface estimation unit 30, represent the same object, and a cluster integration unit 52 that integrates individual clusters in which individual cluster surfaces determined by the identical object determination unit 51 to be due to the same object are set, and outputs the integrated cluster, and that outputs individual clusters in which individual cluster surfaces not determined by the identical object determination unit 51 to be due to the same object are set, as separate integrated clusters.

[0025] The same object determination unit 51 determines whether or not individual cluster surfaces of different individual clusters are caused by the same object for each surface estimated by each individual surface estimation unit of the surface estimation unit 30. For example, when the reference surface is the first cluster surface 511a or the second cluster surface 521a shown in FIG. 11 output from the first surface estimation unit 31, and a comparison surface, which is an individual cluster surface of an individual cluster different from the reference surface, is one of the first cluster surface 511b and the second cluster surface 521b output from the first surface estimation unit 31, or the first cluster surface 512a, the second cluster surface 522a, the first cluster surface 512b, and the second cluster surface 522b output from the second surface estimation unit 32, the same object determination unit 51 determines whether or not the reference surface and the comparison surface are caused by the same object. The same object determination unit 51 determines, for example, four determination conditions in the same object determination process shown in the following conditions 1 to 4 in order, and determines that the reference surface and the comparison surface are caused by the same object when any of the determination conditions 1 to 4 is satisfied. Note that the following explanation shows a determination of whether or not individual cluster surfaces of individual clusters acquired from different individual sensors are caused by the same object, but the same process is also performed to determine whether or not individual cluster surfaces of different individual clusters acquired from one individual sensor are caused by the same object.

[0026] Condition 1 is that the reference surface and the comparison surface intersect, and the angle between the reference surface and the comparison surface is within a predetermined range from a first threshold value to a second threshold value. Here, the range of the angle between the reference surface and the comparison surface is arbitrary, and for example, it is determined that condition 1 is met when the angle between the reference surface and the comparison surface is between 70 degrees and 110 degrees.

[0027] Condition 2 is that when the reference surface and the comparison surface do not intersect, a line of intersection between a reference extension surface obtained by extending the reference surface toward the comparison surface and a comparison extension surface obtained by extending the comparison surface toward the reference surface is found, and either the reference extension length from the reference surface to the line of intersection or the comparison extension length from the comparison surface to the line of intersection is equal to or less than a predetermined third threshold, and the angle between the reference surface and the comparison surface is within a predetermined range from a fourth threshold to a fifth threshold. Here, the range of the reference extension length or the comparison extension length and the angle between the reference surface and the comparison surface are arbitrary. For example, Condition 2 is determined to be met when the reference extension length is within 10% of the length of the reference surface in the extension direction, the comparison extension length is within 10% of the length of the comparison surface in the extension direction, and the angle between the reference surface and the comparison surface is between 70 degrees and 110 degrees. Furthermore, when the reference extension surface intersects with two or more comparison extension surfaces, resulting in two or more intersections, the above condition determination is performed for the combination of the reference surface and the comparison surface with the smallest sum of the reference extension length and the comparison extension length.

[0028] Condition 3 is that the following three conditions, the first, second, and third judgment conditions, are satisfied. The first judgment condition is that the angle between the plane including the reference surface and the plane including the comparison surface is equal to or less than a predetermined sixth threshold. The second judgment condition is that the two points included in the reference surface that are furthest apart in the horizontal direction are defined as the first endpoint and the second endpoint, the two points included in the comparison surface that are furthest apart in the horizontal direction are defined as the third endpoint and the fourth endpoint, the distance between the first endpoint and the plane including the comparison surface is defined as the first endpoint distance, the distance between the second endpoint and the plane including the comparison surface is defined as the second endpoint distance, the distance between the third endpoint and the plane including the reference surface is defined as the third endpoint distance, and the distance between the fourth endpoint and the plane including the reference surface is defined as the fourth endpoint distance, and the largest value among the first endpoint distance, the second endpoint distance, the third endpoint distance, and the fourth endpoint distance is equal to or less than a seventh threshold. The third judgment condition is that at least one of the following is a ratio of the area of ​​the projection figure formed when the reference surface is projected perpendicularly onto the comparison surface to the total area of ​​the comparison surface, and a ratio of the area of ​​the projection figure formed when the comparison surface is projected perpendicularly onto the reference surface to the total area of ​​the reference surface, is equal to or greater than an eighth threshold. For example, the sixth threshold is 15 degrees, the seventh threshold is several centimeters, and the eighth threshold is 70%.

[0029] Condition 4 is that the normal line extending from the reference surface intersects with the normal line extending from the comparison surface, and the angle between the normal line extending from the reference surface and the normal line extending from the comparison surface is within a predetermined range from a ninth threshold to a tenth threshold. For example, it is determined that Condition 4 is met when the normal line extending from the reference surface intersects with the normal line extending from the comparison surface, and the angle between the normal line extending from the reference surface and the normal line extending from the comparison surface is between 160 degrees and 200 degrees.

[0030] Fig. 12 is a diagram for explaining condition 1 of the same object determination process according to embodiment 1. In the left diagram of Fig. 12, a first cluster surface 511c and a second cluster surface 521c are estimated from individual point cloud data classified into individual cluster 331c corresponding to vehicle 313 using individual point cloud data acquired by first sensor 11, and a first cluster surface 512c and a second cluster surface 522c are estimated from individual point cloud data classified into individual cluster 332c corresponding to vehicle 313 using individual point cloud data acquired by second sensor 12. As shown in the right diagram of Figure 12, when second cluster surface 521c is used as the reference surface and first cluster surface 512c is used as the comparison surface, the reference surface and the comparison surface intersect, and the angle between the reference surface and the comparison surface is 90 degrees and is in the range of 70 degrees to 110 degrees, so it is determined that condition 1 is met, and second cluster surface 521c and first cluster surface 512c are determined to be caused by the same object, and individual cluster 331c and individual cluster 332c are determined to be caused by the same object.

[0031] Fig. 13 is a diagram for explaining condition 2 of the same object determination process according to embodiment 1. In the left diagram of Fig. 13, a first cluster surface 511d and a second cluster surface 521d are estimated from individual point cloud data classified into individual cluster 331d corresponding to vehicle 314 using individual point cloud data acquired by first sensor 11, and a first cluster surface 512d and a second cluster surface 522d are estimated from individual point cloud data classified into individual cluster 332d corresponding to vehicle 314 using individual point cloud data acquired by second sensor 12. As shown in the right diagram in Figure 13, when first cluster surface 511c is used as the reference surface and second cluster surface 522d is used as the comparison surface, the reference surface and the comparison surface do not intersect, the reference extension amount from the reference surface to the intersection line is within 10% of the length in the extension direction of the reference surface, and the angle between the reference surface and the comparison surface is 90 degrees and is in the range of 70 degrees to 110 degrees, so it is determined that condition 2 is met, and first cluster surface 511c and second cluster surface 522d are determined to be caused by the same object, and individual cluster 331d and individual cluster 332d are determined to be caused by the same object.

[0032] 14 is a diagram illustrating condition 3 of the same object determination process according to embodiment 1. In the left diagram of FIG. 14, first cluster surface 511e and second cluster surface 521e are estimated from individual point cloud data classified into individual cluster 331e corresponding to vehicle 315 using individual point cloud data acquired by first sensor 11, and first cluster surface 512e and second cluster surface 522e are estimated from individual point cloud data classified into individual cluster 332e corresponding to vehicle 315 using individual point cloud data acquired by second sensor 12. As shown in the right diagram of FIG. 14, when second cluster surface 521e is used as the reference surface and second cluster surface 522e is used as the comparison surface, the angle between the plane including the reference surface and the plane including the comparison surface is 15 degrees or less, and therefore the first determination condition is satisfied. Furthermore, when the two points included in the reference plane that are furthest apart horizontally are defined as the first endpoint and the second endpoint, the two points included in the comparison plane that are furthest apart horizontally are defined as the third endpoint and the fourth endpoint, the distance between the first endpoint and the plane that includes the comparison plane is defined as the first endpoint distance, the distance between the second endpoint and the plane that includes the comparison plane is defined as the second endpoint distance, the distance between the third endpoint and the plane that includes the reference plane is defined as the third endpoint distance, and the distance between the fourth endpoint and the plane that includes the reference plane is defined as the fourth endpoint distance, the largest value among the first endpoint distance, the second endpoint distance, the third endpoint distance, and the fourth endpoint distance is several centimeters or less, so the second determination criterion is met. Furthermore, at least one of the ratio of the area of ​​the projected figure created when the reference plane is projected perpendicularly onto the comparison plane to the total area of ​​the comparison plane and the ratio of the area of ​​the projected figure created when the comparison plane is projected perpendicularly onto the reference plane to the total area of ​​the reference plane is 70% or more, so the third determination criterion is met. Since the three conditions of the first judgment condition, the second judgment condition, and the third judgment condition are satisfied, it is ultimately determined that condition 3 is satisfied, and second cluster surface 521e and second cluster surface 522e are determined to be caused by the same object, and individual cluster 331e and individual cluster 332e are determined to be caused by the same object.

[0033] Fig. 15 is a diagram illustrating condition 4 of the same object determination process according to embodiment 1. In the left diagram of Fig. 15, a first cluster surface 511f and a second cluster surface 521f are estimated from individual point cloud data classified into individual cluster 331f corresponding to vehicle 316 using individual point cloud data acquired by first sensor 11, and a first cluster surface 512f and a second cluster surface 522f are estimated from individual point cloud data classified into individual cluster 332f corresponding to vehicle 316 using individual point cloud data acquired by second sensor 12. As shown in the right diagram in Figure 15, when first cluster surface 511f is used as the reference surface and first cluster surface 512f is used as the comparison surface, normal 611f extended from the reference surface and normal 612f extended from the comparison surface intersect, and the angle between normal 611f extended from the reference surface and normal 612f extended from the comparison surface is 180 degrees, which is in the range of 160 degrees to 200 degrees. Therefore, it is determined that condition 4 is satisfied, and first cluster surface 511f and first cluster surface 512f are determined to be caused by the same object, and individual cluster 331f and individual cluster 332f are determined to be caused by the same object.

[0034] 1 performs a reclustering process, integrating individual clusters in which individual cluster faces determined by the identical object determination unit 51 to be due to the same object into one cluster and outputting the integrated cluster, and outputting individual clusters in which individual cluster faces not determined by the identical object determination unit 51 to be due to the same object as separate integrated clusters. FIG. 16 is a diagram illustrating an example of integrated clusters generated by the reclustering process of the first embodiment. For example, in the situation illustrated in FIG. 5, if the determination result of the identical object determination unit 51 determines that individual clusters 331a and 332a are due to the same object and individual clusters 331b and 332b are due to the same object, as illustrated in FIG. 16, the individual clusters 331a and 332a are integrated into one integrated cluster 331A, and the individual clusters 331b and 332b are integrated into one integrated cluster 331B, and the integrated clusters 331A and 331B are output. As a result, the integrated cluster 331A includes both the individual point cloud data 321a and the individual point cloud data 322a, and the integrated cluster 331B includes both the individual point cloud data 321b and the individual point cloud data 322b.

[0035] Fig. 17 is a diagram showing another example of an integrated cluster generated by the reclustering process of Embodiment 1. For example, in the situation shown in Fig. 5, if the determination result of the identical object determination unit 51 indicates that the individual clusters 331a and 332a are determined to be due to the same object, and the individual clusters 331b and 332b are not determined to be due to the same object, then, as shown in Fig. 17, the individual clusters 331a and 332a are integrated into the integrated cluster 331A, the individual clusters 331b and 332b are not integrated, and the integrated cluster 331A, the individual clusters 331b, and the individual clusters 332b are output as the integrated cluster. As a result, the integrated cluster 331A includes both the individual point cloud data 321a and the individual point cloud data 322a, the individual cluster 331b includes the individual point cloud data 321b, and the individual cluster 332b includes the individual point cloud data 322b.

[0036] In this way, by determining whether individual cluster surfaces of different individual clusters are due to the same object and performing re-clustering processing based on the determination result to obtain an integrated cluster, it is possible to prevent the same object from being recognized as a different object when obtained from individual point cloud data from each of multiple individual sensors, and to prevent erroneous recognition of the number of objects or the shape of the object. Furthermore, when point cloud data representing one object is obtained in a discontinuous manner as multiple individual point cloud data, it is possible to prevent erroneous recognition of the number or size of objects, and to improve the accuracy of estimating the number, position, and shape of objects.

[0037] The shape derivation unit 60 shown in FIG. 1 performs a quadrangle derivation process for each integrated cluster reclustered by the cluster integration unit 52, deriving a second quadrangle that matches the horizontal shape of the detected object from information about the individual cluster surfaces set in the integrated cluster. When the detected object is a vehicle, the shape derivation unit 60 derives a second rectangle from information about the individual cluster surfaces in each integrated cluster. Next, the shape derivation unit 60 obtains the position and shape of the detected object from the position and shape of the estimated second quadrangle. Here, the second quadrangle may be derived by any method, including by incorporating the second quadrangle so as to minimize its area, or by approximating it in some way. Furthermore, if the size of the derived second quadrangle is not within a predetermined range, the shape derivation unit 60 may perform a quadrangle correction process for correcting the second quadrangle based on a predetermined value.

[0038] FIG. 18 is a diagram illustrating an example of a second quadrangle derived by the quadrangle derivation process according to the first embodiment. The diagram on the left of FIG. 18 illustrates that, as a result of the identical object determination process illustrated in FIG. 12, individual cluster 331c and individual cluster 332c are integrated by the reclustering process, and first cluster surface 511c, second cluster surface 521c, first cluster surface 512c, and second cluster surface 522c are included in one integrated cluster. In this case, as illustrated in the diagram on the right of FIG. 18, second rectangle 530, which is a second quadrangle, is derived from first cluster surface 511c, second cluster surface 521c, first cluster surface 512c, and second cluster surface 522c. In the diagram on the right of FIG. 18, the smallest rectangle that contains first cluster surface 511c, second cluster surface 521c, first cluster surface 512c, and second cluster surface 522c is defined as second rectangle 530.

[0039] Using FIGS. 19 and 20, the rectangular correction process in the shape derivation unit 60 will be described. FIG. 19 is a diagram showing another example of the second rectangle derived by the rectangular derivation process of Embodiment 1. In the left diagram of FIG. 19, the first cluster plane 511g and the second cluster plane 521g are estimated from the individual point cloud data classified into individual clusters corresponding to the vehicle 317 using the individual point cloud data acquired by the first sensor 11. In this case, as shown in the right diagram of FIG. 19, a second rectangle, the second rectangle 531, is derived from the first cluster plane 511g and the second cluster plane 521g. In the right diagram of FIG. 19, the smallest rectangle enclosing the first cluster plane 511g and the second cluster plane 521g is defined as the second rectangle 531.

[0040] FIG. 20 is a diagram for explaining the rectangular correction process of Embodiment 1. As shown in the upper left diagram of FIG. 20, when the length of the longest side is x and the length of the shortest side is y on the side of the second rectangle, the second rectangle 531, the condition for performing the rectangular correction process may be, for example, when the length of y is clearly shorter than the length of x, or when the ratio of y to x is less than or equal to an eleventh threshold value determined in advance. Here, as shown in the lower left diagram of FIG. 20, even when only the first cluster plane is obtained from the individual point cloud data acquired by the first sensor 11 and the value of y of the derived second rectangle 532 becomes zero, the rectangular correction process is to be performed. The content of the rectangular correction process may be to change the value of y to a fixed value y' determined in advance according to the magnitude of x. For example, when the units of x and y' are meters, y' = 5 when 0 < x < 3, y' = 2 when 3 ≤ x < 6, and y' = 3 when 6 ≤ x. When the rectangular correction process is performed on the second rectangle 531 and the second rectangle 532 based on the above conditions, each of the second rectangle 531 and the second rectangle 532 is corrected to the second rectangle 533 shown in the upper right diagram of FIG. 20. Here, the length of y' of the second rectangle 533 becomes the value shown in the above conditions. By performing the rectangular correction process, it is possible to prevent a large error in the estimated object and improve the estimation accuracy of the position and shape of the object.

[0041] As described above, object detection device 100 according to the first embodiment is an object detection device 100 that detects an object to be detected using individual point cloud data including position information of a plurality of measurement points output from each of one or more individual sensors, and includes a data acquisition unit 20 that acquires, from the individual sensors, individual point cloud data measured at the same time by each of the individual sensors, and generates individual clusters by clustering each of the individual point cloud data; a plane estimation unit 30 that fits a first quadrangle to each individual cluster, and sets two individual cluster planes, which are a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first quadrangle; The device is provided with an identical object determination unit 51 that determines whether individual cluster faces of individual clusters that are determined to be caused by the same object are caused by the same object, a cluster integration unit 52 that integrates each of the individual clusters in which individual cluster faces that have been determined to be caused by the same object by the identical object determination unit 51 are set and outputs the integrated cluster, and outputs each of the individual clusters in which individual cluster faces that have not been determined to be caused by the same object by the identical object determination unit 51 are set as separate integrated clusters, and a shape derivation unit 60 that derives a second quadrangle that matches the shape of the horizontal plane of the object to be detected from information about the individual cluster faces set in the integrated cluster, so that the position and shape of the object to be detected can be detected with high accuracy.

[0042] Embodiment 2 FIG. 21 is a block diagram showing the configuration of an object detection device 100a according to Embodiment 2. Comparing object detection device 100a according to Embodiment 2 shown in FIG. 21 with object detection device 100 according to Embodiment 1 shown in FIG. 1, surface estimation unit 30 has become surface estimation unit 30a, first surface estimation unit 31 has become first surface estimation unit 31a, second surface estimation unit 32 has become second surface estimation unit 32a, third surface estimation unit 33 has become third surface estimation unit 33a, object detection unit 40 has become object detection unit 40a, cluster determination unit 50 has become cluster determination unit 50a, cluster integration unit 52 has become cluster integration unit 52a, and a height determination unit 53 has been added to cluster determination unit 50a. Other configurations of object detection device 100a according to Embodiment 2 are the same as those of object detection device 100 according to Embodiment 1. Object detection system 200a includes sensor 10 and object detection device 100a. Sensor 10 in object detection system 200a is limited to a three-dimensional sensor.

[0043] The plane estimation unit 30a performs the same plane estimation process as the plane estimation unit 30 in the first embodiment, and also performs horizontal plane estimation process for each individual cluster acquired from each individual data acquisition unit included in the data acquisition unit 20, estimating a cluster horizontal plane from the individual point cloud data included in the individual cluster, and outputs position information of the cluster horizontal plane. FIG. 22 is a diagram illustrating an example of the horizontal plane estimation process in the second embodiment. As shown in the left diagram of FIG. 22, the individual point cloud data 321 is acquired by the first sensor 11 and classified by the first data acquisition unit 21 into an individual cluster 331a corresponding to the vehicle 311, which is the detected object. Then, the first plane estimation unit 31a performs horizontal plane estimation process on the individual cluster 331a. The right diagram of FIG. 22 shows how a cluster horizontal plane 541 is obtained by the horizontal plane estimation process performed on the individual cluster 331a shown in the left diagram of FIG. 22. The first plane estimation unit 31a outputs position information of the cluster horizontal plane 541. Here, the horizontal shape and position of the cluster horizontal plane are, for example, the shape and position of a first rectangle fitted to an individual cluster in the surface estimation process. If the detected object is a vehicle, the horizontal shape and position of the cluster horizontal plane are the shape and position of a first rectangle fitted to the individual cluster corresponding to the vehicle. The height of the cluster horizontal plane in an individual cluster is, for example, a statistical quantity calculated from the heights of the individual point cloud data included in the individual cluster using a predetermined statistical method. The height of the cluster horizontal plane in an individual cluster may be, for example, the average height of the individual point cloud data included in the individual cluster, or the maximum height of the individual point cloud data included in the individual cluster. Furthermore, the height of the cluster horizontal plane in an individual cluster may be, for example, a statistical quantity of the heights of the individual point cloud data included in the individual cluster that fall within a predetermined height range.

[0044] The height determination unit 53 performs a height determination process to determine whether the cluster horizontal planes of different individual clusters estimated by each individual surface estimation unit of the surface estimation unit 30a are caused by the same object. For example, when two different individual clusters are a first individual cluster and a second individual cluster, the height determination unit 53 determines that the first individual cluster and the second individual cluster are caused by the same object when the difference in height between the cluster horizontal plane of the first individual cluster and the cluster horizontal plane of the second individual cluster is equal to or less than a predetermined twelfth threshold. For example, the height determination unit 53 determines that the first individual cluster and the second individual cluster are caused by the same object when the difference in height between the cluster horizontal plane of the first individual cluster and the cluster horizontal plane of the second individual cluster is equal to or less than 100 cm. The height determination unit 53 determines whether the cluster horizontal planes of different individual clusters are caused by the same object, and may also determine from information on the cluster horizontal planes of the individual clusters that the objects indicated by the individual clusters are garbage, cardboard, or the like that will not interfere with vehicle travel. The height determination unit 53 may determine that an individual cluster is caused by an object other than the detection target when the height of the cluster horizontal plane of the individual cluster is equal to or less than a predetermined thirteenth threshold. For example, the height determination unit 53 may determine that an individual cluster is caused by an object other than the detection target when the height of the cluster horizontal plane of the individual cluster is several centimeters or less.

[0045] FIG. 23 is a diagram illustrating an example of height determination processing in the second embodiment. As illustrated on the left side of FIG. 23, individual point cloud data 321 is acquired by first sensor 11 and classified by first data acquisition unit 21 into individual cluster 331a corresponding to vehicle 311, which is the object to be detected. Individual point cloud data 322 is acquired by second sensor 12 and classified by second data acquisition unit 22 into individual cluster 331b corresponding to vehicle 311, which is the object to be detected. After that, horizontal plane estimation processing is performed. As illustrated on the right side of FIG. 23, first plane estimation unit 31a obtains cluster horizontal plane 541 by performing horizontal plane estimation processing on individual cluster 331a, and second plane estimation unit 32a obtains cluster horizontal plane 542 by performing horizontal plane estimation processing on individual cluster 331b. When the difference in height between cluster horizontal plane 541 and cluster horizontal plane 542 is equal to or less than a predetermined twelfth threshold, height determination unit 53 determines that individual cluster 331a and individual cluster 331b are caused by the same object.

[0046] 21 integrates individual clusters in which individual cluster faces determined to be caused by the same object by the same object determination unit 51 are set and in which cluster horizontal planes determined to be caused by the same object by the height determination unit 53 are set, and outputs the integrated cluster, and outputs individual clusters in which individual cluster faces not determined to be caused by the same object by the same object determination unit 51 are set or in which cluster horizontal planes not determined to be caused by the same object by the height determination unit 53 are set, as individual integrated clusters. By the above processing, the cluster integration processing in the cluster integration unit 52a can be performed with higher accuracy.

[0047] Furthermore, the cluster integration unit 52a may not integrate with other individual clusters and not output an individual cluster that is determined by the height determination unit 53 to be due to an object other than the detection target. The height determination unit 53 determines that an individual cluster is due to an object other than the detection target when the height of the cluster horizontal plane of the individual cluster is equal to or less than a predetermined thirteenth threshold, and the cluster integration unit 52a does not integrate with other individual clusters and not output an individual cluster that is determined by the height determination unit 53 to be due to an object other than the detection target, thereby making it possible to exclude objects with low heights from detection targets and to perform the cluster integration process in the cluster integration unit 52a with even higher accuracy.

[0048] As described above, object detection device 100a according to the second embodiment includes: surface estimation unit 30a that sets a cluster horizontal plane for each individual cluster; the horizontal shape and position of the cluster horizontal plane are the shape and position of a first rectangle; the height of the cluster horizontal plane is a statistical quantity calculated from the heights of the individual point cloud data included in the individual cluster by a predetermined statistical method; and height determination unit 53 that determines whether the cluster horizontal planes of different individual clusters are caused by the same object. Cluster integration unit 52a integrates, into one integrated cluster, the individual clusters for which the same object determination unit 51 has set individual cluster surfaces determined to be caused by the same object and for which the height determination unit 53 has set cluster horizontal planes determined to be caused by the same object; and outputs each of the individual clusters for which the same object determination unit 51 has set individual cluster surfaces not determined to be caused by the same object or for which the height determination unit 53 has set cluster horizontal planes not determined to be caused by the same object, as individual integrated clusters. As a result, the position and shape of the detected object can be detected with higher accuracy.

[0049] Embodiment 3 FIG. 24 is a block diagram showing the configuration of object detection device 100b according to Embodiment 3. Comparing object detection device 100b according to Embodiment 2 shown in FIG. 24 with object detection device 100 according to Embodiment 1 shown in FIG. 1, object detection unit 40 has been replaced with object detection unit 40b, a type acquisition unit 70 has been added to object detection unit 40b, and shape derivation unit 60 has been replaced with shape derivation unit 60b. The other configuration of object detection device 100b according to Embodiment 2 is the same as that of object detection device 100 according to Embodiment 1. Object detection system 200b includes sensor 10, type determination sensor 80, and object detection device 100b. Object detection device 100b may be configured by replacing shape derivation unit 60 with shape derivation unit 60b and adding type acquisition unit 70 to object detection device 100a according to Embodiment 2 shown in FIG. 21. The object detection device 100b shown in FIG. 24 will be described below.

[0050] The type determination sensor 80 includes an image sensor and a signal processing device. The image sensor may be, for example, a charge-coupled device (CCD) camera, a complementary metal oxide semiconductor (CMOS) image sensor, or an infrared sensor. The signal processing device processes the image signal from the image sensor and determines the type of object in the image signal. Furthermore, the signal processing device acquires object position information indicating the position of each object by, for example, calibrating the image signal from the image sensor. The type determination sensor 80 outputs type information for each object, including the object type and object position information. The image sensor may be installed anywhere, and may be mounted on a vehicle, mobile, or installed on a structure such as a streetlight or on a road. The object to be determined by type may be an object on the ground, such as a vehicle, a person, an animal, or a structure. The type categories output by the type determination sensor 80 may be categorized as vehicles, people, animals, structures, etc., or vehicles may be further categorized as passenger cars, trucks, motorcycles, etc. The type acquisition unit 70 acquires type information relating to each subject output from the type determination sensor 80, and outputs the information to the shape derivation unit 60b.

[0051] The shape derivation unit 60b performs a rectangle derivation process for each integrated cluster acquired from the cluster integration unit 52, deriving a second rectangle that matches the horizontal plane shape of the detected object from information about the individual cluster surfaces set in the integrated cluster. Furthermore, the shape derivation unit 60b predetermines a reference rectangle size, which is the size of a reference rectangle for each type of subject. The shape derivation unit 60b extracts type information corresponding to the integrated cluster by comparing the position of the second rectangle in the integrated cluster with the subject position information in the type information acquired from the type acquisition unit 70, and acquires the type of subject corresponding to the integrated cluster. Next, when the size of the derived second rectangle in each integrated cluster is smaller than the reference rectangle size for the corresponding type of subject, the shape derivation unit 60b corrects the size of the second rectangle to the reference rectangle size.

[0052] When the detected object is a vehicle, the shape derivation unit 60b uses the second quadrangle as the second rectangle and the reference rectangle size as the reference rectangle size, and derives a second rectangle that matches the shape of the horizontal plane of the detected object from the information on the individual cluster surfaces set in each integrated cluster acquired from the cluster integration unit 52. Furthermore, the shape derivation unit 60b predetermines a reference rectangle size, which is the size of a reference rectangle, for each type of object. The shape derivation unit 60b compares the position of the second rectangle in the integrated cluster with the object position information in the type information acquired from the type acquisition unit 70 to extract type information corresponding to the integrated cluster and acquire the type of object corresponding to the integrated cluster. Next, when the size of the derived second rectangle in each integrated cluster is smaller than the reference rectangle size for the corresponding type of object, the shape derivation unit 60b corrects the size of the second rectangle to the reference rectangle size. The reference rectangle size is (W, L) = (2, 5) when the type of subject is a passenger car, and (W, L) = (2.5, 12) when the type of subject is a truck, where W is the width of the rectangle and L is the length of the rectangle, expressed in meters.

[0053] FIG. 25 is a diagram illustrating a process for changing the size of a second quadrangle to a reference quadrangle size in the third embodiment. In the upper diagram of FIG. 25, a first cluster surface 511h and a second cluster surface 521h are estimated from individual point cloud data classified into an individual cluster corresponding to a vehicle 318 using individual point cloud data acquired by the first sensor 11. The type determination sensor 80 outputs type information for the vehicle 318, including the type of the vehicle 318, "standard-sized vehicle," and subject position information indicating the position of the vehicle 318, and the shape derivation unit 60b acquires the type information. As shown in the middle diagram of FIG. 25, the shape derivation unit 60b derives a second quadrangle, a second rectangle 534, from the first cluster surface 511h and the second cluster surface 521h. Shape derivation unit 60b compares the acquired information on the type of subject with the size of second rectangle 534, and if the size of second rectangle 534 is smaller than the reference rectangle size (W, L) = (2, 5) of the corresponding subject type, "standard-sized vehicle," shape derivation unit 60b corrects the size of second rectangle 534 to (W, L) = (2, 5) and outputs it as the processing result of shape derivation unit 60b as second rectangle 534a, as shown in the lower diagram of Fig. 25. In the lower diagram of Fig. 25, the length of the shape of second rectangle 534 before correction, which is L, in the horizontal plane of second cluster face 521h, is shorter than the length of the reference rectangle size, so the length of the second rectangle is extended to create second rectangle 534a.

[0054] As described above, the object detection device 100b detects an object using individual point cloud data including position information of multiple measurement points output from one or more individual sensors and type information of each object output from the type determination sensor 80. The object detection device 100b includes a type acquisition unit 70 that acquires type information from the type determination sensor 80. The type information includes the type of object and object position information indicating the position of the object. The shape derivation unit 60b extracts type information corresponding to each integrated cluster, acquires the object type corresponding to the integrated cluster, and corrects the size of the derived second rectangle to the reference rectangle size when the size of the derived second rectangle is smaller than a reference rectangle size predetermined for each object type. This allows the size of the derived second rectangle to be determined with higher accuracy. Furthermore, even if the surface estimation unit 30 is unable to estimate a surface suitable for the shape of the object, the size of the second rectangle can be determined with higher accuracy.

[0055] Embodiment 4 26 is a block diagram showing the configuration of a driving assistance system 210 according to the fourth embodiment. The driving assistance system 210 according to the fourth embodiment includes the object detection device 100 according to the first embodiment and the driving assistance information generation device 90. The driving assistance system 210 may include the object detection device 100a according to the second embodiment and the driving assistance information generation device 90, or may include the object detection device 100b according to the third embodiment and the driving assistance information generation device 90. In the following, the driving assistance system 210 will be described as including the object detection device 100 according to the first embodiment and the driving assistance information generation device 90.

[0056] The driving assistance information generation device 90 includes a detection result collection unit 91, a mapping unit 92, a movement range information generation unit 93, a driving assistance information generation unit 94, and a driving assistance information output unit 95. The detection result collection unit 91 collects information on the position and shape of a second rectangle that matches the horizontal shape of each detected object output by the shape derivation unit 60 of the object detection device 100. The mapping unit 92 generates mapping information that maps the positions of the detected objects present in the detection area on a map based on the information on the second rectangles collected by the detection result collection unit 91. The movement range information generation unit 93 uses the mapping information generated by the mapping unit 92 to generate movement range information that indicates the range within which the detected object is expected to move. The driving assistance information generation unit 94 uses the movement range information generated by the movement range information generation unit 93 to generate driving assistance information for the detected object. The driving assistance information output unit 95 outputs the driving assistance information generated by the driving assistance information generation unit 94 to the detected object.

[0057] As described above, the driving assistance system 210 according to embodiment 4 includes the object detection device 100, the detection result collection unit 91 that acquires information about the second rectangle from the shape derivation unit 60, the mapping unit 92 that generates mapping information that maps the position of the detected object on a map based on the information about the second rectangle collected by the detection result collection unit 91, the movement range information generation unit 93 that uses the mapping information to generate movement range information that indicates the range within which the detected object is expected to move, the driving assistance information generation unit 94 that uses the movement range information to generate driving assistance information for the detected object, and the driving assistance information output unit 95 that outputs the driving assistance information to the detected object. As a result, driving assistance for the detected object can be provided using information about the position and shape of the detected object that has been detected with high accuracy.

[0058] FIG. 27 is a schematic diagram showing an example of the hardware configuration of object detection devices 100, 100a, and 100b according to the embodiment. The data acquisition unit 20, the surface estimation units 30 and 30a, the identical object determination unit 51, the cluster integration units 52 and 52a, the height determination unit 53, and the shape derivation units 60 and 60b are implemented by a processor 701, such as a CPU (Central Processing Unit), that executes programs stored in a memory 702. The memory 702 is also used as a temporary storage device for each process executed by the processor 701. The memory 702 is, for example, a non-volatile or volatile semiconductor memory such as RAM, ROM, flash memory, or EPROM, a magnetic disk, an optical disk, or a combination thereof. The processor 701 and the memory 702 are connected to each other via a bus.

[0059] FIG. 28 is a schematic diagram showing another example of the hardware configuration of object detection devices 100, 100a, and 100b according to the embodiment. In FIG. 28, a processing circuit 704 is connected to a bus 703. When processing circuit 704 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each of the functions of object detection devices 100, 100a, and 100b may be realized by processing circuit 704, or all of the functions may be realized by processing circuit 704. Furthermore, some of the functions of object detection devices 100, 100a, and 100b may be realized by dedicated hardware, and other parts may be realized by software or firmware.

[0060] FIG. 29 is a schematic diagram showing an example of a hardware configuration of a driving assistance information generation device 90 according to the fourth embodiment. The detection result collection unit 91, the mapping unit 92, the movement range information generation unit 93, and the driving assistance information generation unit 94 are realized by a processor 701 such as a CPU that executes programs stored in a memory 702. The memory 702 is also used as a temporary storage device for each process executed by the processor 701. The memory 702 is, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM, a magnetic disk, an optical disk, or a combination thereof. The driving assistance information output unit 95 is realized by a transmitter 705. The processor 701, the memory 702, and the transmitter 705 are connected to one another via a bus.

[0061] FIG. 30 is a schematic diagram showing another example of the hardware configuration of the driving assistance information generation device 90 according to the fourth embodiment. The driving assistance information output unit 95 is implemented by a transmitter 705. In FIG. 30, the processing circuit 704 and the transmitter 705 are connected to a bus 703. When the processing circuit 704 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC, an FPGA, or a combination thereof. The detection result collection unit 91, the mapping unit 92, the moving range information generation unit 93, and the driving assistance information generation unit 94 may each be implemented by the processing circuit 704, or the detection result collection unit 91, the mapping unit 92, the moving range information generation unit 93, and the driving assistance information generation unit 94 may be implemented collectively by the processing circuit 704. Alternatively, the detection result collection unit 91, the mapping unit 92, the moving range information generation unit 93, and the driving assistance information generation unit 94 may be partially implemented by dedicated hardware and partially implemented by software or firmware.

[0062] Although the present application describes various exemplary embodiments, the various features, aspects, and functions described in one or more embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are conceivable within the scope of the technology disclosed in this application, including, for example, cases where at least one component is modified, added, or omitted, and cases where at least one component is extracted and combined with a component of another embodiment.

[0063] Although the preferred embodiments have been described above in detail, the present invention is not limited to the above-described embodiments. The above-described embodiments may be modified without departing from the scope of the claims. Various modifications and substitutions can be made to the above.

[0064] Various aspects of the present disclosure are summarized below as appendices.

[0065] (Appendix 1) An object detection device that detects an object to be detected using individual point cloud data including position information of a plurality of measurement points output from one or more individual sensors, a data acquisition unit that acquires the individual point cloud data measured at the same time by each of the individual sensors from the individual sensors and generates individual clusters by clustering each of the individual point cloud data; a plane estimation unit that fits a first quadrangle to each of the individual clusters and sets two individual cluster planes, a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first quadrangle; an identical object determination unit that determines whether the individual cluster faces of the different individual clusters are caused by the same object; a cluster integration unit that integrates the individual clusters in which the individual cluster faces determined by the identical object determination unit to be due to the same object are set, and outputs the integrated cluster, and that outputs the individual clusters in which the individual cluster faces not determined by the identical object determination unit to be due to the same object are set, as individual integrated clusters; and a shape deriving unit that derives a second quadrangle that matches the shape of the horizontal plane of the object to be detected from information on the individual cluster plane set in the integrated cluster. (Appendix 2) The detected object is a vehicle, the first quadrangle is a first rectangle, The object detection device described in Supplementary Note 1, wherein the second quadrangle is a second rectangle. (Appendix 3) The object detection device described in Appendix 1 or 2, characterized in that the surface estimation unit uses the vertex of the first quadrangle that is closest to the individual sensor that acquired the individual point cloud data included in the individual cluster as a reference point, and the two sides of the first quadrangle that sandwich the reference point as the first side and the second side. (Appendix 4) The object detection device described in any one of Appendixes 1 to 3, characterized in that when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the same object determination unit determines that the reference surface and the comparison surface are caused by the same object when the reference surface and the comparison surface intersect and the angle between the reference surface and the comparison surface is within a range from a predetermined first threshold to a predetermined second threshold. (Appendix 5) The object detection device described in any one of Appendix 1 to 4, characterized in that when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the reference surface and the comparison surface do not intersect, and an intersection line is found between a reference extension surface extended from the reference surface in the direction of the comparison surface and a comparison extension surface extended from the comparison surface in the direction of the reference surface, and when either the reference extension amount from the reference surface to the intersection line or the comparison extension amount from the comparison surface to the intersection line is less than a predetermined third threshold, and the angle between the reference surface and the comparison surface is within a range from a predetermined fourth threshold to a fifth threshold, determines that the reference surface and the comparison surface are caused by the same object. (Appendix 6) the same object determination unit determines that the reference surface and the comparison surface are due to the same object when three conditions, namely, a first determination condition, a second determination condition, and a third determination condition, are satisfied when the individual cluster surfaces of the different individual clusters are used as a reference surface and a comparison surface, the first determination condition is that an angle between a plane including the reference surface and a plane including the comparison surface is equal to or smaller than a sixth threshold value determined in advance; the second determination condition is that when two points that are the furthest apart in the horizontal direction among the points included in the reference surface are defined as a first endpoint and a second endpoint, two points that are the furthest apart in the horizontal direction among the points included in the comparison surface are defined as a third endpoint and a fourth endpoint, the distance between the first endpoint and a plane that includes the comparison surface is defined as a first endpoint distance, the distance between the second endpoint and a plane that includes the comparison surface is defined as a second endpoint distance, the distance between the third endpoint and a plane that includes the reference surface is defined as a third endpoint distance, and the distance between the fourth endpoint and a plane that includes the reference surface is defined as a fourth endpoint distance, the largest value among the first endpoint distance, the second endpoint distance, the third endpoint distance, and the fourth endpoint distance is equal to or less than a seventh threshold, The object detection device described in any one of Appendices 1 to 5, characterized in that the third judgment condition is that at least one of the following is equal to or greater than an eighth threshold: the ratio of the area of ​​the projected figure created when the reference surface is projected perpendicularly onto the comparison surface to the total area of ​​the comparison surface; and the ratio of the area of ​​the projected figure created when the comparison surface is projected perpendicularly onto the reference surface to the total area of ​​the reference surface. (Appendix 7) The object detection device described in any one of Appendix 1 to 6, characterized in that when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the same object determination unit determines that the reference surface and the comparison surface are caused by the same object when a normal extended from the reference surface intersects with a normal extended from the comparison surface and the angle formed between the normal extended from the reference surface and the normal extended from the comparison surface is within a predetermined range from a ninth threshold to a tenth threshold. (Appendix 8) the plane estimation unit sets a cluster horizontal plane for each of the individual clusters, the horizontal shape and position of the cluster horizontal plane are the shape and position of the first rectangle, and the height of the cluster horizontal plane is a statistical quantity calculated from the heights of the individual point cloud data included in the individual cluster by a predetermined statistical method; a height determination unit that determines whether the cluster horizontal planes of the different individual clusters are due to the same object; The object detection device according to any one of Supplementary Notes 1 to 7, characterized in that the cluster integration unit integrates the individual clusters in which the same object determination unit has set the individual cluster faces that are determined to be caused by the same object and in which the height determination unit has set the cluster horizontal planes that are determined to be caused by the same object into one integrated cluster and outputs the integrated cluster, and outputs the individual clusters in which the same object determination unit has set the individual cluster faces that are not determined to be caused by the same object or in which the height determination unit has set the cluster horizontal planes that are not determined to be caused by the same object as each individual integrated cluster. (Appendix 9) An object detection device that detects the object to be detected using the individual point cloud data including position information of a plurality of measurement points output from one or more of the individual sensors and type information of each subject output from a type determination sensor, a type acquisition unit that acquires the type information from the type determination sensor, the type information includes a type of the subject and subject position information indicating a position of the subject; The object detection device according to any one of Supplementary Notes 1 to 8, wherein the shape derivation unit extracts, for each of the integrated clusters, the type information corresponding to the integrated cluster, acquires the type of the subject corresponding to the integrated cluster, and, when the size of the derived second rectangle is smaller than a reference rectangle size predetermined for each type of the subject, corrects the size of the second rectangle to the reference rectangle size. (Appendix 10) The object detection device according to any one of appendices 1 to 9, characterized in that the data acquisition unit detects the position of the measurement point by irradiating it with visible light, ultraviolet light, or infrared light, and acquires the individual point cloud data from the individual sensors installed on the road. (Appendix 11) An object detection device according to any one of Supplementary Notes 1 to 10; a detection result collection unit that acquires information about the second quadrangle from the shape derivation unit; a mapping unit that generates mapping information in which the position of the detected object is mapped on a map based on the information about the second rectangle collected by the detection result collection unit; a movement range information generating unit that generates movement range information indicating a range within which the object is expected to move, using the mapping information; a driving assistance information generating unit that generates driving assistance information for the detected object using the movement range information; a driving assistance information output unit that outputs the driving assistance information to the detected object. [Explanation of symbols]

[0066] 10 Sensor, 11 First sensor, 12 Second sensor, 13 Third sensor, 20 Data acquisition unit, 21 First data acquisition unit, 22 Second data acquisition unit, 23 Third data acquisition unit, 30, 30a Surface estimation unit, 31, 31a First surface estimation unit, 32, 32a Second surface estimation unit, 33, 33a Third surface estimation unit, 40, 40a, 40b Object detection unit, 50, 50a Cluster determination unit, 51 Same object determination unit, 52, 52a Cluster integration unit, 53 Height determination unit, 60, 60b Shape derivation unit, 70 Type acquisition unit, 80 Type determination sensor, 90 Driving assistance information generation device, 91 Detection result collection unit, 92 Mapping unit, 93 Movement range information generation unit, 94 Driving assistance information generation unit, 95 Driving assistance information output unit, 100, 100a, 100b Object detection device, 200, 200a, 200b Object detection system, 210 Driving assistance system, 300 Ground, 311, 312, 313, 314, 315, 316, 317, 318 Vehicle, 321, 321a, 321b, 322, 322a, 322b Individual point cloud data, 331a, 331b, 331c, 331d, 331e, 331f, 332a, 332b, 332c, 332d, 332e, 332f Individual cluster, 331A, 331B Integrated cluster, 400, 400a, 400b Reference point, 401, 401a, 401b First vertex, 402, 402a, 402b Second vertex, 403 First rectangle, 510, 510a, 510b, 511a, 511b, 511c, 511d, 511e, 511f, 511g, 511h, 512a, 512b, 512c, 512d, 512e, 512f First cluster face, 520, 520a, 520b, 521a, 521b, 521c, 521d, 521e, 521f, 521g, 521h, 522a, 522b, 522c, 522d, 522e, 522f Second cluster face, 530, 531, 532, 533, 534, 534a Second rectangle, 541, 542 Cluster horizontal plane, 611f, 612f Normal, 701 Processor, 702 Memory, 703 bus, 704 processing circuit, 705 transmitter.

Claims

1. An object detection device that detects an object to be detected using individual point cloud data including position information of a plurality of measurement points output from one or more individual sensors, a data acquisition unit that acquires the individual point cloud data measured at the same time by each of the individual sensors from the individual sensors and generates individual clusters by clustering each of the individual point cloud data; a plane estimation unit that fits a first quadrangle to each of the individual clusters and sets two individual cluster planes, a vertical plane including the first side and a vertical plane including the second side, for two adjacent sides of the first quadrangle; an identical object determination unit that determines whether the individual cluster faces of the different individual clusters are caused by the same object; a cluster integration unit that integrates the individual clusters in which the individual cluster faces determined by the identical object determination unit to be due to the same object are set, and outputs the integrated cluster, and that outputs the individual clusters in which the individual cluster faces not determined to be due to the same object are set, as individual integrated clusters; and a shape deriving unit that derives a second quadrangle that matches the shape of the horizontal plane of the object to be detected from information on the individual cluster plane set in the integrated cluster.

2. The detected object is a vehicle, the first quadrangle is a first rectangle, The object detection device according to claim 1 , wherein the second quadrangle is a second rectangle.

3. The object detection device described in claim 1 or 2, characterized in that the surface estimation unit uses the vertex of the first rectangle that is closest to the individual sensor that acquired the individual point cloud data included in the individual cluster as a reference point, and uses the two sides of the first rectangle that sandwich the reference point as the first side and the second side.

4. The object detection device described in claim 1 or 2, characterized in that when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the same object determination unit determines that the reference surface and the comparison surface are caused by the same object when the reference surface and the comparison surface intersect and the angle between the reference surface and the comparison surface is within a range from a predetermined first threshold to a predetermined second threshold.

5. 3. The object detection device according to claim 1, wherein when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the same object determination unit determines an intersection line between a reference extension surface obtained by extending the reference surface in the direction of the comparison surface and a comparison extension surface obtained by extending the comparison surface in the direction of the reference surface, where the reference surface and the comparison surface do not intersect, and determines that the reference surface and the comparison surface are caused by the same object when either the reference extension amount from the reference surface to the intersection line or the comparison extension amount from the comparison surface to the intersection line is less than a predetermined third threshold, and the angle between the reference surface and the comparison surface is within a range from a predetermined fourth threshold to a predetermined fifth threshold.

6. the same object determination unit determines that the reference surface and the comparison surface are due to the same object when three conditions, namely, a first determination condition, a second determination condition, and a third determination condition, are satisfied when the individual cluster surfaces of the different individual clusters are used as a reference surface and a comparison surface, the first determination condition is that an angle between a plane including the reference surface and a plane including the comparison surface is equal to or smaller than a sixth threshold value determined in advance; the second determination condition is that when two points that are the furthest apart in the horizontal direction among the points included in the reference surface are defined as a first endpoint and a second endpoint, two points that are the furthest apart in the horizontal direction among the points included in the comparison surface are defined as a third endpoint and a fourth endpoint, a distance between the first endpoint and a plane that includes the comparison surface is defined as a first endpoint distance, a distance between the second endpoint and a plane that includes the comparison surface is defined as a second endpoint distance, a distance between the third endpoint and a plane that includes the reference surface is defined as a third endpoint distance, and a distance between the fourth endpoint and a plane that includes the reference surface is defined as a fourth endpoint distance, the largest value among the first endpoint distance, the second endpoint distance, the third endpoint distance, and the fourth endpoint distance is equal to or less than a seventh threshold, The object detection device described in claim 1 or 2, characterized in that the third judgment condition is that at least one of the following is a ratio of the area of ​​the projected figure created when the reference surface is projected perpendicularly onto the comparison surface to the total area of ​​the comparison surface: and a ratio of the area of ​​the projected figure created when the comparison surface is projected perpendicularly onto the reference surface to the total area of ​​the reference surface, is equal to or greater than an eighth threshold.

7. The object detection device described in claim 1 or 2, characterized in that when the individual cluster surfaces of different individual clusters are used as a reference surface and a comparison surface, the same object determination unit determines that the reference surface and the comparison surface are caused by the same object when a normal extended from the reference surface intersects with a normal extended from the comparison surface and the angle formed between the normal extended from the reference surface and the normal extended from the comparison surface is within a range from a predetermined ninth threshold to a tenth threshold.

8. the plane estimation unit sets a cluster horizontal plane for each of the individual clusters, the horizontal shape and position of the cluster horizontal plane are the shape and position of the first rectangle, and the height of the cluster horizontal plane is a statistical quantity calculated from the heights of the individual point cloud data included in the individual cluster by a predetermined statistical method; a height determination unit that determines whether the cluster horizontal planes of the different individual clusters are due to the same object; 3. The object detection device according to claim 1, wherein the cluster integration unit integrates the individual clusters in which the individual object determination unit has set the individual cluster faces determined to be due to the same object and in which the height determination unit has set the cluster horizontal planes determined to be due to the same object into one integrated cluster, and outputs the integrated cluster; and outputs the individual clusters in which the individual object determination unit has set the individual cluster faces that are not determined to be due to the same object, or in which the height determination unit has set the cluster horizontal planes that are not determined to be due to the same object, as separate integrated clusters.

9. an object detection device that detects the object to be detected using the individual point cloud data including position information of a plurality of measurement points output from each of the one or more individual sensors and type information of each subject output from a type determination sensor, a type acquisition unit that acquires the type information from the type determination sensor, the type information includes a type of the subject and subject position information indicating a position of the subject; 3. The object detection device according to claim 1, wherein the shape derivation unit extracts, for each of the integrated clusters, the type information corresponding to the integrated cluster, acquires the type of the subject corresponding to the integrated cluster, and, when the size of the derived second rectangle is smaller than a reference rectangle size predetermined for each type of the subject, corrects the size of the second rectangle to the reference rectangle size.

10. The object detection device according to claim 1 or 2, characterized in that the data acquisition unit detects the position of the measurement point by irradiating it with visible light, ultraviolet light or infrared light, and acquires the individual point cloud data from the individual sensors installed on the road.

11. The object detection device according to claim 1 or 2; a detection result collection unit that acquires the second quadrangle from the shape derivation unit; a mapping unit that generates mapping information in which the position of the detected object is mapped on a map based on the information about the second rectangle collected by the detection result collection unit; a movement range information generating unit that generates movement range information indicating a range within which the object is expected to move, using the mapping information; a driving assistance information generating unit that generates driving assistance information for the detected object using the movement range information; a driving assistance information output unit that outputs the driving assistance information to the detected object.

Images