Object detection device, object detection system, and driving assistance system

Abstract

To provide an object detection device which detects an object with high accuracy while suppressing a processing load.SOLUTION: An object detection device 20 for detecting a target using point group data including positional information of a plurality of measurement points outputted from a road-side sensor 10 installed at a road side comprises: a data acquisition unit 21 for acquiring point group data from the road-side sensor 10; a ground surface estimation unit 22 for generating ground surface data indicating a position of a ground surface from the point group data acquired from the data acquisition unit 21; a ground surface removal unit 24 for generating ground surface removal data, in which measurement data of the measurement points on a ground surface are removed from the point group data acquired from the data acquisition unit 21, using the ground surface data; and an object detection unit 25 for detecting the target from the ground surface removal data. The ground surface estimation unit 22 executes in predetermined timing ground surface estimation processing for acquiring the point group data from the data acquisition unit 21 and generating the ground surface data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present application relates to an object detection device, an object detection system, and a driving assistance system. [Background technology]

[0002] Technologies have been developed that use roadside sensors, such as LiDAR (Light Detection and Ranging) or stereo cameras, installed on the roadside to detect objects such as pedestrians on sidewalks and vehicles and obstacles on roadways, and use the detection results to assist vehicle driving. When detecting objects using roadside sensors, the accuracy of object detection can be improved by removing measurement data of measurement points on the ground from point cloud data acquired by the roadside sensors. For example, proposed methods for removing measurement data of measurement points on the ground from point cloud data acquired by roadside sensors include a method of removing measurement point data below a predetermined height in the point cloud data acquired by the roadside sensor without performing ground estimation processing, and a method of estimating the ground using a Random Sampling Consensus (RANSAC) algorithm each time the roadside sensor acquires point cloud data (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-43475 Summary of the Invention [Problem to be solved by the invention]

[0004] In the object detection device shown in Patent Document 1, if a method is used in which measurement point data below a predetermined height in point cloud data acquired by a roadside sensor is removed as measurement data of measurement points on the ground, the processing load is small but the accuracy of removing the measurement data of measurement points on the ground is low, and if the ground is estimated using a method based on the RANSAC algorithm each time the roadside sensor acquires point cloud data, the accuracy of the ground estimation is high but the processing load is large. Either method has the problem of not being able to detect objects with high accuracy while suppressing the processing load.

[0005] The present application has been made to solve the above-mentioned problems, and aims to provide an object detection device, an object detection system, and a driving assistance system that detect objects with high accuracy while reducing the processing load. [Means for solving the problem]

[0006] The object detection device disclosed in the present application is an object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, and includes a data acquisition unit that acquires the point cloud data from the roadside sensor, a ground surface estimation unit that generates ground surface data indicating the position of the ground surface from the point cloud data acquired from the data acquisition unit and outputs the data, a ground surface data storage unit that stores the ground surface data, a ground surface removal unit that uses the ground surface data acquired from the ground surface data storage unit to generate ground surface removal data by removing measurement data of measurement points on the ground surface from the point cloud data acquired from the data acquisition unit and outputs the data, and an object detection unit that detects an object from the ground surface removal data and outputs the detection result as object detection information, and the ground surface estimation unit performs ground surface estimation processing by acquiring the point cloud data from the data acquisition unit and generating the ground surface data according to a predetermined method. The time interval of the ground surface estimation process is longer than the time interval of the ground surface removal process in which the ground surface removal unit acquires point cloud data from the data acquisition unit and generates ground surface removal data. It is characterized by: [Effects of the Invention]

[0007] Object detection device disclosed in the present application According to It is possible to detect objects with high accuracy while suppressing the processing load. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of a roadside sensor and an object to be detected. [Figure 2] 1 is a block diagram showing a configuration of an object detection device in accordance with Embodiment 1. FIG. [Figure 3] 4 is a flowchart illustrating the operation of the object detection device according to the first embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of an object detection device according to a second embodiment. [Figure 5] 10 is a flowchart illustrating the operation of the object detection device according to the second embodiment. [Figure 6] FIG. 10 is a block diagram showing the configuration of an object detection system according to a third embodiment. [Figure 7] 1 is a schematic diagram illustrating an example of a hardware configuration of an object detection system according to a first embodiment. [Figure 8] FIG. 4 is a schematic diagram showing another example of the hardware configuration of the object detection system in accordance with the first embodiment. [Figure 9] FIG. 10 is a schematic diagram illustrating an example of a hardware configuration of an object detection system according to a second embodiment. [Figure 10] FIG. 10 is a schematic diagram illustrating another example of the hardware configuration of the object detection system according to the second embodiment. [Figure 11] FIG. 11 is a schematic diagram illustrating an example of a hardware configuration of a driving support information generating device according to a third embodiment. [Figure 12] FIG. 11 is a schematic diagram showing another example of the hardware configuration of the driving support information generating device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an object detection device, an object detection system, and a driving assistance system according to embodiments of the present invention will be described in detail with reference to the drawings. Note that the same reference numerals in the various drawings indicate the same or corresponding parts.

[0010] Embodiment 1 FIG. 1 is a diagram showing an example of a roadside sensor 10 and an object to be detected. The roadside sensor 10 is installed, for example, at a height of several meters above the road, with its posture fixed so as to cover the range in which the object is to be detected. The roadside sensor 10 is, for example, a LiDAR or a stereo camera. The roadside sensor 10 is used to detect, for example, a pedestrian 101 or a vehicle 102 on the ground 103 in the detection area.

[0011] FIG. 2 is a block diagram showing the configuration of an object detection device 20 according to the first embodiment. The object detection device 20 detects an object based on information from a roadside sensor 10 that measures the positions of multiple measurement points. The object detection device 20 includes a data acquisition unit 21, a ground estimation unit 22, a ground data storage unit 23, a ground removal unit 24, and an object detection unit 25. The object detection system 100 includes the roadside sensor 10 and the object detection device 20. The roadside sensor 10 outputs point cloud data including measurement data from multiple measurement points. The roadside sensor 10 is installed on the roadside and is, for example, a LiDAR or a stereo camera, and outputs point cloud data including position information of the multiple measurement points. The roadside sensor 10 may output information on the time the position information was acquired along with the position information of the measurement points. The position information output by the roadside sensor 10 may be expressed in a three-dimensional coordinate system of X, Y, and Z, or in a polar coordinate system of yaw angle, pitch angle, and depth. The point cloud data output by the roadside sensor 10 may include the reflected light intensity of the measurement points, or may include color information of the measurement points.

[0012] The data acquisition unit 21 acquires point cloud data including measurement data at multiple measurement points from the roadside sensor 10, and outputs the acquired point cloud data to the ground surface estimation unit 22 and the ground surface removal unit 24. If the roadside sensor 10 is a compound eye camera such as a stereo camera, the roadside sensor 10 may determine and output position information of the multiple measurement points from the acquired multiple image data, or the roadside sensor 10 may output the acquired multiple image data, and the data acquisition unit 21 may determine position information of the multiple measurement points from the image data acquired from the roadside sensor 10.

[0013] The ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 and generates and outputs ground surface data indicating the position of the ground surface from the acquired point cloud data. The ground surface estimation unit 22 detects a plane by fitting a plane to the position coordinates of multiple measurement points in the point cloud data, for example, using a RANSAC algorithm, and outputs information on the position of the detected plane as ground surface data. The area output as ground surface data may be a single area or multiple areas. The ground surface estimation unit 22 may also calculate and output the reliability of the generated ground surface data based on the number of measurement points, the density of the measurement points, the intensity of reflected light from the measurement points, and the area of ​​the plane indicated by the generated ground surface data in the point cloud data used to generate the ground surface data. The reliability of the ground surface data may be calculated using previously generated ground surface data or by comparing the generated ground surface data with the results of a prior survey of the relationship between the ground surface and the roadside sensor 10. The point cloud data may also be subjected to preprocessing, such as downsampling or noise removal, as necessary, and the ground surface data may be obtained from the preprocessed point cloud data. The ground surface data storage unit 23 stores the ground surface data output from the ground surface estimation unit 22. When the reliability of the ground surface data is output from the ground surface estimation unit 22, the ground surface data storage unit 23 also stores the reliability together with the ground surface data.

[0014] The ground removal unit 24 acquires ground data indicating the position of the ground from the ground data storage unit 23, acquires point cloud data from the data acquisition unit 21, compares the position of each measurement point in the point cloud data with the ground data, and generates ground removal data by removing the measurement data of measurement points determined to be on the ground from the point cloud data, and outputs the ground removal data to the object detection unit 25. The ground removal unit 24 determines that a measurement point is on the ground, for example, when the distance between the position of the measurement point and the plane indicated in the ground data is equal to or less than a predetermined threshold value.

[0015] The object detection unit 25 detects objects from the ground-removed data acquired from the ground removal unit 24, and outputs the detection result as object detection information. The detection of objects from the ground-removed data in the object detection unit 25 is performed using known techniques such as Euclidean clustering or machine learning. The objects detected in the object detection unit 25 are, for example, pedestrians 101 or vehicles 102, and the object detection unit 25 also detects the position, size, orientation, etc. of the objects.

[0016] Here, the ground surface estimation unit 22 executes ground surface estimation processing at a predetermined timing, in which it acquires point cloud data from the data acquisition unit 21 and generates ground surface data. If the ground surface estimation unit 22 generates ground surface data every time the data acquisition unit 21 acquires point cloud data from the roadside sensor 10, the processing load will be large. When detecting an object from point cloud data acquired from the roadside sensor 10 installed on the roadside, the position of the ground surface in the point cloud data rarely changes. Therefore, by having the ground surface estimation unit 22 execute ground surface estimation processing at a predetermined timing, in which the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 and generates ground surface data, the processing load can be reduced without significantly reducing the accuracy of the ground surface estimation processing in the ground surface estimation unit 22, and object detection processing can be performed with high accuracy while reducing the processing load. do.

[0017] The ground estimation process in which the ground estimation unit 22 acquires point cloud data from the data acquisition unit 21 and generates ground data may be performed only at any of the following timings 1 to 4, or may be performed at all of the following timings 1 to 4. Timing 1 is when the data acquisition unit 21 acquires point cloud data for the first time. Performing the ground estimation process only when the data acquisition unit 21 acquires point cloud data for the first time can significantly reduce the processing load. Timing 2 is when the ground estimation process is performed at a predetermined time interval. The time interval for the ground estimation process may be several minutes, several hours, or even one day. Performing the ground estimation process at a predetermined time interval can respond to changes in ground information in the point cloud data due to changes in weather or the movement of parked vehicles. Furthermore, the processing load can be reduced by setting the time interval for the ground estimation process longer than the time interval for the ground removal process in which the ground removal unit acquires point cloud data from the data acquisition unit and generates ground removal data.

[0018] At timing 3, the ground surface estimation process is executed when it is confirmed that no moving object is present within the detection range of the roadside sensor 10. The absence of a moving object is confirmed, for example, by video data acquired from a camera that captures the detection range of the roadside sensor 10. When no moving object is present, more position information can be acquired using the ground surface as a measurement point, and the ground surface estimation process can be performed with high accuracy.

[0019] At timing 4, the frequency of the ground surface estimation process is determined based on the reliability of ground surface data previously generated by the ground surface estimation unit 22. For example, when the reliability of ground surface data previously generated by the ground surface estimation unit 22 is low, the frequency of the ground surface estimation process in which the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 to generate ground surface data may be increased, and when the reliability of ground surface data previously generated by the ground surface estimation unit 22 is high, the frequency of the ground surface estimation process in which the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 to generate ground surface data may be decreased. For example, when the reliability of ground surface data previously generated by the ground surface estimation unit 22 is low, the ground surface estimation interval time when the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 to generate ground surface data may be shortened, and when the reliability of ground surface data previously generated by the ground surface estimation unit 22 is high, the ground surface estimation interval time when the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 to generate ground surface data may be lengthened.

[0020] The ground surface estimation unit 22 may execute the ground surface estimation process, which acquires point cloud data from the data acquisition unit 21 and generates ground surface data, multiple times, and store the generated multiple pieces of ground surface data in the ground surface data storage unit 23. The ground surface removal unit 24 may acquire one piece of ground surface data from the multiple pieces of ground surface data stored in the ground surface data storage unit 23 to generate the ground surface removal data. For example, the ground surface estimation unit 22 may store ground surface data including ground surface data creation time information in the ground surface data storage unit 23, and the ground surface removal unit 24 may acquire the most recent ground surface data from the ground surface data storage unit 23 based on the ground surface data creation time information to generate the ground surface removal data. For example, the ground surface estimation unit 22 may store ground surface data including reliability in the ground surface data storage unit 23, and the ground surface removal unit 24 may acquire the most reliable ground surface data from the ground surface data storage unit 23 to generate the ground surface removal data. For example, in rainy weather, the detection performance of the roadside sensor 10 may deteriorate, the number of ground surface measurement points in the point cloud data acquired by the ground surface estimation unit 22 from the data acquisition unit 21 may decrease, and the reliability of the ground surface data generated by the ground surface estimation unit 22 may decrease. Even in such a case, the ground removal unit 24 generates ground removal data using highly reliable ground data generated using point cloud data acquired from the data acquisition unit 21 on sunny days, in contrast to the point cloud data acquired by the ground removal unit 24 from the data acquisition unit 21 on rainy days, thereby enabling accurate removal of measurement data of measurement points on the ground even on rainy days.

[0021] The ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21 and executes ground surface estimation processing multiple times to generate ground surface data, storing each piece of ground surface data in the ground surface data storage unit 23. The ground surface removal unit 24 generates one piece of composite ground surface data from the multiple pieces of ground surface data acquired from the ground surface data storage unit 23, and generates and outputs ground surface removal data by removing measurement data of measurement points on the ground surface from the point cloud data acquired from the data acquisition unit 21, using the plane indicated by the composite ground surface data as the ground surface. For example, the ground surface removal unit 24 may use the average value of the multiple pieces of ground surface data as the composite ground surface data. For example, when the ground surface data is expressed in terms of position and height, the height value of a specified position in the composite ground surface data may be the average of the height values ​​of the specified positions in each piece of ground surface data. When the ground surface data is expressed in terms of angle and distance, the distance value at a specified angle in the composite ground surface data may be the average of the distance values ​​at the specified angle in each piece of ground surface data. By using the average value of the multiple pieces of ground surface data as the composite ground surface data, when each piece of ground surface data contains noise, the noise contained in the composite ground surface data can be reduced by calculating the average value. The number of ground surface data used to generate one piece of synthetic ground surface data may be, for example, several tens to several hundreds. In addition, the process of determining the height of a specified position in the synthetic ground surface data may use all ground surface data determined in the past, or may use ground surface data generated within a predetermined time, or may determine an average value by weighting the data to emphasize ground surface data generated close to the time the synthetic ground surface data is generated, or may determine an average value by weighting the data according to the reliability of the ground surface data.

[0022] 3 is a flowchart for explaining the operation of object detection device 20 according to embodiment 1. Step S01 is a data acquisition step, step S02 is a ground surface estimation confirmation step, step S03 is a ground surface position estimation step, step S04 is a ground surface data storage step, step S05 is a ground surface removal step, and step S06 is an object detection step. Object detection device 20 repeatedly performs the processes from step S01 to step S06.

[0023] In step S01, the data acquisition unit 21 acquires point cloud data, which is the measurement result of the roadside sensor 10, from the roadside sensor 10, and proceeds to step S02. In step S02, the ground surface estimation unit 22 checks whether the conditions for performing the ground surface estimation processing are met, and if the conditions for performing the ground surface estimation processing are met, proceeds to step S03, and if the conditions for performing the ground surface estimation processing are not met, proceeds to step S05. The condition for performing the ground surface estimation processing may be, for example, one of the above-described timings 1 to 4, or may be that any of the conditions for timings 1 to 4 is met.

[0024] In step S03, the ground surface estimation unit 22 acquires point cloud data from the data acquisition unit 21, generates ground surface data indicating the position of the ground surface from the acquired point cloud data, and outputs the data. In step S04 In step S04, the ground data storage unit 23 stores the ground data acquired from the ground estimation unit 22, and the process proceeds to step S05. In step S05, the ground removal unit 24 acquires ground data indicating the position of the ground from the ground data storage unit 23, acquires point cloud data from the data acquisition unit 21, compares the positions of each measurement point in the point cloud data with the ground data, and generates and outputs ground-removed data in which measurement data of measurement points determined to be on the ground is removed from the point cloud data, and the process proceeds to step S06. In step S06, the object detection unit 25 detects an object from the ground-removed data acquired from the ground removal unit 24 and outputs the detection result. If the object detection device 20 continues the object detection process, the process returns to step S01 and continues the process, and if the object detection process ends, the process ends in step S06.

[0025] As described above, the object detection device 20 according to embodiment 1 is an object detection device 20 that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor 10 installed on the roadside, and includes a data acquisition unit 21 that acquires point cloud data from the roadside sensor 10, a ground surface estimation unit 22 that generates and outputs ground surface data indicating the position of the ground from the point cloud data acquired from the data acquisition unit 21, a ground surface data storage unit 23 that stores the ground surface data, a ground surface removal unit 24 that uses the ground surface data acquired from the ground surface data storage unit 23 to generate and output ground surface removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit 21, and an object detection unit 25 that detects an object from the ground surface removal data. The ground surface estimation unit 22 executes ground surface estimation processing at a predetermined timing to acquire point cloud data from the data acquisition unit 21 and generate ground surface data, thereby making it possible to detect objects with high accuracy while suppressing the processing load.

[0026] Embodiment 2 Fig. 4 is a block diagram showing the configuration of object detection device 20a according to embodiment 2. Comparing object detection device 20a according to embodiment 2 shown in Fig. 4 with object detection device 20 according to embodiment 1 shown in Fig. 2, disturbance detection unit 26 is added, and ground surface estimation unit 22 is replaced with ground surface estimation unit 22a. Other configurations of object detection device 20a according to embodiment 2 are the same as those of object detection device 20 according to embodiment 1. Object detection system 100a includes roadside sensor 10, disturbance sensor 11, and object detection device 20a.

[0027] The disturbance sensor 11 measures a disturbance applied to the roadside sensor 10 and outputs the magnitude of the disturbance applied to the roadside sensor 10 as a disturbance value. The disturbance sensor 11 is, for example, an acceleration sensor that measures the acceleration of the roadside sensor 10, an angular velocity sensor that measures the angular velocity of the roadside sensor 10, or a camera fixed to a support device to which the roadside sensor 10 is fixed. When the disturbance sensor 11 is an acceleration sensor, the acceleration of the roadside sensor 10 is output as the disturbance value. When the disturbance sensor 11 is an angular velocity sensor, the angular velocity of the roadside sensor 10 is output as the disturbance value. When the disturbance sensor 11 is a camera, for example, two consecutive frames of image data are acquired and the difference in brightness values ​​between the two frames of image data is output as the disturbance value, or two consecutive frames of image data are acquired and the amount of movement of the entire screen in the two frames of image data is output as the disturbance value. The disturbance detection unit 26 acquires the disturbance value from the disturbance sensor 11 and outputs a disturbance detection signal when the magnitude of the disturbance value exceeds a predetermined disturbance threshold.

[0028] When a disturbance such as an impact or vibration is applied to the roadside sensor 10, the relative positional relationship between the roadside sensor 10 and the ground may change, potentially changing the position of the ground in the point cloud data acquired from the roadside sensor 10. Therefore, if the ground removal unit 24 generates ground removal data using previously generated ground data when the roadside sensor 10 is subjected to a disturbance such as an impact or vibration, the accuracy of the process of removing measurement data of measurement points on the ground from the point cloud data may be reduced. The object detection device 20a according to the second embodiment operates in the same manner as the object detection device 20 according to the first embodiment, and includes a disturbance detection unit 26 that outputs a disturbance detection signal when the magnitude of the disturbance value output from the disturbance sensor 11, which measures a disturbance applied to the roadside sensor 10, exceeds a predetermined disturbance threshold. The ground estimation unit 22a performs ground estimation processing by acquiring point cloud data from the data acquisition unit 21 and generating ground data when the disturbance detection signal is received. This prevents a reduction in the accuracy of the process of removing measurement data of measurement points on the ground from the point cloud data when a disturbance is applied to the roadside sensor 10. When the ground data generated by the ground estimation unit 22a that has acquired the disturbance detection signal is stored in the ground data storage unit 23, past ground data already stored in the ground data storage unit 23 may be discarded.

[0029] 5 is a flowchart illustrating the operation of object detection device 20a according to embodiment 2. In FIG. 5, the processes from step S01 to step S06 are the same as steps S01 to S06 of object detection device 20 according to embodiment 1 shown in FIG. 3. Step S07 is a disturbance value acquisition step, and step S08 is a disturbance detection step. Object detection device 20a repeatedly performs the processes from step S01 to step S06, including steps S07 and S08.

[0030] In step S01, the data acquisition unit 21 acquires point cloud data from the roadside sensor 10, and then proceeds to step S07. In step S07, the disturbance detection unit 26 acquires a disturbance value from the disturbance sensor 11, which measures a disturbance applied to the roadside sensor 10. When the magnitude of the disturbance value exceeds a predetermined disturbance threshold, the disturbance detection unit 26 outputs a disturbance detection signal, and then proceeds to step S08. In step S08, the ground surface estimation unit 22a checks whether the disturbance detection signal has been output from the disturbance detection unit 26. If the disturbance detection signal has been output, the process proceeds to step S03; if the disturbance detection signal has not been output, the process proceeds to step S02. The processing from step S02 to step S06 is the same as the processing in the object detection device 20 shown in FIG. 3.

[0031] As described above, the object detection device 20a according to embodiment 2 includes a disturbance detection unit 26 that acquires a disturbance value from the disturbance sensor 11 that measures a disturbance applied to the roadside sensor 10, and outputs a disturbance detection signal when the magnitude of the disturbance value exceeds a predetermined disturbance threshold, and the ground estimation unit 22a acquires point cloud data from the data acquisition unit 21 when it acquires the disturbance detection signal and generates ground data, thereby preventing a decrease in the accuracy of the process of removing measurement data of measurement points on the ground from the point cloud data when a disturbance is applied to the roadside sensor 10.

[0032] Embodiment 3 6 is a block diagram showing the configuration of a driving assistance system 110 according to the third embodiment. The driving assistance system 110 according to the third embodiment includes the object detection device 20 and the driving assistance information generation device 30 according to the first embodiment. The driving assistance system 110 may also include the object detection device 20a and the driving assistance information generation device 30 according to the second embodiment. In the following, the driving assistance system 110 will be described as including the object detection device 20 and the driving assistance information generation device 30 according to the first embodiment.

[0033] The driving assistance information generation device 30 includes a detection result collection unit 31, a mapping unit 32, a movement range information generation unit 33, a driving assistance information generation unit 34, and a driving assistance information output unit 35. The detection result collection unit 31 collects object detection information output by the object detection unit 25 of the object detection device 20. The mapping unit 32 generates mapping information that maps the positions of objects present in the detection area on a map based on the object detection information collected by the detection result collection unit 31. The movement range information generation unit 33 uses the mapping information generated by the mapping unit 32 to generate movement range information that indicates the range within which the object is expected to move. The driving assistance information generation unit 34 uses the movement range information generated by the movement range information generation unit 33 to generate driving assistance information for the object. The driving assistance information output unit 35 outputs the driving assistance information generated by the driving assistance information generation unit 34 to the object.

[0034] As described above, the driving assistance system 110 according to embodiment 3 includes the object detection device 20, the detection result collection unit 31 that acquires object detection information from the object detection unit 25, the mapping unit 32 that generates mapping information that maps the position of the object on a map based on the object detection information collected by the detection result collection unit 31, the movement range information generation unit 33 that uses the mapping information to generate movement range information that indicates the range within which the object is expected to move, the driving assistance information generation unit 34 that uses the movement range information to generate driving assistance information for the object, and the driving assistance information output unit 35 that outputs the driving assistance information to the object. As a result, driving assistance for the object can be provided using the object detection information generated by the object detection device with a reduced processing load.

[0035] FIG. 7 is a schematic diagram showing an example of the hardware configuration of the object detection system 100 according to the first embodiment. The data acquisition unit 21, the ground surface estimation unit 22, the ground surface removal unit 24, and the object detection unit 25 are realized by a processor 201, such as a CPU (Central Processing Unit), that executes programs stored in a memory 202. The memory 202 is also used as a temporary storage device for each process executed by the processor 201. The memory 202 is, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM, a magnetic disk, an optical disk, or a combination thereof. The ground surface data storage unit 23 is realized by the memory 202. The processor 201, the memory 202, and the roadside sensor 10 are connected to one another via a bus.

[0036] FIG. 8 is a schematic diagram showing another example of the hardware configuration of the object detection system 100 according to the first embodiment. In FIG. 8, a processing circuit 204 and a roadside sensor 10 are connected to a bus 203. When the processing circuit 204 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each of the functions of the object detection device 20 may be realized by the processing circuit 204, or the functions may be realized collectively by the processing circuit 204. Furthermore, some of the functions of the object detection device 20 may be realized by dedicated hardware, and other parts may be realized by software or firmware.

[0037] FIG. 9 is a schematic diagram showing an example of the hardware configuration of an object detection system 100a according to the second embodiment. The data acquisition unit 21, the ground surface estimation unit 22a, the ground surface removal unit 24, the object detection unit 25, and the disturbance detection unit 26 are implemented by a processor 201, such as a CPU (Central Processing Unit), that executes programs stored in a memory 202. The memory 202 is also used as a temporary storage device for each process executed by the processor 201. The memory 202 is, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM, a magnetic disk, an optical disk, or a combination thereof. The ground surface data storage unit 23 is implemented by the memory 202. The processor 201, the memory 202, the roadside sensor 10, and the disturbance sensor 11 are connected to one another via a bus.

[0038] FIG. 10 is a schematic diagram showing another example of the hardware configuration of the object detection system 100a according to the second embodiment. In FIG. 8, the processing circuit 204, the roadside sensor 10, and the disturbance sensor 11 are connected to a bus 203. When the processing circuit 204 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each of the functions of the object detection device 20a may be realized by the processing circuit 204, or the functions may be realized collectively by the processing circuit 204. Furthermore, some of the functions of the object detection device 20a may be realized by dedicated hardware, and the other parts may be realized by software or firmware.

[0039] FIG. 11 is a schematic diagram showing an example of a hardware configuration of a driving assistance information generation device 30 according to the third embodiment. The detection result collection unit 31, the mapping unit 32, the movement range information generation unit 33, and the driving assistance information generation unit 34 are realized by a processor 201 such as a CPU (Central Processing Unit) that executes programs stored in a memory 202. The memory 202 is also used as a temporary storage device for each process executed by the processor 201. The memory 202 is, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM, a magnetic disk, an optical disk, or a combination thereof. The driving assistance information output unit 35 is realized by a transmitter 205. The processor 201, the memory 202, and the transmitter 205 are connected to one another via a bus.

[0040] FIG. 12 is a schematic diagram showing another example of the hardware configuration of the driving assist information generation device 30 according to the third embodiment. The driving assist information output unit 35 is implemented by a transmitter 205. In FIG. 12, the processing circuit 204 and the transmitter 205 are connected to a bus 203. When the processing circuit 204 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The detection result collection unit 31, the mapping unit 32, the movement range information generation unit 33, and the driving assist information generation unit 34 may each be implemented by the processing circuit 204, or the detection result collection unit 31, the mapping unit 32, the movement range information generation unit 33, and the driving assist information generation unit 34 may be implemented collectively by the processing circuit 204. In addition, the detection result collection unit 31, mapping unit 32, movement range information generation unit 33, and driving assistance information generation unit 34 may be partially realized by dedicated hardware, and partially realized by software or firmware.

[0041] Although the present application describes various exemplary embodiments, the various features, aspects, and functions described in one or more embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are conceivable within the scope of the technology disclosed in this application, including, for example, cases where at least one component is modified, added, or omitted, and cases where at least one component is extracted and combined with a component of another embodiment.

[0042] Although the preferred embodiments have been described above in detail, the present invention is not limited to the above-described embodiments. The above-described embodiments may be modified without departing from the scope of the claims. Various modifications and substitutions can be made to the above.

[0043] Various aspects of the present disclosure are summarized below as appendices.

[0044] (Appendix 1) An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on a roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; The object detection device, wherein the ground surface estimation unit executes a ground surface estimation process at a predetermined timing to acquire the point cloud data from the data acquisition unit and generate the ground surface data. (Appendix 2) The object detection device according to claim 1, wherein the ground estimation unit executes the ground estimation process when the data acquisition unit acquires the point cloud data from the roadside sensor for the first time. (Appendix 3) the ground surface estimation unit executes the ground surface estimation process at predetermined time intervals, The object detection device described in Appendix 1 or 2, characterized in that the time interval of the ground estimation processing is longer than the time interval of the ground removal processing in which the ground removal unit acquires the point cloud data from the data acquisition unit and generates the ground removal data. (Appendix 4) The object detection device described in any one of Supplementary Notes 1 to 3, characterized in that the ground estimation unit performs the ground estimation process when it is confirmed that no moving object is present within the detection range of the roadside sensor. (Appendix 5) 5. The object detection device according to any one of claims 1 to 4, wherein the frequency of the ground processing is determined based on the reliability of the ground data generated in the past. (Appendix 6) a disturbance detection unit that acquires a disturbance value from a disturbance sensor that measures a disturbance applied to the roadside sensor, and outputs a disturbance detection signal when the magnitude of the disturbance value exceeds a predetermined disturbance threshold value; 6. The object detection device according to claim 1, wherein the ground estimation unit executes the ground estimation process when the disturbance detection signal is acquired. (Appendix 7) the ground surface estimation unit outputs the ground surface data including a reliability; The object detection device according to any one of appendices 1 to 6, characterized in that the ground removal unit generates ground removal data using the ground data with the highest reliability from among the plurality of ground data stored in the ground data storage unit. (Appendix 8) the ground surface removal unit generates one piece of synthetic ground surface data from the plurality of pieces of ground surface data stored in the ground surface data storage unit, and generates ground surface removal data using a plane indicated by the synthetic ground surface data as the ground surface; When the ground data is represented by a position and a height, the height value of the designated position in the composite ground data is an average of the height values ​​of the designated position in each of the ground data, An object detection device described in any one of Appendices 1 to 5, characterized in that when the ground data is expressed in terms of angle and distance, the distance value at a specified angle in the synthetic ground data is the average of the distance values ​​at the specified angle in each of the ground data. (Appendix 9) The roadside sensor; An object detection system comprising the object detection device according to any one of Supplementary Notes 1 to 5. (Appendix 10) The roadside sensor; the disturbance sensor; An object detection system comprising the object detection device according to Supplementary Note 6. (Appendix 11) An object detection device according to any one of Supplementary Notes 1 to 6; a detection result collection unit that acquires the object detection information from the object detection unit; a mapping unit that generates mapping information in which positions of the objects are mapped on a map based on the object detection information collected by the detection result collection unit; a movement range information generating unit that generates movement range information indicating a range in which the object is expected to move, using the mapping information; a driving assistance information generating unit that generates driving assistance information for the object using the movement range information; a driving assistance information output unit that outputs the driving assistance information to the object. [Explanation of symbols]

[0045] 10 Roadside sensor, 11 Disturbance sensor, 20, 20a Object detection device, 21 Data acquisition unit, 22, 22a Ground estimation unit, 23 Ground data storage unit, 24 Ground removal unit, 25 Object detection unit, 26 Disturbance detection unit, 30 Driving assistance information generation device, 31 Detection result collection unit, 32 Mapping unit, 33 Movement range information generation unit, 34 Driving assistance information generation unit, 35 Driving assistance information output unit, 100, 100a Object detection system, 101 Pedestrian, 102 Vehicle, 103 Ground, 110 Driving assistance system, 201 Processor, 202 Memory, 203 Bus, 204 Processing circuit, 205 Transmitter.

Claims

1. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on a roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; the ground surface estimation unit executes a ground surface estimation process at predetermined time intervals to acquire the point cloud data from the data acquisition unit and generate the ground surface data; An object detection device characterized in that the time interval of the ground estimation processing is longer than the time interval of the ground removal processing in which the ground removal unit acquires the point cloud data from the data acquisition unit and generates the ground removal data.

2. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; The object detection device is characterized in that the ground surface estimation unit executes a ground surface estimation process that acquires the point cloud data from the data acquisition unit and generates the ground surface data when it is confirmed that no moving object is present within the detection range of the roadside sensor.

3. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; the ground surface estimation unit executes a ground surface estimation process at a predetermined timing to acquire the point cloud data from the data acquisition unit and generate the ground surface data; An object detection device, wherein the frequency of the ground surface estimation process is determined based on the reliability of the ground surface data generated in the past.

4. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; the ground surface estimation unit outputs the ground surface data including a reliability; the ground removal unit generates ground removal data using the ground data having the highest reliability from among the plurality of ground data stored in the ground data storage unit; and The object detection device, wherein the ground surface estimation unit executes a ground surface estimation process at a predetermined timing to acquire the point cloud data from the data acquisition unit and generate the ground surface data.

5. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; the ground surface removal unit generates one piece of synthetic ground surface data from the plurality of pieces of ground surface data stored in the ground surface data storage unit, and generates ground surface removal data using a plane indicated by the synthetic ground surface data as the ground surface; When the ground data is represented by a position and a height, the height value of the designated position in the composite ground data is an average of the height values ​​of the designated position in each of the ground data, When the ground data is represented by an angle and a distance, the distance value at a specified angle in the synthetic ground data is an average of the distance values ​​at the specified angle in each of the ground data, the ground surface estimation unit executes a ground surface estimation process at a predetermined timing to acquire the point cloud data from the data acquisition unit and generate the ground surface data. Body detection device.

6. An object detection device that detects an object using point cloud data including position information of a plurality of measurement points output from a roadside sensor installed on the roadside, a data acquisition unit that acquires the point cloud data from the roadside sensor; a ground surface estimation unit that generates and outputs ground surface data indicating a position of the ground surface from the point cloud data acquired from the data acquisition unit; a ground data storage unit that stores the ground data; a ground removal unit that uses the ground data acquired from the ground data storage unit to generate and output ground removal data by removing measurement data of measurement points on the ground from the point cloud data acquired from the data acquisition unit; and an object detection unit that detects the object from the ground-removed data and outputs the detection result as object detection information; a disturbance detection unit that acquires a disturbance value from a disturbance sensor that measures a disturbance applied to the roadside sensor, and outputs a disturbance detection signal when the magnitude of the disturbance value exceeds a predetermined disturbance threshold value; The object detection device is characterized in that the ground surface estimation unit executes a ground surface estimation process, when it acquires the disturbance detection signal from the disturbance detection unit, to acquire the point cloud data from the data acquisition unit and generate the ground surface data.

7. The roadside sensor; An object detection system comprising: the object detection device according to any one of claims 1 to 5.

8. The roadside sensor; the disturbance sensor; An object detection system comprising: the object detection device according to claim 6 .

9. The object detection device according to any one of claims 1 to 5; a detection result collection unit that acquires the object detection information from the object detection unit; a mapping unit that generates mapping information in which positions of the objects are mapped on a map based on the object detection information collected by the detection result collection unit; a movement range information generating unit that generates movement range information indicating a range in which the object is expected to move, using the mapping information; a driving assistance information generating unit that generates driving assistance information for the object using the movement range information; a driving assistance information output unit that outputs the driving assistance information to the object.

10. The object detection device according to claim 6 ; a detection result collection unit that acquires the object detection information from the object detection unit; a mapping unit that generates mapping information in which positions of the objects are mapped on a map based on the object detection information collected by the detection result collection unit; a movement range information generating unit that generates movement range information indicating a range in which the object is expected to move, using the mapping information; a driving assistance information generating unit that generates driving assistance information for the object using the movement range information; a driving assistance information output unit that outputs the driving assistance information to the object.

Images