Obstacle detection system

Abstract

To provide an obstacle detection system which materializes reduction of the burden of arithmetic processing, and furthermore enables misdetection to be suppressed.SOLUTION: The obstacle detection system performs angle calculation for each pixel on the basis of distance image data acquired by a distance image sensor 1 to thereby detect the presence of an obstacle. The distance information of each pixel obtained when a flat ground surface is photographed, is obtained as reference data from the installed position of the distance image sensor, reference data is excluded from distance measurement data acquired by the distance image sensor, and orthogonal coordinate conversion is performed on the distance measurement data of the distance image sensor.SELECTED DRAWING: Figure 1

Description

Technical Field

[0005] , ,

[0004] , , ,

[0001] The present invention relates to an obstacle detection system that detects the presence or absence of obstacles by calculating an angle for each pixel based on distance image data (polar coordinate data) acquired by a distance image sensor.

Background Art

[0002] Patent Document 1 describes a ground detection method and a ground detection device for discriminating the ground from point cloud data. Further, Patent Document 2 describes a construction machine that includes a distance measuring sensor for detecting an obstacle and suppresses false detection due to its own reflection.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the case of determination using point cloud data as in the above Patent Document 1, since it is necessary to perform angle calculations such as orthogonal coordinate conversion for each pixel, the amount of calculation increases. Further, when projecting point cloud data onto a mesh, it is necessary to allocate memory even to a portion where the point cloud is not projected (empty space), so the amount of memory used also increases. For this reason, there is a problem that the burden of arithmetic processing is large. <o000029>

[0005] On the other hand, in Patent Document 2, while suppressing the erroneous determination of the lower traveling body of a construction machine as an obstacle, it is possible to detect an obstacle near the lower traveling body. However, at the site where a construction machine is used, there is a possibility of erroneously detecting a sand pile or a slope of the ground as an obstacle.

[0006] This invention has been made in view of the above circumstances, and its objective is to provide an obstacle detection system that can reduce the burden of computational processing while suppressing false detections. [Means for solving the problem]

[0007] An obstacle detection system according to one aspect of the present invention is an obstacle detection system that detects the presence or absence of an obstacle by performing angle calculations for each pixel based on distance measurement data acquired by a distance image sensor, wherein the distance information of each pixel when a flat ground is photographed from the installation position of the distance image sensor is obtained as reference data, and the reference data and the distance measurement data acquired by the distance image sensor and The difference between the distance measurement data and the reference data is compared pixel by pixel. value Take the difference using the nearest pixels. value The amount of change combines pixels within a predetermined threshold, and the difference value Pixels where the value is approximately 0 are designated as ground pixels. For neighboring pixels of a ground pixel, if that ground pixel is an inflection point of the difference value, it will not be merged with that ground pixel. The method is characterized by excluding the ground pixels and pixels that are coupled with the ground pixels from the distance measurement data, and then performing an orthogonal coordinate transformation on the distance measurement data after exclusion. [Effects of the Invention]

[0008] According to the present invention, by removing reference data from distance measurement data acquired by a distance image sensor, the amount of computation and memory usage can be reduced. Furthermore, by removing reference data from the distance measurement data, false detections can be suppressed by differentiating the ground from the detection target. Therefore, an obstacle detection system can be provided that reduces the burden of computational processing while suppressing false detections. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing a schematic configuration of an obstacle detection system according to an embodiment of the present invention. [Figure 2] This flowchart shows the procedure for creating standard data, which is performed in the standard data creation unit. [Figure 3]This is a schematic diagram illustrating the creation of reference data performed by the reference data creation unit. [Figure 4] This flowchart shows the procedure for selecting pixels to be excluded, which is performed in the excluded pixel selection unit. [Figure 5] This diagram illustrates the first comparison method, showing the detected and undetected items. [Figure 6] This diagram illustrates the differences between detected and undetected objects, illustrating the first comparison method. [Figure 7] This diagram illustrates the differences between detected and undetected objects, illustrating the second comparison method. [Figure 8] This is a schematic diagram illustrating the process of creating a 2D image from actual distance measurement data. [Figure 9] This diagram shows a simplified 2D image representing the difference between virtual ground data and actual distance data. [Figure 10] This figure shows a simplified 2D image of obstacles other than the ground that remain as difference values. [Figure 11] This is a simplified 2D image illustrating areas with large gaps between surrounding pixels. [Figure 12] This diagram is used to determine the reference point for ground pixels and explains the difference in depth values ​​at each pixel. [Figure 13] This diagram shows a simplified 2D image representing the difference between virtual ground data and actual distance data. [Figure 14] This figure shows a simplified 2D image where pixels with small gaps near the ground pixels are combined to form the ground pixels. [Figure 15] This is a simplified 2D image showing the detection target data that was not combined with ground data. [Figure 16] This diagram is intended to explain the exclusion of ground pixels, and it shows a simplified 2D image in which pixels identified as ground pixels have been excluded from the detection target data. [Figure 17] This figure shows the distance measurement data detected by the distance image sensor. [Figure 18] It is a diagram showing the ground ranging data extracted from FIG. 17. [Figure 19] It is a diagram showing the ranging data of the detection target extracted from FIG. 17.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a schematic configuration of an obstacle detection system according to an embodiment of the present invention. This system includes a distance image sensor 1 such as a ToF (Time of Flight) sensor that acquires ranging data, a reference data creation unit 2 that obtains distance information of each pixel when a flat ground (virtual ground) is photographed from the installation position of the distance image sensor 1 as reference data, an excluded pixel selection unit 3 that selects excluded pixels, and a detection processing unit 4 that detects obstacles such as ground irregularities and people.

[0011] The excluded pixel selection unit 3 creates comparison data for comparing the ranging data acquired by the distance image sensor 1 with the distance information of each pixel obtained by the reference data creation unit 2, excludes the ground data from the ranging data based on the created comparison data, and outputs it to the detection processing unit 4. In this way, every time the distance image sensor 1 acquires distance information, the ranging data is compared with the reference data.

[0012] FIG. 2 shows the creation procedure of the reference data executed by the reference data creation unit 2. The creation of this reference data is carried out only once and is not performed for each frame. First, information on the mounting position and angle of the distance image sensor 1 is acquired (step S1). Subsequently, the distance from each pixel from the mounting position and angle of the distance image sensor 1 to the virtual ground is calculated, and reference data is created (step S2). That is, at the time of activation of the distance image sensor, the output result (distance value at each pixel) of the distance image sensor when photographing an ideal (flat) virtual ground without inclination and undulation from the installation position and angle of the distance image sensor is calculated, and the distance information of each pixel is used as reference data.

[0013] Figure 3 is a schematic diagram illustrating the creation of reference data performed by the reference data creation unit 2. Point cloud data is projected onto a mesh, and the unprojected areas become empty space. When creating a 2D image of a virtual ground surface without slope or undulation, the projection onto the plane will have a shape with pincushion distortion, as shown in Figure 3(a). As a result, the 2D image will look like Figure 3(b).

[0014] Figure 4 shows the procedure for selecting pixels to be excluded, which is performed by the exclusion pixel selection unit 3. This process is performed each time distance measurement data is acquired. First, distance measurement data is acquired by the distance image sensor 1 (step S11), and invalid pixels are detected to exclude abnormal values ​​in the distance measurement data (step S12).

[0015] Next, comparison data is created by comparing the reference data created using the procedure in Figure 2 with the distance measurement data created in step S12 (step S13). Subsequently, neighboring pixels with small changes in the comparison data are combined (step S14). Next, ground pixel determination is performed (step S15). In ground pixel determination, pixels with a small difference between the reference data and the distance measurement data (comparison data ≈ 0) are considered ground pixels, and pixels combined with those pixels are also considered ground pixels.

[0016] Specifically, pixels whose relative change amount is within a predetermined threshold are considered ground pixels and excluded from the distance measurement data. Furthermore, pixels whose relative change amount is less than the predetermined threshold are recognized as ground slopes, and pixels whose change amount is greater are detected as obstacles. Subsequently, exclusion pixel determination is performed, and ground pixels and abnormal pixels are excluded from the obstacle detection target (step S16).

[0017] Figures 5 and 6 illustrate the first comparison method, respectively. Assuming that the detected object is a crouching person as shown in Figure 5(a) and the undetected object is a sloping surface such as a sand dune as shown in Figure 5(b), the depth difference from the reference surface will be as shown in Figures 6(a) and (b), respectively. Figure 6(a) shows the depth difference between the crouching person and the ground, which is the reference surface, and as indicated by the arrows, the transition of the distance measurement data is not smooth. In contrast, as shown in Figure 6(b), the depth difference between the sloping surface and the ground shows a smooth transition of distance measurement data, as indicated by the arrows.

[0018] Thus, the first comparison method involves taking the difference between the distance measurement data and the reference data, "reference data - distance measurement data". This first comparison method is suitable when high processing speed is required. However, its effectiveness decreases (the error in the results for each pixel increases) when the resolution of the distance image sensor 1 is low or when the laser irradiation angle is close to horizontal.

[0019] Figure 7 illustrates the second comparison method, showing the difference between detected and undetected objects. This second comparison method involves taking the difference between the distance measurement data and the reference data, and then dividing that difference by the reference data: "([Reference Data] - [Distance Measurement Data]) / [Reference Data]". Figure 7(a) shows the height difference between a crouching person and the reference plane (the ground), and as indicated by the arrows, the transition of the distance measurement data is not smooth.

[0020] In contrast, as shown in Figure 7(b), the difference in height between the inclined surface and the ground results in a smoother transition of distance measurement data, as indicated by the arrows. Thus, while calculating the height difference from the reference plane results in a slower processing speed compared to the first comparison method (1) due to the division, it allows calculations to be performed on the same scale for every pixel.

[0021] Figure 8 is a schematic diagram illustrating the process of creating a 2D image from actual distance measurement data. As shown in Figure 8(a), when the detected object is a crouching person, acquiring distance measurement data yields a 2D image like the one shown in Figure 8(b). This 2D image is an image in which the reflected light from the crouching person is superimposed in the foreground, with the 2D image shown in Figure 3(b) as the background.

[0022] Next, by taking the difference between the actual distance data shown in Figure 8(b) and the virtual ground data shown in Figure 3(b), a 2D image like the one shown in Figure 9 is obtained. If the ground is flat (no slope or undulation), the difference value will be "0". On the other hand, obstacles other than the ground remain as difference values ​​as shown in Figure 10, so check the areas with large gaps with surrounding pixels. Figure 11 shows areas with large gaps with surrounding pixels marked with an "x".

[0023] Furthermore, in areas with large gaps between pixels, pixels whose change amount from nearby pixels falls within a predetermined threshold are considered ground pixels and excluded from the distance measurement data. In other words, pixels whose change amount from nearby pixels is less than a predetermined threshold are recognized as slopes on the ground, while pixels whose change amount is larger are detected as obstacles. To put it another way, obstacles that can be overcome are excluded, while obstacles that cannot be overcome are detected as obstacles.

[0024] More specifically, when a particular ground pixel is used as a reference, the modification of a neighboring pixel to a ground pixel is only permitted if the conditions for modification are met for both the pixels above / below and the pixels to the left / right of that ground pixel. The conditions for modification of neighboring pixels are when one of the difference values ​​between the pixels above / below (or left / right) is the same, and the other is the same, increases (including sudden increases), or decreases. Also, when one of the difference values ​​between the pixels above / below (or left / right) is increasing (including sudden increases) and the other is decreasing. Here, the equivalent difference values ​​of pixels mean that the amount of change between adjacent pixels is within a predetermined threshold.

[0025] In contrast, if the reference ground pixel is at an inflection point (where the difference between the upper and lower pixels increases and increases (or decreases and decreases), or where the difference between the left and right pixels increases and increases (or decreases and decreases)), the neighboring pixels cannot be changed. Also, if the upper and lower (or left and right) pixels include a sharp decrease, the neighboring pixels cannot be changed.

[0026] Figure 12 shows the difference in depth values ​​for each pixel, used to determine the reference ground pixel. Figure 13 is a 2D image showing the difference between virtual ground data and actual distance data. If the difference value is within a predetermined threshold range, it is designated as the reference ground pixel. In Figure 13, pixel 11 is determined to be the reference ground pixel.

[0027] Figure 14 shows the ground pixels, formed by combining pixels 12-1 to 12-4, which have small gaps near the ground pixels, into the ground reference pixels 11. Only data that can be considered as ground pixels is combined, and as shown in Figure 15, data that is not combined with the ground data is designated as detection target data (pixels 13 with a downward-sloping hatch).

[0028] Figure 16 illustrates the exclusion of ground pixels, showing that pixel 13, which was identified as a "ground pixel," has been excluded from the data being detected. In Figure 16, if any gaps or other issues occur, this process does not remove them; instead, the detection process is left to a later stage.

[0029] Figure 17 shows the distance measurement data detected by the distance image sensor 1 described above. Figure 18 shows the distance measurement data of the ground (sloping surface) extracted from Figure 17, and Figure 19 shows the detected object (person) extracted from Figure 17. As shown in Figure 19, by excluding the distance measurement data of the ground (sloping surface) from the distance measurement data detected by the distance image sensor 1, the amount of data to be detected as an obstacle can be significantly reduced.

[0030] As described above, the present invention enables processing without complex calculations, resulting in high throughput. Furthermore, memory usage can be limited to only the number of pixels used by the sensor. Moreover, even when point cloud creation or other processes are performed in subsequent detection processing, the amount of computation can be minimized because data can be thinned out beforehand. In addition, it can suppress the misdetection of sand dunes, ground slopes, etc., as obstacles.

[0031] Therefore, according to the obstacle detection system of the present invention, by excluding data from the ground portion, the amount of computation and memory usage per frame is reduced, thereby alleviating the burden on computational processing, while differentiating the ground from the detection target can suppress false detections.

[0032] The configurations and control procedures described in the above embodiments are merely schematic representations to the extent that the present invention can be understood and implemented. Therefore, the present invention is not limited to the described embodiments and can be modified in various forms as long as it does not deviate from the scope of the technical idea set forth in the claims.

[0033] For example, an obstacle detection system could be installed on construction machinery, and the system could further exclude the distance image data of the construction machinery itself that is captured in the distance image data acquired by the distance image sensor.

[0034] Furthermore, since the light intensity tends to be low in edge data, detection accuracy can be improved by utilizing the attenuation of light intensity values ​​at edges to differentiate the detected object from ground data through edge extraction. Distance image sensors (MEMS: Micro Electro Mechanical Systems) can acquire distance data and light intensity data coaxially. The distance value is measured by the time from laser emission to reception of reflected light, and the light intensity value is measured by the amount of light from the received laser.

[0035] Furthermore, it is possible to differentiate between detection targets and ground data by training an AI with composite data of distance image data and light intensity data. Since significant features can be detected in both distance images and light intensity images, AI training can improve accuracy compared to using data from distance images alone. [Explanation of Symbols]

[0036] 1...Distance image sensor, 2...Reference data creation unit, 3...Exclusion pixel selection unit, 4...Detection processing unit, 11...Pixel (ground reference pixel), 12-1~12-4...Pixel (small gap pixel), 13...Pixel (detection target data)

Claims

1. An obstacle detection system that detects the presence or absence of an obstacle by performing angle calculations for each pixel based on distance measurement data acquired by a distance image sensor, An obstacle detection system characterized by: obtaining distance information for each pixel when photographing a flat ground surface from the installation position of the distance image sensor as reference data; comparing the reference data with the distance measurement data acquired by the distance image sensor for each pixel; taking the difference value between the distance measurement data and the reference data; combining neighboring pixels where the change in the difference value is within a predetermined threshold; designating pixels with a difference value of approximately 0 as ground pixels; not combining neighboring pixels of ground pixels if the ground pixel is an inflection point of the difference value; excluding the ground pixels and pixels that would otherwise be combined with the ground pixels from the distance measurement data; and performing an orthogonal coordinate transformation on the distance measurement data after exclusion.

2. The obstacle detection system according to claim 1, characterized in that each time the distance image sensor acquires distance measurement data, it compares the reference data with the distance measurement data pixel by pixel and excludes the ground pixels and pixels that combine with the ground pixels from the distance measurement data.

3. The obstacle detection system according to claim 1, characterized in that pixels whose difference value change from neighboring pixels is smaller than the predetermined threshold are recognized as slopes on the ground, and pixels whose difference value is larger are detected as obstacles.

4. The obstacle detection system according to claim 1, characterized in that, after taking the difference value between the reference data and the distance measurement data, the difference value is divided by the reference data.

5. The obstacle detection system according to any one of claims 1 to 4 above is mounted on a construction machine, An obstacle detection system characterized by further excluding the distance measurement data of the construction machine itself from the distance measurement data acquired by the distance measurement image sensor.

Images