Object detection device
The object detection device addresses specular reflection issues by converting coordinates of points below the ground surface and using map data to enhance self-position estimation and object recognition accuracy.
Patent Information
- Application Number
- JP2023075613
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
AI Technical Summary
Existing object detection devices face issues with specular reflection, leading to incorrect recognition of object information due to reduced point cloud data and decreased self-position estimation accuracy.
An object detection device that uses a laser beam changing irradiation angles to measure distances and convert coordinates of points below the ground surface, incorporating map data to estimate self-position and account for specular reflections.
This approach maintains the number of points in point cloud data, enhancing self-position estimation accuracy and enabling accurate object recognition.
Smart Images

Figure 2024160519000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an object detection device. [Background technology]
[0002] The object detection device includes a laser range finder and a control device. The laser range finder emits a laser beam at each predetermined irradiation angle. When the laser beam emitted from the laser range finder hits an object, the laser beam is reflected. When the laser range finder receives the laser beam reflected by the object, it calculates the distance to the point where the laser beam hits from the time required from emitting the laser beam to receiving the laser beam. The control device acquires point cloud data from the laser beam. The point cloud data is data in which the positions of the points where the laser beam hits are represented by points. The control device recognizes information about the object from the point cloud data. The information about the object is, for example, the distance to the object or the shape of the object.
[0003] When specular reflection occurs due to the laser light being incident on the water surface, the specularly reflected laser light is reflected by the object and then received by the laser range finder. Therefore, when specular reflection occurs, the time from the emission of the laser light to the reception of the laser light is longer than when specular reflection does not occur. In this case, there is a risk that the control device will not be able to correctly recognize information about the object.
[0004] The object detection device disclosed in Patent Document 1 estimates its own position from the detection result of a laser range finder. The object detection device removes points obtained by specularly reflected laser light from the point cloud data. This prevents the points obtained by specularly reflected laser light from reducing the accuracy of the estimation of the self-position. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2022-129058 A Summary of the Invention [Problem to be solved by the invention]
[0006] In Patent Document 1, points obtained by specularly reflected laser light are removed. This reduces the number of points included in the point cloud data. This may lead to a problem that information about the object cannot be correctly recognized. [Means for solving the problem]
[0007] An object detection device that solves the above problem is an object detection device equipped on a moving body, and includes a laser rangefinder that emits laser light while changing the irradiation angle relative to the vertical direction to measure the distance to the point where the laser light hits, and a control device that acquires point cloud data, which is a collection of points, from the laser rangefinder, and the control device extracts from the point cloud data the points that are located below the ground surface in a coordinate system that represents the positions of the points, and converts the coordinates of the points that are located below the ground surface into coordinates of a position that is plane-symmetrical with respect to the ground surface.
[0008] When a laser beam is irradiated below the laser rangefinder and the laser beam is specularly reflected, the point obtained indicates coordinates below the ground surface. Therefore, by extracting points located below the ground surface in a coordinate system representing the position of a point from the point cloud data, the point obtained by the specularly reflected laser beam can be extracted. The control device converts the coordinates of the points located below the ground surface into coordinates of positions that are symmetrical with respect to the ground surface. The coordinates obtained by this conversion indicate the position of the point where the specularly reflected laser beam hit the object. By converting the coordinates of the points included in the point cloud data, the number of points included in the point cloud data is prevented from decreasing compared to the case of removing points from the point cloud data.
[0009] In the above object detection device, the control device may extract, from the point cloud data, the points located below the ground surface, based on a height from the ground surface to the laser range finder. The above-mentioned object detection device may be provided with a storage device storing map data indicating the environment in which the moving body is used, and the control device may estimate the self-position of the moving body and extract from the point cloud data the points located below the ground surface from the map data.
[0010] The above-mentioned object detection device may be provided with a memory device storing map data indicating the environment in which the moving body is used, the map data including position information of a mirror reflection unit that mirror-reflects the laser light, and the control device may estimate the self-position of the moving body, and when the mirror reflection unit is located within a predetermined range from the self-position, convert the coordinates of the point located below the ground surface into coordinates of a position that is plane-symmetrical with respect to the ground surface. Effect of the Invention
[0011] According to the present invention, a reduction in the number of points included in point cloud data is suppressed. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a schematic diagram of an environment in which a moving object is used. [Diagram 2] FIG. 2 is a schematic configuration diagram of the moving body of FIG. [Diagram 3] 3 is a flowchart showing a self-position estimation control executed by the control device of FIG. 2. [Figure 4] FIG. 2 is a schematic diagram of an environment in which the moving object of FIG. 1 is used. [Diagram 5] 3 is a flowchart showing a self-position estimation control executed by the control device of FIG. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] [First embodiment] A first embodiment of an object detection device will be described. As shown in Fig. 1, an environment A1 in which a moving object 10 is used includes a ground surface 20. The moving object 10 is, for example, a vehicle. The vehicle is, for example, a passenger car or an industrial vehicle. The industrial vehicle is, for example, a forklift, a towing tractor, or a transport vehicle. The ground surface 20 is a road surface on which the moving object 10 moves.
[0014] The environment A1 includes a specular reflection portion 21. The specular reflection portion 21 is located on the ground surface 20. The specular reflection portion 21 is an object that can cause specular reflection. In this embodiment, the specular reflection portion 21 occurs accidentally. The specular reflection portion 21 is, for example, a puddle that exists on the ground surface 20. The environment A1 includes at least one object 31. The object 31 is, for example, a structure that is unlikely to change position. The structure is, for example, a wall, a pillar, or a roof. Although one object 31 is illustrated in FIG. 1, the environment A1 may include multiple objects 31.
[0015] <Object detection device> As shown in FIG. 2, the moving body 10 includes an object detection device 30. The object detection device 30 includes a control device 11. The control device 11 includes a processor 12 and a storage unit 13. The processor 12 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). The storage unit 13 includes a random access memory (RAM) and a read only memory (ROM). The storage unit 13 stores program codes or instructions configured to cause the processor 12 to execute processing. The storage unit 13, i.e., the computer-readable medium, includes any available medium accessible by a general-purpose or dedicated computer. The control device 11 may be configured by a hardware circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control device 11, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or an FPGA, or a combination thereof.
[0016] The object detection device 30 includes a laser range finder 15. The laser range finder 15 may also be referred to as a LIDAR (Laser Imaging Detection and Ranging). The laser range finder 15 detects the position of the object 31 in coordinates of a three-dimensional coordinate system. As shown in FIG. 1, the laser range finder 15 irradiates the surroundings with a laser light L1. The laser range finder 15 irradiates the laser light L1 while changing the irradiation angle with respect to the vertical direction according to the angular resolution. The laser range finder 15 of this embodiment irradiates the laser light L1 while changing the irradiation angle with respect to both the vertical direction and the horizontal direction. The laser range finder 15 measures the distance to the points P1, P2 where the laser light L1 hits from the time it takes to receive the reflected light reflected from the points P1, P2 where the irradiated laser light L1 hits. In detail, the laser range finder 15 measures the distance to the object 31 at the irradiation angle where the laser light L1 was irradiated from the time it takes to receive the laser light L1 when the laser light L1 was irradiated.
[0017] The points P1 and P2 where the laser light L1 hits represent a part of the surface of the object 31. The positions of the points P1 and P2 where the laser light L1 hits can be expressed by coordinates of a polar coordinate system using the irradiation angle of the laser light L1 and the distance to the points P1 and P2 where the laser light L1 hits. The coordinates of the points P1 and P2 where the laser light L1 hits in the polar coordinate system are converted to coordinates of a Cartesian coordinate system. The laser range finder 15 derives the coordinates of the points P1 and P2 where the laser light L1 hits in the sensor coordinate system. The sensor coordinate system is a three-axis Cartesian coordinate system with the laser range finder 15 as the origin. The sensor coordinate system is, for example, a coordinate system in which the horizontal position is represented by the X-axis and the Y-axis which are orthogonal to each other, and the vertical position is represented by the Z-axis which is orthogonal to the X-axis and the Y-axis. The laser range finder 15 outputs points P11 and P21 which indicate the coordinates in the sensor coordinate system of the points P1 and P2 where the laser light L1 hits as point cloud data PC. The sensor coordinate system is a coordinate system that represents the positions of points P11 and P21.
[0018] The object detection device 30 includes a storage device 16. The storage device 16 stores information that can be read by the control device 11. The storage device 16 is, for example, a hard disk drive or a solid state drive. The storage device 16 stores map data D1.
[0019] The map data D1 is data indicating the environment A1. The map data D1 is, for example, data representing the position where the object 31 exists. In this embodiment, the map data D1 is data representing the object 31 existing in the environment A1 and the ground surface 20 in coordinates of a map coordinate system. The map coordinate system is, for example, a three-axis orthogonal coordinate system. The map coordinate system is a coordinate system with an arbitrary point in the environment A1 as its origin. The map coordinate system is, for example, a coordinate system in which a horizontal position is represented by an X-axis and a Y-axis that are orthogonal to each other, and a vertical position is represented by a Z-axis that is orthogonal to the X-axis and the Y-axis.
[0020] The control device 11 estimates the self-position of the moving body 10. The self-position is the position of the moving body 10 on the map data D1. The self-position is a coordinate indicating one point of the moving body 10 in the map coordinate system. The one point of the moving body 10 is arbitrary, but can be, for example, the center position of the moving body 10 in the horizontal direction. The control device 11 estimates the self-position by comparing the point cloud data PC with the map data D1. The self-position estimation control will be described below. The self-position estimation control is repeatedly performed at a predetermined control period.
[0021] <Self-location estimation control> As shown in Fig. 3, in step S1, the control device 11 acquires point cloud data PC from the laser range finder 15. The point cloud data PC is a collection of points P11, P21. The points P11, P21 are data that represent points P1, P2 on the surface of the object 31 where the laser light L1 hits, using coordinates in the sensor coordinate system. Fig. 1 shows an example of the point cloud data PC. The point cloud data PC shown in Fig. 1 shows an example where the laser light L1 hits the object 31. For ease of explanation, Fig. 1 shows only some of the points P11, P21.
[0022] The point cloud data PC includes a first point P11 and a second point P21. The first point P11 is a point P11 obtained by the laser light L1 hitting the object 31 without specular reflection. The second point P21 is a point P21 obtained by the laser light L1 hitting the object 31 after specular reflection at the specular reflection unit 21. As shown in FIG. 1, when the environment A1 includes the specular reflection unit 21, the laser light L1 irradiated to the specular reflection unit 21 may be specularly reflected. The specularly reflected laser light L1 is reflected by the object 31 and received by the laser range finder 15. As a result, the point cloud data PC includes a second point P21. In the example shown in FIG. 1, the first point P11 indicates the coordinates of the location P1. The second point P21 indicates the coordinates of the location P3. The coordinates of the location P3 are specifically determined by the laser light reflected by the location P2.
[0023] Next, in step S2, the control device 11 extracts points P11 and P21 located below the ground surface 20 in the sensor coordinate system from the point cloud data PC. The control device 11 extracts points P11 and P21 located below the ground surface 20 from the point cloud data PC based on the height from the ground surface 20 to the laser range finder 15. The height from the ground surface 20 to the laser range finder 15 can be known in advance from the mounting height of the laser range finder 15. That is, the Z coordinate of the ground surface 20 in the sensor coordinate system can be known in advance. The control device 11 extracts points P11 and P21 included in the point cloud data PC and located below the Z coordinate of the ground surface 20 as points P11 and P21 located below the ground surface 20. In the example shown in FIG. 1, a second point P21 is extracted as points P11 and P21 located below the ground surface 20. The Z coordinate of the ground surface 20 in the sensor coordinate system may be stored in the storage unit 13 or the storage device 16. Up and down refers to up and down in the vertical direction.
[0024] The laser range finder 15 measures the distance to the object 31 at the irradiation angle of the laser light L1 from the time required for the laser light L1 to be received when it is irradiated. As can be seen from FIG. 1, when specular reflection occurs at the specular reflecting portion 21, the reflection angle of the specularly reflected laser light L1 changes from the irradiation angle of the laser light L1. That is, the irradiation angle of the laser light L1 by the laser range finder 15 is different from the angle at which the specularly reflected laser light L1 hits the object 31. The laser range finder 15 measures the distance to the object 31 assuming that the laser light L1 hits the object 31 at the irradiation angle of the laser light L1. When the laser range finder 15 irradiates the laser light L1 downward from the laser range finder 15 and the laser light L1 hits the object 31 through specular reflection, a second point P21 located below the ground surface 20 is obtained.
[0025] A virtual object 32 is a virtual object obtained by turning the object 31 upside down with the ground surface 20 as the boundary. The second point P21 can be considered to be a point obtained when the laser light L1 hits the virtual object 32. When the laser light L1 is specularly reflected, the angle of incidence and the angle of reflection of the laser light L1 are the same. Therefore, the point P2 on the object 31 where the laser light L1 hits and the point P3 on the virtual object 32 where the second point P21 is located are the same point. In other words, when the object 31 is turned upside down with the ground surface 20 as the boundary, the points P2 and P3 overlap.
[0026] Next, in step S3, the control device 11 converts the coordinates in the sensor coordinate system of the second point P21 located below the ground surface 20 into coordinates of a position that is plane-symmetrical with respect to the ground surface 20. That is, the control device 11 converts the Z coordinate into the coordinate of a position that is upside down with the ground surface 20 as a boundary, while maintaining the X and Y coordinates of the second point P21 in the sensor coordinate system. Since the control device 11 recognizes the Z coordinate in the sensor coordinate system of the ground surface 20, it can recognize the dimension of the second point P21 in the Z-axis direction from the ground surface 20. The control device 11 only needs to convert the Z coordinate to a position above the ground surface 20 by the dimension of the second point P21 in the Z-axis direction from the ground surface 20. The second point P21 whose Z coordinate has been converted in step S3 is set as a virtual point P22. The virtual point P22 can be regarded as the coordinate in the sensor coordinate system of the point P2 where the specularly reflected laser light L1 hits the object 31. As a result, the control device 11 obtains point cloud data PC including the first point P11 and the imaginary point P22.
[0027] Next, in step S4, the control device 11 estimates its own position using the point cloud data PC including the first point P11 and the virtual point P22. The control device 11 extracts landmarks having the same shape as the landmarks obtained from the point cloud data PC from the map data D1. The control device 11 recognizes the positions of the landmarks in the map coordinate system from the map data D1. The relative position of the landmarks and the mobile body 10 can be grasped from the detection result of the laser rangefinder 15. Therefore, the control device 11 can estimate its own position by recognizing the positions of the landmarks in the map coordinate system. The landmarks are objects 31 having features that can be identified by the laser rangefinder 15.
[0028] [Operation of the first embodiment] The laser range finder 15 emits a laser beam L1. When the laser beam L1 is specularly reflected, the time from when the laser beam L1 is emitted to when the laser beam L1 is received is longer than when no specular reflection occurs. This causes a shift in the position between the point P2 where the laser beam L1 hits the object 31 and the point P21 recognized by the laser range finder 15.
[0029] When the laser light L1 is specularly reflected by the specular reflector 21 present on the ground surface 20, the obtained point P21 indicates coordinates below the ground surface 20 in the sensor coordinate system. Therefore, when the point P21 located below the ground surface 20 in the sensor coordinate system is extracted from the point cloud data PC, the point P21 obtained by the laser light L1 specularly reflected by the specular reflector 21 present on the ground surface 20 can be extracted. The control device 11 converts the coordinates of the point P21 located below the ground surface 20 into coordinates of a position that is plane-symmetrical with respect to the ground surface 20. The coordinates obtained by this conversion indicate the position of the spot where the specularly reflected laser light L1 hits the object 31.
[0030] [Effects of the first embodiment] (1-1) The control device 11 converts the coordinates of the point P21 located below the ground surface 20 into the coordinates of a position that is plane-symmetric with respect to the ground surface 20. By converting the coordinates of the point P21 included in the point cloud data PC, a decrease in the number of points P11 and P21 included in the point cloud data PC is suppressed compared to the case where points P11 and P21 are removed from the point cloud data PC.
[0031] In the present embodiment, when estimating the self-location using the point cloud data PC, the self-location is performed using the point P21 obtained by mirror reflection, so that the estimation accuracy of the self-location is prevented from decreasing. Furthermore, the number of points P11 and P21 included in the point cloud data PC is reduced, so that the estimation accuracy of the self-location is prevented from decreasing.
[0032] (1-2) The control device 11 extracts, from the point cloud data PC, a point P21 located below the ground surface 20, based on the height from the ground surface 20 to the laser range finder 15. Therefore, it is easy to extract the point P21 located below the ground surface 20.
[0033] [Second embodiment] A second embodiment of the object detection device will be described below, focusing on the differences from the first embodiment.
[0034] 4, the environment A1 includes an object 41. The object 41 is located above the moving object 10. The object 41 is, for example, a roof. The environment A1 includes a specular reflector 42. The specular reflector 42 in this embodiment is continuously present on the ground surface 20. The specular reflector 42 is, for example, intentionally placed by a user of the moving body 10. The specular reflector 42 is, for example, a mirror. The specular reflector 42 is placed such that when the moving body 10 is located under an object 41, the laser light L1 specularly reflected by the specular reflector 42 hits the object 31 serving as a landmark.
[0035] The map data D1 includes data expressing the specular reflection portion 42 in coordinates of a map coordinate system. This data is position information of the specular reflection portion 42. The control device 11 performs self-location estimation control. The self-location estimation control performed in the second embodiment is different from the self-location estimation control in the first embodiment.
[0036] <Self-location estimation control> As shown in FIG. 5, the control device 11 acquires the point cloud data PC from the laser range finder 15 in step S1.
[0037] Next, in step S11, the control device 11 judges whether the specular reflection unit 42 is located within a predetermined range from the self-position. The predetermined range is set, for example, to a range equal to or less than the reachable distance of the laser light L1. If the judgment result of step S11 is positive, the control device 11 performs the process of steps S2 to S4. The processes of steps S2 to S4 are the same as those of the first embodiment. If the judgment result of step S11 is negative, the control device 11 performs the process of step S4. That is, if the specular reflection unit 42 is located within a predetermined range from the self-position, the control device 11 estimates the self-position from the point cloud data PC including the virtual point P22 obtained by converting the coordinates of the second point P21 located below the ground surface 20 into the coordinates of a position that is plane-symmetrical with respect to the ground surface 20. If the specular reflection unit 42 is not located within the predetermined range from the self-position, the control device 11 estimates the self-position from the point cloud data PC acquired from the laser rangefinder 15 without converting the coordinates. The determination in step S11 in the next control cycle is made based on the self-position estimated in step S4.
[0038] [Effects of the second embodiment] (2-1) When the moving body 10 is located under the object 41, the laser light L1 may not hit the landmark object 31 depending on the irradiation angle due to the laser light L1 being blocked by the object 41. In this case, the estimation accuracy of the self-position may be reduced. If the mirror reflector 42 is arranged so that the laser light L1 hits the landmark object 31 by mirror reflection at a position where the laser light L1 is unlikely to hit the landmark object 31, the laser light L1 can be hit on the object 31. Compared to the case where the mirror reflector 42 is not arranged, the number of points P21 included in the point cloud data PC can be increased. Then, when the mirror reflector 42 is located within a predetermined range from the self-position, the control device 11 converts the coordinates of the point P21 located below the ground surface 20 into coordinates of a position that is plane-symmetrical with respect to the ground surface 20. By providing the mirror reflector 42, it is possible to obtain the point P21 that cannot be obtained when the mirror reflector 42 is not provided. Then, the self-position can be estimated using the virtual point P22 obtained from this point P21. This improves the estimation accuracy of the self-position.
[0039] [Example of change] The embodiment can be modified as follows: The embodiment and the following modified examples can be combined with each other to the extent that there is no technical contradiction.
[0040] In each embodiment, the control device 11 may detect an obstacle instead of estimating the self-position. For example, the control device 11 performs clustering. Clustering is grouping the points P11 and P21 that can be regarded as the same object among the points P11 and P21 included in the point cloud data PC. The control device 11 regards the points P11 and P21 grouped by the clustering as one obstacle. When the control device 11 detects an obstacle, the control device 11 may control the moving body 10 to avoid the obstacle. The control device 11 performs clustering using the point cloud data PC including the first point P11 and the virtual point P22. This makes it possible to recognize the position and shape of the obstacle with high accuracy. In this case, the control device 11 may estimate the self-position without using the point cloud data PC. For example, the control device 11 may estimate the self-position by dead reckoning, or may estimate the self-position using a satellite signal transmitted from a GNSS (Global Navigation Satellite System) satellite.
[0041] In each embodiment, the control device 11 may extract a point P21 located below the ground surface 20 from the point cloud data PC of the map data D1. The control device 11 can recognize the height of the ground surface 20 at the position where the object 31 exists from the map data D1. For example, the control device 11 recognizes the height of the ground surface 20 in the sensor coordinate system from the difference between the Z coordinate of the ground surface 20 in the map coordinate system and the origin of the Z axis in the sensor coordinate system. The control device 11 extracts a point P21 located below the ground surface 20 from the point cloud data PC based on the height of the ground surface 20 in the sensor coordinate system. Then, the control device 11 converts the coordinate of the point P21 located below the ground surface 20 into the coordinate of a position that is plane-symmetrical with respect to the ground surface 20. At this time, the control device 11 converts the coordinate of the point P21 using the height of the ground surface 20 at the position where the object 31 exists. By using the map data D1, even if the Z coordinate of the ground surface 20 varies depending on the position, the Z coordinate of the ground surface 20 according to the position can be obtained. Therefore, the imaginary point P22 can be detected with high accuracy. [Explanation of symbols]
[0042] A1...environment, D1...map data, P11, P21...points, PC...point cloud data, 10...mobile body, 11...control device, 15...laser rangefinder, 16...storage device, 20...ground surface, 30...object detection device, 31...object, 42...mirror reflector.
Claims
1. An object detection device provided on a moving object, a laser range finder that irradiates a laser beam while changing an irradiation angle with respect to a vertical direction and measures a distance to a point where the laser beam hits; a control device that acquires point cloud data, which is a set of points, from the laser range finder; The control device includes: extracting, from the point cloud data, the points located below the ground surface in a coordinate system representing the positions of the points; The object detection device converts the coordinates of the point located below the ground surface into coordinates of a position that is plane-symmetric with respect to the ground surface.
2. The object detection device according to claim 1 , wherein the control device extracts the points located below the ground surface from the point cloud data based on a height from the ground surface to the laser range finder.
3. a storage device storing map data indicating an environment in which the moving object is used; The control device includes: Estimating a self-position of the moving body; The object detection device according to claim 1 , wherein the points located below the ground surface are extracted from the point cloud data of the map data.
4. a storage device storing map data indicating an environment in which the moving object is used; the map data includes position information of a mirror reflecting portion that mirror-reflects the laser light, The control device includes: Estimating a self-position of the moving body; 3. The object detection device according to claim 1, wherein when the mirror reflection unit is located within a predetermined range from the self-position, the coordinates of the point located below the ground surface are converted into coordinates of a position that is plane-symmetric with respect to the ground surface.
Citation Information
Patent Citations
Radar system
JP2010127835A
Three-dimensional positioning device and three-dimensional positioning method
JP2020190484A
Road surface reflectivity detection by lidar sensor
US20160178802A1
Driving assistance device and driving assistance method
WO2016117060A1
Autonomous moving body movement assistance device, autonomous moving body, and map information distribution device
WO2020202559A1
