Road surface recognition method and road surface recognition device

Abstract

In a road surface recognition method and a road surface recognition device according to the present invention, a point cloud is generated by clustering a plurality of distance measurement points on the basis of distance measurement point data, acquired by a sensor, relating to the plurality of distance measurement points. A road surface point cloud corresponding to the road surface around the sensor, a first point cloud relating to an object located farther away from the sensor that the road surface point cloud, and a second point cloud relating to a reflected image of the object, reflected by the road surface, are extracted from the point cloud. The shape of the road surface farther from the sensor than the road surface point cloud and before the first point cloud and the second point cloud is estimated on the basis of the positions and shapes of the first point cloud and the second point cloud.

Description

Road surface recognition method and road surface recognition device

[0001] The present disclosure relates to a road surface recognition method and a road surface recognition device.

[0002] A technology has been proposed in which the distance to a target object is measured by measuring reflected signals from the target object using multiple irradiation signals with different depression angles, and each measurement point sequence measured in the circumferential direction for each depression angle is accumulated, and a road surface shape is calculated based on the accumulated measurement point sequence (see Patent Document 1).In this technology, a focus point in one of the measurement point sequences and a pair of adjacent points located on the larger depression angle side and smaller depression angle side of the measurement point sequence are extracted, and the focus point is classified into a road surface point, a candidate road surface point, or a measurement point that does not constitute a road surface based on a shape determination result using the focus point and the pair of adjacent points, and the road surface shape is calculated based on the classification.

[0003] Japanese Patent Application Laid-Open No. 2021-185345

[0004] The technology described in Patent Document 1 has a problem in that it cannot accurately recognize the road surface when the road surface is wet or when an object is reflected on the road surface.

[0005] The present disclosure has been made in view of the above-mentioned problems, and an object of the present disclosure is to provide a road surface recognition method and a road surface recognition device that can accurately recognize a road surface even when an object is reflected on the road surface, such as when the road surface is wet.

[0006] To solve the above-described problems, a road surface recognition method and road surface recognition device according to one aspect of the present disclosure cluster multiple ranging points based on ranging point data relating to multiple ranging points acquired by a sensor to generate a point cloud. From the point cloud, a road surface point cloud corresponding to the road surface around the sensor, a first point cloud relating to objects located farther from the sensor than the road surface point cloud, and a second point cloud relating to images of the objects reflected by the road surface are extracted. Based on the positions and shapes of the first and second point clouds, the shape of the road surface farther from the sensor than the road surface point cloud and closer to the sensor than the first and second point clouds is estimated.

[0007] According to the present disclosure, the road surface can be recognized with high accuracy even when an object is reflected on the road surface, such as when the road surface is wet.

[0008] Fig. 1 is a block diagram showing a configuration of a road surface recognition device according to an embodiment of the present disclosure. Fig. 2 is a flowchart showing processing of the road surface recognition device according to an embodiment of the present disclosure. Fig. 3 is a first schematic diagram showing an example of the positional relationship between a point cloud and a distance measurement sensor. Fig. 4 is a second schematic diagram showing an example of the positional relationship between a point cloud and a distance measurement sensor.

[0009] Next, embodiments of the present disclosure will be described in detail with reference to the drawings. In the description, the same components are designated by the same reference numerals and redundant description will be omitted.

[0010] [Configuration of Road Surface Recognition Device] An example configuration of a road surface recognition device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of a road surface recognition device according to this embodiment. As shown in Fig. 1, the road surface recognition device 1 includes a distance measurement sensor 10 (sensor) and a controller 20.

[0011] The road surface recognition device 1 may be mounted on a vehicle with an automatic driving function, or on a vehicle without an automatic driving function. Furthermore, the road surface recognition device 1 may be mounted on a vehicle capable of switching between automatic driving and manual driving. Furthermore, the automatic driving function may be a driving assistance function that automatically controls only some of the vehicle control functions, such as steering control, braking force control, and driving force control, to assist the driver in driving. In this embodiment, the road surface recognition device 1 is described as being mounted on a vehicle with an automatic driving function.

[0012] Although not shown in Fig. 1, the road surface recognition device 1 may control various actuators such as a steering actuator, an accelerator pedal actuator, and a brake actuator based on the recognition results (position, shape, attitude, etc. of an object). This may enable highly accurate autonomous driving.

[0013] The distance measurement sensor 10 generates distance measurement point data relating to distance measurement points located on the surface of an object. For example, the distance measurement sensor 10 is mounted on a vehicle that travels on a road surface.

[0014] For example, the distance measurement sensor 10 may include a sensor that measures the distance and direction to an object around the vehicle by emitting electromagnetic waves from an emission point around the vehicle and detecting the position of the reflection point based on the reflected wave of the emitted electromagnetic wave. One example of such a sensor is a LIDAR (Laser Imaging Detection and Ranging). A LIDAR is a sensor that emits light (laser light) from an emission point around a predetermined range around the vehicle, detects the position of a reflection point, which is a ranging point, based on the reflected wave, and generates ranging point data related to the ranging point.

[0015] Lidar measures the distance and direction to an object and recognizes the shape of the object by measuring the time it takes for the light (reflected wave) to bounce back after being emitted. Lidar can also obtain the positional relationship of objects in three dimensions. Mapping is also possible using the intensity of the reflected wave.

[0016] For example, the lidar scans the surroundings of the vehicle in the main scanning direction and the sub-scanning direction by changing the light irradiation direction. In particular, the lidar acquires range measurement point data for range measurement points located on the sides of a cone with the apex at the emission point by changing the angle from the horizontal plane (the horizontal plane in the lidar coordinate system) when emitting electromagnetic waves from the emission point. One side of the cone corresponds to one angle when the lidar emits electromagnetic waves.

[0017] For example, the lidar acquires ranging point data for ranging points included on the side surface of a single cone by scanning the side surface of the single cone (scanning in the main scanning direction).The lidar then acquires ranging point data for the side surfaces of multiple cones by scanning the side surface of the single cone (scanning in the sub-scanning direction) while changing the angle from the horizontal plane when emitting electromagnetic waves from the emission point.In this way, the lidar sequentially irradiates multiple ranging points around the vehicle with light.The lidar emits electromagnetic waves and generates ranging point data for ranging points located on the surface of the object based on the reflected waves of the electromagnetic waves from the surface of the object.

[0018] The LIDAR repeatedly emits light to all of the distance measurement points around the vehicle at predetermined time intervals. The LIDAR generates information (distance measurement point information) for each distance measurement point obtained by emitting light. The LIDAR then outputs the distance measurement point data to the controller 20.

[0019] The ranging point information includes position information of the ranging point. The position information is information indicating the position coordinates of the ranging point. The position coordinates may use a polar coordinate system represented by the direction from the lidar to the ranging point (yaw angle, pitch angle) and the distance from the lidar to the ranging point (depth). The position coordinates may use a three-dimensional coordinate system represented by x, y, and z coordinates with the installation position of the lidar as the origin. For example, the x and y coordinates may be coordinates on a horizontal plane in the lidar coordinate system, and the z coordinate may be a coordinate on an axis in the height direction perpendicular to the horizontal plane. Note that the x and y coordinates may be coordinates on a plane parallel to the road surface on which the vehicle is traveling, and the z coordinate may be a coordinate on an axis in the height direction above the road surface.

[0020] The ranging point information may also include time information of the ranging point. The time information is information indicating the time when the position information of the ranging point was generated (when the reflected electromagnetic wave was received). Additionally, the ranging point information may also include information on the intensity of the reflected wave from the ranging point (intensity information).

[0021] Alternatively, the distance measurement sensor 10 may be a stereo camera, which generates distance measurement point data relating to the positions of the distance measurement points using the principles of trigonometry based on a plurality of image data captured by the stereo camera.

[0022] The controller 20 processes the distance measurement point data generated by the distance measurement sensor 10. For example, the controller 20 is a general-purpose computer equipped with a CPU (Central Processing Unit), memory, and input / output units. A computer program for causing the computer to function as the road surface recognition device 1 is installed in the computer. By executing the computer program, the computer functions as multiple information processing circuits equipped in the road surface recognition device 1.

[0023] Although an example is shown here in which the multiple information processing circuits provided in the road surface recognition device 1 are realized by software, it is of course also possible to configure the information processing circuits by preparing dedicated hardware for executing each of the information processes described below.Furthermore, the multiple information processing circuits may be configured by individual hardware.

[0024] The controller 20 includes, as examples of a plurality of information processing circuits (information processing functions), a point cloud acquisition unit 21, a road surface point cloud extraction unit 23, an object point cloud extraction unit 25, a shape estimation unit 27, a determination unit 29, and an output unit 31. The controller 20 may also be expressed as an ECU (Electronic Control Unit).

[0025] The point cloud acquisition unit 21 generates a point cloud by clustering multiple ranging points based on the ranging point data. Each point cloud is a group of ranging points related to one or more ranging point data. For example, the point cloud acquisition unit 21 may simply treat multiple ranging points whose distances between them are equal to or less than a predetermined threshold as belonging to one point cloud and perform clustering.

[0026] Alternatively, the point cloud acquisition unit 21 may generate a point cloud by clustering a plurality of ranging points using various clustering algorithms such as k-means, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), or HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise).

[0027] The road surface point cloud extraction unit 23 extracts a road surface point cloud corresponding to the road surface around the distance measurement sensor 10 from the point cloud generated by the point cloud acquisition unit 21. For example, the distance measurement sensor 10 may extract a point cloud located at a predetermined height lower than the distance measurement sensor 10 as the road surface point cloud. Here, the predetermined height may be determined based on the height of the vehicle on which the distance measurement sensor 10 is mounted. For example, the predetermined height may be the height of the vehicle.

[0028] In addition, the road surface point cloud extraction unit 23 may extract a point cloud that is located at a predetermined height lower than the ranging sensor 10 and within a predetermined distance from the ranging sensor 10 as the road surface point cloud.

[0029] When the road surface is wet, diffuse reflection at the ranging points is weak, and the intensity of the reflected waves from the ranging points is low. Therefore, it is difficult to obtain ranging point data for ranging points located far away from the ranging points on a wet road surface. However, the shorter the distance between the ranging sensor 10 and the ranging point, the greater the intensity of the reflected waves obtained from the ranging point, so it is often possible to obtain ranging point data for ranging points located close to the ranging sensor 10. Therefore, the road surface point cloud extraction unit 23 may extract a point cloud located within a predetermined distance from the ranging sensor 10 as the road surface point cloud.

[0030] Alternatively, the road surface point cloud extraction unit 23 may extract, as the road surface point cloud, a point cloud that is located at a predetermined height lower than the distance measurement sensor 10 and that spreads along the horizontal plane of the distance measurement sensor 10. When the distance measurement sensor 10 is a sensor mounted on a vehicle, the road surface on which the vehicle travels is often located at a predetermined height lower than the distance measurement sensor 10 and that spreads along the horizontal plane of the distance measurement sensor 10. Therefore, the road surface point cloud extraction unit 23 may extract, as the road surface point cloud, a point cloud that resembles a road surface as described above.

[0031] The object point cloud extraction unit 25 extracts, from the point clouds generated by the point cloud acquisition unit 21, a first point cloud relating to an object located farther away from the ranging sensor 10 than the road surface point cloud, and a second point cloud relating to the reflected image of the object by the road surface.

[0032] More specifically, the object point cloud extraction unit 25 sets the distance between the road surface point cloud and the distance measurement sensor 10 as the distance threshold. The object point cloud extraction unit 25 also calculates the distance (point cloud distance) between the point cloud generated by the point cloud acquisition unit 21 and the distance measurement sensor 10. Then, the object point cloud extraction unit 25 extracts, as the target point cloud, a point cloud whose point cloud distance is greater than the distance threshold.

[0033] The object point cloud extraction unit 25 extracts, from the target point cloud, two point clouds that are at approximately the same distance along the horizontal plane from the distance measurement sensor 10 and that are positioned next to each other above and below as viewed from the distance measurement sensor 10, as a first point cloud and a second point cloud. In particular, of the two point clouds, the point cloud located at a higher position is extracted as the first point cloud, and the point cloud located at a lower position is extracted as the second point cloud.

[0034] In addition, the object point cloud extraction unit 25 may extract, as the first point cloud and the second point cloud, two point clouds from the target point cloud that are at approximately the same distance along the horizontal plane from the ranging sensor 10, are located next to each other above and below when viewed from the ranging sensor 10, and have the same shape when flipped upside down in the coordinate system of the ranging sensor 10.

[0035] The shape estimation unit 27 estimates the shape of the road surface farther away than the road surface point cloud and closer to the first point cloud and the second point cloud, based on the positions and shapes of the first point cloud and the second point cloud, as viewed from the distance measurement sensor 10. For example, the shape estimation unit 27 may estimate that the road surface is located in the center of the section between the lower end of the first point cloud and the upper end of the second point cloud.

[0036] This will be explained more specifically using Fig. 3. Fig. 3 is a first schematic diagram showing an example of the positional relationship between the point clouds and the distance measurement sensor. In Fig. 3, it is assumed that the distance measurement sensor 10 is installed at point PS, and that the road surface point cloud GR, the first point cloud G1, and the second point cloud G2 have been extracted. In this case, the shape estimation unit 27 may estimate that the road surface RS is located at a position that divides the line segment connecting point PA at the bottom end of the first point cloud G1 and point PB at the top end of the second point cloud G2 in a one-to-one relationship.

[0037] Furthermore, the shape estimation unit 27 may extract first reference points of an object associated with the first point cloud and second reference points of an object associated with the second point cloud based on the positions and shapes of the first point cloud and the second point cloud. Here, for example, if the road surface is wet and diffuse reflection on the road surface is weak, the object associated with the second point cloud may be a reflected image corresponding to the first reference points. In this case, the shape of the first point cloud and the shape of the second point cloud will be the same.

[0038] The shape estimation unit 27 then estimates the normal direction at a road surface reflection point where a line connecting the distance measurement sensor 10 and the second reference point intersects with the road surface, based on the positional relationship between the first reference point, the second reference point, and the installation location of the distance measurement sensor 10. The shape estimation unit 27 may then estimate the shape based on the estimated normal direction.

[0039] A more specific explanation will be given using FIG. 4. FIG. 4 is a second schematic diagram showing an example of the positional relationship between the point cloud and the distance measurement sensor. In FIG. 3, the distance measurement sensor 10 is installed at point PS, and a road surface point cloud GR, a first point cloud G1, and a second point cloud G2 are extracted. The first and second reference points are points PA and PB, respectively. Furthermore, the plane PL is defined as a plane that is obtained by internally dividing the line segment connecting the first reference point PA and the second reference point PB on a one-to-one basis and perpendicularly intersecting the line segment.

[0040] At this time, the shape estimation unit 27 calculates an intersection PR where the plane PL intersects with the line connecting the points PS and PB, and determines the intersection PR as a road surface reflection point. Then, it calculates a normal line HL of the plane PL at the intersection PR.

[0041] By estimating the road surface reflection points and the normal directions at the road surface reflection points, the shape estimation unit 27 can estimate the shape of the road surface RS. For example, by assuming that the road surface near the distance measurement sensor 10 indicated by the road surface point cloud GR and the road surface near the road surface reflection points are continuous, it is possible to calculate the shape of the road surface from near the distance measurement sensor 10 to near the road surface reflection points. In particular, the shape estimation unit 27 can calculate the gradient of the road surface.

[0042] The determination unit 29 determines whether an object related to the first point cloud or the second point cloud interferes with the traveling of the vehicle. For example, the determination unit 29 determines whether the distance between the bottom end of the first point cloud and the top end of the second point cloud is greater than a predetermined threshold. Here, the predetermined threshold may be determined based on the height of the vehicle on which the distance measuring sensor 10 is mounted. For example, the predetermined threshold may be twice the height of the vehicle.

[0043] Then, if it is determined that the distance between the bottom end of the first point group and the top end of the second point group is greater than a predetermined threshold, the determination unit 29 may determine that the object related to the first point group is in a position that does not interfere with the vehicle's movement.

[0044] Alternatively, the determination unit 29 may determine whether the road surface is wet based on an image of the road surface acquired by an imaging device (not shown). The determination unit 29 may acquire the image of the road surface from an imaging device connected to the controller 20.

[0045] For example, the determination unit 29 may input an image of a road surface into a learning model obtained by performing machine learning based on an image of a wet road surface, and calculate the probability that the road surface reflected in the image is wet from the learning model. If the calculated probability is equal to or greater than a predetermined value, the determination unit 29 may determine that the road surface is wet.

[0046] It should be noted that when the determining unit 29 determines that the road surface is wet, the shape estimating unit 27 may estimate the shape of the road surface.

[0047] The output unit 31 outputs data relating to the shape of the road surface estimated by the shape estimation unit 27. Alternatively, the output unit 31 may output the determination result by the determination unit 29.

[0048] [Processing Procedure of Road Surface Recognition Device 1] Next, a processing procedure of the road surface recognition device 1 according to this embodiment will be described with reference to the flowchart of Fig. 2. Fig. 2 is a flowchart showing processing of the road surface recognition device according to this embodiment. The processing of the road surface recognition device 1 shown in Fig. 2 may be repeatedly executed at a predetermined cycle.

[0049] In step S101, the distance measurement sensor 10 generates distance measurement point data relating to a plurality of distance measurement points.

[0050] In step S103, the point cloud acquisition unit 21 generates a point cloud by clustering the plurality of distance measurement points based on the distance measurement point data.

[0051] In step S105, the road surface point cloud extraction unit 23 extracts a road surface point cloud, and the object point cloud extraction unit 25 extracts a first point cloud and a second point cloud.

[0052] In step S107, the shape estimation unit 27 estimates the shape of the road surface.

[0053] In step S109, the output unit 31 outputs data relating to the shape of the road surface.

[0054] [Effects of the Embodiment] As described in detail above, the road surface recognition method and road surface recognition device according to this embodiment generate a point cloud by clustering multiple ranging points based on ranging point data relating to multiple ranging points acquired by a sensor. From the point cloud, a road surface point cloud corresponding to the road surface around the sensor, a first point cloud relating to objects located farther from the sensor than the road surface point cloud, and a second point cloud relating to images of the objects reflected by the road surface are extracted. Based on the positions and shapes of the first and second point clouds, the shape of the road surface farther from the sensor than the road surface point cloud and closer to the sensor than the first and second point clouds is estimated.

[0055] This allows the road surface to be recognized with high accuracy even when objects are reflected by the road surface, such as when the road surface is wet. In particular, even when a distance measurement point on the road surface located far from the sensor cannot be obtained because the rate of diffuse reflection on the road surface is low, the road surface shape can be estimated based on objects near the road surface and the reflected images of the objects on the road surface.

[0056] Furthermore, the road surface recognition method and road surface recognition device according to this embodiment may estimate that the road surface is located in the center of the section between the lower end of the first point cloud and the upper end of the second point cloud. This reduces the calculation cost required to estimate the position of the road surface. Furthermore, the distance between the object associated with the first point cloud and the road surface (the height at which the object associated with the first point cloud is installed) can be estimated with low calculation cost.

[0057] Furthermore, the road surface recognition method and road surface recognition device according to this embodiment may extract a first reference point of the object and a second reference point of the reflected image corresponding to the first reference point based on the positions and shapes of the first point cloud and the second point cloud. Based on the positional relationship between the first reference point, the second reference point, and the installation location of the sensor, the normal direction at the road surface reflection point where the line connecting the sensor and the second reference point intersects with the road surface may be estimated, and the shape may be estimated based on the normal direction. This allows the road surface shape to be estimated with higher accuracy. In particular, the slope of the road surface can be estimated.

[0058] Furthermore, in the road surface recognition method and road surface recognition device according to this embodiment, distance measurement point data may be acquired by a sensor mounted on a vehicle traveling on a road surface. Then, if the distance between the bottom end of the first point cloud and the top end of the second point cloud is greater than a predetermined threshold, it may be determined that the object is located in a position that does not interfere with the traveling of the vehicle. This makes it possible to determine whether or not an object interferes with the traveling of the vehicle by targeting only obstacles that may interfere with the traveling of the vehicle from among the generated point clouds. As a result, it is possible to reduce the calculation cost required to determine whether or not an object present around the vehicle interferes with the traveling of the vehicle.

[0059] Furthermore, the road surface recognition method and road surface recognition device according to this embodiment may determine whether the road surface is wet based on an image of the road surface acquired by an imaging device, and estimate the shape of the road surface if it is determined that the road surface is wet. As a result, if the road surface is not wet, the process of estimating the shape of the road surface is not performed. As a result, it is possible to reduce erroneous determinations.

[0060] Each of the functions described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), or circuit components arranged to perform the described functions.

[0061] Although the contents of the present disclosure have been described above based on the embodiments, the present disclosure is not limited to these descriptions, and various modifications and improvements are possible, which will be apparent to those skilled in the art. The descriptions and drawings that form part of this disclosure should not be understood as limiting the present disclosure. Various alternative embodiments, examples, and operating techniques will be apparent to those skilled in the art from this disclosure.

[0062] Of course, the present disclosure includes various embodiments not described herein. Therefore, the technical scope of the present disclosure is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description.

[0063] REFERENCE SIGNS LIST 1 Road surface recognition device 10 Distance measurement sensor 20 Controller 21 Point cloud acquisition unit 23 Road surface point cloud extraction unit 25 Object point cloud extraction unit 27 Shape estimation unit 29 Determination unit 31 Output unit GR Road surface point cloud G1 First point cloud G2 Second point cloud

Claims

1. A road surface recognition method for a road surface recognition device including a sensor that generates distance measurement point data for a plurality of distance measurement points and a controller that processes the distance measurement point data, wherein the controller: generates a point cloud by clustering the plurality of distance measurement points based on the distance measurement point data; extracts from the point cloud a road surface point cloud corresponding to the road surface around the sensor, a first point cloud related to an object located farther from the sensor than the road surface point cloud, and a second point cloud related to a reflected image of the object by the road surface; and estimates the shape of the road surface in front of the road surface point cloud and farther from the sensor than the first point cloud and the second point cloud based on the positions and shapes of the first point cloud and the second point cloud.

2. The road surface recognition method according to claim 1, wherein the controller estimates that the road surface is located at the center of a section sandwiched between the lower end of the first point cloud and the upper end of the second point cloud.

3. The road surface recognition method according to claim 1 or 2, wherein the controller: extracts a first reference point of the object and a second reference point of the reflected image corresponding to the first reference point based on the positions and shapes of the first point cloud and the second point cloud; estimates the normal direction at a road surface reflection point where a straight line connecting the sensor and the second reference point intersects the road surface based on the positional relationship among the first reference point, the second reference point, and the installation location of the sensor; and estimates the shape based on the normal direction.

4. The road surface recognition method according to any one of claims 1 to 3, wherein the sensor is mounted on a vehicle traveling on the road surface.

5. The road surface recognition method according to claim 4, wherein the controller determines that the object is in a position where it does not interfere with the travel of the vehicle when the distance between the lower end of the first point cloud and the upper end of the second point cloud is greater than a predetermined threshold value.

6. The road surface recognition method according to any one of claims 1 to 5, wherein the controller: is connected to an imaging device; determines whether the road surface is wet based on an image of the road surface acquired by the imaging device; and estimates the shape when it is determined that the road surface is wet.

7. A road surface recognition device comprising a sensor that generates distance measurement point data for a plurality of distance measurement points, and a controller that processes the distance measurement point data, wherein the controller: clusters the plurality of distance measurement points based on the distance measurement point data to generate a point cloud; extracts from the point cloud a road surface point cloud corresponding to the road surface around the sensor, a first point cloud related to an object located farther than the road surface point cloud as viewed from the sensor, and a second point cloud related to a reflected image of the object by the road surface; and estimates the shape of the road surface in front of the road surface point cloud and farther than the first point cloud and the second point cloud as viewed from the sensor based on the positions and shapes of the first point cloud and the second point cloud.

Images