A method and apparatus for curb detection

Abstract

The application provides a road edge detection method and device, which combines the features of single scanning layer and multi scanning layer in the determination of road edge point cloud data, and effectively improves the distance and accuracy of road edge detection without filtering out the effective point cloud data in the distance due to the sparseness of point cloud data, resulting in a shorter detection distance. The road edge detection method comprises the following steps: determining road edge point cloud data according to the layer features of each point cloud layer data in multiple point cloud layer data and the layer features between the point cloud layer data adjacent to the each point cloud layer data; determining an initial road edge according to the road edge point cloud data, and determining a road edge seed point according to the initial road edge, wherein the initial road edge is used to represent the road edge determined when the road edge detection starts, and the road edge seed point is used to represent the true condition of the road edge; and determining a parameterized equation corresponding to the road edge according to the road edge seed point.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and more particularly, to a method and device for road edge detection. BACKGROUND

[0002] Autonomous driving is a mainstream application in the field of artificial intelligence. Autonomous driving technology relies on computer vision, radar, monitoring devices, and global positioning systems to cooperate, so that motor vehicles can be automatically driven without human active operation. Autonomous vehicles use various computing systems to help transport passengers from one location to another. Some autonomous vehicles can require some initial input or continuous input from an operator, such as a navigator, a driver, or a passenger. Autonomous vehicles allow operators to switch from a manual mode to an autonomous driving mode or a mode between the two. Since autonomous driving technology does not require humans to drive motor vehicles, it can theoretically effectively avoid human driving errors, reduce traffic accidents, and improve highway transportation efficiency. Therefore, autonomous driving technology is increasingly valued.

[0003] For an autonomous driving platform, environmental perception is an interaction port between an intelligent vehicle platform and a surrounding traffic scene, and is also a front-end input of a motion decision, planning, and control system of the intelligent vehicle platform. The performance of environmental perception directly determines the stability of the autonomous driving platform in driving tasks. In environmental perception, robust road edge detection, as one of the core tasks, plays a key role in vehicle collision prevention. SUMMARY

[0004] The present application provides a method and device for road edge detection, which combines features within a single scanning layer and between multiple scanning layers in the determination of road edge point cloud data. The method and device do not filter out valid point cloud data in the distance due to the sparsity of point cloud data, which shortens the detection distance and effectively improves the distance and accuracy of road edge detection.

[0005] In a first aspect, a method for road edge detection is provided, applied to a vehicle including a laser radar, comprising: determining road edge point cloud data according to layer-in features of each point cloud layer data in a plurality of point cloud layer data and layer-in features between the each point cloud layer data and adjacent point cloud layer data of the each point cloud layer data; determining an initial road edge according to the road edge point cloud data, and determining a road edge seed point according to the initial road edge, the initial road edge being used to represent a road edge determined when road edge detection starts, and the road edge seed point being used to represent a true condition of the road edge; and determining a parameterized equation corresponding to the road edge according to the road edge seed point.

[0006] According to the embodiment of the present application, the features of single scanning layer and multi scanning layers are combined in the determination of the road edge point cloud data, which does not filter out the effective point cloud data in the distance due to the sparsity of the point cloud data, and effectively improves the distance and accuracy of the road edge detection.

[0007] In combination with the first aspect, in some implementations of the first aspect, the intra-layer features include at least one of a slope feature, a number feature, or a reflection intensity feature; the slope feature is used to indicate point cloud data in each point cloud layer data whose distribution direction and the driving direction of the vehicle have an included angle less than a third threshold value, the number feature is used to indicate point cloud data in each point cloud layer data whose number of point cloud data in the neighborhood is greater than a fourth threshold value, and the reflection intensity feature is used to indicate point cloud data in each point cloud layer data whose reflectivity is greater than a fifth threshold value.

[0008] In combination with the first aspect, in some implementations of the first aspect, the inter-layer features include at least one of an inter-layer distance feature or an inter-layer number feature; the inter-layer distance feature is used to indicate point cloud data in each point cloud layer data whose distance with adjacent point cloud layer data is less than a sixth threshold value, and the inter-layer number feature is used to indicate point cloud data in each point cloud layer data whose number of point cloud data in the neighborhood of adjacent point cloud layer data is greater than a seventh threshold value.

[0009] According to the embodiment of the present application, the features of single point cloud layer data and the features between adjacent two point cloud layer data are combined in the method for determining the road edge point cloud data, which does not filter out the effective point cloud data in the distance due to the sparsity of the point cloud data, and effectively improves the distance of the road edge detection. The designed features basically conform to the actual features of the vast majority of road edges in the real traffic scene through actual road test data test.

[0010] In combination with the first aspect, in some implementations of the first aspect, the determining the road edge seed points according to the initial road edge includes: determining a first group of seed points and a first group of candidate points according to the initial road edge, each point cloud data in the first group of seed points has a distance less than a first threshold value from the initial road edge, the first group of seed points is used to represent the true condition of the first part of the road edge, and the first group of candidate points is point cloud data in the road edge point cloud data except the first group of seed points; determining a second group of seed points according to the first group of seed points and the first group of candidate points, the second group of seed points is used to represent the true condition of the second part of the road edge; and determining the first group of seed points and the second group of seed points as the road edge seed points.

[0011] According to the embodiment of the present application, the distribution of the point cloud data corresponding to the road edge is predicted according to the first group of seed points and the first group of candidate points, and the road edge seed points are updated through multiple iterations, which further improves the detection distance and detection effect of the road edge detection.

[0012] With reference to the first aspect, in some implementations of the first aspect, determining the second set of seed points according to the first set of seed points and the first set of candidate points comprises: determining a second set of seed points and a second set of candidate points according to the first set of seed points and the first set of candidate points, the second set of candidate points being the point cloud data in the road edge point cloud data other than the first set of seed points and the second set of seed points; and determining the first set of seed points and the second set of seed points as the road edge seed points comprises: determining a third set of seed points according to the first set of seed points, the second set of seed points and the second set of candidate points, and determining the first set of seed points, the second set of seed points and the third set of seed points as the road edge seed points, the third set of seed points being used to represent the true condition of a third part of the road edge.

[0013] According to the embodiments of the present application, the road edge seed points are constantly updated through multiple iterations according to the first set of seed points, the second set of seed points and the second set of candidate points, which further improves the detection distance and detection effect of road edge detection.

[0014] With reference to the first aspect, in some implementations of the first aspect, determining the second set of seed points according to the first set of seed points and the first set of candidate points further comprises: determining a first set of key seed points according to the first set of seed points, the point cloud data in the first set of key seed points corresponding to the grids in the first set of seed points after gridding, the coordinate value of the point cloud data in the first key seed points being the coordinate average of the point cloud data in the corresponding grids in the first set of seed points, and the weight of the point cloud data in the first key seed points being determined by the number of point cloud data in the corresponding grids; determining a first set of key candidate points according to the first set of candidate points, the point cloud data in the first set of key candidate points corresponding to the grids in the first set of candidate points after gridding, the coordinate value of the point cloud data in the first key candidate points being the coordinate average of the point cloud data in the corresponding grids in the first set of candidate points, and the weight of the point cloud data in the first key candidate points being determined by the number of point cloud data in the corresponding grids; and determining the second set of seed points according to the first set of key seed points and the first set of key candidate points.

[0015] According to the embodiments of the present application, since the computational overhead of the road edge prediction model is positively correlated with the input data, the first set of seed points and the first set of candidate points can be gridded to reduce the computational overhead of the road edge prediction model in subsequent steps without affecting the prediction accuracy.

[0016] In some implementations of the first aspect, the determining the second set of seed points according to the first set of key seed points and the first set of key candidate points comprises: determining a first set of predicted points according to the first set of key seed points and the first set of key candidate points by using a road edge prediction model, the first set of predicted points being used to represent a predicted state of the second part of the road edge; determining point cloud data in the first set of key candidate points that satisfies a first condition as point cloud data in the second set of seed points, the first condition being that a deviation between the point cloud data and point cloud data in the first set of predicted points is less than a second threshold; and wherein the road edge prediction model is obtained by machine learning.

[0017] According to embodiments of the present application, the road edge prediction model can be a model established based on Gaussian process regression. Since Gaussian process regression is a non-parametric model, it does not require the road to have a specific shape, and thus can better predict the road edge distribution of a road with large curvature variation or a difficult scene such as a construction section, thereby improving the accuracy in the road edge prediction process.

[0018] In some implementations of the first aspect, the determining the initial road edge according to the road edge point cloud data comprises: determining the initial road edge according to point cloud data in the road edge point cloud data projected on a top view plane.

[0019] According to embodiments of the present application, to reduce the computational cost, the determined road edge point cloud data can be first projected on a top view plane, which can be considered as a horizontal plane or a plane on which the road is located.

[0020] In some implementations of the first aspect, the determining the initial road edge according to the point cloud data in the road edge point cloud data projected on the top view plane comprises: determining a plurality of initial straight line segments according to the point cloud data projected on the top view plane; and determining the initial road edge according to the plurality of initial straight line segments.

[0021] According to embodiments of the present application, a plurality of initial straight line segments can be determined in the projected road edge point cloud data by using Hough transformation, and the plurality of initial straight line segments can be used to represent the road edge. However, since only a part of the road edge distribution in a real road scene conforms to a straight line distribution, the plurality of initial straight line segments need to be screened, and the screened plurality of initial straight line segments need to be merged to form an initial road edge, which can be used as a reference for subsequent methods.

[0022] In some implementations of the first aspect, the method further comprises: determining the plurality of point cloud layer data according to original point cloud data; wherein the original point cloud data is point cloud data obtained by the laser radar, and the plurality of point cloud layer data is two-dimensional point cloud data in a self-vehicle coordinate system.

[0023] According to the embodiments of the present application, the original point cloud data can be point cloud data corresponding to one frame obtained by the laser radar, or can also be point cloud data corresponding to multiple frames. That is, the method for curb detection provided by the embodiments of the present application can output a curb result according to the point cloud data of each frame of the laser radar, or can output a curb result according to the point cloud data of multiple frames, and can be selected according to the actual application scenario.

[0024] In combination with the first aspect, in some implementations of the first aspect, the determining the plurality of point cloud layer data according to the original point cloud data comprises: generating a point cloud feature map according to the point cloud data after spherical projection of the original point cloud data; and determining the plurality of point cloud layer data according to the point cloud feature map.

[0025] According to the embodiments of the present application, the three-dimensional point cloud data is reduced to two-dimensional point cloud data, and the two-dimensional point cloud data is represented in a point cloud feature map, which greatly reduces the calculation cost and improves the overall calculation efficiency of the method.

[0026] In combination with the first aspect, in some implementations of the first aspect, the point cloud feature map comprises a point cloud depth map and a point cloud index map.

[0027] According to the embodiments of the present application, the point cloud feature map can be a point cloud depth map and a point cloud index map, or can also be other types of point cloud feature maps, which are not limited by the present application.

[0028] The second aspect provides a curb detection device, comprising: a processing unit, configured to: determine curb point cloud data according to layer-in-feature of each point cloud layer data in the plurality of point cloud layer data and layer-between-feature between the point cloud layer data adjacent to the each point cloud layer data; determine an initial curb according to the curb point cloud data, and determine a curb seed point according to the initial curb, wherein the initial curb is used to represent a curb determined when curb detection starts, and the curb seed point is used to represent a true condition of the curb; and determine a parameterized equation corresponding to the curb according to the curb seed point.

[0029] In combination with the second aspect, in some possible implementations, the layer-in-feature comprises at least one of a slope feature, a number feature or a reflection intensity feature; wherein the slope feature is used to indicate point cloud data in the each point cloud layer data with an included angle between a distribution direction and a driving direction of the vehicle less than a third threshold value, the number feature is used to indicate point cloud data in the each point cloud layer data with a number of point cloud data in a neighborhood greater than a fourth threshold value, and the reflection intensity feature is used to indicate point cloud data in the each point cloud layer data with a reflectivity greater than a fifth threshold value.

[0030] With reference to the second aspect, in some possible implementations, the inter-layer feature includes at least one of an inter-layer distance feature or an inter-layer number feature; the inter-layer distance feature is used to indicate point cloud data in each point cloud layer data with a distance to adjacent point cloud layer data less than a sixth threshold value, and the inter-layer number feature is used to indicate point cloud data in each point cloud layer data with a number of point cloud data in a neighborhood of adjacent point cloud layer data greater than a seventh threshold value.

[0031] With reference to the second aspect, in some possible implementations, the processing unit is specifically configured to: determine a first group of seed points and a first group of candidate points according to the initial road alignment, each point cloud data in the first group of seed points has a distance to the initial road alignment less than a first threshold value, and the first group of seed points are used to represent a true condition of a first part of the road alignment, and the first group of candidate points are point cloud data in the road alignment point cloud data other than the first group of seed points; determine a second group of seed points according to the first group of seed points and the first group of candidate points, and the second group of seed points are used to represent a true condition of a second part of the road alignment; and determine the first group of seed points and the second group of seed points as the road alignment seed points.

[0032] With reference to the second aspect, in some possible implementations, the processing unit is specifically configured to: determine a second group of seed points and a second group of candidate points according to the first group of seed points and the first group of candidate points, and the second group of candidate points are point cloud data in the road alignment point cloud data other than the first group of seed points and the second group of seed points; and the determining of the first group of seed points and the second group of seed points as the road alignment seed points includes: determining a third group of seed points according to the first group of seed points, the second group of seed points and the second group of candidate points, and determining the first group of seed points, the second group of seed points and the third group of seed points as the road alignment seed points, and the third group of seed points are used to represent a true condition of a third part of the road alignment.

[0033] With reference to the second aspect, in some possible implementations, the processing unit is further specifically configured to: determine a first set of key seed points according to the first set of seed points, point cloud data in the first set of key seed points corresponding to grids in the first set of seed points after gridding, a coordinate value of the point cloud data in the first key seed points being a coordinate mean value of the point cloud data in the corresponding grid in the first set of seed points, and a weight of the point cloud data in the first key seed points being determined according to a quantity of the point cloud data in the corresponding grid; determine a first set of key candidate points according to the first set of candidate points, point cloud data in the first set of key candidate points corresponding to grids in the first set of candidate points after gridding, a coordinate value of the point cloud data in the first key candidate points being a coordinate mean value of the point cloud data in the corresponding grid in the first set of candidate points, and a weight of the point cloud data in the first key candidate points being determined according to a quantity of the point cloud data in the corresponding grid; and determine the second set of seed points according to the first set of key seed points and the first set of key candidate points.

[0034] With reference to the second aspect, in some possible implementations, the processing unit is further specifically configured to: determine a first set of predicted points according to the first set of key seed points and the first set of key candidate points by using a road edge prediction model, the first set of predicted points being used to represent a predicted state of a second part of the road edge; and determine point cloud data in the first set of key candidate points that satisfies a first condition as point cloud data in the second set of seed points, the first condition being that a deviation between the point cloud data and point cloud data in the first set of predicted points is less than a second threshold value; and wherein the road edge prediction model is obtained by machine learning.

[0035] With reference to the second aspect, in some possible implementations, the processing unit is further specifically configured to: determine an initial road edge according to point cloud data in the overhead plane projection of the road edge point cloud data.

[0036] With reference to the second aspect, in some possible implementations, the processing unit is further specifically configured to: determine a plurality of initial straight line segments according to the point cloud data in the overhead plane projection; and determine the initial road edge according to the plurality of initial straight line segments.

[0037] With reference to the second aspect, in some possible implementations, the apparatus further includes an acquisition unit configured to acquire original point cloud data, and the processing unit is further configured to: determine the plurality of point cloud layer data according to the original point cloud data; wherein the original point cloud data is point cloud data acquired by a laser radar, and the plurality of point cloud layer data is two-dimensional point cloud data in a self-vehicle coordinate system.

[0038] With reference to the second aspect, in some possible implementations, the processing unit is further specifically configured to: generate a point cloud feature map according to the point cloud data after the spherical projection of the original point cloud data; and determine the plurality of point cloud layer data according to the point cloud feature map.

[0039] With reference to the second aspect, in some possible implementations, the point cloud feature map comprises a point cloud depth map and a point cloud index map.

[0040] In a third aspect, a control device of a curb detection apparatus is provided, comprising at least one memory and at least one processor, the at least one memory is configured to store a program, and the at least one processor is configured to execute the program to implement the method in the first aspect.

[0041] In a fourth aspect, a chip is provided, comprising at least one processing unit and an interface circuit, the interface circuit is configured to provide program instructions or data for the at least one processing unit, and the at least one processing unit is configured to execute the program instructions to implement the method in the first aspect or support the function implementation of the device in the second aspect.

[0042] In a fifth aspect, a computer readable storage medium is provided, the computer readable medium stores program code for execution by a device, and the program code, when executed by the device, implements the method in the first aspect.

[0043] In a sixth aspect, a terminal is provided, comprising the curb detection apparatus in the second aspect, or the control device in the third aspect, or the chip in the fourth aspect. Further, the terminal can be a smart transportation device (vehicle or unmanned aerial vehicle), a smart home device, a smart manufacturing device, a surveying and mapping device, or a robot, etc. The smart transportation device can be an automated guided vehicle (AGV) or an unmanned transportation vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a functional block diagram of a vehicle to which embodiments of the present application are applicable.

[0045] Figure 2 is a schematic diagram of an autonomous driving system to which embodiments of the present application are applicable.

[0046] Figure 3 is an application schematic diagram of an autonomous driving vehicle on a cloud side according to an embodiment of the present application.

[0047] Figure 4 is a framework diagram of an autonomous driving system according to an embodiment of the present application.

[0048] Figure 5is a flowchart of a method 400 for curb detection provided by an embodiment of the present application.

[0049] Figure 6 is a flowchart of a method 500 for determining multiple point cloud layer data provided by an embodiment of the present application.

[0050] Figure 7 is a schematic diagram of a method 600 for determining curb point cloud data provided by an embodiment of the present application.

[0051] Figure 8 is a schematic diagram of a slope feature provided by an embodiment of the present application.

[0052] Figure 9 is a schematic diagram of multiple point cloud layer data provided by an embodiment of the present application.

[0053] Figure 10 is a schematic diagram of a neighborhood provided by an embodiment of the present application.

[0054] Figure 11 is a flowchart of a method 700 for determining a curb seed point provided by an embodiment of the present application.

[0055] Figure 12 is a schematic diagram of a method for determining a curb seed point provided by an embodiment of the present application.

[0056] Figure 13 is a schematic flowchart of a method for training a curb prediction model provided by an embodiment of the present application.

[0057] Figure 14 is a flowchart of a method for updating a curb seed point provided by an embodiment of the present application.

[0058] Figure 15 is a schematic diagram of determining a key point seed point in a grid provided by an embodiment of the present application.

[0059] Figure 16 is an iteration effect diagram of curb detection provided by an embodiment of the present application.

[0060] Figure 17 is a schematic structural diagram of a curb detection device provided by an embodiment of the present application.

[0061] Figure 18 is a schematic structural diagram of a control device of a curb detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] For ease of understanding, the following first introduces several concepts and terms related to the present application.

[0063] LiDAR: Light Detection and Ranging, a radar system that detects the position, speed and other characteristic quantities of a target by emitting a probe beam. Its working principle is to emit a probe signal (probe beam) to the target, then compare the received signal (target echo) reflected from the target with the emitted signal, and after appropriate processing, obtain the relevant information of the target, such as target distance, direction, height, speed, attitude, and even shape, etc. parameters, so as to detect, track and identify targets such as aircraft and missiles.

[0064] Point cloud data: refers to a set of vectors in a 3D coordinate system. These vectors are usually represented in the form of 3D coordinates of x, y, and z, and generally mainly represent the outer surface geometry of an object. In addition to this, point cloud data can also be attached with RGB information, i.e. color information of each coordinate point, or other information.

[0065] Point cloud layer data: refers to the point cloud data obtained after a single emitted probe beam in the LiDAR detects a target. Each point cloud layer data corresponds to a probe beam in the LiDAR.

[0066] Frame: can be understood as the set of point cloud data obtained when the LiDAR detects a target through a probe beam each time.

[0067] Rasterization: raster is a pixel, and rasterization is the conversion of vector graphics to a bitmap (raster image). The most basic rasterization algorithm renders a 3D scene represented by polygons to a two-dimensional (2-dimension, 2D) surface.

[0068] Point cloud depth image: also called distance image, refers to an image in which the distance (depth) value from the origin of the LiDAR to each point in the scene is taken as the pixel value.

[0069] Point cloud index image: refers to an image in which the labels corresponding to different point cloud data after the conversion of point cloud data from a 3D scene to a 2D scene are taken as pixel values, so as to restore the 2D scene to a 3D scene.

[0070] Self-vehicle coordinate system: refers to a coordinate system commonly used by all sensors in a vehicle. For example, a vehicle can include a LiDAR, a visible light camera and a positioning device, wherein the data obtained by the LiDAR, the visible light camera and the positioning device are in different coordinate systems. In order to facilitate the processor of the vehicle to uniformly process these data, it is necessary to convert these data to the same coordinate system for processing.

[0071] The method and / or device for curb detection provided by the embodiments of the present application can be applied to various vehicles. The method and / or device can be applied to manual driving, assisted driving, and automatic driving. The technical solutions of the embodiments of the present application are described below with reference to the drawings.

[0072] Figure 1 is a functional block diagram of a vehicle to which the embodiments of the present application are applicable. The vehicle 100 can be a manually driven vehicle, or the vehicle 100 can be configured to be in a fully or partially automatic driving mode.

[0073] In one example, the vehicle 100 can control the ego vehicle while in an automatic driving mode, and can determine a current state of the vehicle and its surrounding environment, determine a possible behavior of at least one other vehicle in the surrounding environment, and determine a confidence level corresponding to a likelihood that the other vehicle will perform the possible behavior based on the determined information, and control the vehicle 100 based on the determined information. When the vehicle 100 is in the automatic driving mode, the vehicle 100 can be configured to operate without human interaction.

[0074] The vehicle 100 can include various subsystems, such as a travel system 110, a sensing system 120, a control system 130, one or more peripheral devices 140, a power source 160, a computer system 150, and a user interface 170.

[0075] Optionally, the vehicle 100 can include more or fewer subsystems, and each subsystem can include multiple elements. In addition, each subsystem and element of the vehicle 100 can be interconnected by wire or wirelessly.

[0076] By way of example, the travel system 110 can include components for providing powered movement to the vehicle 100. In one embodiment, the travel system 110 can include an engine 111, a transmission 112, an energy source 113, and wheels 114 / tires. The engine 111 can be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines; for example, a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 111 can convert the energy source 113 into mechanical energy.

[0077] By way of example, the energy source 113 can include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 113 can also provide energy to other systems of the vehicle 100.

[0078] Exemplarily, the transmission 112 can include a gearbox, a differential, and a drive shaft; wherein the transmission 112 can transmit mechanical power from the engine 111 to the wheels 114.

[0079] In one embodiment, the transmission 112 can further include other devices, such as a clutch. Wherein the drive shaft can include one or more shafts that can be coupled to one or more wheels 114.

[0080] Exemplarily, the sensing system 120 can include several sensors that sense information about the environment surrounding the vehicle 100.

[0081] For example, the sensing system 120 can include a positioning system 121 (e.g., a global positioning system (GPS), a Beidou system, or other positioning system), an inertial measurement unit (IMU) 122, a radar 123, a laser rangefinder 124, a camera 125, and a vehicle speed sensor 126. The sensing system 120 can also include sensors that monitor the internal systems of the vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their respective characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the autonomous vehicle 100.

[0082] Wherein the positioning system 121 can be used to estimate the geographic position of the vehicle 100. The IMU 122 can be used to sense changes in position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 can be a combination of an accelerometer and a gyroscope.

[0083] Exemplarily, the radar 123 can utilize radio information to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 123 can also be used to sense the speed and / or direction of advance of the objects.

[0084] Exemplarily, the laser rangefinder 124 can utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 124 can include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0085] Exemplarily, the camera 125 can be used to capture multiple images of the surrounding environment of the vehicle 100. For example, the camera 125 can be a still camera or a video camera.

[0086] For example, the vehicle speed sensor 126 can be used to measure the speed of the vehicle 100. For instance, the vehicle speed can be measured in real time. The measured vehicle speed can be transmitted to the control system 130 to control the vehicle.

[0087] like Figure 1 As shown, the control system 130 controls the operation of the vehicle 100 and its components. The control system 130 may include various components, such as a steering system 131, an accelerator 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

[0088] For example, the steering system 131 can be operated to adjust the forward direction of the vehicle 100. For example, in one embodiment, it can be a steering wheel system. The throttle 132 can be used to control the operating speed of the engine 111 and thus the speed of the vehicle 100.

[0089] For example, braking unit 133 can be used to control the deceleration of vehicle 100; braking unit 133 can use friction to slow down wheel 114. In other embodiments, braking unit 133 can convert the kinetic energy of wheel 114 into electric current. Braking unit 133 can also take other forms to slow down the rotational speed of wheel 114 to control the speed of vehicle 100.

[0090] like Figure 1 As shown, the computer vision system 134 is operable to process and analyze images captured by the camera 125 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic information, road boundaries, and obstacles. The computer vision system 134 may use object recognition algorithms, structure-from-motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 134 may be used to map the environment, track objects, estimate object velocities, and so on.

[0091] For example, the route control system 135 can be used to determine the driving route of the vehicle 100. In some embodiments, the route control system 135 can combine data from sensors, GPS, and one or more predetermined maps to determine the driving route of the vehicle 100.

[0092] like Figure 1 As shown, obstacle avoidance system 136 can be used to identify, assess and avoid or otherwise traverse potential obstacles in the environment of vehicle 100.

[0093] In one instance, the control system 130 may include additional or alternative components besides those shown and described. Alternatively, some of the components shown above may be reduced.

[0094] like Figure 1 As shown, vehicle 100 can interact with external sensors, other vehicles, other computer systems or users through peripheral device 140; wherein peripheral device 140 may include wireless communication system 141, on-board computer 142, microphone 143 and / or speaker 144.

[0095] In some embodiments, peripheral device 140 may provide a means for vehicle 100 to interact with user interface 170. For example, on-board computer 142 may provide information to users of vehicle 100. User interface 116 may also operate on-board computer 142 to receive user input; on-board computer 142 may be operated via touchscreen. In other cases, peripheral device 140 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 143 may receive audio (e.g., voice commands or other audio input) from users of vehicle 100. Similarly, speaker 144 may output audio to users of vehicle 100.

[0096] like Figure 1 The wireless communication system 141 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 141 can use 3G cellular communication; such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE); or 5G cellular communication. The wireless communication system 141 can communicate using Wi-Fi and wireless local area networks (WLANs).

[0097] In some embodiments, the wireless communication system 141 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee protocol; other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 141 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between the vehicle and / or roadside stations.

[0098] like Figure 1As shown, power source 160 can provide power to various components of vehicle 100. In one embodiment, power source 160 can be a rechargeable lithium-ion battery or a lead-acid battery. One or more battery packs of such a battery can be configured as a power source to provide power to various components of vehicle 100. In some embodiments, power source 160 and energy source 113 can be implemented together, as in some all-electric vehicles.

[0099] By way of example, portions or all of the functionality of vehicle 100 can be controlled by computer system 150, which can include at least one processor 151 that executes instructions 153 stored in non-transitory computer readable media, such as memory 152. Computer system 150 can also be a plurality of computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0100] Processor 151 can be any conventional processor, such as a commercially available central processing unit (CPU).

[0101] Alternatively, the processor can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 Although the processor, memory, and other elements of the computer are functionally illustrated as being within the same block, it should be understood that the processor, computer, or memory can actually comprise multiple processors, computers, or memories that can or can not be stored in the same physical housing. For example, memory can be a hard drive or other storage medium located in a housing different from that of the computer. Reference to the processor or computer shall thus be understood to encompass reference to a collection of processors or computers or memories that can or can not operate in parallel, and that can be physically co-located or distributed across a variety of different physical locations. Rather than using a single processor to perform the steps described herein, such as some of the components of the steering assembly and the deceleration assembly, each can have its own processor that performs only the computations related to the functionality specific to that component.

[0102] In various aspects described herein, the processor can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0103] In some embodiments, memory 152 may contain instructions 153 (e.g., program logic) that can be used by processor 151 to perform various functions of vehicle 100, including those described above. Memory 152 may also include additional instructions, such as instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the mobility system 110, sensing system 120, control system 130, and peripheral devices 140.

[0104] For example, in addition to instruction 153, memory 152 may also store data, such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information can be used by vehicle 100 and computer system 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0105] like Figure 1 As shown, the user interface 170 can be used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 170 may include one or more input / output devices within a set of peripheral devices 140, such as a wireless communication system 141, an on-board computer 142, a microphone 143, and a speaker 144.

[0106] In embodiments of this application, computer system 150 can control the functions of vehicle 100 based on input received from various subsystems (e.g., mobility system 110, sensing system 120, and control system 130) and from user interface 170. For example, computer system 150 can utilize input from control system 130 to control braking unit 133 to avoid obstacles detected by sensing system 120 and obstacle avoidance system 136. In some embodiments, computer system 150 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0107] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 152 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0108] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0109] Optionally, vehicle 100 may be an autonomous vehicle traveling on a road, capable of identifying objects in its surrounding environment to determine adjustments to its current speed. These objects may be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object may be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0110] Optionally, the vehicle 100 or a computing device associated with the vehicle 100 (such as...) Figure 1 The computer system 150, computer vision system 134, and memory 152 can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.).

[0111] Optionally, since each identified object depends on the behavior of others, the behavior of a single identified object can also be predicted by considering all identified objects together. Vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine, based on the predicted behavior of the objects, that the vehicle will need to adjust to a steady state (e.g., accelerate, decelerate, or stop). In this process, other factors can also be considered in determining the speed of vehicle 100, such as the lateral position of vehicle 100 on the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0112] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0113] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0114] In one possible implementation, the above Figure 1 The vehicle 100 shown may be an autonomous vehicle. The autonomous driving system is described in detail below.

[0115] Figure 2 This is a schematic diagram of an autonomous driving system applicable to the embodiments of this application.

[0116] like Figure 2The illustrated autonomous driving system includes a computer system 201, which includes a processor 203 coupled to a system bus 205. The processor 203 can be one or more processors, each of which can include one or more processor cores. A video adapter 207 can drive a display 209 coupled to the system bus 205 via the display adapter. The system bus 205 can be coupled via a bus bridge 211 to an input / output (I / O) bus 213, where a memory controller 215 can be coupled to the I / O bus 213. The memory controller 215 can be coupled to a memory 217, such as a random access memory (RAM), and a storage device 219, such as a disk drive or other storage device. The I / O bus 213 can be coupled to an I / O interface 221, which can be coupled to various I / O devices, such as a keyboard 223, a mouse 225, a touchpad, a touch screen, a trackpad, a trackball, a stylus, a microphone, a camera, a speaker, a display screen 227, a printing device, a scanner, a network interface 229, and a disk drive or other storage device. The I / O devices can be built-in or external to the computer system 201. The network interface 229 can be a hardware network interface, such as a network card, and can enable the computer system 201 to communicate with one or more remote computer systems.

[0117] The processor 203 can be any conventional processor, such as a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of such processors.

[0118] Alternatively, the processor 203 can be a special purpose device, such as an application specific integrated circuit (ASIC), or a combination of such a device and a conventional processor. The processor 203 can be a neural network processor or a combination of a neural network processor and a conventional processor.

[0119] Alternatively, in some embodiments, the computer system 201 can be located remotely from the autonomous vehicle and can communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are performed on a processor located within the autonomous vehicle, and others are performed by a remote processor, including taking the actions required to perform a single maneuver.

[0120] The computer system 201 can communicate with a software deployment server 249 via a network interface 229 and a network 227. The network interface 229 can be a hardware network interface, such as a network card. The network 227 can be an external network, such as the Internet, or an internal network, such as an Ethernet or virtual private network (VPN). Alternatively, the network 227 can be a wireless network, such as a WiFi network, a cellular network, or the like.

[0121] As Figure 2As shown, the hard drive interface and the system bus 205 are coupled, and the hardware drive interface 231 can be connected with a hard drive 233, and the system memory 235 is coupled with the system bus 205. The data running in the system memory 235 can include an operating system 237 and an application program 243. The operating system 237 can include a shell 239 and a kernel 241. The shell 239 is an interface between the user and the kernel of the operating system. The shell can be the outermost layer of the operating system; the shell can manage the interaction between the user and the operating system, such as waiting for the user's input, explaining the user's input to the operating system, and processing various outputs of the operating system. The kernel 241 can be composed of those parts of the operating system used to manage memory, files, peripherals, and system resources. Directly interacting with hardware, the operating system kernel usually runs processes and provides communication between processes, provides CPU time slice management, interrupts, memory management, IO management, and the like. The application program 243 includes programs related to controlling the automatic driving of the vehicle, such as programs for managing the interaction between the automatic driving vehicle and the obstacles on the road, programs for controlling the route or speed of the automatic driving vehicle, programs for controlling the interaction between the automatic driving vehicle and other automatic driving vehicles on the road. The application program 243 also exists on the system of the software deployment server 249. In one embodiment, when the automatic driving related program 247 needs to be executed, the computer system 201 can download the application program from the software deployment server 249.

[0122] For example, the application program 243 can also be a program for interacting between the automatic driving vehicle and the lane line on the road, that is, a program for tracking the lane line in real time.

[0123] For example, the application program 243 can also be a program for controlling the automatic driving vehicle to automatically park.

[0124] Exemplarily, the sensor 253 can be associated with the computer system 201, and the sensor 253 can be used to detect the environment around the computer 201.

[0125] For example, the sensor 253 can detect the lane on the road, such as detecting the lane line, and can track the lane line change within a certain range in front of the vehicle in real time during the movement (such as driving) of the vehicle. For another example, the sensor 253 can detect animals, cars, obstacles, and crosswalks, and further the sensor can detect the environment around the above-mentioned animals, cars, obstacles, and crosswalks, such as the environment around the animals, for example, other animals appearing around the animals, weather conditions, brightness of the surrounding environment, and the like.

[0126] Alternatively, if the computer 201 is located on an autonomous vehicle, the sensors may be cameras, infrared sensors, chemical detectors, microphones, etc.

[0127] For example, in a lane tracking scenario, sensor 253 can be used to detect lane lines in front of the vehicle, enabling the vehicle to perceive lane changes during travel and plan and adjust the vehicle's driving in real time accordingly.

[0128] For example, in an automatic parking scenario, sensor 253 can be used to detect the size or position of parking spaces and surrounding obstacles around the vehicle, thereby enabling the vehicle to perceive the distance between the parking spaces and surrounding obstacles, perform collision detection during parking, and prevent the vehicle from colliding with obstacles.

[0129] In one example Figure 1 The computer system 150 shown can also receive information from other computer systems or transfer information to other computer systems. Alternatively, sensor data collected from the sensing system 120 of the vehicle 100 can be transferred to another computer for processing, as described below. Figure 3 Let's take an example to illustrate.

[0130] Figure 3 This is a schematic diagram illustrating the application of a cloud-based command-driven autonomous vehicle according to an embodiment of this application.

[0131] like Figure 3 As shown, data from computer system 312 can be transmitted via a network to cloud-side server 320 for further processing. The network and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, intranet, virtual private network, wide area network, local area network, private network using proprietary communication protocols of one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations thereof; such communication can be conducted by any device capable of transmitting data to and from other computers, such as modems and wireless interfaces.

[0132] In one example, server 320 may include a server with multiple computers, such as a load balancing server cluster, which exchanges information with different nodes in the network for the purpose of receiving, processing, and transmitting data from computer system 312. The server may be configured similarly to computer system 312, having a processor 330, memory 340, instruction set 350, and data storage 360.

[0133] For example, the data 360 of server 320 may include information related to road conditions around the vehicle. For instance, server 320 may receive, detect, store, update, and transmit information related to road conditions around the vehicle.

[0134] For example, the relevant information of the road situation around the vehicle includes other vehicle information and obstacle information around the vehicle.

[0135] Figure 4 is a framework diagram of an automatic driving system according to an embodiment of the present application. Some modules in the framework diagram of the automatic driving system can be located in the processor 203 shown in FIG. 1. Figure 2

[0136] As shown in FIG. 2, the automatic driving system can include an environment perception module, a vehicle positioning module, a decision planning module, and a vehicle control module. Figure 4

[0137] The environment perception module can perform road edge detection, lane line detection, target obstacle detection, and traffic sign detection according to sensor data obtained by sensors, for determining the driving environment in which the vehicle is currently located. The vehicle positioning module can determine the current driving posture (position and driving speed, etc.) of the vehicle according to sensor data obtained by sensors. The decision planning module can determine the driving strategy (driving angle and driving speed, etc.) of the vehicle at the next moment according to the results output by the environment perception module and the vehicle positioning module. The vehicle control module can determine the vehicle control amount (parameters required to be adjusted by each component) according to the driving strategy, so as to realize automatic driving.

[0138] It should be understood that the environment perception module is an interaction port of the intelligent vehicle platform and the surrounding traffic scene, and is also the front-end input of the motion decision, planning and control system of the intelligent vehicle platform. The performance of the environment perception module directly determines the stability of the automatic driving platform in executing driving tasks. In the environment perception module, robust road edge (road boundary) detection, as one of the core tasks, plays a key role in vehicle collision prevention.

[0139] The present application provides a method and device for road edge detection based on a laser radar. The actual data test through road test can realize robust and accurate road edge detection in real traffic scenes, greatly improves the detection distance of the road edge, and can be applied to an automatic driving system or an advanced driving assistance system (ADAS) of a vehicle.

[0140] Figure 5 is a flowchart of a method 400 for road edge detection according to an embodiment of the present application. Figure 5 ​​The method shown can be executed by an automatic driving device, which can be a cloud service device, a terminal device, or a system composed of a cloud service device and a terminal device. For example, the method 400 can be run on a device with sufficient computing power to execute the automatic driving method, such as a vehicle-mounted computer with a computing power capable of meeting an update rate of 30 Hz or higher. For example, the method 400 can be executed by Figure 1 the computer system 150 in the cloud service device 100, Figure 2 the computer system 201 in the cloud service device 200, or Figure 3 the computer system 312 or the server 320 in the cloud service device 300.

[0141] The method 400 includes steps S410 to S440. The steps S410 to S440 are described in detail below.

[0142] S420, determining the road edge point cloud data according to the intra-layer features of each point cloud layer data in the plurality of point cloud layer data and the inter-layer features between the point cloud layer data adjacent to the point cloud data.

[0143] The intra-layer features of each point cloud layer data in the plurality of point cloud layer data and the inter-layer features between the point cloud layer data adjacent to each point cloud layer data are determined according to the plurality of detection beams of the laser radar. Since the intra-layer features of single scanning and the inter-layer features of multiple scanning are combined in the determination process of the road edge point cloud data, the effective point cloud data in the distance is not filtered out due to the sparsity of the point cloud data, which shortens the detection distance and effectively improves the distance of the road edge detection.

[0144] S430, determining the initial road edge according to the road edge point cloud data, and determining the road edge seed point according to the initial road edge.

[0145] The point cloud data in the road edge point cloud data is the point cloud data related to the road edge, the initial road edge is used to represent the road edge determined at the beginning of the road edge detection, and the road edge seed point is used to represent the true condition of the road edge. The straight lines extracted from the road edge point cloud data are screened and merged as the initial road edge. The data correlation is performed on the road edge point cloud data and the initial road edge, the distance between the point cloud data in the road edge point cloud data and the initial road edge is less than a first threshold value, which is considered as a successful correlation and is taken as a first road edge seed point, otherwise, the correlation fails and is taken as a first road edge candidate point. Moreover, the road edge seed point can be predicted by a road edge prediction model according to the first road edge seed point and the first seed candidate point, so as to constantly update the point cloud data of the road edge seed point.

[0146] It should be understood that in the part of generating the initial road edge, since there is no direct restriction on the number of line segments extracted by the straight line extraction from the road edge point cloud data, the adaptability to complex roads (curved roads or multi-fork intersections) is better, and it is not only suitable for a single road structure.

[0147] S440, determining a parameterized equation corresponding to the road edge according to the road edge seed point.

[0148] The parameterized equation of the road edge is obtained by fitting the point cloud data in the road edge seed point, and the parameterized equation can be applied to an automatic driving system or an ADAS of a vehicle to represent a real road edge.

[0149] The method 400 can further include a step S410 of determining a plurality of point cloud layer data according to original point cloud data.

[0150] The original point cloud data can be point cloud data obtained by a laser radar of a vehicle, specifically, point cloud data obtained by a plurality of laser emitting units in the laser radar detecting a target through a detection beam.

[0151] It should be understood that the original point cloud data can be point cloud data corresponding to one frame obtained by the laser radar, or can also be point cloud data corresponding to multiple frames. That is, the road edge detection method provided in the embodiments of the present application can output a road edge result according to point cloud data of each frame of the laser radar, or can output a road edge result according to point cloud data of multiple frames, which can be selected according to actual application scenarios.

[0152] Figure 6 is a flowchart of a method 500 for determining a plurality of point cloud layer data provided in the embodiments of the present application, which can correspond to the step S410 in the method 400 described above.

[0153] The method 500 includes steps S510 to S550. The steps S510 to S550 are described in detail below.

[0154] S510, obtaining original point cloud data.

[0155] The spatial environment of the target region is detected by emitting a detection beam by a plurality of laser emitting units in the laser radar, and original point cloud data of the spatial environment of the target region is obtained, which can be three-dimensional point cloud data.

[0156] S520, denoising the original point cloud data.

[0157] Some point cloud data in the original point cloud data is not in the range of interest for the driving task, for example, point cloud data with coordinates below the ground or point cloud data with coordinates much higher than the ground. The data points or noise points in the original point cloud data can be simply and preliminarily filtered out through their three-dimensional spatial coordinates.

[0158] S530, coordinate system conversion.

[0159] The original point cloud data is generated in the coordinate system of the laser radar and needs to be converted to the ego-vehicle coordinate system. There is a rotation and translation relationship between the coordinate system of the laser radar and the ego-vehicle coordinate system in three-dimensional space, which can be converted by using a 4x4 homogeneous coordinate transformation matrix. The 4x4 homogeneous coordinate transformation matrix includes a 3x3 space rotation matrix and a 3x1 space translation vector. It should be understood that the embodiments of the present application only take the 4x4 homogeneous coordinate transformation matrix as an example, and in actual application, other ways can also be used to realize the conversion between the coordinate system of the laser radar and the ego-vehicle coordinate system.

[0160] S540, the original point cloud data is reduced in dimension, and a point cloud feature map is generated.

[0161] Since the original point cloud data is three-dimensional point cloud data, a large amount of computing cost may be consumed in the process of processing (feature extraction) of the three-dimensional point cloud data, resulting in too low overall computing efficiency of the method. Therefore, the three-dimensional point cloud data can be reduced in dimension to two-dimensional point cloud data, and the two-dimensional point cloud data is represented in the point cloud feature map, greatly reducing the computing cost and improving the overall computing efficiency of the method. For example, the three-dimensional point cloud data can be converted into two-dimensional point cloud data by using spherical projection. The spherical projection formula is as follows:

[0162]

[0163] wherein (x, y, z) is the spatial coordinates of the three-dimensional point cloud data acquired by the laser radar, (u, v) is the coordinates of the corresponding pixel of the three-dimensional point cloud data in the point cloud feature map after spherical projection, Row is the number of laser emitting units in the laser radar, Col is the number of intervals in the horizontal direction, f down is the vertical field of view angle of the laser radar below the horizontal scanning, f is the vertical field of view angle of the laser radar (including the vertical field of view angle f down of the horizontal below scanning and the vertical field of view angle f up of the horizontal above scanning), and Rang is the spatial distance of the point cloud data from the laser radar.

[0164] In one embodiment, the point cloud feature map can be a point cloud depth map and a point cloud index map, or can also be other types of point cloud feature maps, which are not limited by the present application.

[0165] S550, a plurality of point cloud layer data is determined.

[0166] The two-dimensional plurality of point cloud layer data in the ego-vehicle coordinate system can be determined according to the original point cloud data and the generated point cloud feature map.

[0167] Figure 7This is a schematic diagram of a method 600 for determining roadside point cloud data provided in an embodiment of this application, which can correspond to step S420 in the method 400 described above.

[0168] The technical solution provided in this embodiment uses lidar to acquire point cloud data, and combines the intra-layer features of each point cloud layer with the inter-layer features between the point cloud layers adjacent to that point cloud layer to extract the roadside point cloud data.

[0169] The intra-layer features may include at least one of slope features, number features, or reflection intensity features. The inter-layer features may include at least one of inter-layer distance features or inter-layer number features.

[0170] Intra-layer features:

[0171] Slope characteristics: such as Figure 8 As shown, this is used to indicate point cloud data whose distribution direction and the vehicle's driving direction are at an angle θ less than a third threshold. The distribution direction can be understood as the direction formed by the line connecting the first point cloud data point to its adjacent point cloud data point within a single point cloud layer. The first point cloud data point can be any point cloud data point within the single point cloud layer. In actual traffic scenarios, vehicles travel between the roadside curbs on both sides; therefore, the angle between the distribution direction of the curb point cloud data and the vehicle's driving direction fluctuates within a relatively small range. For a single point cloud layer in the vehicle coordinate system, if the angle difference between the direction formed by the line connecting it to its adjacent point cloud data point and the driving direction in the two-dimensional plane is within a certain range, it can be identified as curb point cloud data.

[0172] Number feature: Used to indicate the number of point cloud data in a neighborhood of a single point cloud layer that is greater than a fourth threshold. For example... Figure 9 As shown, based on the characteristics of point cloud data acquired by lidar, the density value of the point cloud data obtained when the probe beam illuminates the curb is greater than the density value of the point cloud data obtained when it illuminates the road surface. Therefore, the number of point cloud data in the neighborhood of each point cloud data in the curb point cloud data must be greater than the fourth threshold. Figure 10 As shown in (a), the neighborhood of a single point cloud layer can be understood as two adjacent regions within the point cloud layer. For example, within region 3 of the first point cloud layer, the neighborhood of the point cloud data is region 2 and region 4. Alternatively, it can be understood as multiple adjacent regions within the point cloud layer. For example, within region 3 of the first point cloud layer, the neighborhood of the point cloud data is region 1, region 2, region 3, and region 4.

[0173] Reflection intensity feature: used to indicate the point cloud data whose reflection intensity is greater than a fifth threshold in the single point cloud layer data. Through this feature, some road edge noise points with low reflection intensity can be filtered out through the reflection intensity feature. For example, some road edge noise points with low reflection intensity introduced by rainwater due to rainy days, dust in the air and other factors can be filtered out. Alternatively, in another scheme, in an actual traffic road, the material of the road edge is generally a material with high reflection intensity, and therefore, the point cloud data whose reflection intensity is greater than the fifth threshold can also be determined as road edge point cloud data.

[0174] Interlayer feature:

[0175] Interlayer distance feature: used to indicate the point cloud data whose distance from the adjacent point cloud layer data is less than a sixth threshold. As shown in FIG. 6, each laser emitting unit in the laser radar has a fixed vertical illumination angle (an angle with the direction perpendicular to the road edge), and therefore, the point cloud data on the horizontal road surface is distributed in a ring shape. However, since the road edge is a plane perpendicular to the road surface, the distance between the point cloud data of the adjacent two point cloud layer data becomes small. Therefore, if the distance between the point cloud data and the adjacent point cloud layer data is less than the sixth threshold, it can be determined that the point cloud data is road edge point cloud data. Figure 9

[0176] Interlayer number feature: used to indicate the point cloud data whose number of point cloud data in the neighborhood of the adjacent point cloud layer data is greater than a seventh threshold. As shown in FIG. 7, according to the characteristics of the point cloud data obtained by the laser radar, the density value of the point cloud data obtained by the detection beam irradiated on the road edge is greater than that of the point cloud data obtained by the detection beam irradiated on the road surface. Therefore, the number of point cloud data in the neighborhood of the corresponding adjacent point cloud layer data of each point cloud data in the road edge point cloud data must be greater than the seventh threshold. As shown in (b) of FIG. 8, the neighborhood in the adjacent point cloud layer data can be understood as two adjacent regions of the point cloud data in the adjacent point cloud layer data, for example, the point cloud data is in region 1 of the first point cloud layer data, and the neighborhood is region 2 of the second point cloud layer data and region 2 of the third point cloud layer data. Alternatively, it can also be understood as a plurality of adjacent regions of the point cloud data in the adjacent point cloud layer data, for example, the point cloud data is in region 3 of the first point cloud layer data, and the neighborhood is region 1, region 2 and region 3 of the second point cloud layer data and region 1, region 2 and region 3 of the third point cloud layer data. Figure 9 Figure 10

[0177] ​​​It should be understood that the method for determining the road edge point cloud data provided in the embodiment combines the features of a single point cloud layer data and the features between two adjacent point cloud layer data, and the effective point cloud data in the distance is not filtered out due to the sparsity of the point cloud data, so that the detection distance is shortened, and the distance of the road edge detection is effectively improved. The designed features basically conform to the actual features of most road edges in a real traffic scene.

[0178] Figure 11 is a flowchart of a method 700 for determining a road edge seed point provided by the embodiment of the present application, which can correspond to step S430 in the method 400 described above.

[0179] The method 700 includes steps S710 to S730. The steps S710 to S730 are described in detail below.

[0180] S710, determining an initial road edge according to the road edge point cloud data.

[0181] To reduce the calculation cost, the determined road edge point cloud data can be projected in a top view plane, as shown in (a) of FIG. 7, the top view plane can be considered as a horizontal plane or a plane where the road is located. A plurality of initial straight line segments can be determined in the projected road edge point cloud data by using the Hough transform, as shown in (b) of FIG. 7. Figure 12 Figure 12 The plurality of initial straight line segments can be used to determine the initial condition of the road edge. However, since only a part of the road edge distribution in the real road scene conforms to the straight line distribution, the plurality of initial straight line segments needs to be screened, and the screened plurality of initial straight line segments needs to be merged to form an initial road edge, as shown in (c) of FIG. 7. Figure 12

[0182] It should be understood that the Hough transform is only a method for extracting straight line segments by transforming space, and in actual application, the plurality of initial straight line segments can also be obtained by other methods, which are not limited by the present application.

[0183] The screening process of the plurality of initial straight line segments can include removing the straight line segments with shorter length or larger slope difference from the vehicle driving direction.

[0184] The merging process of the screened plurality of initial straight line segments can include merging the straight line segments with smaller slope difference and closer starting point distance into new straight line segments, and finally obtaining the initial road edge.

[0185] ​​The initial straight line segment is extracted by projecting the road edge point cloud data in the top view plane, which has less overhead than directly calculating in three-dimensional space. Meanwhile, in the part generated by the initial road edge, there is no direct restriction on the number of initial straight line segments, which is more adaptable to complex roads and not only suitable for a single road structure, such as straight roads.

[0186] S720, determining a first group of seed points and a first group of candidate points according to the initial road edge.

[0187] The first group of seed points and the first group of candidate points can be determined according to the distance between the initial road edge and the first group of seed points. The first group of seed points is used to represent the true state of the first part of the road edge, and the first group of candidate points is the point cloud data in the road edge point cloud data except the first group of seed points. The point cloud data in the road edge point cloud data with a distance less than a first threshold value from the initial road edge can be used as the point cloud data of the first group of seed points, and the point cloud data with a distance greater than or equal to the first threshold value can be used as the point cloud data in the first group of candidate points, as shown in (d) of FIG. 6. Figure 12 The first group of seed points can be used as road edge seed points and can be used to represent the true state of the road edge.

[0188] S730, determining road edge seed points according to the first group of seed points and the first group of candidate points.

[0189] The distribution of the point cloud data corresponding to the road edge can be predicted by the road edge prediction model according to the first group of seed points and the first group of candidate points, and the road edge seed points can be updated through multiple iterations to further improve the detection distance and detection effect of the road edge detection. For example, a second group of seed points and a second group of candidate points can be determined according to the first group of seed points and the first group of candidate points, the first group of seed points and the second group of seed points are determined as the road edge seed points, the second group of seed points is used to represent the true state of the second part of the road edge, and the second group of candidate points is the point cloud data in the road edge point cloud data except the first group of seed points and the second group of seed points. A third group of seed points and a third group of candidate points are determined according to the first group of seed points, the second group of seed points and the second group of candidate points, the first group of seed points, the second group of seed points and the third group of seed points are determined as the road edge seed points, and the road edge seed points are updated through multiple iterations according to the method, the third group of seed points is used to represent the true state of the third part of the road edge, and the third group of candidate points is the point cloud data in the road edge point cloud data except the first group of seed points, the second group of seed points and the third group of seed points. It should be understood that the first part of the road edge, the second part of the road edge and the third part of the road edge can be different, and the first part of the road edge, the second part of the road edge and the third part of the road edge can be continuous and used to represent a continuous road edge.

[0190] Figure 13 is a schematic flowchart of a training method of a road edge prediction model provided by an embodiment of the present application, which can be applied to Figure 11At S730 of the method 700 shown, the lane seed point is constantly updated for multiple iterations. The model can be a model established based on Gaussian process regression.

[0191] Figure 13 The method shown can be executed by a computer system 150 in the Figure 1 computer system 201 in the Figure 2 computer system 312 or the server 320 in the Figure 3 or the like device with strong computing power. Figure 13 The method shown includes steps S801 to S804, which will be introduced in detail below.

[0192] S801, label the lane ground truth.

[0193] The lane ground truth is manually labeled in the point cloud data obtained by the laser radar, and the lane ground truth is the real value corresponding to the lane in the point cloud data obtained by the laser radar.

[0194] It should be understood that in the labeling process, multiple road structure scenes and multiple lane types should be selected to ensure the generality of the data and to increase the robustness and adaptability of the neural network strong model.

[0195] S802, construct a training data set, and the training data set includes multiple manually labeled lane ground truths.

[0196] S803, train the parameters of the lane prediction model.

[0197] Since the lane prediction model can be a model established based on Gaussian process regression, Gaussian process regression is a non-parametric model, but its kernel function includes hyperparameters, which need to be obtained through training. The hyperparameters of the kernel function can be trained by using the maximum marginal log-likelihood of the lane ground truth in the training data set.

[0198] It should be understood that since Gaussian process regression is a non-parametric model, it does not require the road to have a specific shape, and therefore, for difficult scenes such as roads with large curvature changes or construction sections, Gaussian process regression can better predict the lane distribution of the road.

[0199] Alternatively, the parameters of the lane prediction model can also be obtained by other machine learning methods, and the present application does not limit this, which can be adjusted according to the actual production or design needs.

[0200] S804, construct a lane prediction model.

[0201] A roadside prediction model is constructed based on the hyperparameters of the kernel function of the Gaussian process regression obtained through training. This model can be used to update roadside seed points, thereby improving the detection distance and detection performance of roadside detection. The roadside prediction model belongs to the field of machine learning, but it can also be obtained through other methods; this application does not impose any restrictions on this.

[0202] Figure 14 This is a flowchart of a method for updating a curb seed point provided in an embodiment of this application, which can correspond to step S730 in the method 700 described above.

[0203] Method 800 includes steps S810 to S830. Steps S810 to S830 are described in detail below.

[0204] S810, determine the first set of key seed points and the first set of key candidate points.

[0205] Since the computational cost of the curb prediction model is positively correlated with the input data, in order to reduce the computational cost of the curb prediction model in subsequent steps without affecting the prediction accuracy, the first set of seed points and the first set of candidate points can be rasterized, and the first set of key seed points and the first set of key candidate points can be determined based on the rasterized first set of seed points and the first set of candidate points.

[0206] like Figure 15 As shown, keypoints can be determined based on seed points within a grid. The coordinates of the keypoints can be the mean coordinates of the point cloud data within the grid. Simultaneously, to avoid the influence of noise on the roadside prediction results, the weights of the keypoints can be determined based on the number of point cloud data points within the grid, generating a corresponding weight matrix. Each keypoint is represented by (x, y, weight). That is, the number of point cloud data points can be input into the sample weight function, designed as shown in the following formula. It should be understood that the above scheme can also be used to determine the corresponding keypoints for candidate points within a grid.

[0207]

[0208] Where n is the number of point cloud data in the raster, and n1 and n2 are the thresholds (upper and lower limits) of the point cloud data in the raster, respectively.

[0209] S820, determine the curb prediction point based on the curb prediction model.

[0210] Based on the above Figure 13 The method shown trains a roadside prediction model that generates roadside prediction points, which can be used to represent the predicted condition of the roadside. The roadside prediction model can be pre-trained and set up as an offline component in autonomous driving systems or ADAS.

[0211] The key seed points are used as training data of the road edge prediction model, and the key candidate points are used as test data of the road edge prediction model. In the prediction process, the x coordinate value of the road edge prediction point is selected as the x coordinate value of the key candidate point, and the y coordinate value is the value obtained by the road edge prediction model.

[0212] S830, updating the road edge seed points according to the road edge prediction points.

[0213] If the deviation between the road edge prediction point and the key candidate point is less than the threshold value, the candidate points in the grid corresponding to the key candidate point are updated as the road edge seed points and removed from the candidate points; otherwise, they are kept as candidate points.

[0214] The deviation between the road edge prediction point and the key candidate point can be calculated by the following formula:

[0215]

[0216] wherein y test is the y coordinate value of the key candidate point, is the y coordinate value of the road edge prediction point, and C is the variance value of the road edge prediction point, is the noise variance value of the hyperparameter in the kernel function, and T is the deviation threshold value, which can be a preset value.

[0217] It should be understood that S820 and S830 can be iterative steps, and the training data and test data of the road edge prediction model can be updated by the road edge prediction points, so that the road edge seed points are updated multiple times. For example, in the initial state, the first group of key seed points and the first group of key candidate points are used as the training data and test data of the road edge prediction model, respectively. According to the obtained first group of road edge prediction points, the point cloud data in the first group of key candidate points with a deviation less than the threshold value from the first group of road edge prediction points can be used as the second group of key seed points, and the remaining key candidate points can be used as the second group of key candidate points. At this time, the road edge seed points can include point cloud data in the grid corresponding to the first group of key seed points and the second group of key seed points. In the first iteration, the first group of key seed points and the second group of key seed points can be used as the training data of the road edge prediction model, and the second group of key candidate points can be used as the test data of the road edge prediction model. According to the obtained second group of road edge prediction points, the point cloud data in the second group of key candidate points with a deviation less than the threshold value from the second group of road edge prediction points can be used as the third group of key seed points, and the remaining key candidate points can be used as the third group of key candidate points. Through the above steps, the road edge seed points can be updated multiple times to obtain a longer road edge detection distance. Figure 16 As shown in FIG. 6, through multiple iterations, the point cloud data in the road edge seed points is continuously increased, and the distance of the road edge detection is continuously increased.

[0218] Meanwhile, whether the sparse point cloud data in the distance is a noise point can be determined through the road edge prediction model, without losing effective point cloud data, and the road edge detection accuracy is improved. Since the road edge seed point is updated in multiple iterations, the detection effect is still good on complex road sections such as large-bend road sections, low road edge road sections in urban areas, and road sections in construction areas.

[0219] S840, determining whether to exit the iteration.

[0220] The condition for ending the iteration can be set according to actual design requirements. When the condition is met, the iteration is exited, and the current determined road edge seed point is output. When the condition is not met, steps S820 and S830 are continued.

[0221] For example, the condition for ending the iteration can be whether the number of point cloud data in the candidate point is less than or equal to a number threshold. If it is less than or equal to the number threshold, the iteration is exited and the determined road edge seed point is output. If it is greater than the number threshold, whether the number of iterations is greater than or equal to an iteration threshold is determined. If it is greater than or equal to the iteration threshold, the iteration is exited and the determined road edge seed point is output. Otherwise, the updated key seed point and the key candidate point are taken as the training data and the test data of the road edge prediction model, respectively, until the iteration exit condition is met. Alternatively, the condition for ending the iteration can also be that the number of point cloud data in the road edge seed point is greater than a number threshold. The present application does not make any limitation thereon, and the condition for ending the iteration can be adjusted according to actual conditions.

[0222] It should be understood that the output road edge seed point can be fitted with the point cloud data in the road edge seed point through S430 in the method 400 to obtain a parameterized equation of the road edge, which can be applied to an automatic driving system or an ADAS of a vehicle.

[0223] The road edge detection method provided by the embodiments of the present application basically meets the vast majority of road structures of highway sections and urban road sections, and is suitable for complex road scenes. In the generation process of the initial road edge, a clustering algorithm is not used to remove the sparse point cloud data in the distance. Whether the sparse point cloud data in the distance is a noise point is determined through the subsequent road edge prediction model, without losing effective point cloud data. Meanwhile, the input data is rasterized and the key points and weights are extracted through the road edge prediction model, which improves the iteration calculation efficiency of the road edge prediction model. The average output frequency of the road edge detection result can reach 50 Hz. Based on actual road test data, the road edge is detected by a 40-line laser radar, and the average detection distance can reach more than 50 m.

[0224] Figure 17 is a schematic block diagram of the road edge detection device 900 provided by the embodiments of the present application. Figure 17The shown road edge detection apparatus 900 comprises a processing unit 910 and an acquisition unit 920.

[0225] The processing unit 910 and the acquisition unit 920 can be configured to perform the road edge detection method of the embodiments of the present application, for example, any one of the methods 400-800 described above.

[0226] The processing unit 910 can be configured to determine road edge point cloud data according to the features of a single point cloud layer data in the plurality of point cloud layer data and the features between two adjacent point cloud layer data, determine an initial road edge according to the road edge point cloud data, and determine a road edge seed point according to the initial road edge, the road edge seed point being used to represent the true condition of the road edge, and determine a parameterized equation of the road edge according to the road edge seed point.

[0227] In one embodiment, the processing unit 910 is specifically configured to determine a first group of seed points and a first group of candidate points according to the initial road edge, each point cloud data in the first group of seed points being less than a first threshold value from the initial road edge, determine a second group of seed points according to the first group of seed points and the first group of candidate points, and determine the first group of seed points and the second group of seed points as the road edge seed points.

[0228] In one embodiment, the processing unit 910 is further specifically configured to determine a second group of seed points and a second group of candidate points according to the first group of seed points and the first group of candidate points, and determine the first group of seed points and the second group of seed points as the road edge seed points, including determining a third group of seed points according to the first group of seed points, the second group of seed points and the second group of candidate points, and determining the first group of seed points, the second group of seed points and the third group of seed points as the road edge seed points.

[0229] In one embodiment, the processing unit 910 is further specifically configured to determine a first group of key seed points according to the first group of seed points, the point cloud data in the first group of key seed points corresponding to a grid in the gridded first group of seed points, the coordinate value of the point cloud data in the first key seed point being the coordinate average of the point cloud data in the corresponding grid in the first group of seed points, and the weight of the point cloud data in the first key seed point being determined by the number of point cloud data in the corresponding grid, determine a first group of key candidate points according to the first group of candidate points, the point cloud data in the first group of key candidate points corresponding to a grid in the gridded first group of candidate points, the coordinate value of the point cloud data in the first key candidate point being the coordinate average of the point cloud data in the corresponding grid in the first group of candidate points, and the weight of the point cloud data in the first key candidate point being determined by the number of point cloud data in the corresponding grid, and determine the second group of seed points according to the first group of key seed points and the first group of key candidate points.

[0230] In an embodiment, the processing unit 910 is further specifically configured to: determine a first set of predicted points according to the first set of key seed points and the first set of key candidate points through a road edge prediction model; determine point cloud data in the first set of key candidate points that satisfies a first condition as point cloud data in the second set of seed points, the first condition being that a deviation between the point cloud data and point cloud data in the first set of road edge predicted points is less than a second threshold; wherein the road edge prediction model is obtained according to a data set, the data set including a plurality of training data, each training data including the road edge ground truth, the road edge ground truth being a true value corresponding to the road edge in point cloud data obtained by the lidar.

[0231] In an embodiment, the processing unit 910 is further specifically configured to: determine an initial road edge according to point cloud data in a top view plane projection of the road edge point cloud data.

[0232] In an embodiment, the processing unit 910 is further specifically configured to: determine a plurality of initial straight line segments according to the point cloud data in the top view plane projection; and determine the initial road edge according to the plurality of initial straight line segments.

[0233] In an embodiment, the features of the single point cloud layer data include a slope feature, a number feature, and a reflection intensity feature; wherein the slope feature is used to indicate point cloud data in the single point cloud layer data whose distribution direction and the driving direction of the vehicle form an angle less than a third threshold value, the number feature is used to indicate point cloud data in the single point cloud layer data whose number of point cloud data in a neighborhood is greater than a fourth threshold value, and the reflection intensity feature is used to indicate point cloud data in the single point cloud layer data whose reflectivity is greater than a fifth threshold value.

[0234] In an embodiment, the features between the two adjacent point cloud layer data include an inter-layer distance feature and an inter-layer number feature; wherein the inter-layer distance feature is used to indicate point cloud data whose distance between adjacent point cloud layer data is less than a sixth threshold value, and the inter-layer number feature is used to indicate point cloud data whose number of point cloud data in a neighborhood of adjacent point cloud layer data is greater than a seventh threshold value.

[0235] In an embodiment, the acquisition unit 920 is configured to acquire original point cloud data, and the processing unit 910 is further configured to: determine the plurality of point cloud layer data according to the original point cloud data; wherein the original point cloud data is point cloud data obtained by a lidar, and the plurality of point cloud layer data is two-dimensional point cloud data in a self-vehicle coordinate system.

[0236] In an embodiment, the processing unit 910 is further specifically configured to: generate a point cloud feature map according to point cloud data after spherical projection of the original point cloud data; and determine the plurality of point cloud layer data according to the point cloud feature map.

[0237] In one embodiment, the point cloud feature map comprises a point cloud depth map and a point cloud index map.

[0238] Figure 18 Fig. 1 is a schematic structural diagram of a control device of a curb detection device provided by an embodiment of the present application.

[0239] The control device 1300 of the detection device comprises at least one processing unit 1310 and an interface circuit 1320. Optionally, it can also comprise a memory 1330 for storing programs.

[0240] When the programs run in the at least one processing unit 1310, the at least one processing unit 1310 is used to execute the method of curb detection described above.

[0241] In one design, there can be multiple processing units in the control device in terms of function division, different processing units perform different control functions, and multiple processing units communicate with the processing unit performing central control to communicate information and data with the processing unit. For example, a first processing unit is used to control a laser radar, a second processing unit is used to process raw point cloud data acquired by the laser radar, a third processing unit is used to perform analog-to-digital conversion, such as an analog-to-digital converter ADC circuit, a fourth processing unit is used to perform photoelectric signal conversion, such as a photodiode circuit, and / or a fifth processing unit performs digital signal processing.

[0242] These processing units can be various types of processors, integrated circuits, field programmable gate arrays FPGA, etc., and the present application does not specifically limit the form of integration on the chip as long as the above functions can be realized. For the convenience of description, the processing unit can also be referred to as a processor. Further, the above processing units can be integrated on one chip or distributed on multiple chips, and the present application does not specifically limit it, and the specific design is used as the criterion.

[0243] The embodiments of the present application also provide a computer readable storage medium having program instructions, when the program instructions are executed directly or indirectly, the method described above is realized.

[0244] In the embodiments of the present application, a computer program product comprising instructions is also provided, when it runs on a computing device, the computing device executes the method described above, or the computing device realizes the function of the device described above.

[0245] The embodiment of the present application further provides a chip system, which comprises at least one processing unit and an interface circuit, the interface circuit is used for providing program instructions or data for the at least one processing unit, and the at least one processor is used for executing the program instructions to realize the method in the foregoing.

[0246] The embodiment of the present application further provides a terminal, which comprises the road edge detection device in the foregoing. Further, the terminal can be a smart transportation device (a vehicle or a drone), a smart home device, a smart manufacturing device, a surveying and mapping device or a robot, etc. The smart transportation device can be an automated guided vehicle (AGV) or a unmanned vehicle, for example.

[0247] Those skilled in the art can understand that the units, modules and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0248] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0249] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0250] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0251] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of curb detection, applied in a vehicle, the vehicle comprising a lidar, characterized in that, The method comprises: determining a plurality of point cloud layer data from original point cloud data, the original point cloud data being point cloud data acquired by the laser radar, the plurality of point cloud layer data being two-dimensional point cloud data in a self-vehicle coordinate system; determining road edge point cloud data according to intra-layer features of each point cloud layer data in the plurality of point cloud layer data and inter-layer features between the each point cloud layer data and adjacent point cloud layer data of the each point cloud layer data; determining an initial road edge according to the road edge point cloud data, and determining a road edge seed point according to the initial road edge, the initial road edge being used to represent a road edge determined when road edge detection starts, and the road edge seed point being used to represent a true condition of the road edge; determining a parameterized equation corresponding to the road edge according to the road edge seed point; wherein the determining of the road edge seed point according to the initial road edge comprises: determining a first group of seed points and a first group of candidate points according to the initial road edge, each point cloud data in the first group of seed points being less than a first threshold value in distance from the initial road edge, the first group of seed points being used to represent a true condition of a first part of the road edge, and the first group of candidate points being point cloud data in the road edge point cloud data other than the first group of seed points; determining the first group of seed points as the road edge seed point.

2. The method of claim 1, wherein: the intra-layer features comprise at least one of a slope feature, a number feature or a reflection intensity feature; wherein the slope feature is used to indicate point cloud data in the each point cloud layer data having a distribution direction and an angle with a driving direction of the vehicle less than a third threshold value, the number feature is used to indicate point cloud data in the each point cloud layer data having a number of point cloud data in a neighborhood greater than a fourth threshold value, and the reflection intensity feature is used to indicate point cloud data in the each point cloud layer data having a reflectivity greater than a fifth threshold value.

3. The method of claim 1 or 2, wherein: the inter-layer features comprise at least one of an inter-layer distance feature or an inter-layer number feature; wherein the inter-layer distance feature is used to indicate point cloud data in the each point cloud layer data having a distance from adjacent point cloud layer data less than a sixth threshold value, and the inter-layer number feature is used to indicate point cloud data in the each point cloud layer data having a number of point cloud data in a neighborhood of the adjacent point cloud layer data greater than a seventh threshold value.

4. The method of claim 1, wherein, determining the first group of seed points as the road edge seed point comprises: determining a second group of seed points from the first group of seed points and the first group of candidate points, the second group of seed points being used to represent a true condition of a second part of the road edge; determining the first group of seed points and the second group of seed points as the road edge seed point.

5. The method of claim 4, wherein, the determining of the second group of seed points from the first group of seed points and the first group of candidate points comprises: determining a second group of seed points and a second group of candidate points from the first group of seed points and the first group of candidate points, the second group of candidate points being point cloud data in the road edge point cloud data other than the first group of seed points and the second group of seed points; the determining of the first group of seed points and the second group of seed points as the road edge seed point comprises: According to the first group of seed points, the second group of seed points and the second group of candidate points, a third group of seed points is determined, and the first group of seed points, the second group of seed points and the third group of seed points are determined as the road edge seed points, and the third group of seed points are used to represent the true condition of the third part of the road edge.

6. The method according to claim 4 or 5, characterized in that, The determining the second group of seed points according to the first group of seed points and the first group of candidate points further includes: According to the first group of seed points, a first group of key seed points is determined, the point cloud data in the first group of key seed points corresponds to the grid in the first group of seed points after rasterization, the coordinate value of the point cloud data in the first group of key seed points is the coordinate mean value of the point cloud data in the corresponding grid in the first group of seed points, and the weight of the point cloud data in the first group of key seed points is determined by the number of point cloud data in the corresponding grid. According to the first group of candidate points, a first group of key candidate points is determined, the point cloud data in the first group of key candidate points corresponds to the grid in the first group of candidate points after rasterization, the coordinate value of the point cloud data in the first group of key candidate points is the coordinate mean value of the point cloud data in the corresponding grid in the first group of candidate points, and the weight of the point cloud data in the first group of key candidate points is determined by the number of point cloud data in the corresponding grid. The second group of seed points is determined according to the first group of key seed points and the first group of key candidate points.

7. The method of claim 6, wherein, The determining the second group of seed points according to the first group of key seed points and the first group of key candidate points includes: According to the first group of key seed points and the first group of key candidate points, a first group of predicted points is determined through a road edge prediction model, and the first group of predicted points are used to represent the predicted condition of the second part of the road edge. The point cloud data in the first group of key candidate points that satisfies a first condition is determined as the point cloud data in the second group of seed points, and the first condition is that the deviation between the point cloud data in the first group of predicted points is less than a second threshold. The road edge prediction model is obtained through machine learning.

8. The method of claim 1, wherein, The determining the initial road edge according to the road edge point cloud data includes: The initial road edge is determined according to the point cloud data in the top view plane projection of the road edge point cloud data.

9. The method of claim 8, wherein, The determining the initial road edge according to the point cloud data in the top view plane projection of the road edge point cloud data includes: A plurality of initial straight line segments are determined according to the point cloud data in the top view plane projection. The initial road edge is determined according to the plurality of initial straight line segments.

10. The method of claim 1, wherein, The determining the plurality of point cloud layer data according to the original point cloud data includes: A point cloud feature map is generated according to the point cloud data after spherical projection of the original point cloud data; The plurality of point cloud layer data is determined according to the point cloud feature map.

11. The method of claim 10, wherein, The point cloud feature map includes a point cloud depth map and a point cloud index map.

12. A curb detection device characterized by comprising: It includes: An acquisition unit is configured to acquire original point cloud data; A processing unit is configured to: determine a plurality of point cloud layer data according to the original point cloud data, the original point cloud data being point cloud data acquired by a laser radar, and the plurality of point cloud layer data being two-dimensional point cloud data in a self-vehicle coordinate system. determine the road edge point cloud data according to the intra-layer feature of each point cloud layer data in the plurality of point cloud layer data and the inter-layer feature between the adjacent point cloud layer data of the each point cloud layer data; determine an initial road edge according to the road edge point cloud data, and determine a road edge seed point according to the initial road edge, the initial road edge being used to represent the road edge determined when the road edge detection starts, and the road edge seed point being used to represent the real condition of the road edge; determine a parameterized equation corresponding to the road edge according to the road edge seed point wherein the processing unit is specifically configured to: determine a first group of seed points and a first group of candidate points according to the initial road edge, each point cloud data in the first group of seed points being less than a first threshold value in distance from the initial road edge, the first group of seed points being used to represent the real condition of a first part of the road edge, and the first group of candidate points being the point cloud data in the road edge point cloud data except the first group of seed points; determine the first group of seed points as the road edge seed points.

13. The apparatus according to claim 12, wherein the intra-layer feature comprises at least one of a slope feature, a number feature or a reflection intensity feature; wherein the slope feature is used to indicate the point cloud data in the each point cloud layer data with a distribution direction and an angle of less than a third threshold value from a driving direction of a vehicle, the number feature is used to indicate the point cloud data in the each point cloud layer data with a number of point cloud data in a neighborhood greater than a fourth threshold value, and the reflection intensity feature is used to indicate the point cloud data in the each point cloud layer data with a reflectivity greater than a fifth threshold value.

14. The apparatus according to claim 12 or 13, wherein the inter-layer feature comprises at least one of an inter-layer distance feature or an inter-layer number feature; wherein the inter-layer distance feature is used to indicate the point cloud data in the each point cloud layer data with a distance from the adjacent point cloud layer data less than a sixth threshold value, and the inter-layer number feature is used to indicate the point cloud data in the each point cloud layer data with a number of point cloud data in a neighborhood of the adjacent point cloud layer data greater than a seventh threshold value.

15. The apparatus of claim 12, wherein, the processing unit is specifically configured to: determine a second group of seed points according to the first group of seed points and the first group of candidate points, the second group of seed points being used to represent the real condition of a second part of the road edge; determine the first group of seed points and the second group of seed points as the road edge seed points.

16. The apparatus of claim 15, wherein, the processing unit is further specifically configured to: determine a second group of seed points and a second group of candidate points according to the first group of seed points and the first group of candidate points, the second group of candidate points being the point cloud data in the road edge point cloud data except the first group of seed points and the second group of seed points; the determination of the first group of seed points and the second group of seed points as the road edge seed points comprises: determine a third group of seed points according to the first group of seed points, the second group of seed points and the second group of candidate points, determine the first group of seed points, the second group of seed points and the third group of seed points as the road edge seed points, and the third group of seed points being used to represent the real condition of a third part of the road edge.

17. The apparatus of claim 15 or 16, wherein, the processing unit is further specifically configured to: determining a first set of key seed points according to the first set of seed points, point cloud data in the first set of key seed points corresponding to a grid in the first set of seed points after gridding, a coordinate value of the point cloud data in the first set of key seed points being a coordinate mean value of point cloud data in the grid in the first set of seed points corresponding to the point cloud data, and a weight of the point cloud data in the first set of key seed points being determined according to a number of point cloud data in the corresponding grid; determining a first set of key candidate points according to the first set of candidate points, point cloud data in the first set of key candidate points corresponding to a grid in the first set of candidate points after gridding, a coordinate value of the point cloud data in the first set of key candidate points being a coordinate mean value of point cloud data in the grid in the first set of candidate points corresponding to the point cloud data, and a weight of the point cloud data in the first set of key candidate points being determined according to a number of point cloud data in the corresponding grid; determining the second set of seed points according to the first set of key seed points and the first set of key candidate points.

18. The apparatus of claim 17, wherein, The processing unit is further specifically configured to: determine a first set of predicted points according to the first set of key seed points and the first set of key candidate points by using a road edge prediction model, the first set of predicted points being used to represent a predicted state of a second part of the road edge; determine point cloud data in the first set of key candidate points that satisfies a first condition as point cloud data in the second set of seed points, the first condition being that a deviation between the point cloud data and point cloud data in the first set of predicted points is less than a second threshold value; wherein the road edge prediction model is obtained by machine learning.

19. The apparatus of claim 12, wherein, The processing unit is further specifically configured to: determine an initial road edge according to point cloud data in a top view plane projection of the road edge point cloud data.

20. The apparatus of claim 19, wherein, The processing unit is further specifically configured to: determine a plurality of initial straight line segments according to the point cloud data in the top view plane projection; determine the initial road edge according to the plurality of initial straight line segments.

21. The apparatus of claim 12, wherein, The processing unit is further specifically configured to: generate a point cloud feature map according to point cloud data after spherical projection of the original point cloud data; determine the plurality of point cloud layer data according to the point cloud feature map.

22. The apparatus of claim 21, wherein, The point cloud feature map includes a point cloud depth map and a point cloud index map.

23. A control device of a curb detection device, characterized by The terminal includes at least one memory and at least one processor, the at least one memory being used to store a program, and the at least one processor being used to run the program to implement the method in any one of claims 1 to 11.

24. A chip, characterized by The terminal includes at least one processing unit and an interface circuit, the interface circuit being used to provide program instructions or data for the at least one processing unit, and the at least one processing unit being used to execute the program instructions to implement the method in any one of claims 1 to 11.

25. A computer-readable storage medium, characterized in that, The computer readable medium stores program code for execution by a device, the program code, when executed by the device, implementing the method in any one of claims 1 to 11.

26. A terminal, characterized by The terminal includes the road edge detection apparatus in any one of claims 12 to 22, or the control apparatus in claim 23, or the chip in claim 24.

27. The terminal of claim 26, wherein, The terminal is a smart transportation device, a smart manufacturing device, a smart home device, or a surveying and mapping device. The terminal is a smart transportation device, a smart manufacturing device, a smart home device, or a surveying and mapping device.

Images