Road edge detection method and road edge detection device applied to vehicle

Abstract

The present invention provides a road edge detection method and a road edge detection device applied to a vehicle. The method comprises: determining target point cloud data according to point cloud data acquired by a radar sensor and image information collected by an image sensor; performing feature extraction on the target point cloud data to determine first road edge information corresponding to the target point cloud data; performing road edge recognition on the image information to obtain second road edge information and lane line information corresponding to the image information; and determining the target road edge and target lane line according to the first road edge information, the second road edge information and the lane line information. The present invention can improve the detection accuracy of the road edge.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a road edge detection method and a road edge detection device applied to a vehicle. Background Art

[0002] Self-driving cars can provide higher safety, productivity and traffic rate, and will play an important role in the future urban transportation system. In most autonomous driving or assisted driving scenarios, the perception of the surrounding environment is a crucial task, and a single sensor has different disadvantages in environmental perception. Therefore, multi-sensor fusion has become a necessary means to improve the effect of the perception system.

[0003] At present, a multi-sensor fusion method is generally used to detect the road edge, that is, a data-level fusion road edge detection method. The data-level fusion road edge detection method is to transfer all raw data to a processor for data processing to determine the road edge.

[0004] However, the above data-level fusion road edge detection method has the problem of low detection accuracy. Summary of the invention

[0005] The embodiments of the present invention provide a road edge detection method and a road edge information detection device applied to a vehicle, so as to solve the problem of low road edge detection accuracy in the detection method of the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a road edge detection method, comprising:

[0007] Determine the target point cloud data based on the point cloud data acquired by the radar sensor and the image information collected by the image sensor;

[0008] Extracting features of the target point cloud data to determine first road edge information corresponding to the target point cloud data;

[0009] Performing road edge recognition on the image information to obtain second road edge information and lane line information corresponding to the image information;

[0010] A target road edge and a target lane line are determined according to the first road edge information, the second road edge information and the lane line information.

[0011] In a second aspect, an embodiment of the present invention provides a road edge detection device for a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the first aspect or any possible implementation method of the first aspect are implemented.

[0012] The embodiment of the present invention provides a road edge detection method and a road edge detection device applied to a vehicle, which determines the target point cloud data according to the point cloud data acquired by the radar sensor and the image information collected by the image sensor, then extracts features from the target point cloud data to determine the first road edge information corresponding to the target point cloud data, and performs road edge recognition on the image information to obtain the second road edge information and lane line information corresponding to the image information, and finally determines the target road edge and the target lane line according to the first road edge information, the second road edge information and the lane line information. The present invention combines the data collected by the radar sensor and the image sensor to determine the target point cloud data, and further performs road edge fusion on the first road edge information corresponding to the target point cloud data and the second road edge information corresponding to the image sensor, so as to correct the fused road edge information through the target lane line information corresponding to the target point cloud data, determine the target road edge and the target lane line, and improve the detection accuracy of the target road edge and the target lane line. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0014] Figure 1 This is a centralized fusion structure diagram provided by an embodiment of the present invention;

[0015] Figure 2 is a distributed fusion structure diagram provided by an embodiment of the present invention;

[0016] Figure 3 is a hybrid fusion structure diagram provided by an embodiment of the present invention;

[0017] Figure 4 is a flow chart of an implementation method of a road edge detection method provided by an embodiment of the present invention;

[0018] Figure 5 is a diagram of an improved hybrid fusion structure provided by an embodiment of the present invention;

[0019] Figure 6 is a schematic diagram of the positional relationship among a radar, a vehicle and a camera provided by an embodiment of the present invention;

[0020] Figure 7 is a flow chart of an implementation method of a road edge detection method provided by another embodiment of the present invention;

[0021] Figure 8is a schematic diagram of the geometric relationship between the data points and the vehicle coordinate system and the sensor coordinate system provided by an embodiment of the present invention;

[0022] Fig. 9 is a schematic diagram of the geometric relationship of vehicle turning provided by an embodiment of the present invention;

[0023] Fig.10 is a schematic diagram of data time synchronization provided by an embodiment of the present invention;

[0024] Fig.11 is a schematic diagram of a forward radar coordinate system provided by an embodiment of the present invention;

[0025] Fig.12 is a schematic diagram of the positional relationship among an image coordinate system, a camera coordinate system, and a vehicle coordinate system provided by an embodiment of the present invention;

[0026] Fig.13 is a schematic diagram of the positional relationship between an image coordinate system and a pixel coordinate system provided by an embodiment of the present invention;

[0027] Fig.14 is a schematic diagram of a grid corresponding to a target lane line and a grid corresponding to target road edge information provided by an embodiment of the present invention;

[0028] Fig.15 It is a schematic diagram of the overlap matching of the grid corresponding to the target lane line and the grid corresponding to the target road edge information provided by an embodiment of the present invention;

[0029] Fig.16 is a schematic diagram of a grid corresponding to a target road edge provided by an embodiment of the present invention;

[0030] Fig.17 is a schematic structural diagram of a road edge detection device provided by an embodiment of the present invention;

[0031] Fig.18 It is a schematic structural diagram of a road edge detection device applied to a vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0034] Self-driving cars can provide higher safety, productivity and traffic rate, and will play an important role in the future urban transportation system. In most autonomous driving or assisted driving scenarios, the perception of the surrounding environment is a crucial task. Single sensors (such as lidar, millimeter wave radar, camera and ultrasound) have different disadvantages in environmental perception, so multi-sensor fusion has become a necessary means to improve environmental perception.

[0035] According to the degree of data processing by local sensors in multi-sensor fusion, the multi-sensor fusion methods can be divided into centralized, distributed and hybrid. Figure 1-Figure 3 The above three fusion methods are described as follows: Figure 1 This is a centralized fusion structure diagram. Figure 1 It can be seen that the centralized method sends all the information from the sensor to the domain controller, which then performs data association, measurement fusion, and target tracking, and finally obtains the location and status information of the target, and finally makes a decision. The advantage of the centralized method is the high accuracy of data processing, while the disadvantage is that a large amount of data can easily cause excessive communication load, and high performance requirements are placed on the controller processing. Figure 2 This is a distributed fusion structure diagram. Figure 2 It can be seen that the distributed method is to detect and track the target observation results of each sensor locally, and then send them to the domain controller to obtain the local track information of multi-target tracking. The advantages of the distributed method are low demand for communication bandwidth and fast calculation speed, but the disadvantage is that the tracking accuracy is far less than that of the centralized method. Figure 3 It is a hybrid fusion structure diagram. Figure 3 It can be seen that the hybrid type is a hybrid structure composed of different sensor data requirements, which has the advantages of both centralized and distributed structures and makes up for the shortcomings of both.

[0036] However, Figure 3 In the hybrid fusion structure shown, the fusion of different types of sensors, such as the fusion of image sensors (cameras) and radar sensors (corner radars, forward radars), is mainly concentrated at the target level, and the 5R1V fusion solution is mainly concentrated at the data level. The detection accuracy of the road edge is relatively low, where 5R1V refers to the sensor configuration of 5 millimeter-wave radars and 1 forward-looking multi-function camera. Figure 3 The hybrid fusion structure shown in the figure proposes an improved hybrid fusion structure and a road edge detection method to further improve the road edge detection accuracy.

[0037] See also Figure 4 and Figure 5 , Figure 4 A road edge detection method according to an embodiment of the present invention is provided as a flowchart for implementing the road edge detection method. Figure 5The improved hybrid fusion structure shown, Figure 5 In the improved hybrid fusion structure shown, the sensors used for road edge detection include radar sensors and image sensors, wherein the radar sensors include forward radar sensors and side radar sensors, and the number of side radar sensors can be multiple, such as 2, 4 or more. Preferably, the number of side radar sensors is 4. In this case, Figure 5 The hybrid fusion structure shown can be called a 5R1V fusion structure, where "5R" refers to five radar sensors, namely one forward radar sensor and four side radar sensors, and "1V" refers to one image sensor. This structure is mainly aimed at L3-level high-speed driving scenarios and can obtain more accurate and reliable road edge detection results. In some embodiments, the radar sensor can be a millimeter-wave radar sensor, and the image sensor is a camera.

[0038] The following will be Figure 5 Taking the hybrid fusion structure shown in the figure as an example, the specific process of the road edge detection method of each embodiment of the present invention is introduced. It includes the following steps:

[0039] Step S101: determining target point cloud data according to the point cloud data acquired by the radar sensor and the image information collected by the image sensor;

[0040] Step S102: extracting features from the target point cloud data to determine first road edge information corresponding to the target point cloud data;

[0041] Step S103: performing road edge recognition on the image information to obtain second road edge information and lane line information corresponding to the image information;

[0042] Step S104: determining a target road edge and a target lane line according to the first road edge information, the second road edge information and the lane line information.

[0043] Specifically, Figure 6 The image shows the positional relationship between the radar, camera, and vehicle. One forward radar sensor and one camera are installed at the front of the vehicle, and four high-resolution side radar sensors (corner radars) are installed at the four sides of the vehicle. Figure 6The specific process of the present invention is as follows: during the driving process of the vehicle, the forward radar sensor collects point cloud data of the corresponding detection area, the four lateral radar sensors collect point cloud data of the corresponding detection area respectively, and the camera collects image information within the camera range. The point cloud data obtained by the radar sensor and the image information collected by the image sensor are used to determine the target point cloud data, and then feature extraction is performed on the target point cloud data to determine the first road edge information corresponding to the target point cloud data and to identify the road edge of the image information to obtain the second road edge information and lane line information corresponding to the image information. Finally, the target road edge and the target lane line are determined based on the first road edge information, the second road edge information and the lane line information. Among them, the second road edge information is determined by detecting and tracking the image information obtained by the camera. In addition, the execution of step S102 and step S103 is not limited in order, and they can be executed simultaneously.

[0044] Compared with the prior art, the road edge detection method provided by the embodiment of the present invention acquires point cloud data through a radar sensor and collects a target image through an image sensor, then fuses the target image with the point cloud data at the data level to determine the target point cloud data, and then performs feature extraction on the target point cloud data to obtain road edge information (i.e., first road edge information), and at the same time, performs road edge recognition on the target image to obtain another road edge information (i.e., second road edge information) and lane line information, and then fuses the first road edge information and the second road edge information at the target level, and then corrects the fused road edge information through the lane line information. Since the target image collected by the image sensor not only assists the radar sensor in identifying the road edge and lane line at the target level, but also assists the radar sensor in judging the point cloud data at the data level to determine the target point cloud data, and performs road edge recognition based on the target point cloud data, the obtained detection result is more accurate, and the detection accuracy of the target road edge and the target lane line is improved.

[0045] In one embodiment, the above step S101 may include the following steps:

[0046] Step S201: Synchronize the point cloud data and the image information to obtain synchronized point cloud data and synchronized image information;

[0047] The synchronization processing includes time synchronization processing and space synchronization processing.

[0048] Step S202: rasterizing the static point cloud data in the synchronized point cloud data to obtain a first raster map, and rasterizing the synchronized image information to obtain a second raster map;

[0049] Step S203: fusing the first raster image and the second raster image according to a preset fusion method to obtain a fused raster image;

[0050] Step S204: modifying the attributes of the point cloud data based on the fused raster image to obtain target point cloud data.

[0051] Specifically, the following Figure 5 The 5R1V hybrid fusion structure shown is used as an example to describe the specific implementation process of the obstacle detection method, that is, the radar sensor includes a forward radar sensor and a side radar sensor, and when the radar sensor is a millimeter wave radar sensor.

[0052] When obstacle detection is performed using multiple sensors, the data of the multiple sensors need to meet the requirements of time synchronization and space synchronization. Therefore, before processing the sensor data, the data of each sensor needs to be synchronized first.

[0053] Fig.10 The schematic diagram of the time synchronization of the data collected by the radar sensor and the image sensor in one embodiment of the present invention. The present invention synchronizes the radar sensor and the image sensor with each other by using GPS. After the unified time synchronization, the radar sensor and the image sensor synchronize their time according to Lagrange interpolation. Fig.10 It can be seen that the data collected by each sensor has a GPS timestamp. The GPS timestamp of the radar sensor can be considered as the time when the domain controller obtains the point cloud data collected by the radar sensor in the current reporting cycle, and the GPS timestamp of the image sensor can be considered as the time when the domain controller obtains the image information collected by the image sensor in the current reporting cycle. When each sensor has a corresponding GPS timestamp, the Lagrange difference is used to synchronize the time of each sensor. The process of synchronizing the time of each sensor is common knowledge and will not be repeated here.

[0054] After time synchronization, the data of each sensor after time synchronization is further synchronized in space. Spatial synchronization mainly maps the data collected by each sensor into a unified coordinate system. In this embodiment, the unified coordinate system is a coordinate system centered on the rear axle of the vehicle (hereinafter uniformly described as the vehicle coordinate system). For different sensors, the process of spatial synchronization is as follows:

[0055] (1) For the forward millimeter-wave radar, based on the conversion relationship between the forward millimeter-wave radar coordinate system and the vehicle coordinate system, the point cloud data corresponding to the forward millimeter-wave radar is converted to the vehicle coordinate system to obtain the synchronous data corresponding to the forward millimeter-wave radar.

[0056] Forward millimeter wave radar coordinate system X R Y R Z R -OR like Fig.11 As shown, the installation position of the forward millimeter-wave radar is defined as the coordinate origin O R The directions of the three coordinate axes are the same as those of the vehicle coordinate system. The detection direction of the forward millimeter-wave radar is the X-axis direction. R O R Y R is the forward millimeter-wave radar detection plane, Y R O R Z R The target data output by the forward millimeter-wave radar includes distance, speed, relative angle, etc., which is the X coordinate system of the forward millimeter-wave radar. R O R Y R Two-dimensional information within the surface. R O R Z R Plane and Y W O W Z W The planes are parallel and the distance is X0, X R O R Y R Plane and X W O W Y W The planes are parallel and the distance is H. For the forward millimeter-wave radar target P(R,α), the transformation relationship between the forward millimeter-wave radar coordinate system and the vehicle coordinate system is:

[0057]

[0058] Where R represents the target distance and α represents the azimuth.

[0059] (2) For the side millimeter-wave radar, based on the conversion relationship between the side millimeter-wave radar coordinate system and the vehicle coordinate system, the point cloud data corresponding to the side millimeter-wave radar is converted into the vehicle coordinate system to obtain the synchronous data corresponding to the side millimeter-wave radar.

[0060] Lateral millimeter wave radar coordinate system X Ri Y Ri Z Ri -O Ri , i=1,2,3,4 and the conversion relationship between the vehicle coordinate system is:

[0061]

[0062] Among them, (X i ,Y i ,Z i ) is the installation position of the lateral millimeter-wave radar in the vehicle coordinate system, The azimuth and elevation angles of the target detected by the i-th lateral millimeter-wave radar, ω i ,φ i Azimuth and elevation angles for the side-facing millimeter-wave radar installation.

[0063] (3) For image sensors, taking cameras as an example, spatial synchronization is mainly based on the conversion relationship between the image coordinate system and the pixel coordinate system, the conversion relationship between the camera coordinate system and the image coordinate system, the conversion relationship between the world coordinate system and the camera coordinate system, and the conversion relationship between the world coordinate system and the pixel coordinate system. The image information after time synchronization is converted into the vehicle coordinate system to obtain the image information after spatial synchronization.

[0064] The specific implementation process of spatial synchronization of image information can be referred to Fig.12 and Fig.13 , Fig.12 It is a schematic diagram of the positional relationship among the image coordinate system, the camera coordinate system and the vehicle coordinate system. Fig.13 It is a schematic diagram of the positional relationship between the image coordinate system and the pixel coordinate system. Based on the linear camera model, the coordinates of each point in the image are first determined through the conversion relationship between the camera coordinate system and the image coordinate system, and then the corresponding coordinates of each point in the image projected into the world coordinate system are obtained through the conversion relationship between the image coordinate system and the pixel coordinate system and the conversion relationship between the pixel coordinate system and the world coordinate system. The world coordinate system is the vehicle coordinate system. The conversion between the vehicle coordinate system and the camera coordinate system is completed through the above conversion process to realize the three-dimensional reconstruction of the point coordinates in the plane image.

[0065] Image coordinate system xoy: The coordinate system of the imaging plane after the camera projects the object in the three-dimensional real environment through perspective projection. Define the intersection of the optical axis and the imaging plane as the coordinate origin O, and the imaging plane as the coordinate system plane. The image information stored in the computer is based on the pixel coordinate system, which defines the upper left corner vertex of the image as the origin of the pixel coordinate system uO0v, such as Fig.13 As shown. The origin O of the image coordinate system is located at the pixel point (u0, v0) in the pixel coordinate system, so the conversion relationship between the image coordinate system and the pixel coordinate system is:

[0066]

[0067] Among them, dx and dy represent the physical size of each pixel in the x and y directions of the image coordinate system respectively.

[0068] Camera coordinate system X c Y c Z c -O c :The center of the camera optical lens is the origin O c , with the optical axis of the camera as Z cThe coordinate system established by the axis. Its coordinate axis is parallel to the image coordinate axis, so the conversion relationship between the camera coordinate system and the image coordinate system (f is the focal length of the camera) is:

[0069]

[0070] World coordinate system (X W Y W Z W ): As a reference coordinate system, it is used to describe the installation position of the radar and the camera (i.e., the camera in this application), as well as the position of other objects in space. The conversion relationship between the world coordinate system and the camera coordinate system is:

[0071]

[0072] The rotation matrix R is a 3×3 unit orthogonal matrix, which represents the rotation relationship between the camera coordinate system and the world coordinate system. c A vector suitable for describing the translation relationship of the camera coordinate system relative to the world coordinate system.

[0073] Finally, the conversion relationship between the world coordinate system and the pixel coordinate system is obtained as follows:

[0074]

[0075]

[0076] Where M1 is the camera intrinsic parameter matrix, and M2 is the camera extrinsic parameter matrix.

[0077] After the data of each sensor is synchronized in time and space, it is necessary to fuse the data of each sensor at the data level. When performing data fusion, first, the static point cloud data in the synchronized point cloud data is rasterized to obtain a first raster map, and the synchronized image information is rasterized to obtain a second raster map, and then the first raster map and the second raster map are fused according to a preset fusion method to obtain a fused raster map, and the point cloud data is attributed based on the fused raster map to obtain the target point cloud data, thereby realizing the process of correcting the point cloud data using image information, that is, executing the above-mentioned steps S202 to S204.

[0078] Before rasterizing the point cloud data, it is necessary to separate the point cloud data into dynamic and static data to obtain dynamic point cloud data and static point cloud data. The dynamic and static separation of point cloud data is mainly based on the vehicle projection speed and the point cloud Doppler speed. Specifically, the implementation of dynamic and static separation includes the following steps:

[0079] (11) obtaining the measured Doppler velocity of each data point in the point cloud data;

[0080] (12) Calculate the target Doppler speed for each data point based on the current vehicle speed;

[0081] (13) calculating the difference between the measured Doppler velocity and the target Doppler velocity;

[0082] (14) When the absolute value of the difference is greater than a preset threshold, the data point is marked as a moving point. When the absolute value of the difference is less than or equal to the preset threshold, the data point is marked as a static point. Thus, the point cloud data is divided into dynamic point cloud data corresponding to the moving point and static point cloud data corresponding to the static point.

[0083] In the above implementation process, the measured Doppler velocity is the velocity information carried in the point cloud data, and the target Doppler velocity is related to the current vehicle speed. Depending on the vehicle driving state, the vehicle speed is represented in different ways. In this embodiment, the vehicle driving state includes: a straight driving state and a non-straight driving state. Based on different driving states, the current vehicle speed is represented in different ways, so the target Doppler velocity is calculated in different ways. The following describes the process of performing dynamic and static separation under different vehicle driving states.

[0084] (1) When the vehicle is in a straight-line driving state, the vehicle speed is projected to the direction of the line connecting the point and the radar center, that is, the point cloud Doppler can be expressed as: V di =V ego cosθ i , where V di V is the point cloud Doppler velocity inferred from the vehicle speed, i.e., the target Doppler velocity; ego is the vehicle speed, θ i is the sum of the azimuth and installation angle of the i-th point cloud, combined with Figure 8 , taking the right front radar sensor in the radar sensor as an example, the installation angle α is the angle between the connecting line of the vehicle coordinate system origin E and the right front radar sensor coordinate system origin S and the vehicle coordinate system Y axis, and the azimuth angle γ is the angle passed by the normal vector y axis of the right front radar sensor coordinate system origin S when the connecting line of the i-th data point and the right front radar sensor coordinate system origin S rotates along the minimum path, wherein the azimuth angle γ is positive when rotating counterclockwise, and negative when rotating clockwise; then the difference between the Doller velocity and the target is calculated, namely: If the speed difference is greater than the preset threshold, it is a moving point, otherwise it is a static point, where is the point cloud Doppler velocity actually measured by the radar.

[0085] (2) When the vehicle is not traveling in a straight line, V diThe calculation is determined by the linear speed of the vehicle. The linear speed calculation formula of the vehicle is: V = ωR, where ω is the angular velocity (yawRate), R is the turning radius, and V is the linear speed. Since the turning radius of the left and right wheels is different and there is a difference of one wheelbase, the linear speeds of the left and right wheels are different when turning, and β is the wheel angle. It was found through engineering experiments that when the vehicle speed during turning is the inner wheel speed, the error between the actual Doppler value of the target point and the theoretical value is small and stable. Combined with Fig. 9 , we can get So V di =V y cosθ i +V x sinθ i , where V x is the linear velocity in the X-axis direction of the vehicle coordinate system, V y is the linear velocity in the Y-axis direction of the vehicle coordinate system.

[0086] Furthermore, since the accuracy of the dynamic and static separation of the point cloud data directly affects the accuracy of the drivable area detection and the judgment of the target obstacle attributes, the present invention performs dynamic and static separation on the point cloud data obtained by each radar sensor, that is, the point cloud data obtained by the forward millimeter-wave radar and the point cloud data obtained by the lateral millimeter-wave radar are respectively separated to obtain the dynamic point cloud data and static point cloud data corresponding to the forward millimeter-wave radar, and the dynamic point cloud data and static point cloud data corresponding to the lateral millimeter-wave radar. In one embodiment, the step of performing dynamic and static separation on the point cloud data is performed before the synchronization processing, that is, during the synchronization processing, the dynamic point cloud data and static point cloud data obtained by each radar sensor are synchronized.

[0087] In step S202, the static point cloud data in the synchronized point cloud data is rasterized to obtain a first raster map, and the synchronized image information is rasterized to obtain a second raster map.

[0088] Among them, the process of rasterizing the point cloud data acquired by each radar sensor can include the following steps: rasterizing the detection area of ​​the radar sensor, counting the number of static points in each grid, when the number of static points contained in the grid is greater than the first target threshold, the grid is an occupied grid, and the attribute of the grid is marked as occupied, when the number of static points contained in the grid is less than or equal to the first target threshold, the grid is an invalid grid, and the attribute of the grid is marked as invalid, thereby obtaining a first grid map composed of occupied grids and invalid grids. In this process, the process of marking the attributes of each grid is the process of rasterizing the point cloud data.

[0089] When the radar sensor includes multiple, the number of the first grid map is multiple. Further, taking the radar sensor including the forward radar sensor and the side radar sensor as an example, the first grid map includes the forward grid map and the side grid map, wherein the forward grid map is the first grid map corresponding to the point cloud data obtained by the forward radar sensor, and the side grid map is the first grid map corresponding to the point cloud data obtained by the side radar sensor. Further, when the number of the side radar sensors is 4, the number of the side grid maps is also 4. For the forward radar sensor, the detection area of ​​the forward radar sensor is gridded, and the number of static points in each grid is counted. When the number of static points contained in the grid is greater than the first target threshold, the grid is an occupied grid, otherwise the grid is an invalid grid, and the forward grid map is determined by the occupied grid and the invalid grid. Similarly, based on the above operation steps, the grid map corresponding to the side radar sensor, that is, the side grid map, can be determined, and the specific steps are not repeated.

[0090] The process of rasterizing the image information collected by the image sensor includes the following steps: rasterizing the camera collection area, and when a non-moving target falls into a grid among the multiple grids corresponding to the camera collection area, the grid is marked as an occupied grid, otherwise it is marked as an invalid grid, thereby obtaining a second grid map composed of occupied grids and invalid grids. Among them, the non-moving target is a stationary target, such as a stationary vehicle, bushes or lamp poles. Furthermore, before the step of rasterizing the image information, it also includes a target tracking process, that is, target tracking is performed on the image information collected by the camera to obtain lane lines and target information. Then, the lane lines and target information are synchronized in data time and data time in turn to obtain synchronized image information.

[0091] Optionally, the above step S203 may include the following steps:

[0092] (1) for each grid, determining the grid occupancy value by using the grid occupancy result in the first grid map and the preset weight of the area to which the grid belongs, and the grid occupancy result in the second grid map and the preset weight of the area to which the grid belongs;

[0093] (2) Taking the occupancy value of the grid and the number of grid images as the quotient, we get the average occupancy value of the grid;

[0094] (3) When the average occupancy value of the grid is greater than the second target threshold, the grid is an occupied grid, otherwise the grid is an invalid grid, and a fused grid map is determined by the occupied grids and the invalid grids.

[0095] Specifically, continue with Figure 5Taking the improved hybrid fusion structure of 5R1V as an example, a total of 6 sensors are included, namely 1 forward millimeter-wave radar, 4 side millimeter-wave radars and 1 camera, which includes 5 first grid images (recorded as 1 forward grid image and 4 side grid images respectively) and 1 second grid image. When performing grid fusion, first, according to the detection accuracy of each sensor, a weight weight[i] is set a priori for each sensor in different areas, where i is the sensor number; then, each grid j of the forward grid image, the side grid image and the second grid image is traversed, and the attributes of each grid in the forward grid image, the side grid image and the second grid image (i.e., occupied or invalid) are counted, and the target occupancy value cellValue of each grid in the fused grid image to be obtained is calculated based on the statistical results. The calculation formula of the target occupancy value is: Where N represents the number of sensors, cell j [i] represents the occupancy value of the jth grid in the grid map of the i-th sensor, where when the grid attribute is occupied, the occupancy value corresponding to the grid is 1, and when the grid attribute is invalid, the occupancy value corresponding to the grid is 0; then, the average occupancy value is calculated, and the specific calculation formula is: avgcellValue = cellValue / sensorNum, sensorNum is the number of sensors; finally, if cellValue is greater than the first preset threshold, the jth grid of the fused grid map is occupied, otherwise it is invalid, thereby obtaining a fused grid map composed of occupied grids and invalid grids. Among them, the process of calculating the attributes of each grid in the fused grid map through the first grid map and the second grid map is the process of data fusion.

[0096] Optionally, the above step S204 may include the following steps:

[0097] (1) When the data point in the point cloud data corresponds to an occupied grid in the fused grid map and the current attribute of the data point is a moving point, the attribute of the data point is corrected to a static point;

[0098] (2) The point cloud data corresponding to the corrected static point and the static point cloud data are used together as the target point cloud data.

[0099] Specifically, if a data point in the point cloud data falls into an occupied grid in the fused grid map, then no matter what the attribute of the data point is before, it will be set as a static point, wherein the attributes of the point cloud include static points and moving points.

[0100] Through the above steps S201 to S204, the process of fusing the image information collected by the camera with the point cloud data obtained by the radar sensor at the data level and correcting the point cloud data using the image information is realized.

[0101] In one embodiment, step S102 includes: accumulating the target point cloud data to determine the drivable area of ​​the lane where the vehicle is located; performing feature extraction in the drivable area to obtain road edge information and lane line information corresponding to the target point cloud data.

[0102] Specifically, the present invention can intuitively determine the drivable area of ​​the lane where the vehicle is located by performing point cloud fusion on the point cloud data obtained by the radar sensor and the image information obtained by the image sensor, and accumulating the target point cloud data determined after the fusion, and then distinguish between the data with different characteristics in the drivable area to extract the road edge information and lane line information, which can effectively improve the recognition accuracy of the lane information.

[0103] In one embodiment, the present invention determines the target road edge by correcting the grid corresponding to the target road edge information using the grid corresponding to the target lane line. Specifically, step S103 includes:

[0104] Step S301: extracting a target lane line corresponding to the lane line information, and based on a preset lane line equation, performing ordinate sampling on the target lane line at a preset distance to obtain a plurality of ordinates, wherein the preset lane line equation is an equation corresponding to the target lane line;

[0105] Step S302: multiple sampling points are determined according to multiple ordinates and a preset lane line equation, and the horizontal coordinate and the vertical coordinate of each sampling point in the multiple sampling points are rounded to obtain a grid corresponding to the target lane line.

[0106] Specifically, the target lane line is first determined, and then the grid corresponding to the target lane line is determined based on the target lane line. Fig.14 , the lane line information is preprocessed, and the target lane line (the dotted line in the figure on the left side of the Y axis) can be directly determined. Then, the ordinate is sampled on the target lane line at a preset distance to obtain multiple ordinates (y values). Since the target lane line has been determined, the multiple ordinates can be substituted into the curve equation corresponding to the target lane line to obtain the abscissa corresponding to each ordinate, and the corresponding sampling point (the dot in the figure on the left side of the Y axis) is determined by the abscissa and ordinate. Then, the abscissa and ordinate of the above sampling points are rounded to obtain the grid corresponding to the target lane line (the grid in the figure on the left), and the grid on the right is the grid corresponding to the edge of the target road.

[0107] Step S303: Overlapping and matching the grid corresponding to the target lane line with the grid corresponding to the target road edge information, and determining the overlapped grids, grid similarity, and grid overlap, wherein the grid corresponding to the target road edge information is determined by performing road edge fusion of the first road edge information and the second road edge information;

[0108] Step S304: Correct the overlapped grids according to the grid similarity and grid overlap to obtain the target road edge.

[0109] Specifically, combined Figure 14-16 The specification describes the implementation process of determining the target road edge. Fig.14 The grid on the right side of the middle is the grid corresponding to the target road edge information determined by road edge fusion of the first road edge information and the second road edge information. Fig.14 The grid in the left figure is translated to the right so that the grid in the left figure coincides with the grid on the right, and the coincident grid is obtained, that is, Fig.15 The grid on the right side of the arrow. Then calculate the similarity and overlap between the grid corresponding to the target lane line and the grid corresponding to the target road edge information, and correct the overlapped grids by similarity and overlap to obtain the grid corresponding to the target road edge ( Fig.16 The target road edge can be determined by the grid corresponding to the target road edge. Optionally, the correction can add or delete the superimposed grid.

[0110] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0111] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0112] Fig.17 A schematic diagram of the structure of a road edge detection device provided by an embodiment of the present invention is shown. For the convenience of description, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0113] like Fig.17 As shown, a road edge detection device comprises:

[0114] The point cloud data determination module 171 is used to determine the target point cloud data according to the point cloud data acquired by the radar sensor and the image information collected by the image sensor;

[0115] A feature extraction module 172, used to extract features from the target point cloud data and determine first road edge information corresponding to the target point cloud data;

[0116] A road edge recognition module 173 is used to perform road edge recognition on the image information to obtain second road edge information and lane line information corresponding to the image information;

[0117] The target lane information determination module 174 is used to determine the target road edge and the target lane line according to the first road edge information, the second road edge information and the lane line information.

[0118] In a possible implementation, the target road edge determination module 174 includes: a ordinate selection submodule, which is used to extract the target lane line corresponding to the lane line information, and based on the preset lane line equation, perform ordinate sampling on the target lane line at a preset distance to obtain multiple ordinates, wherein the preset lane line equation is the equation corresponding to the target lane line; a lane line grid determination submodule, which is used to determine multiple sampling points according to the multiple ordinates and the preset lane line equation, and round the horizontal coordinate and the vertical coordinate of each sampling point in the multiple sampling points to obtain a grid corresponding to the target lane line; a grid coincidence matching submodule, which is used to coincide and match the grid corresponding to the target lane line with the grid corresponding to the target road edge information, and determine the coincident grid, grid similarity and grid coincidence, wherein the grid corresponding to the target road edge information is determined by road edge fusion of the first road edge information and the second road edge information; a grid correction submodule, which is used to correct the coincident grid according to the grid similarity and the grid coincidence to obtain the target road edge.

[0119] In one possible implementation, the point cloud data determination module 171 includes: a synchronization processing submodule, which is used to synchronize the point cloud data and the image information to obtain synchronized point cloud data and synchronized image information; a rasterization submodule, which is used to rasterize the static point cloud data in the synchronized point cloud data to obtain a first raster map, and to rasterize the synchronized image information to obtain a second raster map; a fusion submodule, which is used to fuse the first raster map with the second raster map according to a preset fusion method to obtain a fused raster map; and a correction submodule, which is used to correct the point cloud data according to the fused raster map to obtain target point cloud data.

[0120] In a possible implementation, before the rasterization submodule, it also includes: a dynamic and static separation submodule, which is used to separate the point cloud data acquired by the radar sensor into dynamic and static point cloud data to obtain dynamic point cloud data and static point cloud data; correspondingly, the rasterization submodule includes: rasterizing the detection area of ​​the radar sensor, counting the number of static points in each grid, when the number of static points contained in the grid is greater than the first target threshold, the grid is an occupied grid, otherwise the grid is an invalid grid, and the first grid map is determined by the occupied grids and the invalid grids.

[0121] In a possible implementation, the fusion submodule includes: an occupancy value calculation unit, which is used to determine the occupancy value of the grid for each grid through the occupancy result of the grid in the first grid map and the preset weight of the area to which the grid belongs, as well as the occupancy result of the grid in the second grid map and the preset weight of the area to which the grid belongs; an average value calculation unit, which is used to obtain the average occupancy value of the grid by taking the quotient of the occupancy value of the grid and the number of grid maps; and a fused grid determination unit, which is used to determine that the grid is an occupied grid when the average occupancy value of the grid is greater than a second target threshold, otherwise the grid is an invalid grid, and determine the fused grid map through the occupied grids and the invalid grids.

[0122] In one possible implementation, the correction submodule includes: a judgment unit, which is used to correct the attribute of the data point to a static point when the grid in the fused grid map corresponding to the data point in the point cloud data is an occupied grid and the current attribute of the data point is a moving point; a target point cloud determination unit, which is used to use the point cloud data corresponding to the corrected static point and the static point cloud data together as target point cloud data.

[0123] In a possible implementation, the dynamic-static separation submodule includes: a measured value acquisition unit, used to obtain the measured Doppler velocity of each data point in the point cloud data; a target value calculation unit, used to calculate the target Doppler velocity of each data point according to the current vehicle speed; a difference calculation unit, used to calculate the difference between the measured Doppler velocity and the target Doppler velocity; a judgment unit, used to mark the data point as a dynamic point when the absolute value of the difference is greater than a preset threshold, and mark the data point as a static point when the absolute value of the difference is less than or equal to the preset threshold, and the point cloud data is divided into dynamic point cloud data corresponding to the moving point and static point cloud data corresponding to the static point.

[0124] In a possible implementation, the target value calculation unit includes: when the vehicle is in a straight-line driving state, the target Doppler velocity of each data point is calculated according to the current vehicle speed:

[0125] V di =V ego cosθ i

[0126] Among them, V di is the target Doppler velocity; V ego is the vehicle speed, θ iis the sum of the azimuth angle and the installation angle of the i-th data point. The installation angle is the angle between the connecting line of the origin of the vehicle coordinate system and the origin of the radar sensor or image sensor coordinate system and the Y axis of the vehicle coordinate system. The azimuth angle is the angle passed by the normal vector y axis of the i-th data point and the origin of the radar sensor or image sensor coordinate system rotated to the origin of the radar sensor or image sensor coordinate system in the shortest path. When rotating counterclockwise, the azimuth angle is positive, and when rotating clockwise, the azimuth angle is negative. When the vehicle is not driving in a straight line, the target Doppler velocity of each data point is calculated according to the current vehicle speed as follows:

[0127] V di =V y cosθ i +V x sinθ i

[0128] Among them, V di is the target Doppler velocity, V x V is the linear velocity of the vehicle in the X-axis direction of the vehicle coordinate system. y is the linear velocity of the vehicle in the Y-axis direction of the vehicle coordinate system, θ i is the sum of the azimuth angle and the installation angle of the i-th data point.

[0129] In a possible implementation, the radar sensor is a millimeter wave radar sensor and includes one forward radar sensor and four side radar sensors, and the image sensor includes one camera.

[0130] Fig.18 Schematic diagram of a road edge detection device applied to a vehicle provided by an embodiment of the present invention. Fig.18 As shown, the road edge detection device 18 applied to a vehicle in this embodiment includes: a processor 180, a memory 181, and a computer program 182 stored in the memory 181 and executable on the processor 180. When the processor 180 executes the computer program 182, the steps in the above-mentioned road edge detection method embodiments are implemented, such as Figure 4 Alternatively, when the processor 180 executes the computer program 182, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Fig.17 Functionality of modules / units 171 to 174 shown.

[0131] Exemplarily, the computer program 182 may be divided into one or more modules / units, one or more modules / units are stored in the memory 181 and executed by the processor 180 to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 182 in the road edge detection device 18 applied to the vehicle. For example, the computer program 182 may be divided into Fig.17 Modules / units 171 to 174 are shown.

[0132] The road edge detection device 18 applied to the vehicle may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The road edge detection device 18 applied to the vehicle may include, but is not limited to, a processor 180 and a memory 181. Those skilled in the art will appreciate that Fig.18 This is only an example of a road edge detection device 18 applied to a vehicle and does not constitute a limitation on the road edge detection device 18 applied to a vehicle. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0133] The processor 180 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0134] The memory 181 may be an internal storage unit of the road edge detection device 18 applied to the vehicle, such as a hard disk or memory of the road edge detection device 18 applied to the vehicle. The memory 181 may also be an external storage device of the road edge detection device 18 applied to the vehicle, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the road edge detection device 18 applied to the vehicle. Further, the memory 181 may also include both an internal storage unit of the road edge detection device 18 applied to the vehicle and an external storage device. The memory 181 is used to store computer programs and other programs and data required by the terminal. The memory 181 may also be used to temporarily store data that has been output or is to be output.

[0135] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and the computer program that can be completed by instructing the relevant hardware through a computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned lane information detection method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A road edge detection method, characterized in that: include: Determine the target point cloud data based on the point cloud data acquired by the radar sensor and the image information collected by the image sensor; Performing feature extraction on the target point cloud data to determine first road edge information corresponding to the target point cloud data; Performing road edge recognition on the image information to obtain second road edge information and lane line information corresponding to the image information; Determine a target road edge and a target lane line according to the first road edge information, the second road edge information and the lane line information; Wherein, determining a target road edge and a target lane line according to the first road edge information, the second road edge information and the lane line information includes: Extracting a target lane line corresponding to the lane line information, and based on a preset lane line equation, performing ordinate sampling on the target lane line at a preset distance to obtain a plurality of ordinates, wherein the preset lane line equation is an equation corresponding to the target lane line; Determine a plurality of sampling points according to the plurality of ordinates and the preset lane line equation, and round the abscissa and ordinate of each of the plurality of sampling points to obtain a grid corresponding to the target lane line; Overlapping and matching the grid corresponding to the target lane line with the grid corresponding to the target road edge information, and determining the overlapped grid, grid similarity, and grid overlap, wherein the grid corresponding to the target road edge information is determined by performing road edge fusion of the first road edge information and the second road edge information; The overlapped grids are corrected according to the grid similarity and the grid overlap to obtain a target road edge.

2. The method according to claim 1, characterized in that The step of determining target point cloud data based on the point cloud data acquired by the radar sensor and the image information collected by the image sensor includes: Synchronously processing the point cloud data and the image information to obtain synchronized point cloud data and synchronized image information; rasterizing the static point cloud data in the synchronized point cloud data to obtain a first raster map, and rasterizing the synchronized image information to obtain a second raster map; fusing the first raster image with the second raster image according to a preset fusion method to obtain a fused raster image; The point cloud data is corrected according to the fused grid image to obtain the target point cloud data.

3. The method according to claim 2, characterized in that Before rasterizing the static point cloud data in the synchronized point cloud data to obtain the first raster map, the method further includes: Separating the point cloud data acquired by the radar sensor into dynamic and static point cloud data to obtain dynamic point cloud data and static point cloud data; Correspondingly, the step of rasterizing the static point cloud data in the synchronized point cloud data to obtain a first raster map includes: The detection area of ​​the radar sensor is rasterized, and the number of static points in each grid is counted. When the number of static points contained in the grid is greater than a first target threshold, the grid is an occupied grid, otherwise the grid is an invalid grid, and the first grid map is determined by the occupied grids and the invalid grids.

4. The method according to claim 2, characterized in that: The step of fusing the first raster image with the second raster image according to a preset fusion method to obtain a fused raster image includes: Calculate the target occupancy value of each corresponding grid in the fused grid image in sequence according to the weights of the first grid image and the second grid image in the fused grid image and the occupancy value of each grid in the first grid image and the second grid image; Taking the target occupancy value as a quotient of the sum of the numbers of the first grid image and the second grid image, and sequentially calculating the average occupancy value of each grid corresponding to the fused grid image; When the average occupancy value is greater than a second target threshold, the corresponding grid in the fused grid image is marked as an occupied grid, otherwise it is marked as an invalid grid, and a fused grid image consisting of the occupied grid and the invalid grid is obtained.

5. The method according to claim 3, characterized in that: The step of correcting the point cloud data according to the fused grid image to obtain target point cloud data includes: When a data point in the point cloud data corresponds to an occupied grid in the fused grid image and a current attribute of the data point is a moving point, correcting the attribute of the data point to a static point; The point cloud data corresponding to the corrected static point and the static point cloud data are used together as the target point cloud data.

6. The method according to claim 3, characterized in that The step of separating the point cloud data acquired by the radar sensor from dynamic and static points to obtain dynamic point cloud data and static point cloud data includes: Obtaining a measured Doppler velocity for each data point in the point cloud data; Calculate the target Doppler velocity for each data point based on the current vehicle velocity; Calculating the difference between the measured Doppler velocity and the target Doppler velocity; When the absolute value of the difference is greater than a preset threshold, the data point is marked as a moving point; when the absolute value of the difference is less than or equal to the preset threshold, the data point is marked as a static point, and the point cloud data is divided into dynamic point cloud data corresponding to the moving point and static point cloud data corresponding to the static point.

7. The method according to claim 6, characterized in that The step of calculating the target Doppler speed of each data point according to the current vehicle speed includes: When the vehicle is in a straight-line driving state, the target Doppler velocity of each data point is calculated according to the current vehicle speed as follows: V di =V ego cosθ i Among them, V di is the target Doppler velocity; V ego is the vehicle speed, θ i is the sum of the azimuth angle and the installation angle of the i-th data point. The installation angle is the angle between the connecting line of the origin of the vehicle coordinate system and the origin of the radar sensor or image sensor coordinate system and the Y axis of the vehicle coordinate system. The azimuth angle is the angle passed by the y axis of the normal vector of the connecting line of the i-th data point and the origin of the radar sensor or image sensor coordinate system rotated to the origin of the radar sensor or image sensor coordinate system in the shortest path. When rotating counterclockwise, the azimuth angle is positive, and when rotating clockwise, the azimuth angle is negative. When the vehicle is in a non-straight-line driving state, the target Doppler velocity of each data point is calculated according to the current vehicle speed as follows: V di =V y cosθ i +V x sinθ i Among them, V di is the target Doppler velocity, V x V is the linear velocity of the vehicle in the X-axis direction of the vehicle coordinate system. y is the linear velocity of the vehicle in the Y-axis direction of the vehicle coordinate system, θ i is the sum of the azimuth angle and the installation angle of the i-th data point.

8. The method according to any one of claims 1 to 7, characterized in that: The radar sensor is a millimeter wave radar sensor and includes 1 forward radar sensor and 4 side radar sensors, and the image sensor includes 1 camera.

9. A road edge detection device for a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the road edge detection method according to any one of claims 1 to 8 are implemented.

Images