Lane line three-dimensional coordinate determination method, computer device, storage medium and vehicle

Abstract

The present application relates to the technical fields of automatic driving and unmanned driving, and specifically provides a lane line three-dimensional coordinate determination method, computer equipment, storage medium and vehicle, aiming to solve the problem of accurately obtaining lane line three-dimensional coordinates. The method provided by the present application comprises: obtaining lane line pixel point two-dimensional coordinates, point cloud data within a vehicle setting range, and angular velocity obtained by an IMU on the vehicle and vehicle rear axle center speed obtained by a wheel speed meter; compensating the attitude and position of the point cloud data according to the angular velocity and the vehicle rear axle center speed respectively, and removing motion distortion; obtaining ground point cloud data according to the point cloud data after removing motion distortion. According to the ground point cloud data and the two-dimensional coordinates, the depth value of the pixel point is obtained, and according to the depth value and the two-dimensional coordinates, the three-dimensional coordinates of the pixel point are determined. Through the above method, the influence of ground unevenness and point cloud motion distortion on the ground point cloud can be reduced, so that accurate depth values can be obtained, and the accuracy of the lane line three-dimensional coordinates can be improved.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving and driverless technology, specifically to a method for determining the three-dimensional coordinates of lane lines, a computer device, a storage medium, and a vehicle. Background Technology

[0002] Autonomous driving technology is a technology that replaces the driver in controlling the vehicle. In the field of autonomous driving, how to locate the vehicle and constrain its pose based on lane lines is crucial. One of the key steps in achieving vehicle positioning based on lane lines is to reconstruct the three-dimensional coordinates of the lane lines from the two-dimensional coordinates of the lane lines used for vehicle positioning.

[0003] Currently, methods for reconstructing the 3D lane coordinates from the 2D coordinates of lane lines used for vehicle localization mainly employ three approaches: methods based on inverse perspective projection, 3D target detection methods within a multi-sensor fusion framework based on bird's-eye view, and unsupervised multi-sensor fusion methods. However, methods based on inverse perspective projection require estimating the vanishing point of the onboard camera, resulting in low accuracy of the reconstructed 3D lane coordinates. 3D target detection methods within a multi-sensor fusion framework based on bird's-eye view require extensive training data to achieve good results, making them cumbersome. Unsupervised multi-sensor fusion methods suffer from poor plane equation fitting due to point cloud motion distortion during vehicle movement and uneven ground surfaces, leading to low accuracy of the reconstructed 3D lane coordinates.

[0004] Accordingly, a new solution is needed in this field to address the aforementioned problems. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, this invention is proposed to solve, or at least partially solve, the technical problem of how to accurately obtain the three-dimensional coordinates of lane lines.

[0006] In a first aspect, a method for determining the three-dimensional coordinates of a lane line is provided, the method comprising:

[0007] Obtain the two-dimensional coordinates of lane line pixels after performing lane line detection on images captured from vehicles;

[0008] Obtain point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frames collected from the vehicle;

[0009] The angular velocity obtained from the vehicle's IMU and the rear axle center velocity obtained from the wheel speed meter are acquired. The attitude and position of the point cloud data are compensated based on the angular velocity and the rear axle center velocity to remove motion distortion of the point cloud data.

[0010] Ground point cloud data is obtained from the point cloud data after removing motion distortion;

[0011] The depth value of lane line pixels is obtained by using the line-surface intersection method based on ground point cloud data and two-dimensional coordinates. The three-dimensional coordinates of lane line pixels are then determined based on the depth value and two-dimensional coordinates.

[0012] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, the step of "obtaining point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle" specifically includes:

[0013] Based on the timestamp of the image and the timestamp of the LiDAR point cloud frame collected by the vehicle, obtain the nearest neighbor LiDAR point cloud frame that is closest to the image in time.

[0014] Obtain point cloud data located within a set distance from the vehicle in the forward direction from the nearest neighbor lidar point cloud frame.

[0015] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, the step of "obtaining point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle" further includes:

[0016] Acquire point cloud data where the distance in the vehicle's forward direction is less than a preset distance threshold and the yaw angle is within a preset angle range;

[0017] Point cloud data is used as point cloud data located within a set distance from the vehicle.

[0018] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, after the step of "obtaining point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle", the method further includes:

[0019] Uniform sampling of point cloud data is performed to reduce the amount of point cloud data.

[0020] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, the step of "obtaining the angular velocity from the vehicle's IMU and the rear axle center velocity from the wheel speedometer" specifically includes:

[0021] Acquire IMU data and wheel speed meter data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame;

[0022] Based on IMU data and wheel speed meter data, the angular velocity and the rear axle center velocity of the vehicle are obtained respectively.

[0023] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, the step of "acquiring IMU data and wheel speed sensor data whose start time is before the scan start timestamp of the nearest neighbor lidar point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor lidar point cloud frame" specifically includes:

[0024] Acquire IMU data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame;

[0025] Obtain wheel speed meter data whose start and stop times are the same as the start and stop times of the IMU data.

[0026] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, the step of "compensating for the attitude and position of the point cloud data based on the angular velocity and the rear axle center velocity of the vehicle, respectively, to remove the motion distortion of the point cloud data" specifically includes:

[0027] The attitude and position in the IMU coordinate system are obtained based on the angular velocity and the rear axle center velocity of the vehicle, respectively.

[0028] Based on the attitude and position in the IMU coordinate system, the attitude and position of the point cloud data are compensated separately to remove motion distortion of the point cloud data.

[0029] In one technical solution of the above-mentioned method for determining the three-dimensional coordinates of lane lines, before the step of "obtaining ground point cloud data based on point cloud data with motion distortion removed", the method further includes:

[0030] Data cleaning is performed on the point cloud data after motion distortion removal to remove non-numerical points and / or zero-value points and / or invalid points from the point cloud data.

[0031] In a second aspect, a computer device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the method for determining the three-dimensional coordinates of lane lines as described in any of the above-described technical solutions.

[0032] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method for determining the three-dimensional coordinates of lane lines as described in any of the above-described technical solutions.

[0033] In a fourth aspect, a vehicle is provided, the vehicle comprising the computer equipment described in the above-described computer equipment technical solution.

[0034] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0035] In one embodiment of the present invention, lane line detection can be performed on images acquired by the vehicle to obtain the two-dimensional coordinates of lane line pixels. Simultaneously, point cloud data located within a set distance from the vehicle in the forward direction can be obtained from the LiDAR point cloud frame acquired by the vehicle. This point cloud data is used to acquire ground point cloud data, and then the depth value of the lane line pixels is obtained based on this ground point cloud data. Since the height of the ground plane near the vehicle in front of it typically does not change abruptly, acquiring the ground point cloud data using the aforementioned point cloud data can reduce errors caused by uneven ground, improve the accuracy of the ground point cloud data, and thus improve the accuracy of the lane line pixel depth value. Furthermore, before acquiring the ground point cloud data using the aforementioned point cloud data, the point cloud data can be compensated based on the angular velocity obtained from the vehicle's IMU and the rear axle center velocity obtained from the wheel speedometer to remove motion distortion of the point cloud data. In this process, higher-frequency IMU data is used to compensate for the pose of the point cloud, and the vehicle's rear axle center velocity is used to compensate for the position of the point cloud. Compared to the commonly used motion compensation based on a uniform velocity assumption, this method improves the removal of motion distortion in the point cloud data, resulting in more accurate surface point cloud data. This further improves the accuracy of the depth values ​​of lane line pixels obtained from the ground point cloud data. Finally, based on the depth values ​​of the lane line pixels and the aforementioned two-dimensional coordinates, the three-dimensional coordinates of the lane line pixels can be accurately obtained.

[0036] In another technical solution of the present invention, after acquiring point cloud data located within a set distance from the vehicle in the forward direction of the vehicle, the number of point cloud data can be significantly reduced by uniformly sampling the point cloud data. By reducing the number of point cloud data that needs to be processed, the real-time performance of the method for determining the three-dimensional coordinates of the lane line is improved without affecting the accuracy of determining the three-dimensional coordinates of the lane line pixels. Attached Figure Description

[0037] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0038] Figure 1 This is a schematic flowchart of the main steps of a method for determining the three-dimensional coordinates of a lane line according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic flowchart of the main steps of a method for removing motion distortion from point cloud data according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the main structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0041] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0042] In the description of this invention, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. Orientational terms used herein, such as "forward," are based on the vehicle's aft-rear direction after the component is mounted to the vehicle.

[0043] The terminology involved in this invention will be explained below.

[0044] An Inertial Measurement Unit (IMU) is a device that measures an object's three-axis attitude angles (or angular velocities) and acceleration. Typically, an IMU contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometers detect the object's acceleration signals along three independent axes of the carrier's coordinate system, while the gyroscopes detect the carrier's angular velocity signals relative to the navigation coordinate system. By measuring the object's angular velocity and acceleration in three-dimensional space, the object's attitude can be calculated.

[0045] The following describes an embodiment of the method for determining the three-dimensional coordinates of lane lines provided by the present invention.

[0046] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of the method for determining the three-dimensional coordinates of lane lines according to the present invention. Figure 1 As shown, the method for determining the three-dimensional coordinates of the lane line in this embodiment of the invention mainly includes the following steps S101 to S105.

[0047] Step S101: Obtain the two-dimensional coordinates of the lane line pixels obtained after performing lane line detection on the image captured by the vehicle.

[0048] When a vehicle is traveling in a lane, an image containing lane lines can be captured by an image acquisition device such as a camera. Lane line detection can then be performed on this image to obtain the two-dimensional coordinates of the lane line pixels. The method for lane line detection in the captured vehicle image can employ methods commonly used by those skilled in the art, such as the Sobel edge detection algorithm or the Canny edge detection algorithm. This invention does not limit the method used to obtain the two-dimensional coordinates of the lane line pixels.

[0049] Step S102: Obtain point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle.

[0050] During vehicle operation, road surfaces may become uneven. However, the elevation of the ground level closest to the vehicle in front of it typically remains relatively constant. Therefore, by acquiring point cloud data within a predetermined distance from the vehicle in the forward direction, errors caused by uneven ground can be reduced, improving the accuracy of the ground point cloud data. Simultaneously, the LiDAR point cloud frame includes the surrounding environment. To reduce the amount of point cloud data that needs processing, only the data within a predetermined distance from the vehicle in the forward direction can be acquired. A single LiDAR point cloud frame acquired by the vehicle can contain approximately 20,000 point cloud data points. By acquiring only the data within a predetermined distance in the forward direction, the number of point cloud data points requiring processing can be reduced to 3,000, thereby increasing the processing speed.

[0051] Step S103: Obtain the angular velocity from the IMU on the vehicle and the rear axle center velocity from the wheel speed meter. Compensate the attitude and position of the point cloud data based on the angular velocity and the rear axle center velocity to remove motion distortion of the point cloud data.

[0052] Since wheel speed sensors are typically mounted at the center of the rear axle of a vehicle, the speed at the rear axle center can be obtained using them. As the vehicle moves, the LiDAR also moves, causing changes in the LiDAR's coordinate system and resulting in motion distortion in the point cloud data. Removing motion distortion from the point cloud data involves transforming all point cloud data to the LiDAR coordinate system at a specific moment in the current point cloud frame, thus obtaining a point cloud frame without motion distortion.

[0053] Step S104: Obtain ground point cloud data based on the point cloud data after removing motion distortion.

[0054] The lidar point cloud frame contains the environment surrounding the vehicle, which includes the ground and surrounding buildings and vehicles. Therefore, it is necessary to extract ground point cloud data from the point cloud data. Methods for obtaining ground point cloud data include algorithms such as the RANSAC algorithm, which will not be elaborated here.

[0055] Step S105: Obtain the depth value of the lane line pixel based on the ground point cloud data and two-dimensional coordinates using the line-surface intersection method. Determine the three-dimensional coordinates of the lane line pixel based on the depth value and two-dimensional coordinates.

[0056] In this embodiment, the planar equation of the ground can be obtained from ground point cloud data. A line is drawn connecting the optical center of the camera and the lane line pixels. The equation of this line is determined based on the coordinates of the camera's optical center and the two-dimensional coordinates of the lane line pixels. The depth value of the lane line pixels can be obtained from the intersection of the ground planar equation and the equation of the line. Based on the two-dimensional coordinates and depth value of each lane line pixel, the three-dimensional coordinates of each lane line pixel in the camera coordinate system are obtained. That is, a three-dimensional coordinate system is formed by the two-dimensional coordinates and depth value, and this three-dimensional coordinate system is the three-dimensional coordinate of the lane line pixels in the camera coordinate system. Furthermore, after obtaining the three-dimensional coordinates of the lane line pixels in the camera coordinate system, a coordinate system transformation can be performed on these three-dimensional coordinates according to actual needs to obtain the three-dimensional coordinates of the lane line pixels in other three-dimensional coordinate systems, such as the world coordinate system.

[0057] Based on the methods described in steps S101 to S105 above, by acquiring point cloud data located within a set distance from the vehicle in the forward direction, errors caused by uneven ground are reduced, and the accuracy of the ground point cloud data is improved. When removing motion distortion from the point cloud, higher-frequency IMU data is used to compensate for the point cloud's attitude, and the vehicle's rear axle center velocity is used to compensate for the point cloud's position. Compared to the commonly used uniform velocity assumption for motion compensation, this improves the effect of removing motion distortion from the point cloud data, thereby accurately obtaining the ground point cloud, acquiring accurate depth values, and ultimately improving the accuracy of determining the three-dimensional coordinates of lane line pixels.

[0058] The following provides further explanation of steps S102 to S104.

[0059] I. Explanation of step S102.

[0060] The ground on which vehicles travel is often uneven, and the farther away from the vehicle, the higher the likelihood and degree of road undulation.

[0061] In one possible embodiment of the present invention, the step of "acquiring point cloud data located within a set distance from the vehicle in the forward direction of the vehicle from the lidar point cloud frame collected by the vehicle (the aforementioned step S102)" specifically includes the following steps 11 to 12:

[0062] Step 11: Based on the timestamp of the image and the timestamp of the LiDAR point cloud frame collected by the vehicle, obtain the nearest neighbor LiDAR point cloud frame that is closest to the image in time.

[0063] The timestamp of the image refers to the time when the vehicle acquired the image, and the timestamp of the LiDAR point cloud frame refers to the time when the LiDAR acquired the point cloud frame. The acquisition time of the nearest neighbor LiDAR point cloud frame, which is closest in time to the image, can be either before or after the image's acquisition time.

[0064] Step 12: Obtain point cloud data located within a set distance from the vehicle in the forward direction from the nearest neighbor lidar point cloud frame.

[0065] The vehicle's forward direction is the direction in which the vehicle is moving. In the lidar coordinate system, the X-axis points in the vehicle's forward direction. By cropping the nearest neighbor lidar point cloud frames, point cloud data within a set distance from the vehicle in the vehicle's forward direction can be obtained.

[0066] By obtaining the nearest LiDAR point cloud frame that is closest to the image in time using the methods described in steps 11 and 12 above, the matching degree between the LiDAR point cloud frame and the points on the image is ensured. By obtaining point cloud data located within a set distance from the vehicle in the forward direction from the nearest neighbor LiDAR point cloud frame, the amount of point cloud data is reduced, ensuring the timeliness of point cloud data processing. At the same time, since only point cloud data within the set distance of the vehicle is processed, the error caused by the distance from the vehicle being too far or the ground being uneven is reduced, and accurate depth values ​​can be obtained subsequently.

[0067] In one possible embodiment of the present invention, the step of "acquiring point cloud data located within a set distance from the vehicle in the forward direction of the vehicle from the lidar point cloud frame collected by the vehicle (the aforementioned step S102)" further includes the following steps 21 to 22:

[0068] Step 21: Acquire point cloud data where the distance in the vehicle's forward direction is less than a preset distance threshold and the yaw angle is within a preset angle range.

[0069] In the lidar coordinate system, the X-axis points in the vehicle's forward direction, and point cloud data with a distance less than a preset distance threshold in that direction refers to point cloud data on the X-axis whose distance from the origin is less than or equal to the distance threshold. In the vehicle coordinate system, the Y-axis points in the vehicle's forward direction, and point cloud data with a distance less than a preset distance threshold in that direction refers to point cloud data on the Y-axis whose distance from the origin is less than or equal to the distance threshold. In the lidar coordinate system, the yaw angle refers to the angle of rotation in the plane formed by ZOX; in the vehicle coordinate system, the yaw angle refers to the angle of rotation in the plane formed by ZOY. The values ​​of the preset distance threshold and the preset angle range can be set by those skilled in the art according to actual needs; this embodiment does not limit the values ​​of the preset distance threshold and the preset angle range.

[0070] As an example, the preset distance threshold is 10 meters and the preset angle range is (-30°, 30°). By cropping point cloud data that are more than 10 meters away or whose yaw angle is outside the preset angle range, point cloud data that are less than the preset distance threshold and whose yaw angle is within the preset angle range in the forward direction of the vehicle can be obtained.

[0071] Step 22: Use the point cloud data as the point cloud data located within a set distance from the vehicle.

[0072] By using the methods in steps 21 to 22 above, point cloud data with a distance less than a preset distance threshold in the forward direction of the vehicle and a yaw angle within a preset angle range are used as point cloud data within a set distance range from the vehicle. This allows the subsequent acquisition of ground point cloud data to only process point cloud data with a small distance in the forward direction of the vehicle, thereby reducing errors caused by uneven ground.

[0073] In a point cloud frame acquired by a lidar, the number of points in the point cloud is typically between tens of thousands and hundreds of thousands. To improve the processing speed of point cloud data, some point cloud data can be removed.

[0074] In one possible embodiment of the present invention, after the step of "acquiring point cloud data located within a set distance from the vehicle in the forward direction of the vehicle from the lidar point cloud frame collected by the vehicle (as described in step S102)," the method further includes reducing the amount of point cloud data by the following method:

[0075] Uniform sampling of point cloud data is performed to reduce the amount of point cloud data.

[0076] Uniform sampling of point cloud data can be performed using conventional uniform sampling methods such as farthest point sampling. This embodiment of the invention does not limit the specific method of uniform sampling.

[0077] By uniformly sampling the point cloud data, the amount of point cloud data can be significantly reduced. By reducing the amount of point cloud data that needs to be processed, the real-time performance of the method for determining the three-dimensional coordinates of lane lines is ensured.

[0078] II. Explanation of step S103.

[0079] In one possible embodiment of the present invention, the step of “acquiring the angular velocity obtained from the IMU on the vehicle and the rear axle center velocity obtained from the wheel speed gauge (the aforementioned step S103)” specifically includes the following steps 31 to 32:

[0080] Step 31: Obtain IMU data and wheel speed meter data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame.

[0081] To obtain the pose of the point cloud from the start of scanning to the stop of scanning of the nearest neighbor LiDAR point cloud frame, it is necessary to obtain IMU data and wheel speed meter data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame.

[0082] For example, all IMU data and wheel velocity meter data can be saved to the IMU dataset and wheel velocity meter dataset. After determining the start timestamp and scan stop timestamp of the nearest neighbor lidar point cloud frame, data that meets the requirements can be found from the IMU dataset and wheel velocity meter dataset.

[0083] Step 32: Based on the IMU data and wheel speed meter data, obtain the angular velocity and the rear axle center velocity of the vehicle, respectively.

[0084] By using the methods described in steps 31 to 32 above, the angular velocity and the vehicle rear axle center velocity are obtained by using a higher frequency IMU and wheel speed meter, respectively, which improves the accuracy of the obtained angular velocity and vehicle rear axle center velocity.

[0085] In one possible embodiment of the present invention, the step of "acquiring IMU data and wheel speed meter data whose start time is before the scan start timestamp of the nearest neighbor lidar point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor lidar point cloud frame (as described in step 31)" specifically includes the following steps 41 to 42:

[0086] Step 41: Obtain IMU data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame.

[0087] To obtain the pose of the point cloud in the IMU coordinate system from the start of scanning to the end of scanning of the nearest neighbor LiDAR point cloud frame, it is necessary to obtain IMU data whose start time is before the scan start timestamp of the nearest neighbor LiDAR point cloud frame and whose stop time is after the scan stop timestamp of the nearest neighbor LiDAR point cloud frame.

[0088] Step 42: Obtain wheel speed meter data with start and stop times that are the same as the start and stop times of the IMU data.

[0089] Since the pose and position of the point cloud should be correlated at a certain moment, wheel speedometer data with start and stop times that are the same as the start and stop times of the IMU data can be obtained to determine the position of the point cloud at that moment.

[0090] By using the methods in steps 41 to 42 above, firstly, IMU data that meets the requirements is found, and then wheel speed meter data with the same start and stop times is obtained based on the start and stop times of the IMU data. This not only ensures the matching degree between IMU data and wheel speed meter data, but also reduces the workload of finding wheel speed meter data that meets the requirements, thus improving the efficiency of obtaining IMU data and wheel speed meter data.

[0091] See appendix Figure 2 , Figure 2 This is a schematic flowchart illustrating the main steps of the method for removing motion distortion from point cloud data according to the present invention. Figure 2 As shown, in one possible embodiment of the present invention, the step of "compensating for the attitude and position of the point cloud data according to the angular velocity and the rear axle center velocity of the vehicle, respectively, to remove the motion distortion of the point cloud data (the aforementioned step S103)" specifically includes the following steps S201 to S202:

[0092] Step S201: Obtain the attitude and position in the IMU coordinate system based on the angular velocity and the rear axle center velocity of the vehicle.

[0093] In this embodiment, each data acquisition moment of the IMU can be obtained according to the data acquisition frequency of the IMU, i.e., the IMU acquisition moment, and the attitude of each IMU acquisition moment can be obtained according to the angular velocity. Then, the position can be determined according to the center velocity of the rear axle of the vehicle.

[0094] Step S202: Based on the attitude and position in the IMU coordinate system, compensate for the attitude and position of the point cloud data respectively to remove motion distortion of the point cloud data.

[0095] By performing coordinate system transformation on the IMU pose based on the extrinsic parameters between the IMU and the lidar, the lidar pose and position at each IMU acquisition time can be obtained. For each point cloud in the current point cloud frame, the acquisition time of each point cloud is acquired separately. Then, using the lidar pose and position at the IMU acquisition time, time interpolation is performed on the acquisition time of each point cloud to obtain the lidar pose and position of each point cloud at the acquisition time. Finally, based on the lidar pose and position of each point cloud at the acquisition time, all point clouds are transformed to the lidar coordinate system at any time in the current point cloud frame, thereby removing motion distortion of the point cloud data. In a preferred embodiment of the present invention, all point clouds can be transformed to the lidar coordinate system of the last point cloud data in the current point cloud frame.

[0096] By using the methods described in steps S201 to S203 above, the attitude and position of the point cloud data are compensated according to the attitude and position in the IMU coordinate system. Compared with the motion compensation based on the assumption of uniform motion commonly used in the field, the point cloud data can be more accurate by using angular velocity and rear axle center velocity of the vehicle with higher frequency to compensate for the attitude and position of the point cloud data.

[0097] III. Explanation of step S104.

[0098] During the data acquisition process, lidar may generate abnormal point cloud data due to factors such as ambient light interference and excessive distance. In order to ensure the reliability of the point cloud data, it is necessary to remove these abnormal point cloud data.

[0099] In one possible embodiment of the present invention, before the step of "obtaining ground point cloud data based on point cloud data with motion distortion removed (the aforementioned step S104)", the point cloud data with motion distortion removed may be cleaned to remove non-numerical points and / or zero-value points and / or invalid points in the point cloud data, thereby improving the accuracy of the point cloud data.

[0100] Non-numerical points refer to coordinate points that are undefined or cannot be represented; zero-value points refer to points with a value of zero; and invalid points refer to points that are too far or too close, or whose data is invalid due to reflection from the object's surface. Methods for removing non-numerical points and / or zero-value points and / or invalid points can employ conventional data cleaning algorithms in this field, which will not be elaborated upon here.

[0101] By cleaning point cloud data, non-numerical points and / or zero-value points and / or invalid points are removed, thereby reducing noise interference in the point cloud data. Using cleaned point cloud data to obtain ground point cloud data is more accurate.

[0102] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0103] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0104] Furthermore, the present invention also provides a computer device.

[0105] See appendix Figure 3 , Figure 3 This is a schematic diagram of the main structure of an embodiment of a computer device according to the present invention. Figure 3 As shown, the computer device in this embodiment of the invention mainly includes a storage device 31 and a processor 32. The storage device 31 can be configured to store a program for executing the lane line three-dimensional coordinate determination method of the above-described method embodiments. The processor 32 can be configured to execute the program in the storage device, which includes, but is not limited to, the program for executing the lane line three-dimensional coordinate determination method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0106] In some possible implementations, the computer device may include multiple storage devices and multiple processors. The program executing the lane line three-dimensional coordinate determination method of the above method embodiments can be divided into multiple subroutines. Each subroutine can be loaded and run by a processor to perform different steps of the lane line three-dimensional coordinate determination method of the above method embodiments. Specifically, each subroutine can be stored in different storage devices, and each processor can be configured to execute programs in one or more storage devices to jointly implement the lane line three-dimensional coordinate determination method of the above method embodiments. That is, each processor executes different steps of the lane line three-dimensional coordinate determination method of the above method embodiments to jointly implement the lane line three-dimensional coordinate determination method of the above method embodiments.

[0107] The aforementioned multiple processors can be processors deployed on the same device. For example, the aforementioned controller can be a high-performance device composed of multiple processors, and the aforementioned multiple processors can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors can also be processors deployed on different devices.

[0108] Furthermore, the present invention also provides a computer-readable storage medium.

[0109] In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for performing the method for determining the three-dimensional coordinates of lane lines in the above-described method embodiment. This program may be loaded and run by a processor to implement the method for determining the three-dimensional coordinates of lane lines. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium may be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0110] Furthermore, the present invention also provides a vehicle. In one embodiment of the vehicle according to the present invention, the vehicle may include the computer equipment described in the above-described computer equipment embodiments. In this embodiment, the vehicle may be an autonomous vehicle, an unmanned vehicle, or the like. Moreover, according to the type of power source, the vehicle in this embodiment may be a gasoline vehicle, an electric vehicle, a hybrid vehicle using a mixture of electric and gasoline power, or a vehicle using other new energy sources, etc.

[0111] The technical solution of the present invention has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for determining the three-dimensional coordinates of a lane line, characterized in that, The method includes: Obtain the two-dimensional coordinates of lane line pixels after performing lane line detection on images captured from vehicles; Obtain point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frames collected from the vehicle; The angular velocity obtained from the vehicle's IMU and the rear axle center velocity obtained from the wheel speed meter are acquired. The attitude and position of the point cloud data are compensated based on the angular velocity and the rear axle center velocity to remove motion distortion of the point cloud data. Ground point cloud data is obtained from the point cloud data after removing motion distortion; The depth value of the lane line pixel is obtained by using the line-plane intersection method based on the ground point cloud data and the two-dimensional coordinates. The three-dimensional coordinates of the lane line pixel are determined based on the depth value and the two-dimensional coordinates. The method for intersecting lines and planes includes: Based on the ground point cloud data, obtain the plane equation of the ground; Connect the optical center of the camera to the lane line pixels; The equation of the connecting line is determined based on the optical center coordinates of the camera and the two-dimensional coordinates of the lane line pixels. The depth value of the lane line pixel is obtained based on the coordinates of the intersection point of the plane equation of the ground and the equation of the connecting line.

2. The method for determining the three-dimensional coordinates of a lane line according to claim 1, characterized in that, The steps of "obtaining point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle" specifically include: Based on the timestamp of the image and the timestamp of the LiDAR point cloud frame collected by the vehicle, obtain the nearest neighbor LiDAR point cloud frame that is closest in time to the image; Obtain point cloud data located within a set distance from the vehicle in the forward direction from the nearest neighbor lidar point cloud frame; And / or, The step of "obtaining point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected by the vehicle" also includes: Acquire point cloud data where the distance in the vehicle's forward direction is less than a preset distance threshold and the yaw angle is within a preset angle range; The point cloud data is used as the point cloud data located within a set distance from the vehicle.

3. The method for determining the three-dimensional coordinates of a lane line according to claim 2, characterized in that, The specific steps for "obtaining the angular velocity from the vehicle's IMU and the rear axle center velocity from the wheel speedometer" include: Acquire IMU data and wheel speed meter data with a start time before the scan start timestamp of the nearest neighbor lidar point cloud frame and a stop time after the scan stop timestamp of the nearest neighbor lidar point cloud frame; Based on the IMU data and the wheel speedometer data, the angular velocity and the rear axle center velocity of the vehicle are obtained respectively.

4. The method for determining the three-dimensional coordinates of a lane line according to claim 3, characterized in that, The steps of "acquiring IMU data and wheel speed meter data with a start time before the scan start timestamp of the nearest neighbor lidar point cloud frame and a stop time after the scan stop timestamp of the nearest neighbor lidar point cloud frame" specifically include: Acquire IMU data with a start time before the scan start timestamp of the nearest neighbor lidar point cloud frame and a stop time after the scan stop timestamp of the nearest neighbor lidar point cloud frame; Obtain wheel speed meter data whose start and stop times are the same as the start and stop times of the IMU data.

5. The method for determining the three-dimensional coordinates of a lane line according to claim 1, characterized in that, After the step of "acquiring point cloud data located within a set distance from the vehicle in the forward direction from the lidar point cloud frame collected from the vehicle", the method further includes: The point cloud data is sampled uniformly to reduce the amount of point cloud data.

6. The method for determining the three-dimensional coordinates of a lane line according to claim 1, characterized in that, The step of "compensating for the attitude and position of the point cloud data based on the angular velocity and the rear axle center velocity of the vehicle, respectively, to remove motion distortion of the point cloud data" specifically includes: Based on the angular velocity and the rear axle center velocity of the vehicle, the attitude and position in the IMU coordinate system are obtained respectively; Based on the attitude and position in the IMU coordinate system, the attitude and position of the point cloud data are compensated to remove motion distortion of the point cloud data.

7. The method for determining the three-dimensional coordinates of a lane line according to claim 1, characterized in that, Prior to the step of "obtaining ground point cloud data based on the motion-distorted point cloud data", the method further includes: The motion distortion-removed point cloud data is cleaned to remove non-numerical points and / or zero-value points and / or invalid points.

8. A computer device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the method for determining the three-dimensional coordinates of the lane line as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the method for determining the three-dimensional coordinates of lane lines as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle includes the computer equipment as described in claim 8.

Images