A method, device and medium for detecting lane lines and vehicles based on laser radar

Abstract

This application discloses a method, device, and medium for lane line and vehicle detection based on LiDAR. The method includes: acquiring several sample data sets; wherein the sample data sets include sample point cloud data and sample two-dimensional images; performing data transformation processing on the sample point cloud data to obtain corresponding sample bird's-eye view images; and annotating lane lines and vehicles in the sample two-dimensional images to obtain a training two-dimensional image; determining the training two-dimensional image as the target image, inputting the sample bird's-eye view images into a preset training multi-scale convolutional neural network model for training until a converged detection model is obtained; acquiring the application point cloud data of the road to be detected, and determining the lane lines and vehicles of the road to be detected based on the application point cloud data and the application detection model. This application achieves lane line and vehicle recognition based on LiDAR data through the above method, thereby advancing intelligent driving technology.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a method, device and medium for lane line and vehicle detection based on lidar. Background Technology

[0002] With the rapid urbanization in China, new infrastructure construction has become a public focus. Among the sub-sectors of new infrastructure, 5G, the Internet of Things, satellite internet, artificial intelligence, data centers, and intelligent transportation infrastructure are all closely related to intelligent driving technology. Intelligent driving, situated at the intersection of these major fields, will serve as a key application scenario to drive the development of new infrastructure. Perception is crucial in intelligent driving technology, and its perception devices are abundant, perception data modalities are diverse, perception data volume is large, and task output is complex. Deep learning technology has been widely applied in this area, for example, in computer vision, natural language processing, and recommendation systems.

[0003] The successful application of these machine learning techniques relies on a foundation of massive amounts of data. However, in real-world applications, a large number of sensing devices inevitably increases terminal costs, making it more difficult to implement intelligent driving technology. Using a single sensing device to achieve intelligent driving technology is becoming a trend. LiDAR (Light Detection and Ranging) offers advantages such as high sensing accuracy, high recognition accuracy, and more direct data. Processing point cloud data places less pressure on the chips and algorithms of intelligent driving systems, and is safer and more reliable. Therefore, how to identify lane lines and vehicles based on LiDAR data to advance intelligent driving technology has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a method, device, and medium for detecting lane lines and vehicles based on LiDAR, in order to solve the following technical problem: how to identify lane lines and vehicles based on LiDAR data in order to advance intelligent driving technology.

[0005] In a first aspect, embodiments of this application provide a method for lane and vehicle detection based on lidar, characterized in that the method includes: acquiring several sample data sets; wherein the sample data sets include sample point cloud data and sample two-dimensional images; performing data conversion processing on the sample point cloud data to obtain a corresponding sample bird's-eye view; and annotating the lane lines and vehicles in the sample two-dimensional images to obtain a two-dimensional image to be trained; determining the two-dimensional image to be trained as the target image, inputting the sample bird's-eye view into a preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained; acquiring the point cloud data to be applied for the road to be detected, and determining the lane lines and vehicles of the road to be detected based on the point cloud data to be applied and the detection model to be applied.

[0006] In one implementation of this application, data transformation processing is performed on the sample point cloud data to obtain the corresponding sample bird's-eye view. Specifically, this includes: determining the sample near-ground cloud data in the sample point cloud data according to a preset angle threshold; mapping the sample near-ground cloud data to the camera coordinate system corresponding to the sample two-dimensional image according to a preset extrinsic parameter matrix to obtain sample mapped point cloud data; performing bird's-eye projection on the sample mapped point cloud data to obtain an initial sample bird's-eye view; and performing a scale transformation on the initial sample bird's-eye view based on the sample two-dimensional image so that the transformed sample bird's-eye view has the same spatial dimension as the sample two-dimensional image.

[0007] In one implementation of this application, the near-ground point cloud data of the sample point cloud data is determined according to a preset angle threshold. Specifically, this includes: determining the spatial vertical coordinates of the near-ground point cloud data of the sample based on the height of the lidar above the ground corresponding to the sample point cloud data; determining the range of the near-ground point cloud data of the sample based on the lower limit angle of the lidar scan and the preset angle threshold; and determining the near-ground point cloud data of the sample point cloud data based on the spatial vertical coordinates and the range of the near-ground point cloud data of the sample.

[0008] In one implementation of this application, before mapping the sample near-ground cloud data to the camera coordinate system corresponding to the sample two-dimensional image according to the preset extrinsic parameter matrix, the method further includes: determining the camera parameters and spatial parameters of the camera corresponding to the sample two-dimensional image; wherein, the camera parameters include: camera focal length parameters and camera optical center parameters; the spatial parameters include: the rotation angle and translation distance of the lidar relative to the camera; based on the camera parameters and space, determining the spatial coordinate transformation relationship between the sample near-ground cloud data and each point in the sample two-dimensional image, and determining the extrinsic parameter matrix based on the spatial coordinate transformation relationship.

[0009] In one implementation of this application, point cloud data of the road to be detected is acquired, and lane lines and vehicles of the road to be detected are determined based on the point cloud data and the detection model to be detected. Specifically, this includes: determining the near-point cloud data to be detected in the point cloud data; determining the mapped point cloud data to be detected in the camera coordinate system corresponding to the near-point cloud data based on a preset extrinsic matrix; performing bird's-eye view projection and scale transformation on the mapped point cloud data to be detected to obtain a bird's-eye view; and inputting the bird's-eye view into the detection model to be detected to determine the lane lines and vehicles of the road to be detected.

[0010] In one implementation of this application, the multi-scale convolutional neural network model to be trained includes: an input layer, a preset number of skip connection layers, a splicing layer, a large-size target output layer, a lane line output layer, a medium-size target output layer, and a small-size target output layer.

[0011] In one implementation of this application, the skip connection layer is composed of a convolutional layer and is used to extract features from the input data; the lane line output layer is used to output the detection results of the lane lines; the large-size target output layer, the medium-size target output layer and the small-size target output layer are used to output the detection results of vehicles or other targets; other targets include at least: people and signs.

[0012] In one implementation of this application, the sample point cloud data in the sample data group has the same timestamp as the sample two-dimensional image.

[0013] Secondly, embodiments of this application also provide a lane line and vehicle detection device based on lidar, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: acquire a plurality of sample data sets; wherein the sample data sets include sample point cloud data and sample two-dimensional images; perform data conversion processing on the sample point cloud data to obtain a corresponding sample bird's-eye view; and annotate lane lines and vehicles in the sample two-dimensional images to obtain a two-dimensional image to be trained; determine the two-dimensional image to be trained as the target image, input the sample bird's-eye view into a preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained; acquire the point cloud data to be applied for the road to be detected, and determine the lane lines and vehicles of the road to be detected based on the point cloud data to be applied and the detection model to be applied.

[0014] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for lane and vehicle detection based on LiDAR, storing computer-executable instructions. The computer-executable instructions are configured to: acquire several sample data sets; wherein the sample data sets include sample point cloud data and sample two-dimensional images; perform data conversion processing on the sample point cloud data to obtain a corresponding sample bird's-eye view; and annotate lane lines and vehicles in the sample two-dimensional images to obtain a two-dimensional image to be trained; determine the two-dimensional image to be trained as the target image, input the sample bird's-eye view into a preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained; acquire the point cloud data to be applied for the road to be detected, and determine the lane lines and vehicles on the road to be detected based on the point cloud data to be applied and the detection model to be applied.

[0015] This application provides a method, device, and medium for lane line and vehicle detection based on LiDAR. By using a smart driving terminal to collect point cloud data during driving, a newly proposed multi-scale deep convolutional network is used to detect lane lines and vehicles. Deep learning technology is used in conjunction with the LiDAR of the smart driving terminal for model iterative updates. This method differs from traditional camera recognition methods, reducing the hardware equipment of the smart driving terminal and lowering the hardware cost of smart driving, thus providing an alternative approach to smart driving. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart illustrating a lane line and vehicle detection method based on LiDAR, provided for an embodiment of this application;

[0018] Figure 2 A near-ground cloud data determination map provided in this application embodiment;

[0019] Figure 3 This is a schematic diagram of a jump-layer connection layer provided in an embodiment of this application;

[0020] Figure 4 A schematic diagram of a multi-scale convolutional neural network model to be trained provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the internal structure of a lane line and vehicle detection device based on LiDAR, provided for an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a method, device, and medium for detecting lane lines and vehicles based on LiDAR, in order to solve the following technical problem: how to identify lane lines and vehicles based on LiDAR data in order to advance intelligent driving technology.

[0024] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 This is a flowchart illustrating a lane line and vehicle detection method based on LiDAR, provided as an embodiment of this application. Figure 1 As shown in the figure, the lane line and vehicle detection method based on LiDAR provided in this application embodiment specifically includes the following steps:

[0026] Step 101: Obtain several sample data sets.

[0027] In one embodiment of this application, the sample data set includes sample point cloud data and sample two-dimensional images. The sample point cloud data originates from a LiDAR sensor installed on the vehicle, and the sample two-dimensional images originate from a camera installed on the vehicle. It is understood that the sample point cloud data and sample two-dimensional images in the sample data set of this application share the same timestamp.

[0028] Step 102: Perform data transformation processing on the sample point cloud data to obtain the corresponding sample bird's-eye view; and annotate the lane lines and vehicles in the sample two-dimensional image to obtain the two-dimensional image to be trained.

[0029] In one embodiment of this application, after acquiring several sample data groups, the sample point cloud data is first processed by data transformation to obtain the corresponding sample bird's-eye view.

[0030] Specifically, firstly, based on a preset angle threshold, the near-point cloud data of the sample point cloud data is determined, such as... Figure 2 As shown, Figure 2 This application provides a near-ground cloud data determination map.

[0031] Based on the height of the lidar above the ground corresponding to the acquired sample point cloud data, the spatial vertical coordinates of the sample near-ground point cloud data are determined; based on the lower limit angle of the lidar scan and the preset angle threshold, the range of the sample near-ground point cloud data is determined; based on the spatial vertical coordinates and the range of the sample near-ground point cloud data, the sample near-ground point cloud data in the sample point cloud data is determined.

[0032] Furthermore, based on the preset extrinsic parameter matrix, the near-field point cloud data of the sample is mapped to the camera coordinate system corresponding to the sample two-dimensional image to obtain the sample mapped point cloud data; the sample mapped point cloud data is subjected to bird's-eye view projection to obtain the initial sample bird's-eye view, and the initial sample bird's-eye view is scaled based on the sample two-dimensional image so that the transformed sample bird's-eye view has the same spatial dimension as the sample two-dimensional image.

[0033] In one embodiment of this application, before mapping the sample near-ground cloud data to the camera coordinate system corresponding to the sample two-dimensional image according to the preset extrinsic parameter matrix, it is necessary to determine the extrinsic parameter matrix.

[0034] Specifically, the camera parameters and spatial parameters of the camera corresponding to the sample two-dimensional image are determined. The camera parameters include: camera focal length parameters and camera optical center parameters. The spatial parameters include: the rotation angle and translation distance of the lidar relative to the camera. Based on the camera parameters and space, the spatial coordinate transformation relationship between the sample near-ground cloud data and each point in the sample two-dimensional image is determined, and the extrinsic parameter matrix is ​​determined based on the spatial coordinate transformation relationship.

[0035] Step 103: Determine the two-dimensional image to be trained as the target image, input the sample bird's-eye view into the preset multi-scale convolutional neural network model to be trained for training, until a converged detection model to be applied is obtained.

[0036] In one embodiment of this application, after determining the sample bird's-eye view and the two-dimensional image to be trained, the two-dimensional image to be trained is determined as the target image. The sample bird's-eye view is input into a preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained.

[0037] It's important to note that Convolutional Neural Networks (CNNs) primarily consist of input layers, hidden layers, fully connected layers, and an output layer. The input layer can process multi-dimensional data. Commonly, a one-dimensional CNN receives a one-dimensional or two-dimensional array as its input layer; a two-dimensional array may contain multiple channels. A two-dimensional CNN receives a two-dimensional or three-dimensional array as its input layer; and a three-dimensional CNN receives a four-dimensional array as its input layer. Hidden layers mainly include convolutional layers and pooling layers. The function of a convolutional layer is to extract features from the input data. It contains multiple convolutional kernels, each element of which corresponds to a weight coefficient and a bias. The pooling layer, after feature extraction by the convolutional layers, performs feature selection and information filtering on the output feature map to avoid overfitting in the CNN. The fully connected layer is mainly responsible for transforming the two-dimensional feature map output by the convolution into a one-dimensional vector, thereby realizing an end-to-end learning process. The output layer uses a logistic function or normalization function to output classification labels. In object recognition problems, the output layer can be designed to output the center coordinates, size, and classification of the object. In image semantic segmentation, the output layer directly outputs the classification result for each pixel. When updating the parameters of a deep neural network, the gradient of each layer is usually calculated from the output layer back to the input layer. Since the gradient is usually a value less than 1, it becomes increasingly smaller as the number of layers increases. Eventually, the vanishing gradient problem occurs. When the gradient approaches zero, the network can no longer update its learning.

[0038] To address this issue, embodiments of this application introduce skip connection layers: In short, shallow layer inputs are added to the intermediate layers of a deep network, making the gradient "path" shorter. It's similar to providing a composite path, adding a "shortcut" to the original "long path." Skip connections essentially provide an extra "shortcut" for calculating gradients. Skip connection layers are as follows... Figure 3 As shown, Figure 3 This is a schematic diagram of a skip connection layer provided in an embodiment of this application. CL is an abbreviation for convolutional layer.

[0039] In one embodiment of this application, the multi-scale convolutional neural network model to be trained includes: an input layer, a preset number of skip connection layers, a splicing layer, a large-size target output layer, a lane line output layer, a medium-size target output layer, and a small-size target output layer.

[0040] In one embodiment of this application, the skip connection layer is composed of a convolutional layer and is used to extract features from the input data; the lane line output layer is used to output the detection results of the lane lines; the large-size target output layer, the medium-size target output layer and the small-size target output layer are used to output the detection results of vehicles or other targets; other targets include at least: people and signs.

[0041] Understandably, the multi-scale convolutional neural network model to be trained in this application removes pooling layers and fully connected layers, and adds more convolutional layers to extract features. Finally, feature maps of different scales complete different tasks through the output layer. Figure 4 This is a schematic diagram of a multi-scale convolutional neural network model to be trained, provided in an embodiment of this application. In this model, SCL is a skip connection layer, IL is the input layer, OUT is the output layer, and CN is the concatenation layer. Figure 4 As shown, the multi-scale convolutional neural network model to be trained in this application contains a total of four output layers. Figure 4 The dashed line encloses the backbone network, which optionally uses a 50-layer SCL for feature extraction. Optionally, the fourth and third output layer branches diverge at the 10th and 25th SCL layers of the backbone network to detect small and medium-sized targets. Optionally, there are two outputs at the 50th SCL layer of the backbone network: one for detecting large targets, and the other, which is processed by the CN layer to combine features from the 10th and 25th layers, for detecting lane lines.

[0042] Step 104: Obtain the point cloud data of the road to be detected, and determine the lane lines and vehicles of the road to be detected based on the point cloud data and the detection model to be detected.

[0043] In one embodiment of this application, after the detection model to be applied is obtained, the point cloud data of the road to be detected is determined, and based on the point cloud data and the detection model to be applied, the lane lines and vehicles of the road to be detected are determined.

[0044] Specifically, the near-point cloud data to be applied is determined from the point cloud data to be applied; based on the preset extrinsic matrix, the mapping point cloud data to be applied in the camera coordinate system corresponding to the near-point cloud data to be applied is determined; the mapping point cloud data to be applied is subjected to bird's-eye view projection and scale transformation to obtain the bird's-eye view image to be applied; the bird's-eye view image to be applied is input into the detection model to be applied to determine the lane lines and vehicles of the road to be detected.

[0045] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a lane marking and vehicle detection device based on lidar, the structure of which is as follows: Figure 2 As shown.

[0046] Figure 5 This is a schematic diagram of the internal structure of a lane marking and vehicle detection device based on LiDAR, provided as an embodiment of this application. Figure 5 As shown, the device includes:

[0047] At least one processor 501;

[0048] And a memory 502 that is communicatively connected to at least one processor;

[0049] The memory 502 stores instructions executable by at least one processor, which are executed by at least one processor 501 to enable at least one processor 501 to:

[0050] Acquire several sample data sets; where each sample data set includes sample point cloud data and sample two-dimensional images;

[0051] The sample point cloud data is transformed to obtain the corresponding sample bird's-eye view; and the lane lines and vehicles in the sample two-dimensional image are labeled to obtain the two-dimensional image to be trained.

[0052] The two-dimensional image to be trained is determined as the target image. The bird's-eye view of the sample is input into the preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained.

[0053] Acquire the point cloud data of the road to be detected, and determine the lane lines and vehicles on the road to be detected based on the point cloud data and the detection model to be detected.

[0054] Some embodiments of this application provide corresponding to Figure 1A non-volatile computer storage medium for lane line and vehicle detection based on LiDAR, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0055] Acquire several sample data sets; where each sample data set includes sample point cloud data and sample two-dimensional images;

[0056] The sample point cloud data is transformed to obtain the corresponding sample bird's-eye view; and the lane lines and vehicles in the sample two-dimensional image are labeled to obtain the two-dimensional image to be trained.

[0057] The two-dimensional image to be trained is determined as the target image. The bird's-eye view of the sample is input into the preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained.

[0058] Acquire the point cloud data of the road to be detected, and determine the lane lines and vehicles on the road to be detected based on the point cloud data and the detection model to be detected.

[0059] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0060] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0061] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0066] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0067] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0069] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for lane line and vehicle detection based on lidar, characterized in that, The method includes: Acquire several sample data sets; wherein, the sample data sets include sample point cloud data and sample two-dimensional images; The sample point cloud data is processed by data transformation to obtain the corresponding sample bird's-eye view; and the lane lines and vehicles in the sample two-dimensional image are labeled to obtain the two-dimensional image to be trained. The two-dimensional image to be trained is determined as the target image. The sample bird's-eye view is input into the preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained. Acquire the point cloud data of the road to be detected and the lane lines and vehicles of the road to be detected based on the point cloud data and the detection model to be detected. The sample point cloud data is subjected to data transformation processing to obtain the corresponding sample bird's-eye view, specifically including: Based on a preset angle threshold, the near-point cloud data of the sample point cloud data is determined; Based on the preset extrinsic parameter matrix, the near-field point cloud data of the sample is mapped to the camera coordinate system corresponding to the two-dimensional image of the sample to obtain the sample mapped point cloud data. The sample mapping point cloud data is projected into an aerial view to obtain an initial sample aerial view. The initial sample aerial view is then scaled based on the sample two-dimensional image so that the transformed sample aerial view has the same spatial dimension as the sample two-dimensional image.

2. The lane line and vehicle detection method based on lidar according to claim 1, characterized in that, Based on a preset angle threshold, the near-field point cloud data of the sample point cloud data is determined, specifically including: Based on the height of the lidar above the ground corresponding to the acquired sample point cloud data, the spatial vertical coordinates corresponding to the sample near-ground point cloud data are determined. The range of the near-ground point cloud data of the sample is determined based on the lower limit angle of the LiDAR scan and the preset angle threshold. Based on the spatial vertical coordinates and the range of the sample near-ground point cloud data, the sample near-ground point cloud data in the sample point cloud data is determined.

3. The lane line and vehicle detection method based on lidar according to claim 1, characterized in that, Before mapping the sample near-ground cloud data to the camera coordinate system corresponding to the sample two-dimensional image according to a preset extrinsic parameter matrix, the method further includes: Determine the camera parameters and spatial parameters of the camera corresponding to the sample two-dimensional image; wherein, the camera parameters include: camera focal length parameters and camera optical center parameters; the spatial parameters include: the rotation angle and translation distance of the lidar relative to the camera; Based on the camera parameters and the space, the spatial coordinate transformation relationship between the sample near-ground cloud data and each point in the sample two-dimensional image is determined, and based on the spatial coordinate transformation relationship, the extrinsic parameter matrix is ​​determined.

4. The lane line and vehicle detection method based on lidar according to claim 1, characterized in that, Acquire point cloud data of the road to be detected, and based on the point cloud data and the detection model to be detected, determine the lane lines and vehicles of the road to be detected, specifically including: Identify the near-point cloud data to be applied from the point cloud data to be applied; Based on the preset extrinsic parameter matrix, determine the applied mapping point cloud data in the camera coordinate system corresponding to the near-point cloud data to be applied; The point cloud data to be mapped is subjected to bird's-eye view projection and scale transformation to obtain the bird's-eye view map to be applied. The bird's-eye view to be applied is input into the detection model to determine the lane lines and vehicles of the road to be detected.

5. The lane line and vehicle detection method based on lidar according to claim 1, characterized in that, The multi-scale convolutional neural network model to be trained includes: an input layer, a preset number of skip connection layers, a splicing layer, a large-size target output layer, a lane line output layer, a medium-size target output layer, and a small-size target output layer.

6. The lane line and vehicle detection method based on lidar according to claim 5, characterized in that, The skip connection layer is composed of convolutional layers and is used to extract features from the input data; the lane line output layer is used to output the detection results of the lane lines. The large-size target output layer, medium-size target output layer, and small-size target output layer are used to output the detection results of vehicles or other targets; The other targets include at least: people and signs.

7. The lane line and vehicle detection method based on lidar according to claim 1, characterized in that, The sample point cloud data in the sample data group has the same timestamp as the sample two-dimensional image.

8. A lane marking and vehicle detection device based on lidar, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Acquire several sample data sets; wherein, the sample data sets include sample point cloud data and sample two-dimensional images; The sample point cloud data is processed by data transformation to obtain the corresponding sample bird's-eye view; and the lane lines and vehicles in the sample two-dimensional image are labeled to obtain the two-dimensional image to be trained. The two-dimensional image to be trained is determined as the target image. The sample bird's-eye view is input into the preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained. Acquire the point cloud data of the road to be detected and the lane lines and vehicles of the road to be detected based on the point cloud data and the detection model to be detected. The sample point cloud data is subjected to data transformation processing to obtain the corresponding sample bird's-eye view, specifically including: Based on a preset angle threshold, the near-point cloud data of the sample point cloud data is determined; Based on the preset extrinsic parameter matrix, the near-field point cloud data of the sample is mapped to the camera coordinate system corresponding to the two-dimensional image of the sample to obtain the sample mapped point cloud data. The sample mapping point cloud data is projected into an aerial view to obtain an initial sample aerial view. The initial sample aerial view is then scaled based on the sample two-dimensional image so that the transformed sample aerial view has the same spatial dimension as the sample two-dimensional image.

9. A non-volatile computer storage medium for lane marking and vehicle detection based on lidar, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: Acquire several sample data sets; wherein, the sample data sets include sample point cloud data and sample two-dimensional images; The sample point cloud data is processed by data transformation to obtain the corresponding sample bird's-eye view; and the lane lines and vehicles in the sample two-dimensional image are labeled to obtain the two-dimensional image to be trained. The two-dimensional image to be trained is determined as the target image. The sample bird's-eye view is input into the preset multi-scale convolutional neural network model to be trained for training until a converged detection model to be applied is obtained. Acquire the point cloud data of the road to be detected and the lane lines and vehicles of the road to be detected based on the point cloud data and the detection model to be detected. The sample point cloud data is subjected to data transformation processing to obtain the corresponding sample bird's-eye view, specifically including: Based on a preset angle threshold, the near-point cloud data of the sample point cloud data is determined; Based on the preset extrinsic parameter matrix, the near-field point cloud data of the sample is mapped to the camera coordinate system corresponding to the two-dimensional image of the sample to obtain the sample mapped point cloud data. The sample mapping point cloud data is projected into an aerial view to obtain an initial sample aerial view. The initial sample aerial view is then scaled based on the sample two-dimensional image so that the transformed sample aerial view has the same spatial dimension as the sample two-dimensional image.

Images