Lane line detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a lane line detection method and device, electronic equipment and a storage medium, and relates to the technical field of lane line detection, and the method comprises the steps: obtaining target point cloud data corresponding to a to-be-detected road, and converting the target point cloud data into grid feature data; inputting the grid feature data into a target detection model to obtain a coded value and a confidence coefficient which are output by the target detection model and correspond to the grid feature data; wherein the target detection model comprises a dynamic snakelike convolution layer and a long short-term memory network layer; and determining the lane line of the to-be-detected road based on the coded value and the confidence coefficient. According to the technical scheme of the invention, the features are processed through the combination of the dynamic snake-shaped convolutional layer and the long-short-term memory network layer, so that the problems that the detection lines are too thick and large, the detection lines are broken, missing detection and false detection occur and the like due to the limitation of the slender and continuous features of the lane lines in the existing lane line detection segmentation model are solved, the slender features are identified, and the detection efficiency is improved. And the accuracy of lane line detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of lane line detection, and in particular to a lane line detection method, device, electronic equipment and storage medium. Background Art

[0002] Currently, the parking, driving and obstacle avoidance functions of autonomous driving technology all require the detection of road information, including road boundary lines, lane lines and other information, in order to determine the drivable area and the vehicle's driving route.

[0003] The current lane line detection models based on radar or vision all use various segmentation models to segment the results. Limited by the slender and continuous characteristics of lane lines, the effects of many existing lane line detection models are not perfect, and there will be problems such as detection lines being too thick, detection lines being disconnected, and missed detections and false detections. Summary of the invention

[0004] The present invention provides a lane line detection method, device, electronic equipment and storage medium, which improve the detection effect of the lane line by identifying the slender and continuous features of the lane line.

[0005] According to one aspect of the present invention, a lane line detection method is provided, comprising:

[0006] Acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0007] Inputting the grid feature data into a target detection model to obtain a coding value and a confidence level corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic snake convolution layer and a long short-term memory network layer;

[0008] Based on the code value and the confidence level, a lane line of the road to be detected is determined.

[0009] According to another aspect of the present invention, there is provided a lane line detection device, comprising:

[0010] A point cloud data acquisition module is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0011] A feature data processing module, used for inputting the grid feature data into a target detection model to obtain a coding value and a confidence level corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic snake convolution layer and a long short-term memory network layer;

[0012] A lane line determination module is used to determine the lane line of the road to be detected based on the coding value and the confidence level.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line detection method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line detection method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; the grid feature data is input into the target detection model to obtain the encoding value and confidence corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic serpentine convolution layer and a long short-term memory network layer; based on the encoding value and confidence, the lane line of the road to be detected is determined. The technical solution of the present invention processes the features through the combination of the dynamic serpentine convolution layer and the long short-term memory network layer, which solves the existing lane line detection and segmentation model. Due to the slender and continuous characteristics of the lane line, there will be problems such as the detection line is too thick, the detection line is disconnected, and missed detections and false detections. It realizes the recognition of slender features and improves the accuracy of lane line detection.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of a lane line detection method provided in Embodiment 1 of the present invention;

[0022] Figure 2 A flow chart of a lane line detection method provided in Embodiment 2 of the present invention;

[0023] Figure 3 A structural diagram of a target detection model provided in Embodiment 2 of the present invention;

[0024] Figure 4 A flowchart of another lane line detection method provided in Embodiment 2 of the present invention;

[0025] Figure 5 A visualization diagram of target point cloud data provided by the second embodiment of the present invention;

[0026] Figure 6 A visualization diagram of the detection results provided in the second embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a lane line detection device provided in Embodiment 3 of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1A flowchart of a lane line detection method provided in Example 1 of the present invention. This embodiment can be applied to lane line detection to avoid problems such as detection lines being too thick, detection lines being disconnected, missed detections, and false detections in the detection results. The method can be executed by a lane line detection device, which can be implemented in the form of hardware and / or software, and can be configured on a vehicle or in the cloud.

[0033] like Figure 1 As shown, the method includes:

[0034] S110, obtaining target point cloud data corresponding to the road to be detected, and converting the target point cloud data into raster feature data.

[0035] Among them, the road to be detected refers to the road or area where lane line detection is required; the target point cloud data can be the point cloud data corresponding to the lane to be detected, and the analysis and processing of the target point cloud data can determine the lane line, boundary line, etc. corresponding to the lane to be detected.

[0036] In the embodiment of the present invention, in the application scenarios of automatic driving and parking, the point cloud data corresponding to the road to be detected can be collected as the target point cloud data by the electronic device for collecting point cloud data. After collecting the target point cloud data, the target point cloud data can be divided into multiple point cloud grids according to the region, and the features of the point cloud data in each point cloud grid are the grid feature data.

[0037] On the basis of the above technical solution, the step of obtaining target point cloud data corresponding to the road to be detected and converting the target point cloud data into grid feature data includes:

[0038] The laser radar is used to collect the unprocessed point cloud data corresponding to the road to be detected, and the invalid point cloud data in the unprocessed point cloud data is screened and processed to obtain the target point cloud data; the grid division parameters are determined, and the target point cloud data is divided into multiple point cloud grids based on the grid division parameters; for each of the point cloud grids, the target point cloud data in the point cloud grid is processed to obtain the grid feature data corresponding to each of the point cloud grids.

[0039] The point cloud data to be processed can be understood as the unprocessed point cloud data collected by the laser radar; the grid division parameters include the grid division size and / or the grid division range.

[0040] Specifically, the laser radar equipment of the vehicle is used to scan along the road to be inspected to obtain the three-dimensional point cloud data of the road and its surrounding environment, that is, the point cloud data to be processed. The point cloud data to be processed contains a large amount of three-dimensional coordinate information, including the position and shape of obstacles, vehicles, pedestrians and other objects on the road surface.

[0041] Furthermore, the collected point cloud data to be processed is preprocessed to eliminate noise and redundant information in the data, including but not limited to filtering, denoising and other operations to improve the quality of the point cloud data.

[0042] After the point cloud to be processed is preprocessed, invalid point cloud data can be screened out. Invalid point cloud data can be understood as point cloud data that has no value for subsequent lane line detection and analysis. For example, invalid point cloud data can be point cloud data that is 3 meters above the ground, which can be removed from the point cloud data to be processed by the point cloud removal algorithm. In this embodiment, an example is given to illustrate invalid point cloud data, and no limitation is imposed on the invalid point cloud data itself.

[0043] After filtering out invalid point cloud data, the target point cloud data obtained will only contain valid three-dimensional coordinate information, reflecting the actual situation of the road to be detected and its surrounding environment. The target point cloud data can be used for subsequent lane line detection and recognition.

[0044] In order to reduce computational complexity and improve data processing efficiency and other factors, the target point cloud data can be further divided into multiple point cloud grids, each point cloud grid includes a certain amount of target point cloud data, and the basis for division can be grid division parameters, which include but are not limited to grid division size and grid division range.

[0045] Specifically, the grid size and shape can be determined according to the grid division size and division range, wherein the grid division size and division range can determine the appropriate grid size and shape according to the distribution and density of the point cloud data. In the three-dimensional space where the target point cloud data is located, a grid structure can be created according to the determined grid size and shape, and then the target point cloud data can be allocated to the corresponding grid according to its coordinates.

[0046] For each point cloud grid, the target point cloud data in the point cloud grid can be determined first, and then the grid feature data of the point cloud grid can be generated according to the number of all target point cloud data in the grid, the mean height value, maximum value, minimum value, mean color value, standard deviation, etc. of the points.

[0047] In a preferred embodiment, after filtering out the invalid point cloud data in the point cloud data to be processed, the method further includes: converting the point cloud data to be processed into point cloud data matching the target vehicle coordinate information according to the target vehicle coordinate information.

[0048] It can be understood that the target vehicle can be the vehicle currently traveling on the road to be inspected. Since the laser radar position for collecting point cloud data and the center point of the vehicle are not at the same position, in order to finally determine the position of the lane line relative to the center point of the vehicle, the point cloud data to be processed can be converted into point cloud data that matches the target vehicle coordinate information based on the target vehicle coordinate information. For example, the coordinates of the point cloud data to be processed can be transformed based on the difference between the laser radar position and the center point of the vehicle.

[0049] S120, inputting the grid feature data into a target detection model, and obtaining the encoding value and confidence level of the grid corresponding to the grid feature data output by the target detection model.

[0050] The target detection model includes a dynamic snake convolution layer and a long short-term memory network layer. The target prediction model can be a pre-trained neural network model. The encoding value is used to represent the lane line corresponding to the grid feature data. The encoding value of the grid feature data located on the same lane line is the same. The confidence can be understood as the probability that the target point cloud data corresponding to the grid feature data is located on the lane line.

[0051] Specifically, the raster feature data is input into the target detection model, and the target detection model can process the raster feature data to predict the coding value and confidence level corresponding to the raster feature data.

[0052] S130. Determine a lane line of the road to be detected based on the coding value and the confidence level.

[0053] It can be understood that if the confidence is high, it can be considered that the target point cloud corresponding to the raster feature data is the target point cloud located on the lane line. Therefore, a confidence threshold can be preset to judge the confidence of the raster point cloud data. If the output confidence is greater than the preset confidence threshold, it can be considered to be the target point cloud data located on the lane line.

[0054] In order to further distinguish the target point cloud data corresponding to different lane lines, the lane lines may be determined according to the coding values, such as the coding values ​​of the same lane lines are the same.

[0055] After determining the target point cloud data located on the lane line, the target point cloud data corresponding to the same lane line can be connected to obtain the coordinate information of the lane line of the road to be detected, providing a data basis for application scenarios such as autonomous driving.

[0056] On the basis of the above technical solution, the coded value includes a coded value corresponding to each of the point cloud grids, and the confidence level includes a confidence level corresponding to each of the point cloud grids; and determining the lane line of the road to be detected based on the coded value and the confidence level includes:

[0057] For each point cloud grid, determine the code value and confidence level corresponding to each point cloud grid, and if the confidence level of the point cloud grid is greater than a preset threshold, determine the point cloud grid as a target grid;

[0058] The target grids are clustered based on the code values ​​of the target grids, and the lane lines of the road to be detected are determined based on the clustering results; wherein a plurality of the target grids with the same code values ​​correspond to the same lane line.

[0059] Exemplarily, there are multiple point cloud grids, for example, the coding value and confidence of point cloud grid 1 are 1 and 0.9 respectively, the coding value and confidence of point cloud grid 2 are 1 and 0.95 respectively, and the coding value and confidence of point cloud grid 3 are 2 and 0.93 respectively; the preset confidence threshold is 0.8. Accordingly, the target point cloud data corresponding to point cloud grid 1 and point cloud grid 2 are both point cloud data located on lane line 1, and the target point cloud data corresponding to point cloud grid 3 is point cloud data located on lane line 2.

[0060] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; the grid feature data is input into the target detection model to obtain the encoding value and confidence corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic serpentine convolution layer and a long short-term memory network layer; based on the encoding value and confidence, the lane line of the road to be detected is determined. The technical solution of the present invention processes the features through the combination of the dynamic serpentine convolution layer and the long short-term memory network layer, which solves the existing lane line detection and segmentation model. Due to the slender and continuous characteristics of the lane line, there will be problems such as the detection line is too thick, the detection line is disconnected, and missed detections and false detections. It realizes the recognition of slender features and improves the accuracy of lane line detection.

[0061] Embodiment 2

[0062] Figure 2 This is a flow chart of a lane detection method provided in the second embodiment of the present invention. This embodiment further illustrates the structure of the target detection model and the processing process of the target detection model on the grid feature data based on the above embodiment. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0063] like Figure 2 As shown, the method includes:

[0064] S210, obtaining target point cloud data corresponding to the road to be detected, and converting the target point cloud data into raster feature data.

[0065] It should also be noted that the target detection model includes a dynamic snake convolution layer, a long short-term memory network layer and a classifier. The dynamic snake convolution layer includes a down-sampling dynamic snake convolution layer and an up-sampling dynamic snake convolution layer. The down-sampling dynamic snake convolution layer and the up-sampling dynamic snake convolution layer are connected through a long short-term memory network layer, and the classifier is connected to the up-sampling dynamic snake convolution layer. Figure 3 , is a structural diagram of a target detection model provided in Example 2 of the present invention, Encoder is composed of a downsampling dynamic snake convolution layer, Lstm is a long short-term memory network layer, Decoder is composed of an upsampling dynamic snake convolution layer, and Head refers to a classifier.

[0066] Specifically, the dynamic snake-shaped convolution layer can adaptively focus on the small and curved local features of the tubular structure and enhance the perception of the geometric structure. The dynamic snake-shaped convolution can enhance the model's perception of slender and weak local features and adapt to complex and changeable global morphological features. In this scheme, as the main structure of the lane line detection model, it also has strong extraction capabilities.

[0067] Long Short-Term Memory (LSTM) is a time-recurrent network. When LSTM is applied to image detection models, its advantage is that it can connect contextual features. For the lane line task of this solution, it is more conducive to extracting lane line features from the front to the back of the road over a long distance.

[0068] S220: Input the grid feature data into the downsampled dynamic serpentine convolution layer to obtain downsampled grid data features.

[0069] Among them, the downsampled dynamic snake-shaped convolution layer can be understood as a convolution layer used to perform feature recognition and extraction on the grid feature data and perform downsampling. Correspondingly, the obtained feature is the downsampled grid data feature.

[0070] S230: Input the downsampled raster data features into the long short-term memory network layer to obtain the raster data features to be processed.

[0071] Among them, the features of the raster data to be processed refer to the features obtained after processing by Lstm. The advantage of Lstm is that it can connect context features. For the lane line detection task of this application, it is more conducive to extracting the lane line features of the road from front to back over a long distance, and can accurately identify the connection between the target point cloud data corresponding to the same lane line.

[0072] S240: Input the to-be-processed raster data features into the up-sampled dynamic serpentine convolution layer to obtain up-sampled raster data features.

[0073] Among them, the up-sampled dynamic serpentine convolution layer can be understood as a convolution layer used to perform feature recognition and extraction on the features of the raster data to be processed, and to perform up-sampling. Correspondingly, the obtained features are the up-sampled raster data features.

[0074] S250: Fusing the downsampled raster data features and the upsampled raster data features to obtain target raster data features.

[0075] Specifically, the downsampled and upsampled features are merged, and the fusion can be performed by cropping, splicing, adding, etc. The fused features are the target raster data features, which contain important information about the lane line and help improve the detection accuracy of the lane line.

[0076] S260: Input the target raster data feature into the classifier to obtain the encoding value and the confidence level output by the classifier.

[0077] Specifically, the target grid data features are input into the classifier to obtain the prediction results output by the classifier, namely, the coding value and confidence. For example, the head part includes two outputs, namely: 1. The confidence of whether each grid feature data is a lane line, with a value of 0 to 1; 2. The coding value of each grid feature data, the coding value on the same line is the same.

[0078] Based on the above technical solution, the number of the downsampling dynamic serpentine convolution layer and the upsampling dynamic serpentine convolution layer is at least two.

[0079] Specifically, in order to identify features in the raster feature data, especially some slender features, so as to determine whether the corresponding target point cloud data is located on the lane line, it can be processed by multiple layers of dynamic serpentine convolution layers, that is, the number of downsampling dynamic serpentine convolution layers and upsampling dynamic serpentine convolution layers is at least two layers.

[0080] Preferably, the downsampling dynamic snake convolution layer is three layers, and the upsampling dynamic snake convolution layer is three layers. Figure 3 , the Encoder includes three layers of DsConv, each sampling layer consists of down-sampling dynamic snake convolution, and the Decoder includes three layers of DsConv, each sampling layer consists of up-sampling dynamic snake convolution. Results refers to the output results.

[0081] It should be noted that the features of each layer of DsConv downsampling in the Encoder must be concat-connected with the corresponding DsConv downsampling features in the Decoder, that is, the features are spliced ​​and fused.

[0082] S270. Determine a lane line of the road to be detected based on the coding value and the confidence level.

[0083] In an optional embodiment, if Figure 4 FIG. 1 is a flow chart of another lane line detection method provided in Embodiment 2 of the present invention. The lane line detection model is the model proposed in this solution, which specifically includes:

[0084] 1. Pre-processing: divide the point cloud data into grids and calculate the preliminary grid feature data;

[0085] 2. Reasoning: Input the grid feature data into the model and output the confidence and encoding of the lane line.

[0086] 3. Post-processing: Based on the confidence information, threshold screening is performed, and the rasters above the confidence threshold are clustered based on density (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) according to the coding values ​​to obtain the results of road boundaries and lane lines.

[0087] 4. Visualization of results.

[0088] like Figure 5 As shown in FIG. 1 , a visualization diagram of target point cloud data provided by Embodiment 2 of the present invention is shown. Figure 6 As shown, it is a visualization diagram of the detection results provided by the second embodiment of the present invention. Figure 6 Road boundaries and lane markings are shown in Figure 6 The upper and lower lines in the middle are road boundaries, and the four lines in the middle are lane lines. The radar can effectively detect everything within its detection range and can be actually applied to mass production projects of autonomous driving.

[0089] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; the grid feature data is input into the target detection model to obtain the encoding value and confidence corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic serpentine convolution layer and a long short-term memory network layer; based on the encoding value and confidence, the lane line of the road to be detected is determined. The technical solution of the present invention processes the features through the combination of the dynamic serpentine convolution layer and the long short-term memory network layer, which solves the existing lane line detection and segmentation model. Due to the slender and continuous characteristics of the lane line, there will be problems such as the detection line is too thick, the detection line is disconnected, and missed detections and false detections. It realizes the recognition of slender features and improves the accuracy of lane line detection.

[0090] Embodiment 3

[0091] Figure 7 This is a schematic diagram of the structure of a lane line detection device provided in Embodiment 3 of the present invention. Figure 7 As shown, the device comprises:

[0092] The point cloud data acquisition module 310 is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0093] The feature data processing module 320 is used to input the grid feature data into the target detection model to obtain the encoding value and confidence corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic snake convolution layer and a long short-term memory network layer;

[0094] The lane line determination module 330 is used to determine the lane line of the road to be detected based on the coding value and the confidence level.

[0095] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; the grid feature data is input into the target detection model to obtain the encoding value and confidence corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic serpentine convolution layer and a long short-term memory network layer; based on the encoding value and confidence, the lane line of the road to be detected is determined. The technical solution of the present invention processes the features through the combination of the dynamic serpentine convolution layer and the long short-term memory network layer, which solves the existing lane line detection and segmentation model. Due to the slender and continuous characteristics of the lane line, there will be problems such as the detection line is too thick, the detection line is disconnected, and missed detections and false detections. It realizes the recognition of slender features and improves the accuracy of lane line detection.

[0096] Based on the above device, the point cloud data acquisition module 310 is specifically used for:

[0097] Collecting the to-be-processed point cloud data corresponding to the to-be-detected road by means of a laser radar, filtering out the invalid point cloud data in the to-be-processed point cloud data, and obtaining the target point cloud data;

[0098] Determine a grid division parameter, and divide the target point cloud data into a plurality of point cloud grids based on the grid division parameter;

[0099] For each of the point cloud grids, the target point cloud data in the point cloud grid is processed to obtain the grid feature data corresponding to each of the point cloud grids;

[0100] The grid division parameters include grid division size and / or grid division range.

[0101] On the basis of the above device, the code value includes the code value corresponding to each of the point cloud grids, and the confidence includes the confidence corresponding to each of the point cloud grids; the lane line determination module 330 is specifically used to:

[0102] For each point cloud grid, determine the code value and confidence level corresponding to each point cloud grid, and if the confidence level of the point cloud grid is greater than a preset threshold, determine the point cloud grid as a target grid;

[0103] The target grids are clustered based on the code values ​​of the target grids, and the lane lines of the road to be detected are determined based on the clustering results; wherein a plurality of the target grids with the same code values ​​correspond to the same lane line.

[0104] On the basis of the above device, after filtering the invalid point cloud data in the point cloud data to be processed, the method further includes:

[0105] According to the target vehicle coordinate information, the to-be-processed point cloud data is converted into point cloud data matching the target vehicle coordinate information.

[0106] On the basis of the above device, the target detection model further includes a classifier, and the dynamic snake convolution layer includes a down-sampling dynamic snake convolution layer and an up-sampling dynamic snake convolution layer;

[0107] The down-sampling dynamic serpentine convolution layer and the up-sampling dynamic serpentine convolution layer are connected through the long short-term memory network layer; and the classifier is connected to the up-sampling dynamic serpentine convolution layer.

[0108] Based on the above device, the feature data processing module 320 is specifically used for:

[0109] Inputting the grid feature data into the downsampled dynamic snake convolution layer to obtain downsampled grid data features;

[0110] Inputting the downsampled raster data features into the long short-term memory network layer to obtain the raster data features to be processed;

[0111] Inputting the to-be-processed raster data features into the up-sampled dynamic serpentine convolution layer to obtain up-sampled raster data features;

[0112] The downsampled raster data features are merged with the upsampled raster data features to obtain target raster data features;

[0113] The target raster data feature is input into the classifier to obtain the encoding value and the confidence level output by the classifier.

[0114] Based on the above device, the number of the downsampling dynamic serpentine convolution layer and the upsampling dynamic serpentine convolution layer is at least two.

[0115] The lane line detection device provided in the embodiment of the present invention can execute the lane line detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Embodiment 4

[0117] Figure 8 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0118] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0119] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the lane line detection method.

[0121] In some embodiments, the lane detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the lane detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the lane detection method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A lane line detection method, characterized in that: include: Acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data; Inputting the grid feature data into a target detection model to obtain a coding value and a confidence level corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic snake convolution layer and a long short-term memory network layer; Based on the code value and the confidence level, a lane line of the road to be detected is determined.

2. The method according to claim 1, characterized in that The step of acquiring target point cloud data corresponding to the road to be detected and converting the target point cloud data into grid feature data includes: Collecting the to-be-processed point cloud data corresponding to the to-be-detected road by means of a laser radar, filtering out the invalid point cloud data in the to-be-processed point cloud data, and obtaining the target point cloud data; Determine a grid division parameter, and divide the target point cloud data into a plurality of point cloud grids based on the grid division parameter; For each of the point cloud grids, the target point cloud data in the point cloud grid is processed to obtain the grid feature data corresponding to each of the point cloud grids; The grid division parameters include grid division size and / or grid division range.

3. The method according to claim 2, characterized in that The coded value includes a coded value corresponding to each of the point cloud grids, and the confidence includes a confidence corresponding to each of the point cloud grids; and determining the lane line of the road to be detected based on the coded value and the confidence includes: For each point cloud grid, determine the code value and confidence level corresponding to each point cloud grid, and if the confidence level of the point cloud grid is greater than a preset threshold, determine the point cloud grid as a target grid; The target grids are clustered based on the code values ​​of the target grids, and the lane lines of the road to be detected are determined based on the clustering results; wherein a plurality of the target grids with the same code values ​​correspond to the same lane line.

4. The method according to claim 2, characterized in that: After the invalid point cloud data in the to-be-processed point cloud data are screened, the method further includes: According to the target vehicle coordinate information, the to-be-processed point cloud data is converted into point cloud data matching the target vehicle coordinate information.

5. The method according to claim 1, characterized in that The target detection model also includes a classifier, and the dynamic snake convolution layer includes a down-sampling dynamic snake convolution layer and an up-sampling dynamic snake convolution layer; The down-sampling dynamic serpentine convolution layer and the up-sampling dynamic serpentine convolution layer are connected through the long short-term memory network layer; and the classifier is connected to the up-sampling dynamic serpentine convolution layer.

6. The method according to claim 5, characterized in that The step of inputting the grid feature data into the target detection model to obtain the encoding value and confidence level corresponding to the grid feature data output by the target detection model comprises: Inputting the grid feature data into the downsampled dynamic snake convolution layer to obtain downsampled grid data features; Inputting the downsampled raster data features into the long short-term memory network layer to obtain the raster data features to be processed; Inputting the to-be-processed raster data features into the up-sampled dynamic serpentine convolution layer to obtain up-sampled raster data features; The downsampled raster data features are merged with the upsampled raster data features to obtain target raster data features; The target raster data feature is input into the classifier to obtain the encoding value and the confidence level output by the classifier.

7. The method according to claim 6, characterized in that The number of the down-sampling dynamic serpentine convolution layer and the up-sampling dynamic serpentine convolution layer is at least two.

8. A lane line detection device, characterized in that: include: A point cloud data acquisition module is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data; A feature data processing module, used for inputting the grid feature data into a target detection model to obtain a coding value and a confidence level corresponding to the grid feature data output by the target detection model; wherein the target detection model includes a dynamic snake convolution layer and a long short-term memory network layer; A lane line determination module is used to determine the lane line of the road to be detected based on the coding value and the confidence level.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line detection method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line detection method according to any one of claims 1 to 7 when executed.

Images