Road line detection method and device based on radar neural network, equipment and medium

Abstract

The invention discloses a road line detection method, device and equipment based on a radar neural network and a medium, and relates to the technical field of road line detection, and the method comprises the steps: obtaining target point cloud data corresponding to a to-be-detected road, and converting the target point cloud data into grid feature data; inputting the grid feature data into a radar neural network model to obtain a confidence coefficient and a coding value which are output by the radar neural network model and correspond to the grid feature data; and according to the confidence coefficient and the coding value, based on a density clustering algorithm, clustering the grid special data, and based on a clustering result, detecting a road line corresponding to a road to be detected. According to the technical scheme, the grid feature data corresponding to the point cloud data is processed through the radar neural network, long-distance detection of road line detection is achieved, and the detection effect is stable and accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of road line detection, and in particular to a road line detection method, device, equipment and medium based on radar neural network. Background Art

[0002] Currently, the parking, driving and obstacle avoidance functions of autonomous driving technology all require the detection of road information, including road boundary lines, lane lines and other information, in order to determine the drivable area and the vehicle's driving route.

[0003] However, there is still no mature neural network for current radar-based detection solutions, and traditional point cloud segmentation algorithms are often used. However, their disadvantages are high time consumption, low accuracy, and difficulty in keeping up with new technology iterations.

[0004] Vision-based detection solutions already have relatively mature neural networks that can output relatively accurate road information detection results. However, due to the shortcomings of the camera such as short detection distance and small detection range, vision can only detect road information in a small range, and its effect is limited in complex scenarios such as intersections, U-turns, and congestion. Summary of the invention

[0005] The present invention provides a road line detection method, device, equipment and medium based on radar neural network to achieve.

[0006] According to one aspect of the present invention, a road line detection method based on radar neural network is provided, comprising:

[0007] Acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0008] Inputting the grid feature data into a radar neural network model, and obtaining a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model;

[0009] According to the confidence and the coding value, the point cloud grids corresponding to the grid feature data are clustered based on a density clustering algorithm, and the road line corresponding to the road to be detected is determined based on the clustering result.

[0010] According to another aspect of the present invention, a road line detection device based on radar neural network is provided, comprising:

[0011] A point cloud data acquisition module is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0012] A feature data processing module, used for inputting the grid feature data into a radar neural network model to obtain a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model;

[0013] The density clustering module is used to cluster the point cloud grids corresponding to the grid feature data based on the confidence and the coding value based on the density clustering algorithm, and the road line corresponding to the road to be detected is based on the clustering result.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the road line detection method based on radar neural network described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the road line detection method based on radar neural network described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; inputs the grid feature data into the radar neural network model to obtain the confidence and code value corresponding to the grid feature data output by the radar neural network model; clusters the grid feature data based on the confidence and code value based on the density clustering algorithm, and determines the road line corresponding to the road to be detected based on the clustering result. The technical solution of the present invention processes the grid feature data corresponding to the point cloud data through the radar neural network, and then clusters based on the code value corresponding to the grid feature data to determine the lane line of the road to be detected, which solves the problems of high time consumption, low precision, and difficulty in following up new technology iterations of the traditional point cloud segmentation algorithm, and the problem that the vision-based detection solution is limited by the short detection distance and small detection range of the camera, and realizes long-distance detection of road line detection, and the detection effect is stable and accurate.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a road line detection method based on radar neural network provided in Embodiment 1 of the present invention;

[0023] Figure 2 A flowchart of a road line detection method based on radar neural network provided in the second embodiment of the present invention;

[0024] Figure 3 A structural diagram of a radar neural network model provided in Embodiment 2 of the present invention;

[0025] Figure 4 A flowchart of another road line detection based on radar neural network provided in the second embodiment of the present invention;

[0026] Figure 5 A visualization diagram of target point cloud data provided by the second embodiment of the present invention;

[0027] Figure 6 A visualization diagram of the detection results provided in the second embodiment of the present invention;

[0028] Figure 7 A schematic diagram of the structure of a road line detection device based on radar neural network provided in Embodiment 3 of the present invention;

[0029] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 This is a flowchart of a road line detection method based on a radar neural network provided in the first embodiment of the present invention. This embodiment is applicable to the detection of lane lines and road boundary lines. The method can be performed by a road line detection device based on a radar neural network. The device can be implemented in the form of hardware and / or software. The device can be configured in the automatic driving system of a vehicle. Figure 1 As shown, the method includes:

[0034] S110, obtaining target point cloud data corresponding to the road to be detected, and converting the target point cloud data into raster feature data.

[0035] Among them, the road to be detected refers to the road or area where lane lines and road boundary lines need to be detected; the target point cloud data can be the point cloud data corresponding to the lane to be detected. By analyzing and processing the target point cloud data, the lane lines, boundary lines, etc. corresponding to the lane to be detected can be determined.

[0036] In the embodiment of the present invention, in the application scenarios of automatic driving and parking, the point cloud data corresponding to the road to be detected can be collected as the target point cloud data by the electronic device for collecting point cloud data. After collecting the target point cloud data, the target point cloud data can be divided into multiple point cloud grids according to the region, and the features of the point cloud data in each point cloud grid are the grid feature data.

[0037] In an embodiment of the present invention, the target point cloud data corresponding to the road to be detected is obtained, and the target point cloud data is converted into grid feature data, including: collecting the processed point cloud data corresponding to the road to be detected by means of a laser radar, filtering and processing the invalid point cloud data in the processed point cloud data, and obtaining the target point cloud data; determining grid division parameters, and dividing the target point cloud data into a plurality of point cloud grids based on the grid division parameters; for each of the point cloud grids, processing the target point cloud data in the point cloud grid, and obtaining the grid feature data corresponding to each of the point cloud grids.

[0038] The grid division parameters include grid division size and / or grid division range.

[0039] Specifically, the laser radar equipment of the vehicle is used to scan along the road to be inspected to obtain the three-dimensional point cloud data of the road and its surrounding environment, that is, the point cloud data to be processed. The point cloud data to be processed contains a large amount of three-dimensional coordinate information, including the position and shape of obstacles, vehicles, pedestrians and other objects on the road surface.

[0040] Furthermore, the collected point cloud data to be processed is preprocessed to eliminate noise and redundant information in the data, including but not limited to filtering, denoising and other operations to improve the quality of the point cloud data.

[0041] After the point cloud to be processed is preprocessed, invalid point cloud data can be screened out. Invalid point cloud data can be understood as point cloud data that has no value for subsequent lane line detection and analysis. For example, invalid point cloud data can be point cloud data that is 3 meters above the ground, which can be removed from the point cloud data to be processed by the point cloud removal algorithm. In this embodiment, an example is given to illustrate invalid point cloud data, and no limitation is imposed on the invalid point cloud data itself.

[0042] After filtering out invalid point cloud data, the target point cloud data obtained will only contain valid three-dimensional coordinate information, reflecting the actual situation of the road to be detected and its surrounding environment. The target point cloud data can be used for subsequent lane line detection and recognition.

[0043] In order to reduce computational complexity and improve data processing efficiency and other factors, the target point cloud data can be further divided into multiple point cloud grids, each point cloud grid includes a certain amount of target point cloud data, and the basis for division can be grid division parameters, which include but are not limited to grid division size and grid division range.

[0044] Specifically, the grid size and shape can be determined according to the grid division size and division range, wherein the grid division size and division range can determine the appropriate grid size and shape according to the distribution and density of the point cloud data. In the three-dimensional space where the target point cloud data is located, a grid structure can be created according to the determined grid size and shape, and then the target point cloud data can be allocated to the corresponding grid according to its coordinates.

[0045] For each point cloud grid, the target point cloud data in the point cloud grid can be determined first, and then the grid feature data of the point cloud grid can be generated according to the number of all target point cloud data in the grid, the mean height value, maximum value, minimum value, mean color value, standard deviation, etc. of the points.

[0046] On the basis of the above technical solution, after filtering out the invalid point cloud data in the point cloud data to be processed, it also includes: according to the target vehicle coordinate information, converting the point cloud data to be processed into point cloud data matching the target vehicle coordinate information.

[0047] It can be understood that the target vehicle can be the current vehicle traveling on the road to be detected and needs to perform road line detection. Since the laser radar position for collecting point cloud data and the center point of the vehicle are not at the same position, in order to finally determine the position of the lane line relative to the center point of the vehicle, the point cloud data to be processed can be converted into point cloud data that matches the target vehicle coordinate information based on the target vehicle coordinate information. For example, the coordinates of the point cloud data to be processed can be transformed based on the difference between the laser radar position and the center point of the vehicle.

[0048] S120, inputting the grid feature data into a radar neural network model, and obtaining a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model.

[0049] Among them, the radar neural network model can be understood as a pre-trained neural network model, the coding value is used to represent the lane line or road boundary corresponding to the raster feature data, the coding value of the raster feature data located on the same lane line or road boundary is the same, and the confidence can be understood as the probability that the target point cloud data corresponding to the raster feature data is located at the lane line or road boundary.

[0050] It is understandable that the radar neural network model can process the grid feature data of the point cloud data collected by the radar as input to obtain the corresponding confidence and coding value. Since the point cloud data is collected by radar and the radar detection distance is far, the corresponding present invention can determine the road line corresponding to the road to be monitored based on the grid feature data corresponding to the point cloud data at a longer distance. The detection distance exceeds the effective range of visual detection. For example, the lane line and road boundary line of the road within a forward distance of 50 meters can be detected. In addition, the use of a traditional point cloud segmentation algorithm is avoided by means of a neural network model, which has the disadvantages of being time-consuming, low-precision, and difficult to follow up with new technology iterations.

[0051] S130. Clustering the point cloud grids corresponding to the grid feature data based on the confidence level and the coding value based on a density clustering algorithm, and determining a road line corresponding to the road to be detected based on the clustering result.

[0052] Among them, the density-based spatial clustering algorithm (DBSCAN) is used to cluster according to the confidence and the coding value. The road line can be a lane line or a road boundary line.

[0053] It can be understood that each point cloud grid has its corresponding grid feature data, as well as confidence and coding value; the coding value and confidence corresponding to each point cloud grid can be determined for each point cloud grid, and if the confidence of the point cloud grid is greater than a preset threshold, the point cloud grid is determined as a target grid; the target grids are clustered based on the coding values ​​of the target grids, and the road lines of the road to be detected are determined based on the clustering results, and multiple target grids with the same coding value correspond to the same lane line.

[0054] Exemplarily, there are multiple point cloud grids, for example, the coding value and confidence of point cloud grid 1 are x1 and 0.9 respectively, the coding value and confidence of point cloud grid 2 are x1 and 0.95 respectively, the coding value and confidence of point cloud grid 3 are x2 and 0.93 respectively, the coding value and confidence of point cloud grid 4 are y2 and 0.93 respectively, and the coding value and confidence of point cloud grid 5 are y2 and 0.7 respectively; the preset confidence threshold is 0.8, x represents the lane line, and y represents the road boundary line. Therefore, the target point cloud data corresponding to point cloud grid 1 and point cloud grid 2 are both point cloud data located on lane line 1, the target point cloud data corresponding to point cloud grid 3 is the point cloud data located on lane line 2; the target point cloud data corresponding to point cloud grid 4 is the point cloud data located on the right road boundary; the confidence of point cloud grid 5 is 0.7, which is less than the preset confidence threshold, so it can be deleted and cannot be used as the target point cloud grid.

[0055] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; inputs the grid feature data into the radar neural network model to obtain the confidence and code value corresponding to the grid feature data output by the radar neural network model; clusters the grid feature data based on the confidence and code value based on the density clustering algorithm, and determines the road line corresponding to the road to be detected based on the clustering result. The technical solution of the present invention processes the grid feature data corresponding to the point cloud data through the radar neural network, and then clusters based on the code value corresponding to the grid feature data to determine the lane line of the road to be detected, which solves the problems of high time consumption, low precision, and difficulty in following up new technology iterations of the traditional point cloud segmentation algorithm, and the problem that the vision-based detection solution is limited by the short detection distance and small detection range of the camera, and realizes long-distance detection of road line detection, and the detection effect is stable and accurate.

[0056] Embodiment 2

[0057] Figure 2 This is a flowchart of a road line detection method based on radar neural network provided in the second embodiment of the present invention. This embodiment is a preferred embodiment of the above embodiment. Figure 2 As shown, the method includes:

[0058] S210, obtaining target point cloud data corresponding to the road to be detected, and converting the target point cloud data into raster feature data.

[0059] S220, inputting the grid feature data into at least two downsampling layers in sequence to obtain downsampling features output by each downsampling layer.

[0060] In an embodiment of the present invention, the radar neural network model includes: a downsampling layer, an upsampling layer and a classifier, the downsampling layer is connected to the upsampling layer; the classifier is connected to the upsampling layer; the downsampling layer includes a residual network, and the number of layers of the downsampling layer and / or the upsampling layer is at least two.

[0061] refer to Figure 3 , which is a structural diagram of the radar neural network model provided in the second embodiment of the present invention, wherein the Encoder part includes 2 downsamplings, a total of 3 network layers, and each sampling layer is composed of a ResNet (residual network) module. The features of the final output of each ResNet are concatenated (Concat) as the output of the Encoder. The Decoder includes 2 upsamplings (Upsample), a total of 3 network layers. The main function is to restore the model features to the original grid size.

[0062] Among them, the downsampling feature refers to the feature output by each downsampling layer.

[0063] S230, fusing the down-sampled features to obtain features to be processed, and inputting the features to be processed into the up-sampling layer to obtain up-sampled features output by the up-sampling layer.

[0064] The upsampling layer refers to the features obtained after the upsampling layer processes the features to be processed.

[0065] S240: Input the up-sampled feature processing into the classifier to obtain the confidence and encoding value corresponding to the raster feature data.

[0066] The confidence includes the road boundary confidence and the lane line confidence, and the code value includes the road boundary code value and the lane line code value. The classifier refers to the head layer, which is used to predict the confidence and code value corresponding to the raster feature data according to the upsampled features.

[0067] S250, clustering the point cloud grids corresponding to the grid feature data based on the confidence level and the coding value based on a density clustering algorithm, and determining a road line corresponding to the road to be detected based on the clustering result.

[0068] Specifically, the code value and confidence level corresponding to each point cloud grid may be determined. If the confidence level of the point cloud grid is greater than a preset threshold, the point cloud grid is determined as a target grid. The target grid is clustered based on the code value of the target grid.

[0069] Further, clustering the point cloud grids corresponding to the grid feature data based on the confidence and the coding value based on a density clustering algorithm, and clustering the road line corresponding to the road to be detected based on the clustering result, includes:

[0070] Each target point cloud grid is used as a sample point, and clustering is performed according to the code value of the target grid. The neighborhood radius and density threshold of the density clustering algorithm are preset, and a sample point is randomly selected. If the sample point is determined to be a core sample point based on the neighborhood radius and the density threshold; the density reachable sample point corresponding to the core sample point is determined, and the reachable sample point is assigned to the cluster corresponding to the core sample point, wherein the reachable sample point is a sample point that has not been assigned a cluster; the road line corresponding to the road to be detected is determined based on the cluster corresponding to the core sample point.

[0071] On the basis of the above technical solution, it also includes: if the sample point is a non-core sample point, repeatedly executing the process of randomly selecting a sample point and determining whether the sample point is a core sample point based on the neighborhood radius and the density threshold until the sample point is a core sample point.

[0072] Specifically, the neighborhood radius (Eps) and density threshold (MinPts) of the preset density clustering algorithm are the key parameters of the DBSCAN algorithm, which determine the clustering results. It includes the following steps: 1. Mark all sample points as unvisited; 2. Randomly select a sample point p. If p has at least MinPts sample points within the radius r, then point p is used as the core point, otherwise a new point p that meets the conditions is selected. 3. Find all sample points that are density-reachable with the core point p. When the sample point is not divided into other clusters, divide them into a cluster and change their status to visited. Find another core point p in the samples in the unvisited state, and then repeat step 3 as another cluster; until all sample points are processed. Each cluster consisting of a core sample point and its density-reachable sample points represents a potential road line. These clusters can be further processed according to the location, shape and other characteristics of the sample points in the cluster to extract accurate road line information.

[0073] In an optional embodiment, if Figure 4 FIG. 1 is a flowchart of another road line detection based on radar neural network provided in Embodiment 2 of the present invention. Specifically, it includes:

[0074] Among them, the road detection part is changed from the traditional algorithm to the neural network model, which includes:

[0075] 1. Pre-processing: divide the point cloud data into grids and calculate the preliminary grid feature data;

[0076] 2. Reasoning: Input the raster feature data into the model and output the following four sets of data, namely the confidence and coding of the road boundary, and the confidence and coding of the lane line.

[0077] 3. Post-processing: Based on the confidence information, perform threshold screening, and perform DBSCAN clustering on the rasters above the confidence threshold according to the coding values ​​to obtain the results of road boundary lines and lane lines.

[0078] The neural network model used has a backbone network including Encoder and Decoder, and the Head part includes 4 output items. The Head part includes 4 outputs, namely: 1. The confidence of whether each grid is a road boundary line, with a value of 0 to 1; 2. The encoding value of each grid corresponding to each road boundary line, and the encoding values ​​on the same line are the same; 3. The confidence of whether each grid is a lane line, with a value of 0 to 1; 4. The encoding value of each grid corresponding to each lane line, and the encoding values ​​on the same line are the same.

[0079] like Figure 5 As shown in FIG. 1 , a visualization diagram of target point cloud data provided by Embodiment 2 of the present invention is shown. Figure 6 As shown, it is a visualization diagram of the detection results provided by the second embodiment of the present invention. Figure 6 Road boundaries and lane markings are shown in Figure 6 The upper and lower lines in the middle are road boundaries, and the four lines in the middle are lane lines. The radar can effectively detect everything within its detection range and can be actually applied to mass production projects of autonomous driving.

[0080] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; inputs the grid feature data into the radar neural network model to obtain the confidence and code value corresponding to the grid feature data output by the radar neural network model; clusters the grid feature data based on the confidence and code value based on the density clustering algorithm, and determines the road line corresponding to the road to be detected based on the clustering result. The technical solution of the present invention processes the grid feature data corresponding to the point cloud data through the radar neural network, and then clusters based on the code value corresponding to the grid feature data to determine the lane line of the road to be detected, which solves the problems of high time consumption, low precision, and difficulty in following up new technology iterations of the traditional point cloud segmentation algorithm, and the problem that the vision-based detection solution is limited by the short detection distance and small detection range of the camera, and realizes long-distance detection of road line detection, and the detection effect is stable and accurate.

[0081] The technical solution of the embodiment of the present invention can use radar point cloud as the input feature of the neural network; can detect the road within a distance of 50m in the front, which exceeds the effective range of visual detection; can cluster the feature outputs of different lane lines through a clustering algorithm, which is more convenient for analyzing the drivable area. It has the beneficial effects of long detection distance, stable detection effect, and no influence of the scene.

[0082] Embodiment 3

[0083] Figure 7 FIG. 1 is a schematic diagram of a road line detection device based on a radar neural network provided in Embodiment 3 of the present invention. Figure 7 As shown, the device comprises:

[0084] The point cloud data acquisition module 310 is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data;

[0085] The feature data processing module 320 is used to input the grid feature data into the radar neural network model to obtain the confidence and coding value corresponding to the grid feature data output by the radar neural network model;

[0086] The density clustering module 330 is used to cluster the point cloud grids corresponding to the grid feature data based on the confidence and the coding value based on the density clustering algorithm, and to obtain the road line corresponding to the road to be detected based on the clustering result.

[0087] The technical solution of the embodiment of the present invention obtains the target point cloud data corresponding to the road to be detected, and converts the target point cloud data into grid feature data; inputs the grid feature data into the radar neural network model to obtain the confidence and code value corresponding to the grid feature data output by the radar neural network model; clusters the grid feature data based on the confidence and code value based on the density clustering algorithm, and determines the road line corresponding to the road to be detected based on the clustering result. The technical solution of the present invention processes the grid feature data corresponding to the point cloud data through the radar neural network, and then clusters based on the code value corresponding to the grid feature data to determine the lane line of the road to be detected, which solves the problems of high time consumption, low precision, and difficulty in following up new technology iterations of the traditional point cloud segmentation algorithm, and the problem that the vision-based detection solution is limited by the short detection distance and small detection range of the camera, and realizes long-distance detection of road line detection, and the detection effect is stable and accurate.

[0088] Optionally, the point cloud data acquisition module 310 includes:

[0089] A screening module is used to collect the to-be-processed point cloud data corresponding to the to-be-detected road through a laser radar, and to screen and process the invalid point cloud data in the to-be-processed point cloud data to obtain the target point cloud data;

[0090] A division module, used for determining a grid division parameter, and dividing the target point cloud data into a plurality of point cloud grids based on the grid division parameter;

[0091] A feature determination module, for processing the target point cloud data in the point cloud grid for each of the point cloud grids, to obtain the grid feature data corresponding to each of the point cloud grids;

[0092] The grid division parameters include grid division size and / or grid division range.

[0093] Optionally, after filtering out the invalid point cloud data in the point cloud data to be processed, the method further includes: converting the point cloud data to be processed into point cloud data matching the target vehicle coordinate information according to the target vehicle coordinate information.

[0094] Optionally, the radar neural network model includes: a downsampling layer, an upsampling layer and a classifier, the downsampling layer is connected to the upsampling layer; the classifier is connected to the upsampling layer; the downsampling layer includes a residual network, and the number of layers of the downsampling layer and / or the upsampling layer is at least two.

[0095] Optionally, the feature data processing module 320 is specifically used to:

[0096] Inputting the grid feature data into at least two downsampling layers in sequence to obtain downsampling features output by each sampling layer;

[0097] The down-sampled features are fused to obtain features to be processed, and the features to be processed are input into the up-sampling layer to obtain up-sampled features output by the up-sampling layer;

[0098] The upsampled feature processing is input into the classifier to obtain the confidence and coding value corresponding to the raster feature data, wherein the confidence includes the road boundary confidence and the lane line confidence, and the coding value includes the road boundary coding value and the lane line coding value.

[0099] Optionally, the density clustering module 330 is specifically used to:

[0100] Taking each point cloud grid corresponding to the grid feature data as a sample point, presetting the neighborhood radius and density threshold of the density clustering algorithm, randomly selecting a sample point, and determining the sample point as a core sample point based on the neighborhood radius and the density threshold;

[0101] Determine the density-reachable sample points corresponding to the core sample points, and assign the reachable sample points to the clusters corresponding to the core sample points, wherein the reachable sample points are sample points that have not been assigned to the clusters;

[0102] The cluster corresponding to the core sample point determines the road line corresponding to the road to be detected.

[0103] Optionally, the density clustering module 330 is further used to:

[0104] If the sample point is a non-core sample point, the process of randomly selecting a sample point and determining whether the sample point is a core sample point based on the neighborhood radius and the density threshold is repeatedly performed until the sample point is a core sample point.

[0105] The road line detection device based on radar neural network provided in the embodiment of the present invention can execute the road line detection method based on radar neural network provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0106] Embodiment 4

[0107] Figure 8A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0108] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0109] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a road line detection method based on a radar neural network.

[0111] In some embodiments, the road line detection method based on the radar neural network can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the road line detection method based on the radar neural network described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the road line detection method based on the radar neural network in any other appropriate manner (for example, by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0114] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0116] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0117] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0118] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0119] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A road line detection method based on radar neural network, characterized in that: include: Acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data; Inputting the grid feature data into a radar neural network model to obtain a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model; According to the confidence and the coding value, the point cloud grids corresponding to the grid feature data are clustered based on a density clustering algorithm, and the road line corresponding to the road to be detected is determined based on the clustering result.

2. The method according to claim 1, characterized in that The step of acquiring target point cloud data corresponding to the road to be detected and converting the target point cloud data into grid feature data includes: Collecting the to-be-processed point cloud data corresponding to the to-be-detected road by means of a laser radar, filtering out the invalid point cloud data in the to-be-processed point cloud data, and obtaining the target point cloud data; Determine a grid division parameter, and divide the target point cloud data into a plurality of point cloud grids based on the grid division parameter; For each of the point cloud grids, the target point cloud data in the point cloud grid is processed to obtain the grid feature data corresponding to each of the point cloud grids; The grid division parameters include grid division size and / or grid division range.

3. The method according to claim 2, characterized in that After the invalid point cloud data in the to-be-processed point cloud data are screened, the method further includes: According to the target vehicle coordinate information, the to-be-processed point cloud data is converted into point cloud data matching the target vehicle coordinate information.

4. The method according to claim 1, characterized in that: The radar neural network model comprises: A downsampling layer, an upsampling layer and a classifier, wherein the downsampling layer is connected to the upsampling layer; and the classifier is connected to the upsampling layer; The downsampling layer includes a residual network, and the number of layers of the downsampling layer and / or the upsampling layer is at least two.

5. The method according to claim 4, characterized in that The step of inputting the grid feature data into a radar neural network model to obtain a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model includes: Inputting the grid feature data into at least two downsampling layers in sequence to obtain downsampling features output by each sampling layer; The down-sampled features are fused to obtain features to be processed, and the features to be processed are input into the up-sampling layer to obtain up-sampled features output by the up-sampling layer; The upsampled feature processing is input into the classifier to obtain the confidence and coding value corresponding to the raster feature data, wherein the confidence includes the road boundary confidence and the lane line confidence, and the coding value includes the road boundary coding value and the lane line coding value.

6. The method according to claim 4, characterized in that The step of clustering the point cloud grids corresponding to the grid feature data based on the confidence level and the coding value based on a density clustering algorithm, and clustering the road line corresponding to the road to be detected based on the clustering result, includes: Taking the code value corresponding to each of the grid feature data as a sample point, presetting the neighborhood radius and density threshold of the density clustering algorithm, randomly selecting a sample point, and determining the sample point as a core sample point based on the neighborhood radius and the density threshold; Determine the density-reachable sample points corresponding to the core sample points, and assign the reachable sample points to the clusters corresponding to the core sample points, wherein the reachable sample points are sample points that have not been assigned to the clusters; A road line corresponding to the road to be detected is determined based on the cluster corresponding to the core sample point.

7. The method according to claim 6, characterized in that Also includes: If the sample point is a non-core sample point, the process of randomly selecting a sample point and determining whether the sample point is a core sample point based on the neighborhood radius and the density threshold is repeatedly performed until the sample point is a core sample point.

8. A road line detection device based on radar neural network, characterized in that: include: A point cloud data acquisition module is used to acquire target point cloud data corresponding to the road to be detected, and convert the target point cloud data into raster feature data; A feature data processing module, used for inputting the grid feature data into a radar neural network model to obtain a confidence level and a coding value corresponding to the grid feature data output by the radar neural network model; The density clustering module is used to cluster the point cloud grids corresponding to the grid feature data based on the confidence and the coding value based on the density clustering algorithm, and the road line corresponding to the road to be detected is based on the clustering result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the road line detection method based on radar neural network according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the road line detection method based on a radar neural network according to any one of claims 1 to 7 when executed.

Images