Lane line data processing method, device, equipment and storage medium

Abstract

The lane line data processing method, device, equipment and storage medium provided by the present disclosure relate to the field of data processing technology, and in particular to the field of vehicle networking, intelligent transportation, intelligent search, and autonomous driving technology. The specific implementation scheme is: by obtaining a lane line data point set, each lane line data point of the lane line data point set carries position information, and then based on the position information carried by each lane line data point, the lane line skeleton map corresponding to the lane line data point set is determined, and then the grid points in the lane line skeleton map are clustered to obtain at least one type of lane line grid points, and finally at least one type of lane line grid points are fitted to obtain lane lines. This technical solution can accurately determine the position of lane lines in the map, thereby improving the accuracy of the map to a certain extent and enhancing the user experience during navigation.

Description

Technical Field

[0001] The present disclosure relates to the fields of vehicle networking, intelligent transportation, intelligent search, and autonomous driving technology in data processing, and in particular to a lane line data processing method, device, equipment, and storage medium. Background Art

[0002] High-precision maps are used in scenarios such as autonomous driving and drones, providing very accurate and important prior knowledge. Unlike traditional navigation maps, high-precision maps require rapid and accurate feedback updates on changes in road elements within the coverage area.

[0003] In the related technology, in the process of producing high-precision maps, the location data sent back by users is collected, and then the map is constructed or updated using the location data. However, due to the large amount of data processed, this solution may have the problem of inaccurate lane line positions, and thus cannot guarantee the safety of the unmanned driving process. Summary of the invention

[0004] The present invention provides a lane line data processing method, device, equipment and storage medium.

[0005] According to a first aspect of the present disclosure, a lane line data processing method is provided, comprising:

[0006] Acquire a lane line data point set, each lane line data point in the lane line data point set carrying position information;

[0007] Determine a lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point;

[0008] Performing lane line clustering on the grid points in the lane line skeleton image to obtain at least one type of lane line grid points;

[0009] Fitting the at least one type of lane line grid points to obtain at least one type of lane line.

[0010] According to a second aspect of the present disclosure, a lane line data processing device is provided, comprising:

[0011] An acquisition unit, used to acquire a lane line data point set, each lane line data point in the lane line data point set carries position information;

[0012] A determination unit, configured to determine a lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point;

[0013] A clustering unit, used to perform lane line clustering on the grid points in the lane line skeleton image to obtain at least one type of lane line grid points;

[0014] The fitting unit is used to fit the at least one type of lane line grid points to obtain at least one type of lane line.

[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0021] According to the technical solution disclosed in the present invention, the problem of inaccurate lane line positions that may be processed during map updating is solved, thereby improving the safety of the unmanned driving process.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 It is a schematic diagram of an application scenario applicable to the embodiment of the present disclosure;

[0025] Figure 2 is a flowchart of a lane line data processing method provided by the first embodiment of the present disclosure;

[0026] Figure 3 is a flow chart of a lane line data processing method provided in the second embodiment of the present disclosure;

[0027] Figure 4 is a flowchart of a lane line data processing method provided in the third embodiment of the present disclosure;

[0028] Figure 5 is a structural schematic diagram of a lane line data processing device provided by an embodiment of the present disclosure;

[0029] Figure 6 is a schematic block diagram of an example electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] The technical solution disclosed in the present invention is applied to map updates, especially in the update scenario of high-precision maps. Due to the high labor cost of map collection and map construction and the long update cycle, using the feedback data to update the existing map is a low-cost solution to maintain the current status of the map. However, due to the large amount of data processed by this solution, the processed lane line position may be inaccurate, resulting in an inaccurate updated map, and thus the safety of the unmanned driving process cannot be guaranteed.

[0032] In response to the above problems, the technical conception process of the technical solution disclosed in the present invention is as follows: the inventor discovered that in the process of map construction using feedback data, a lane line skeleton map can be determined based on a grid map, and then the lane line at each position is determined by clustering at each grid point, and finally fitted. This method can improve the accuracy of the lane lines in the constructed map, thereby improving the safety of the unmanned driving process to a certain extent.

[0033] Based on the above technical conception process, the embodiment of the present disclosure provides a lane line data processing method, which obtains a lane line data point set, each lane line data point of which carries position information, and then determines the lane line skeleton map corresponding to the lane line data point set based on the position information carried by each lane line data point, and then performs lane line clustering on the grid points in the lane line skeleton map to obtain at least one type of lane line grid points, and finally fits at least one type of lane line grid points to obtain lane lines. This technical solution can accurately determine the position of lane lines in the map, thereby improving the accuracy of the map to a certain extent and enhancing the user experience during navigation.

[0034] The present invention discloses a lane line data processing method, device, equipment and storage medium, which are applied to the fields of vehicle networking, intelligent transportation, intelligent search and autonomous driving technology in data processing to improve the position of lane lines in a map, thereby improving the accuracy of the map and enhancing the user's navigation experience.

[0035] It should be noted that the probability graph model in this embodiment is not a probability model for a specific object and cannot reflect the probability information of a specific object. It should be noted that the lane line data point set in this embodiment is from a public data set.

[0036] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0037] For example, Figure 1 Schematic diagram of an application scenario applicable to the embodiment of the present disclosure. Figure 1 As shown, the application scenario may include: a driving lane 100 , vehicles 111 to 114 driving on the driving lane 100 , and a processing device 12 .

[0038] Optionally, the vehicle in the embodiment of the present disclosure may be an autonomous driving vehicle or a manned vehicle, and both vehicles are equipped with an on-board terminal and an on-board sensor, and the on-board terminal may be used to exchange information with the processing device 12. The on-board sensor is used to collect road data of the driving lane 100. Exemplarily, the on-board sensor may be a camera, and the road data collected by the camera may be a driving lane image, and accordingly, lane line data may be obtained by performing image preprocessing on the driving lane image. The on-board sensor may be a radar, a detection sensor, etc., which may obtain lane line data on the driving lane by detection, etc.

[0039] It can be understood that since the vehicle-mounted terminal can carry the collected location information when controlling the vehicle-mounted sensor to collect lane line data, the lane line data points in this embodiment all carry location information.

[0040] In an embodiment of the present disclosure, the processing device 12 can receive lane line data transmitted back by at least one of vehicles 111 to 114, obtain a lane line data point set, and then execute the technical solution provided in the embodiment of the present disclosure to obtain the lane line.

[0041] It can be understood that the embodiment of the present application is explained by collecting lane line data from vehicle 111 to vehicle 114. For example, vehicle 111 is an autonomous driving vehicle. The on-board terminal in the embodiment of the present application can also be called an on-board computing platform, which can control the processor to periodically run autonomous driving related task programs when the autonomous driving function is activated. The programs include but are not limited to perception, decision-making, planning and control task programs. It can be understood that the various task programs can be executed in parallel, and this is not limited at this time.

[0042] In actual applications, when an autonomous driving vehicle is driving on a road, the on-board terminal needs to use the on-board computing resources to perform autonomous driving tasks, such as changing lanes when encountering obstacles, driving according to traffic regulations at intersections, giving way to pedestrians, etc., to meet the functions of autonomous driving.

[0043] Optionally, in actual applications, some vehicles are also provided with a driving computer or an on-board unit (OBU), and some vehicles are equipped with a user terminal such as a mobile phone, and a user holding the user terminal, etc. The driving computer or OBU of the vehicle can communicate with the processing device 12, etc.

[0044] Understandably, Figure 1 The objects in the application scenario shown are only schematic illustrations, and the disclosed embodiment does not specifically limit the objects included in the application scenario. In the application scenario, vehicles can communicate wirelessly with each other, and each vehicle can communicate wirelessly with the processing device 12, etc.

[0045] It should also be noted that the processing device in the embodiments of the present disclosure may be a terminal device, a server or a virtual machine, etc., or a distributed computer system composed of one or more servers and / or computers, etc. Among them, the terminal device includes but is not limited to: smart phones, laptops, desktop computers, platform computers, vehicle-mounted devices, smart wearable devices, etc., which are not limited in the embodiments of the present disclosure. The server can be an ordinary server or a cloud server. The cloud server is also called a cloud computing server or a cloud host, which is a host product in the cloud computing service system. The server can also be a server of a distributed system, or a server combined with a blockchain.

[0046] It is worth noting that the product implementation form of the present disclosure is a program code contained in the platform software and deployed on a processing device (which may also be hardware with computing capabilities such as a computing cloud or a mobile terminal). Figure 1 In the system structure diagram shown, the program code of the present disclosure can be stored inside the processing device. When running, the program code runs in the host memory and / or GPU memory of the processing device.

[0047] In the embodiments of the present disclosure, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0048] Next, combined with the above Figure 1 The application scenarios shown are used to describe the technical solutions of the present disclosure in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0049] For example, Figure 2 is a flow chart of a lane line data processing method provided by the first embodiment of the present disclosure. Figure 1 The processing device in the example is used as the execution subject for explanation. Figure 2 As shown, the lane line data processing method may include the following steps:

[0050] S201. Obtain a lane line data point set, each lane line data point in the lane line data point set carries position information.

[0051] In the embodiments of the present disclosure, the processing device can receive lane line data point sets from other devices, or read lane line data point sets from a database stored in its own storage (in this case, a database is deployed in the processing device), or directly receive lane line data point sets from a vehicle traveling on the driving lane.

[0052] It is understandable that in the embodiment of the present disclosure, the lane line data point set acquired by the processing device may be a pre-processed data point set or an unprocessed data point set, and this embodiment does not limit this.

[0053] S202: Based on the position information carried by each lane line data point, determine a lane line skeleton graph corresponding to the lane line data point set.

[0054] Exemplarily, in this embodiment, since each lane line data point carries location information, it can reflect the specific location of the collected data. Therefore, by performing position processing on each lane line data point in the lane line data point set, the approximate location distribution of the lane line can be determined, and then the lane line skeleton map corresponding to the lane line data point set can be obtained.

[0055] In an example of this embodiment, the processing device may perform rasterization processing on each lane line data point in the lane line data point set, determine the probability distribution of the lane line based on the form of a raster graph, and finally determine the lane line skeleton graph.

[0056] S203: performing lane line clustering on the grid points in the lane line skeleton image to obtain at least one type of lane line grid points.

[0057] Optionally, after determining the lane line skeleton map, lane line clustering processing may be performed on the grid points in the lane line skeleton map to determine the type of lane line that may exist at each grid point, and then determine the grid points of at least one type of lane line.

[0058] In practical applications, lane lines all have one or more elements, such as line type, color, thickness, etc. Different elements may correspond to different lane line types. Therefore, the processing device may cluster the grid points in the lane line skeleton map based on at least one lane line element, and then determine at least one type of lane line grid points.

[0059] S204: Fit at least one type of lane line grid points to obtain at least one type of lane line.

[0060] In the embodiment of the present disclosure, this step mainly involves fitting the output lane line grid points into lane lines. Specifically, after determining at least one type of lane line grid points, line segment fitting can be performed for each type of lane line grid points to determine the lane lines corresponding to each type of lane line grid points, thereby realizing the update of the lane lines in the map.

[0061] Exemplarily, in a possible design of this embodiment, the specific implementation of this step is as follows:

[0062] First, the length of the dividing line is determined according to the curvature of the curve formed by each type of lane line grid points. Then, based on each dividing line length, the curve formed by each type of lane line grid points is divided to obtain at least one lane line segment corresponding to each type of lane line grid points. Finally, line segment fitting is performed on at least one lane line segment corresponding to each type of lane line grid points to obtain at least one type of lane line.

[0063] Optionally, for each type of lane grid point determined in S203, the segmentation length may be determined according to the curvature. A larger curvature may be segmented into more segments, while a smaller curvature (straight line) may be segmented into fewer segments, so as to ensure the fitting effect.

[0064] In the disclosed embodiment, a lane line data point set is obtained, each lane line data point of which carries position information, and then based on the position information carried by each lane line data point, a lane line skeleton map corresponding to the lane line data point set is determined, and then grid points in the lane line skeleton map are clustered to obtain at least one type of lane line grid points, and finally at least one type of lane line grid points are fitted to obtain lane lines. This technical solution can accurately determine the position of lane lines in the map, thereby improving the accuracy of the map to a certain extent and enhancing the user experience during navigation.

[0065] exist Figure 2 Based on the illustrated embodiment, the lane line data processing method provided by the embodiment of the present disclosure is introduced in more detail below.

[0066] For example, Figure 3 FIG. 2 is a flow chart of a lane line data processing method provided by the second embodiment of the present disclosure. Figure 3 As shown, in the embodiment of the present disclosure, the above S203 can be implemented by the following steps:

[0067] S301. According to the position of each grid point in the lane line skeleton map, query the attribute dictionary to determine the lane line element of each grid point.

[0068] The attribute dictionary stores the correspondence between the position of each grid point and the lane line element.

[0069] In an embodiment of the present disclosure, the processing device can determine the position of each grid point and the lane line element at each grid point in the process of processing each lane line data point in the lane line data point set, that is, in the process of generating grid points, and establish a correspondence between the position of each grid point and the lane line element. Finally, the position of each grid point can be used as an index to store the lane line element corresponding to the position of each grid point in a pre-constructed or directly constructed attribute dictionary for subsequent use.

[0070] Accordingly, in the process of lane line clustering of grid points in the lane line skeleton map, the processing device can query the attribute dictionary based on the position of each grid point, determine the lane line element corresponding to the position of each grid point, and provide the possibility for subsequent lane line clustering.

[0071] S302: performing lane line clustering on the grid points in the lane line skeleton image based on the lane line element of each grid point to obtain at least one type of lane line grid points.

[0072] Optionally, the processing device can distinguish different lane lines in this step. Exemplarily, the processing device can determine the lane line type at each grid point in the lane line skeleton image based on the lane line elements, and then determine at least one type of lane line grid point corresponding to the lane line skeleton image by clustering the lane line types.

[0073] For example, in a possible design of this embodiment, the lane line element includes: a lane line angle; for example, the lane line angle may be an angle based on the true north direction. Accordingly, S302 may be implemented by the following steps:

[0074] A1. Perform lane line clustering on each grid point in the lane line skeleton image according to the lane line angle, and determine at least one type of lane line angle corresponding to each grid point.

[0075] Optionally, in this step, the processing device can find the corresponding lane line angle in the attribute dictionary based on the position of each grid point in the lane line skeleton map, and then determine the lane line angle type existing at each grid point in the lane line skeleton map based on the lane line angle, that is, determine how many lane lines in different directions pass through the grid point.

[0076] Exemplarily, the processing device may use a 1D clustering method to determine at least one type of lane line angle corresponding to each grid point. The specific implementation is as follows: sorting the lane line angle of each grid point in the lane line skeleton map, obtaining the sorted skeleton map grid points and the lane line angle difference between adjacent skeleton map grid points, clustering the sorted skeleton map grid points according to the lane line angle difference between adjacent skeleton map grid points and the preset angle difference, and determining at least one type of lane line angle corresponding to each grid point.

[0077] Among them, in this embodiment, it is assumed that the preset angle difference is 10 degrees, and -pi (-π, -180 degrees) and pi (π, 180 degrees) are to be clustered together. Therefore, after sorting the lane line angle of each grid point in the lane line skeleton map, the sorted skeleton map grid points can be obtained first, and then the lane line angle difference between adjacent skeleton map grid points is calculated for the sorted skeleton map grid points, and the lane line grid points with a lane line angle difference less than 10 degrees are clustered into one category. Finally, after all the sorted skeleton map grid points are clustered, at least one type of lane line angle corresponding to each grid point can be determined.

[0078] A2. According to each type of lane line angle, a depth-first search is performed on the grid points in the lane line skeleton map to obtain the lane line grid points corresponding to each type of lane line angle.

[0079] In this embodiment, because the lane line skeleton map is basically connected together, a depth-first search can be started for an edge point of the lane line skeleton map, and points whose lane line angles with the previous grid point are less than a preset angle difference can be dynamically connected to achieve clustering.

[0080] It can be understood that depth-first search (DFS) is an algorithm used to traverse or search the lane line skeleton map data structure. It can start from any grid point in the lane line skeleton map and traverse all grid points in the lane line skeleton map, that is, to ensure that all grid points in the lane line skeleton map have been visited, and determine the lane line grid points corresponding to each type of lane line angle.

[0081] A3. Traverse the lane line divergence points in the lane line grid points corresponding to each type of lane line angle to obtain at least one type of lane line grid points corresponding to the lane line skeleton diagram.

[0082] In this embodiment, after determining the lane line grid points corresponding to each type of lane line angle, the processing device can further determine whether there is a lane line divergence point among the lane line grid points corresponding to each type of lane line angle. If there is a lane line divergence point, the multiple lane lines after the lane line divergence point are divided into different lane line types.

[0083] For example, the processing device can use a shell model to find lane divergence points, and set the diverged lane lines as another type according to the actual situation, and output them as lane line grid points of different types. The shell model is a model for identifying lane line divergence points, and divides the diverged lane lines into different types, thereby improving the accuracy of lane line types.

[0084] In an example of this embodiment, the lane line element further includes: lane line color and / or lane line type; accordingly, before the processing device traverses the lane line divergence points in the lane line grid points corresponding to each type of lane line angle and obtains at least one type of lane line grid points corresponding to the lane line skeleton graph, it may also first perform the following operations:

[0085] Lane line grid points corresponding to each type of lane line angle are clustered based on lane line color and / or lane line type.

[0086] In actual applications, since different colors and different line types can also distinguish different lane lines, the processing device can find the corresponding lane line color and / or lane line type in the attribute dictionary based on the position of each grid point in the lane line skeleton map, and then determine the lane line type at each grid point in the lane line skeleton map based on the lane line color and / or lane line type, that is, different lane line colors and different lane line types belong to different lane line types.

[0087] Correspondingly, the processing device may also perform lane line clustering on the lane line grid points corresponding to each type of lane line angle based on the lane line color and / or lane line type to determine more lane line types.

[0088] For example, the attribute dictionary stores each detected lane line element with the location of the data point, for example, each is stored as a list under a different key: {pos:{color:[],markings:[],headings:[]}}. Among them, pos represents the location of the grid point, color represents the color of the lane line, markings represents the line type of the lane line (solid or dashed), and headings represents the angle of the lane line.

[0089] In the embodiment of the present disclosure, the lane line element of each grid point is determined by querying the attribute dictionary according to the position of each grid point in the lane line skeleton map, and then the grid points in the lane line skeleton map are clustered based on the lane line element of each grid point to obtain at least one type of lane line grid points. In this technical solution, the lane line grid points in the lane line grid map are clustered based on the lane line element, which improves the accuracy of lane line classification.

[0090] For example, Figure 4 FIG. 1 is a flow chart of a lane line data processing method provided in the third embodiment of the present disclosure. Figure 4 As shown, in the embodiment of the present disclosure, the above S202 can be implemented by the following steps:

[0091] S401: Create an initial grid map based on the position information and lane equation carried by each lane data point.

[0092] The value of each grid point in the initial grid image is the probability that a lane line exists at the grid point.

[0093] In this embodiment, the lane line equations of certain driving lanes may be pre-stored in the processing device. Therefore, after obtaining the lane line data point set, an initial grid map may be established based on the position information and lane line equation carried by each lane line data point in the lane line data point set.

[0094] Exemplarily, the size of the initial grid map may be 0.5 m, and each grid position in the initial grid map can reflect the position of the lane line.

[0095] Optionally, the processing device can determine the location area of ​​the lane line based on the location information carried by each lane line data point in the lane line data point set, and then rasterize the location area of ​​the lane line to obtain a grid map, and then use the number of data points in each grid point to determine the probability of the existence of a lane line at each grid point, and use it as the value of the grid point.

[0096] S402: Probabilistically update each grid point of the initial grid map using a probability graph model to obtain a lane line grid map.

[0097] Among them, the probabilistic graphical model (PGM) is a probability distribution represented by a graph. Among them, the probabilistic undirected graphical model is also called the Markov random field, which represents a joint probability distribution and can represent the dependency relationship between random variables.

[0098] In this embodiment, a probability graph model is used to update the probability of each grid point in the initial grid graph. That is, for each grid point with a value in the initial grid graph, loopy belief propagation is used to calculate the probability propagation of other grids with data to the current grid, without probabilistically penalizing unobserved grid points, thereby avoiding the risk of grid points without lane lines being worn out (especially in elevated scenarios).

[0099] For example, for elevated scenes, lane lines may have multiple layers, and the processing device does not perform probability penalties on the grid points corresponding to the lane lines on the bottom layer, that is, when the lane lines in different directions are layered, the lane line duplication phenomenon that occurs when the lane lines on different layers are merged into the same layer can be avoided. This method relies on the Gaussian characteristics of sensor noise and utilizes the fact that multiple observations will be concentrated at the real position, so that an accurate lane line grid map can be determined.

[0100] S403 , based on the probability of each grid point in the lane line grid map, the lane line grid map is processed to obtain a lane line skeleton map.

[0101] Exemplarily, after the probability of each grid point in the lane line grid map is determined, a lane line skeleton map is obtained through binarization processing, denoising processing, skeleton map extraction and other operations.

[0102] Optionally, in this step, the processing device may first determine a grid binary image corresponding to the lane line grid map based on the probability of each grid point in the lane line grid map and a preset probability threshold, then denoise the grid binary image to obtain a processed grid binary image, and finally perform skeleton image extraction on the processed grid binary image to obtain a lane line skeleton image.

[0103] In this embodiment, after determining the lane line grid map, firstly, the grid points whose probabilities exceed a certain threshold can be extracted through binarization processing. For example, the grid points whose probabilities exceed the threshold are set to 1, and the grid points whose probabilities do not exceed the threshold are set to 0 to obtain a grid binary map. Then, the grid binary map is denoised using algorithms such as opening operation and median filtering to obtain a processed grid binary map. Finally, the skeleton map is extracted based on the thinning algorithm of the openCV extension package ximgproc to obtain a lane line skeleton map.

[0104] Among them, the opening operation refers to removing isolated grid points to reduce noise, and the median filter mainly performs smoothing on the grid area. It sets the probability value of each grid to the median of all probability values ​​in the grid neighborhood grid, which can also reduce noise. It can be understood that the embodiment of the present disclosure does not limit the algorithm of denoising processing, which can be determined according to actual needs.

[0105] OpenCV is a cross-platform computer vision and machine learning software library based on open source distribution. Ximgproc is an image processing module in this extension module. The thinning algorithm of ximgproc can be used to extract lane line skeleton images.

[0106] In the embodiment of the present disclosure, an initial grid map is established based on the position information and lane line equation carried by each lane line data point, and a probability map model is used to update the probability of each grid point in the initial grid map to obtain a lane line grid map. Based on the probability of each grid point in the lane line grid map, the lane line grid map is processed to obtain a lane line skeleton map. This technical solution can obtain an accurate lane line skeleton map through a grid probability processing method, laying a foundation for the subsequent determination of accurate lane lines.

[0107] Figure 5 Schematic diagram of the structure of a lane line data processing device provided in an embodiment of the present disclosure. The lane line data processing device provided in this embodiment can be Figure 1 A processing device in a processing device or a device in a processing device. Figure 5 As shown, the lane line data processing device 500 provided in the embodiment of the present disclosure may include:

[0108] An acquisition unit 501 is used to acquire a lane line data point set, each lane line data point in the lane line data point set carries position information;

[0109] A determination unit 502, configured to determine a lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point;

[0110] A clustering unit 503 is used to perform lane line clustering on the grid points in the lane line skeleton image to obtain at least one type of lane line grid points;

[0111] The fitting unit 504 is used to fit the at least one type of lane line grid points to obtain at least one type of lane line.

[0112] In a possible design of this embodiment, the clustering unit 503 includes:

[0113] A query module, for querying an attribute dictionary according to the position of each grid point in the lane line skeleton map, and determining the lane line element of each grid point, wherein the attribute dictionary stores the corresponding relationship between the position of each grid point and the lane line element;

[0114] The clustering module is used to perform lane line clustering on the grid points in the lane line skeleton image based on the lane line element of each grid point to obtain at least one type of lane line grid points.

[0115] Optionally, the lane line elements include: lane line angle;

[0116] The clustering module comprises:

[0117] A determination submodule, configured to perform lane line clustering on each grid point in the lane line skeleton image according to the lane line angle, and determine at least one type of lane line angle corresponding to each grid point;

[0118] A search submodule, for performing a depth-first search on the grid points in the lane line skeleton image according to each type of lane line angle, to obtain the lane line grid points corresponding to each type of lane line angle;

[0119] The traversal submodule is used to traverse the lane line divergence points in the lane line grid points corresponding to each type of lane line angle, and obtain at least one type of lane line grid points corresponding to the lane line skeleton diagram.

[0120] Wherein, the determination submodule is used to:

[0121] Sorting lane line angles of each grid point in the lane line skeleton image to obtain sorted skeleton image grid points and lane line angle differences between adjacent skeleton image grid points;

[0122] The sorted skeleton image grid points are clustered according to the lane line angle difference between adjacent skeleton image grid points and the preset angle difference, and at least one type of lane line angle corresponding to each grid point is determined.

[0123] Optionally, the lane line element further includes: lane line color and / or lane line type;

[0124] The clustering module is further used to perform lane line clustering on lane line grid points corresponding to each type of lane line angle based on lane line color and / or lane line type.

[0125] In another possible design of this embodiment, the determining unit 502 includes:

[0126] An establishment module is used to establish an initial grid map according to the position information and lane line equation carried by each lane line data point, wherein the value of each grid point in the initial grid map is the probability that a lane line exists at the grid point;

[0127] An updating module, used for performing probability update on each grid point of the initial grid map by using a probability map model to obtain a lane line grid map;

[0128] A processing module is used to process the lane line grid map based on the probability of each grid point in the lane line grid map to obtain the lane line skeleton map.

[0129] Wherein, the processing module includes:

[0130] A binarization submodule, for determining a grid binary image corresponding to the lane line grid image according to the probability of each grid point in the lane line grid image and a preset probability threshold;

[0131] A denoising submodule, used for performing denoising processing on the raster binary image to obtain a processed raster binary image;

[0132] The extraction submodule is used to extract the skeleton image of the processed grid binary image to obtain the lane line skeleton image.

[0133] In another possible design of this embodiment, the fitting unit 504 includes:

[0134] A determination module, used to determine the length of the dividing line according to the curvature of the curve formed by the grid points of each type of lane line;

[0135] A segmentation module, configured to segment the curve formed by each type of lane grid points based on the length of each segmentation line, to obtain at least one lane line segment corresponding to each type of lane grid points;

[0136] The fitting module is used to perform line segment fitting on at least one lane line segment corresponding to each type of lane line grid point to obtain at least one type of lane line.

[0137] The lane line data processing device provided in this embodiment can be used to execute the lane line data processing method executed by the processing device in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be elaborated here.

[0138] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0139] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0140] Figure 6is a schematic block diagram of an example electronic device used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0141] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0142] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0143] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, for example, the lane line data processing method. For example, in some embodiments, the lane line data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the lane line data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the lane line data processing method in any other appropriate manner (e.g., by means of firmware).

[0144] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0146] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0148] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0149] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0150] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0151] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A lane line data processing method, comprising: Acquire a lane line data point set, each lane line data point in the lane line data point set carrying position information; Determine a lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point; Performing lane line clustering on the grid points in the lane line skeleton image based on at least one lane line element to obtain at least one type of lane line grid points; Fitting the at least one type of lane line grid points to obtain at least one type of lane line; The lane line element includes: a lane line angle; the lane line clustering of the grid points in the lane line skeleton image based on at least one lane line element to obtain at least one type of lane line grid points includes: Performing lane line clustering on each grid point in the lane line skeleton image according to the lane line angle, and determining at least one type of lane line angle corresponding to each grid point; According to each type of lane line angle, a depth-first search is performed on the grid points in the lane line skeleton image to obtain the lane line grid points corresponding to each type of lane line angle; The lane line divergence points in the lane line grid points corresponding to each type of lane line angle are traversed to obtain at least one type of lane line grid points corresponding to the lane line skeleton graph.

2. The method according to claim 1, wherein: The lane line clustering of the grid points in the lane line skeleton image based on at least one lane line element to obtain at least one type of lane line grid points also includes: According to the position of each grid point in the lane line skeleton map, an attribute dictionary is queried to determine the lane line element of each grid point, and the attribute dictionary stores the corresponding relationship between the position of each grid point and the lane line element.

3. The method according to claim 1, wherein: The lane line clustering of each grid point in the lane line skeleton image according to the lane line angle, and determining at least one type of lane line angle corresponding to each grid point, comprises: Sorting lane line angles of each grid point in the lane line skeleton image to obtain sorted skeleton image grid points and lane line angle differences between adjacent skeleton image grid points; The sorted skeleton image grid points are clustered according to the lane line angle difference between adjacent skeleton image grid points and the preset angle difference, and at least one type of lane line angle corresponding to each grid point is determined.

4. The method according to claim 1, wherein the lane line element further comprises: Lane marking color and / or lane marking style; Before traversing the lane line divergence points in the lane line grid points corresponding to each type of lane line angle to obtain at least one type of lane line grid points corresponding to the lane line skeleton graph, the method further includes: Lane line grid points corresponding to each type of lane line angle are clustered based on lane line color and / or lane line type.

5. The method according to any one of claims 1 to 4, wherein determining the lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point comprises: An initial grid map is established based on the position information and lane line equation carried by each lane line data point, wherein the value of each grid point in the initial grid map is the probability that a lane line exists at the grid point; Using a probability graph model to perform probability update on each grid point of the initial grid graph to obtain a lane line grid graph; The lane line grid map is processed based on the probability of each grid point in the lane line grid map to obtain the lane line skeleton map.

6. The method according to claim 5, wherein: The processing of the lane line grid map based on the probability of each grid point in the lane line grid map to obtain the lane line skeleton map includes: Determine a grid binary image corresponding to the lane line grid image according to the probability of each grid point in the lane line grid image and a preset probability threshold; Performing denoising processing on the raster binary image to obtain a processed raster binary image; A skeleton image is extracted from the processed grid binary image to obtain the lane line skeleton image.

7. The method according to any one of claims 1 to 4 and 6, wherein: The fitting of the at least one type of lane line grid points to obtain at least one type of lane line includes: Determine the length of the dividing line according to the curvature of the curve formed by each type of lane line grid points; The curve formed by each type of lane line grid points is divided based on the length of each dividing line to obtain at least one lane line segment corresponding to each type of lane line grid point; Line segment fitting is performed on at least one lane line segment corresponding to each type of lane line grid point to obtain at least one type of lane line.

8. A lane line data processing device, comprising: An acquisition unit, used to acquire a lane line data point set, each lane line data point in the lane line data point set carries position information; A determination unit, configured to determine a lane line skeleton graph corresponding to the lane line data point set based on the position information carried by each lane line data point; A clustering unit, configured to perform lane line clustering on the grid points in the lane line skeleton image based on at least one lane line element to obtain at least one type of lane line grid points; A fitting unit, used for fitting the at least one type of lane line grid points to obtain at least one type of lane line; The lane line element includes: a lane line angle; the clustering unit includes a clustering module, and the clustering module includes: A determination submodule, configured to perform lane line clustering on each grid point in the lane line skeleton image according to the lane line angle, and determine at least one type of lane line angle corresponding to each grid point; A search submodule, for performing a depth-first search on the grid points in the lane line skeleton image according to each type of lane line angle, to obtain the lane line grid points corresponding to each type of lane line angle; The traversal submodule is used to traverse the lane line divergence points in the lane line grid points corresponding to each type of lane line angle, and obtain at least one type of lane line grid points corresponding to the lane line skeleton diagram.

9. The device according to claim 8, wherein: The clustering unit further includes: The query module is used to query the attribute dictionary according to the position of each grid point in the lane line skeleton map to determine the lane line element of each grid point, and the attribute dictionary stores the corresponding relationship between the position of each grid point and the lane line element.

10. The device according to claim 8, wherein: The determining submodule is used to: Sorting lane line angles of each grid point in the lane line skeleton image to obtain sorted skeleton image grid points and lane line angle differences between adjacent skeleton image grid points; The sorted skeleton image grid points are clustered according to the lane line angle difference between adjacent skeleton image grid points and the preset angle difference, and at least one type of lane line angle corresponding to each grid point is determined.

11. The device according to claim 8, wherein the lane line element further comprises: Lane marking color and / or lane marking style; The clustering module is further used to perform lane line clustering on lane line grid points corresponding to each type of lane line angle based on lane line color and / or lane line type.

12. The device according to any one of claims 8 to 11, wherein the determining unit comprises: An establishment module is used to establish an initial grid map according to the position information and lane line equation carried by each lane line data point, wherein the value of each grid point in the initial grid map is the probability that a lane line exists at the grid point; An updating module, used for performing probability update on each grid point of the initial grid map by using a probability map model to obtain a lane line grid map; A processing module is used to process the lane line grid map based on the probability of each grid point in the lane line grid map to obtain the lane line skeleton map.

13. The device according to claim 12, wherein: The processing module comprises: A binarization submodule, for determining a grid binary image corresponding to the lane line grid image according to the probability of each grid point in the lane line grid image and a preset probability threshold; A denoising submodule, used for performing denoising processing on the raster binary image to obtain a processed raster binary image; The extraction submodule is used to extract the skeleton image of the processed grid binary image to obtain the lane line skeleton image.

14. The device according to any one of claims 8 to 11 and 13, wherein: The fitting unit comprises: A determination module, used to determine the length of the dividing line according to the curvature of the curve formed by the grid points of each type of lane line; A segmentation module, configured to segment the curve formed by each type of lane grid points based on the length of each segmentation line, to obtain at least one lane line segment corresponding to each type of lane grid points; The fitting module is used to perform line segment fitting on at least one lane line segment corresponding to each type of lane line grid point to obtain at least one type of lane line.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images