Method, device, equipment and product for generating lane lines on high-precision maps

Abstract

This disclosure provides methods, devices, equipment, and products for generating lane markings on high-precision maps. These methods relate to computer technology, specifically artificial intelligence technology, and are applicable to autonomous driving scenarios. The specific implementation involves acquiring road surface point cloud data; determining a road surface grid set based on the road surface point cloud data; optimizing each road surface grid in the set based on a preset optimization function to obtain the grid equation corresponding to each road surface grid; and generating lane markings based on the grid equations corresponding to each road surface grid. This implementation improves the accuracy of the generated lane markings.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically the field of image processing technology, and can be applied to autonomous driving scenarios. Background Art

[0002] High-definition maps, also known as high-precision maps, are used by self-driving cars. They contain precise vehicle location information and rich road element data. They help cars predict complex road conditions, such as slope, curvature, and heading, to better avoid potential risks. Lane information in HD maps is particularly important.

[0003] In autonomous driving scenarios, it's often necessary to generate high-precision lane markings on maps to assist with driving. For example, traditional visual 3D reconstruction can be used to generate 3D lane markings. However, in practice, it's been found that the accuracy of the reconstructed lane markings using traditional visual 3D reconstruction methods is poor. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, device, and product for generating lane lines on a high-precision map.

[0005] According to one aspect of the present disclosure, a method for generating map lane lines is provided, comprising: acquiring road surface point cloud data; determining a road surface grid set based on the road surface point cloud data; optimizing each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid; and generating lane lines based on the grid equation corresponding to each road surface grid.

[0006] According to another aspect of the present disclosure, a device for generating map lane lines is provided, comprising: a data acquisition unit configured to acquire road surface point cloud data; a set determination unit configured to determine a road surface grid set based on the road surface point cloud data; an equation determination unit configured to optimize each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid; and a lane line generation unit configured to generate lane lines based on the grid equations corresponding to each road surface grid.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above methods for generating map lane lines.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute any one of the above methods for generating map lane lines.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any of the above methods for generating map lane lines when executed by a processor.

[0010] According to the technology disclosed herein, a method for generating map lane lines is provided, which can improve the accuracy of the generated lane lines.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0013] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0014] Figure 2 is a flowchart of an embodiment of a method for generating map lane lines according to the present disclosure;

[0015] Figure 3 is a schematic diagram of an application scenario of the method for generating map lane lines according to the present disclosure;

[0016] Figure 4 is a flowchart of another embodiment of a method for generating map lane lines according to the present disclosure;

[0017] Figure 5 is a schematic structural diagram of an embodiment of an apparatus for generating map lane lines according to the present disclosure;

[0018] Figure 6 It is a block diagram of an electronic device used to implement the method for generating map lane lines according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0022] The terminal devices 101, 102, and 103 interact with the server 105 via the network 104 to receive or send messages, etc. The terminal devices 101, 102, and 103 may be electronic devices installed in each vehicle that can acquire road images. While the vehicle is traveling, the terminal devices 101, 102, and 103 can capture images of the various roads the vehicle passes through and upload these images to the server 105 via the network 104. Furthermore, the terminal devices 101, 102, and 103 may also be equipped with positioning capabilities. For example, the terminal devices 101, 102, and 103 can acquire positioning data while the vehicle is traveling based on a global navigation satellite system and an inertial measurement unit, and upload this positioning data to the server 105 via the network 104.

[0023] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be the above-mentioned electronic devices installed in each vehicle that can obtain road images and positioning data, including but not limited to mobile phones, on-board computers, on-board tablets, vehicle control devices, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0024] Server 105 can be a server that provides various services. For example, server 105 can obtain road image data and vehicle positioning data sent by terminal devices 101, 102, and 103, and generate road surface point cloud data based on the road image data and vehicle positioning data. Server 105 can then determine a road surface grid set based on the road surface point cloud data, optimize each road surface grid in the set using a preset optimization function, and obtain a grid equation corresponding to each road surface grid. Based on the grid equations corresponding to each road surface grid, server 105 can then generate lane markings on the road.

[0025] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitations are given here.

[0026] It should be noted that the method for generating map lane lines provided in the embodiment of the present disclosure can be executed by the server 105 , and the device for generating map lane lines can be set in the server 105 .

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0028] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for generating map lane lines according to the present disclosure. The method for generating map lane lines in this embodiment includes the following steps:

[0029] Step 201: Acquire road surface point cloud data.

[0030] In this embodiment, the execution subject (such as Figure 1 The server 105 in the embodiment can receive road images and vehicle positioning data transmitted by electronic devices mounted on each vehicle, and then determine road surface point cloud data based on these road images and vehicle positioning data. The road surface point cloud data can be data corresponding to each point on the road. The data corresponding to each point can include, but is not limited to, three-dimensional coordinate data, color data, reflection intensity data, etc., which is not limited in this embodiment. Determining road surface point cloud data based on road images and vehicle positioning data can be achieved using existing visual three-dimensional reconstruction technology, and the specific reconstruction method is not further described here.

[0031] Step 202: Determine a road surface grid set based on the road surface point cloud data.

[0032] In this embodiment, after obtaining the road surface point cloud data, the executing entity may preferably preprocess the road surface point cloud data to obtain preprocessed road surface point cloud data, and then determine a road surface grid set based on the preprocessed road surface point cloud data. Alternatively, the executing entity may directly determine the road surface grid set based on the road surface point cloud data, which is not limited in this embodiment. The road surface grids in the road surface grid set may be grids obtained by dividing the road corresponding to the road surface point cloud data.

[0033] In some optional implementations of this embodiment, determining a road surface grid set based on road surface point cloud data may include: determining initial information corresponding to the road based on the road surface point cloud data, which initial information may include but is not limited to road profile, road conditions, road angle, etc.; determining the number and size of grids based on the initial information corresponding to the road; and generating a road surface grid set corresponding to the road based on the determined number and size of grids. By implementing this optional implementation, the general condition of the road can be determined based on the road surface point cloud data, and the number and size of grid divisions can be determined in combination with the initial information of the road. Generating grids based on this number and size can improve the accuracy of the resulting road surface grids.

[0034] In this implementation, for curved roads or roads with a certain angle, the road surface can be divided into several grids, so that the plane angle within each grid is more accurate, in line with the requirements of driving direction and road characteristics, and then based on the optimization of this grid, more accurate lane lines can be obtained.

[0035] Step 203 : Optimize each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid.

[0036] In this embodiment, for each road surface grid in the road surface grid set, the execution entity can construct the plane equation of the road surface grid based on the point cloud coordinates located on the road surface grid. A plane equation refers to the equation corresponding to all points in space that are located on the same plane. For example, if the road surface grid is a triangular grid, the execution entity can construct the plane equation of the road surface grid based on the three vertices of the grid. Furthermore, the execution entity can pre-set an optimization function for optimizing each road surface grid in the road surface grid set. Based on the preset optimization function, the plane equation corresponding to the road surface grid can be optimized, ultimately obtaining the plane equation corresponding to each road surface grid, that is, the grid equation corresponding to each road surface grid.

[0037] Among them, since the collection of road point cloud data is realized by the electronic equipment installed on each vehicle, and for each vehicle, the road image and positioning data collected by the vehicle are obtained as the vehicle's driving trajectory changes. Therefore, for each vehicle, the closer the point cloud data is to the vehicle's driving trajectory, the higher the accuracy. In addition, the greater the density of the point cloud data, the higher the accuracy of the point cloud data. Therefore, when constructing a preset optimization function, for the interior of the road grid, the density of the point cloud data within the road grid and the distance between the point cloud data and the vehicle's driving trajectory can be taken into account. In addition, for the relationship between road grids, since the road needs to maintain the smoothness of the intersection between grids and the smoothness along the vehicle's driving direction, therefore, when constructing a preset optimization function, for the relationship between road grids, the smoothness characteristics of the grid intersection and the driving characteristics of the vehicle's driving direction can be taken into account.

[0038] In some optional implementations of this embodiment, optimizing each road surface grid in a road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid may include: constructing a preset optimization function based on internal and external characteristics of the road surface grid, wherein the internal characteristics of the road surface grid may include, but are not limited to, the density of point cloud data in the road surface grid and the distance between the point cloud data and the vehicle trajectory; and the external characteristics of the road surface grid may include, but are not limited to, smoothness characteristics corresponding to road surface grid intersections and smoothness characteristics of road surface grid vertices along the vehicle's travel direction; determining initial equations corresponding to each road surface grid; and optimizing each initial equation based on the preset optimization function to obtain a grid equation corresponding to each road surface grid. By implementing this optional implementation, the execution entity may determine an optimization function based on both internal and external characteristics of the road surface grid based on road characteristics and acquired characteristics of the point cloud data, and optimize each initial equation corresponding to each road surface grid based on the optimization function to obtain a grid equation corresponding to each road surface grid. The grid equation obtained in this manner can more accurately reflect road characteristics.

[0039] Step 204 : Generate lane lines based on the grid equations corresponding to the road surface grids.

[0040] In this embodiment, after obtaining the grid equations corresponding to each road surface grid, the execution entity can back-project the two-dimensional lane lines based on these grid equations to generate three-dimensional lane lines. Specifically, when determining road surface point cloud data based on road images and vehicle positioning data transmitted by electronic devices mounted on each vehicle, the execution entity can analyze the road images, identify lane lines within the road images, and generate two-dimensional lane lines based on the lane lines within the road images. The execution entity then uses the grid equations corresponding to each road surface grid to back-project the two-dimensional lane lines onto the road surface grids, constructing three-dimensional lane lines corresponding to the two-dimensional lane lines, thereby generating more accurate three-dimensional lane line information.

[0041] Continue to see Figure 3 , which shows a schematic diagram of an application scenario of the method for generating map lane lines according to the present disclosure. Figure 3 In an application scenario, the execution entity may first obtain crowdsourced road data based on information interaction with electronic devices mounted on a vehicle. The crowdsourced road data may include road images and vehicle positioning data. The execution entity may then determine road surface point cloud data 301 based on the road images and vehicle positioning data. Furthermore, based on recognition of the road images, two-dimensional lane line coordinates 306 may be obtained. Subsequently, the execution entity may divide the road surface into multiple triangular meshes based on the road surface point cloud data 301, obtaining a road surface mesh set 302. Subsequently, the execution entity may determine the plane equation 303 corresponding to each triangular mesh in the road surface mesh set 302. Furthermore, the execution entity may construct a preset optimization function based on the constraints 304 and optimize the plane equations 303 corresponding to each triangular mesh based on the preset optimization function, obtaining optimized plane equations 305. Three-dimensional lane lines 307 may then be generated based on the two-dimensional lane line coordinates 306 and the optimized plane equations 305.

[0042] The method for generating map lane lines provided by the above-mentioned embodiment of the present disclosure can determine a road surface grid set based on road surface point cloud data, and optimize each road surface grid based on a preset optimization function to obtain a grid equation corresponding to each road surface grid. Based on the optimized grid equation corresponding to each road surface grid, corresponding lane lines are generated, thereby improving the accuracy of the generated lane lines.

[0043] Continue to see Figure 4 , which shows a process 400 of another embodiment of the method for generating map lane lines according to the present disclosure. Figure 4 As shown, the method for generating map lane lines in this embodiment may include the following steps:

[0044] Step 401: Acquire road surface point cloud data.

[0045] In this embodiment, for the detailed description of step 401 , please refer to the detailed description of step 201 , which will not be repeated here.

[0046] Step 402 : For each point cloud coordinate in the road surface point cloud data, determine a quality score of the point cloud coordinate based on the distance between the point cloud coordinate and the vehicle driving trajectory and the density corresponding to the point cloud coordinate.

[0047] In this embodiment, after acquiring the road point cloud data, the execution entity can construct a point cloud image of the road, and the execution entity can mark the vehicle driving trajectory in the point cloud image based on the vehicle driving trajectory corresponding to the road point cloud data. Afterwards, for each point cloud data in the road point cloud data, the execution entity can calculate the distance between the point cloud coordinates and the vehicle driving trajectory, as well as the density within a preset range around the point cloud coordinates. The smaller the distance, the higher the accuracy of the point cloud coordinates, and the greater the density, the higher the accuracy of the point cloud coordinates. Therefore, the execution entity can determine the quality score of the point cloud coordinates based on the distance between the point cloud coordinates and the vehicle driving trajectory and the density corresponding to the point cloud coordinates. The quality score is used to describe the accuracy of the point cloud coordinates. The higher the quality score, the higher the accuracy of the point cloud coordinates.

[0048] Step 403 : determining target point cloud coordinates from the road surface point cloud data based on the quality scores of the respective point cloud coordinates.

[0049] In this embodiment, the execution entity sorts the point cloud coordinates according to their quality scores from high to low, and selects a preset number of point cloud coordinates in the sorted order as target point cloud coordinates. The target point cloud coordinates are used as the basis for point cloud hole filling preprocessing.

[0050] Step 404 : determining the point cloud hole filling coordinates based on the height value of the target point cloud coordinates.

[0051] In this embodiment, after obtaining the coordinates of each target point cloud, the execution entity can use the height values ​​of each target point cloud coordinate to calculate a corresponding height index value. The height index value may include, but is not limited to, the average height value, the median height value, the mode height value, etc., which is not limited in this embodiment. The execution entity can then use the height index value as the height value of the point cloud hole filling point to obtain a number of point cloud hole filling coordinates.

[0052] Step 405: Update the road surface point cloud data based on the hole filling coordinates of the point cloud.

[0053] In this embodiment, after obtaining the point cloud hole filling coordinates, the execution entity can add the point cloud hole filling coordinates to the road surface point cloud data to update the road surface point cloud data and obtain updated road surface point cloud data.

[0054] In some optional implementations of this embodiment, updating the road surface point cloud data based on the point cloud hole filling coordinates may include: determining the quality score information and neighborhood height information corresponding to the point cloud hole filling coordinates; determining the target point cloud hole filling coordinates from the point cloud hole filling coordinates based on the quality score information and neighborhood height information; and updating the road surface point cloud data based on the target point cloud hole filling coordinates.

[0055] In this implementation, after obtaining the point cloud hole filling coordinates, the execution entity may also determine the quality score information and neighborhood height information corresponding to the point cloud hole filling coordinates. The quality score information may include the quality score corresponding to each point cloud hole filling coordinate. For each point cloud hole filling coordinate, the quality score of the point cloud hole filling coordinate may be determined based on the distance between the point cloud hole filling coordinate and the vehicle driving trajectory and the density corresponding to the point cloud hole filling coordinate. The neighborhood height information may include the consistency parameter of the height between each point cloud hole filling coordinate and the surrounding neighborhood points. For each point cloud hole filling coordinate, the execution entity may first determine the neighborhood point set corresponding to the point cloud hole filling coordinate. Afterwards, the execution entity may calculate the consistency parameter of the height value based on the height value of the point cloud hole filling coordinate and the height value of each neighborhood point in the corresponding neighborhood point set. The consistency parameter of the height value may include but is not limited to the height standard deviation, height variance, height range, etc., which is not limited in this embodiment.

[0056] Afterwards, the execution entity can determine the target point cloud hole-filling coordinates with higher reliability from the point cloud hole-filling coordinates based on the quality score information and the neighborhood height information, and add the target point cloud hole-filling coordinates to the road surface point cloud data to update the road surface point cloud data. Specifically, the execution entity can determine the coordinates with higher quality scores among the point cloud hole-filling coordinates based on the quality score information as the target point cloud hole-filling coordinates, and the execution entity can determine the coordinates among the point cloud hole-filling coordinates with higher height consistency with the surrounding neighborhood points based on the neighborhood height information as the target point cloud hole-filling coordinates.

[0057] Step 406 : Determine each road surface plane based on the road surface point cloud data.

[0058] In this embodiment, the execution entity may first construct an initial point cloud image based on the road surface point cloud data, and then divide the initial point cloud image into various road surface planes within the initial point cloud image. Optionally, the execution entity may divide the initial point cloud image corresponding to the road surface point cloud data into various road surface planes along the vehicle's travel direction. Each road surface plane may serve as the basis for subsequent road surface grid division.

[0059] Step 407: For each road surface plane, determine the road surface grid corresponding to the road surface plane.

[0060] In this embodiment, after the execution entity divides the initial point cloud image into a number of road surface planes based on the road surface point cloud data, it can further divide each road surface plane into a road surface grid corresponding to the road surface plane. The shape of the road surface grid can be triangular or other shapes, which is not limited in this embodiment.

[0061] Step 408 : Determine a road surface grid set based on the road surface grids corresponding to each road surface plane.

[0062] In this embodiment, the execution entity may group the road surface grids corresponding to the road surface planes into a road surface grid set. Optionally, the execution entity may also establish a correspondence between the road surface planes and the road surface grids corresponding to the road surface planes during the process of grouping the road surface grid set.

[0063] Step 409 : Generate an optimization function based on at least one of the following constraints: a plane fitting constraint, an edge connection constraint, and a plane smoothness constraint.

[0064] In this embodiment, a plane fitting constraint can be used to optimize the height values ​​of the vertices of each road surface grid, which can be determined based on the distance from each point cloud coordinate in the point cloud data to the plane and the weight of each road surface grid plane. An edge constraint can be used to optimize the smoothness of the intersections of each road surface plane, which can be determined based on the point cloud coordinates of the intersections. A plane smooth constraint can be used to optimize the smoothness of the vertices of each road surface grid along the vehicle's travel direction, which can be determined based on the coordinates of the vertices of each road surface grid along the vehicle's travel direction.

[0065] In some optional implementations of this embodiment, the plane fitting constraint is determined based on the following steps: for each road surface grid in the road surface grid set, determine the weight corresponding to the road surface grid and the height value fitting error corresponding to the road surface grid; and perform weighted summation of the weights and height value fitting errors corresponding to each road surface grid to obtain the plane fitting constraint.

[0066] In this implementation, the weight corresponding to the pavement grid is used to describe the reliability of the pavement grid. The higher the reliability of the pavement grid, the higher the weight corresponding to the pavement grid. Optionally, the weight corresponding to the pavement grid can be set manually or automatically generated based on certain factors, which is not limited in this embodiment. Furthermore, the height value fitting error corresponding to the pavement grid is used to describe the distance parameter of the distance between the point cloud coordinates corresponding to the pavement grid and the grid plane when the height values ​​of the three vertices corresponding to the pavement grid are changed. The distance parameter may include but is not limited to the sum of the squares of the distances, the sum of the distances, etc., which is not limited in this embodiment. Specifically, the plane fitting constraint can be calculated based on the following formula:

[0067]

[0068] Among them, L fit refers to the plane fitting constraint, K represents the total number of road surface grids, k represents the kth road surface grid, γ k represents the weight of the kth road surface grid, Represents the height value fitting error of the kth road surface grid.

[0069] In this embodiment, if the plane fitting constraint is used to optimize each pavement grid in the pavement grid set, the following steps can be performed: the execution subject determines the normal vector formula of each pavement grid in the pavement grid set; based on the normal vector formula of the pavement grid, determines the plane equation corresponding to the pavement grid; controls the horizontal and vertical coordinates of the vertices of the pavement grid to remain unchanged, changes the height value of the vertex of the pavement grid, calculates the height value fitting error corresponding to the height value, substitutes the weight of the pavement grid and the height value fitting error into the corresponding optimization function, optimizes the height value, and obtains a pavement grid with optimized height values ​​of each vertex.

[0070] In some other optional implementations of this embodiment, for each road surface grid in the road surface grid set, a weight corresponding to the road surface grid is determined, including: for each road surface grid in the road surface grid set, determining the amount of point cloud data in the road surface grid and the distance between the road surface grid and the vehicle's driving trajectory; and determining the weight corresponding to the road surface grid based on the amount and distance of the point cloud data of the road surface grid.

[0071] In this implementation, the weight corresponding to the above-mentioned road surface grid can be determined based on the number of point cloud data in the road surface grid and the distance between the road surface grid and the vehicle's driving track. Specifically, the execution entity can calculate the quantity reliability of the point cloud data based on the number of point cloud data in the road surface grid, and the distance reliability based on the distance between the road surface grid and the vehicle's driving track in the road surface grid. The final weight corresponding to the road surface grid is obtained by combining the quantity reliability and the distance reliability. Among them, the more the number of point cloud data, the higher the quantity reliability and the higher the weight. The smaller the distance between the road surface grid and the vehicle's driving track, the higher the distance reliability and the higher the weight. For example, if the calculated quantity reliability is α and the distance reliability is β, the weight can be α×β.

[0072] In some other optional implementations of this embodiment, the edge constraint is determined based on the following steps: for each road surface grid in the road surface grid set, determine the road surface plane corresponding to the road surface grid; determine the edge point pair that matches the road surface plane corresponding to the road surface grid; for any edge point in the edge point pair, replace the coordinates of the edge point with the merged coordinates; and determine the edge constraint based on the merged coordinates and the original coordinates of the unreplaced coordinates in the edge point pair.

[0073] In this implementation, when forming the pavement grid set as described above, the correspondence between the pavement plane and the pavement grid can be established in advance. The execution entity can determine the corresponding pavement plane for each pavement grid, and select edge point pairs whose point cloud coordinates are less than a threshold value at the intersection of the pavement plane and other pavement planes. Afterwards, the execution entity can replace the coordinates of any edge point in the edge point pair with the merged coordinates. The merged coordinates can be the coordinates of the pavement grid vertex that is closest to the edge point. For the merged coordinates obtained by replacing the coordinates in the edge point pair, an edge constraint can be constructed based on the fact that the height value between the merged coordinate and the original coordinate of the other non-replaced coordinate is equal or approximately equal.

[0074] In some other optional implementations of this embodiment, the plane smoothness constraint is determined based on the following steps: for each road surface grid in the road surface grid set, determine the forward vertex and backward vertex of each vertex of the road surface grid along the vehicle's travel direction; and determine the plane smoothness constraint based on the forward vertex and backward vertex of each vertex of each road surface grid.

[0075] In this implementation, for each road surface grid in the road surface grid set, the execution entity can calculate the corresponding plane smoothness constraint according to the following formula:

[0076]

[0077] Among them, L smooth refers to the plane smooth constraint, i refers to the vertex of the road mesh, F i Refers to the forward vertex of the road mesh along the vehicle's travel direction, B i Refers to the vertex of the road mesh along the vehicle's travel direction, z i Refers to the height value of the vertex of the road surface grid, Refers to the height value of the forward vertex of the road mesh along the direction of vehicle travel. Refers to the height value of the vertex of the road surface grid along the vehicle's travel direction.

[0078] Step 410 : Optimize each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid.

[0079] In this embodiment, the preset optimization function is preferably determined by the above-mentioned plane fitting constraint, edge connection constraint, and plane smoothness constraint, and can be specifically determined by the following formula:

[0080] L total =L fit +ω1Lsmooth +ω2L connection

[0081] Among them, L total Refers to the preset optimization function, L fit refers to the plane fitting constraint, L smooth Refers to the plane smooth constraint, L connection refers to the edge constraint, and ω1 and ω2 refer to the parameters that are continuously updated during the optimization process.

[0082] In this embodiment, the execution entity may use the above-mentioned optimization function to optimize the initial plane equation corresponding to each road surface grid to obtain the optimized grid equation corresponding to each road surface grid.

[0083] Step 411: Obtain a two-dimensional lane line coordinate set.

[0084] In this embodiment, the execution entity can obtain a two-dimensional lane line coordinate set from a road image. The road image can be the road image mentioned above as the basis for generating road surface point cloud data, or it can be a road image uploaded by other electronic devices. This embodiment does not limit this.

[0085] Step 412 : For each two-dimensional lane line coordinate in the two-dimensional lane line coordinate set, determine the three-dimensional coordinate corresponding to the two-dimensional lane line coordinate based on the two-dimensional lane line coordinate and the grid equation corresponding to each road surface grid.

[0086] In this embodiment, the execution entity may first determine each two-dimensional lane segment based on each two-dimensional lane line coordinate in the two-dimensional lane line coordinate set. Then, for each two-dimensional lane segment, the existing perspective projection transformation is used to project the two-dimensional lane segment to obtain a three-dimensional lane segment. Here, each three-dimensional coordinate in a three-dimensional lane segment is the three-dimensional coordinate corresponding to each two-dimensional lane line coordinate in the two-dimensional lane segment. The execution entity may then determine the center point coordinates of each three-dimensional lane segment, and based on each center point coordinate and the grid equation corresponding to each road surface grid, determine the road surface grid corresponding to each three-dimensional lane segment. Then, using the grid equation corresponding to the road surface grid, the three-dimensional coordinates of each three-dimensional lane segment are calibrated to obtain more accurate three-dimensional coordinates.

[0087] Step 413: Generate three-dimensional lane lines based on each three-dimensional coordinate.

[0088] In this embodiment, the execution entity may first connect each three-dimensional lane line segment based on each three-dimensional coordinate, and then use the existing lane line series connection algorithm to connect each three-dimensional lane line segment in series to obtain a three-dimensional lane line.

[0089] The method for generating map lane lines provided by the above-mentioned embodiments of the present disclosure can also first divide a larger road surface plane based on road point cloud data, and then determine a corresponding road surface grid for each road surface plane. This method of dividing the resulting road surface grid sets can improve the accuracy of the divided road surface grids. Furthermore, an optimization function can be determined based on plane constraints, edge constraints, and plane smoothness constraints, so that the optimization of the road surface grid takes into account factors such as road surface grid weights, height value fitting errors, smoothness of road surface intersections, and smoothness of vertices along the vehicle's travel direction, thereby improving the accuracy of the optimized road surface grid. Furthermore, during point cloud preprocessing, the density of point cloud coordinates and the distance between the point cloud coordinates and the vehicle's travel trajectory can be used to determine point cloud hole-filling coordinates, thereby improving the hole-filling effect. Furthermore, the grid equation corresponding to each road surface grid is used to determine the three-dimensional coordinates corresponding to the two-dimensional lane line coordinates to generate three-dimensional lane lines, thereby obtaining high-precision lane lines. This method of generating lane lines can generate high-precision lane lines without the need for lidar, reducing hardware costs. It can also be used to acquire road images in crowdsourcing scenarios, thus expanding its applicability.

[0090] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for generating map lane lines. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to electronic devices such as servers.

[0091] like Figure 5 As shown, the device 500 for generating map lane lines in this embodiment includes: a data acquisition unit 501, a set determination unit 502, an equation determination unit 503 and a lane line generation unit 504.

[0092] The data acquisition unit 501 is configured to acquire road surface point cloud data.

[0093] The set determining unit 502 is configured to determine a road surface grid set based on the road surface point cloud data.

[0094] The equation determining unit 503 is configured to optimize each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid.

[0095] The lane line generating unit 504 is configured to generate lane lines based on the grid equations corresponding to the road surface grids.

[0096] In some optional implementations of this embodiment, the set determination unit 502 is further configured to: determine each road surface plane based on the road surface point cloud data; for each road surface plane, determine the road surface grid corresponding to the road surface plane; and determine the road surface grid set based on the road surface grids corresponding to each road surface plane.

[0097] In some optional implementations of this embodiment, it further includes: an optimization function determination unit configured to generate an optimization function based on at least one of the following constraints: a plane fitting constraint, an edge connection constraint, and a plane smoothness constraint.

[0098] In some optional implementations of this embodiment, the optimization function determination unit is further configured to: for each road surface grid in the road surface grid set, determine the weight corresponding to the road surface grid and the height value fitting error corresponding to the road surface grid; and perform weighted summation of the weights and height value fitting errors corresponding to each road surface grid to obtain a plane fitting constraint.

[0099] In some optional implementations of this embodiment, the optimization function determination unit is further configured to: for each road surface grid in the road surface grid set, determine the number of point cloud data in the road surface grid and the distance between the road surface grid and the vehicle's driving trajectory; and determine the weight corresponding to the road surface grid based on the number and distance of the point cloud data of the road surface grid.

[0100] In some optional implementations of this embodiment, the optimization function determination unit is further configured to: for each road surface grid in the road surface grid set, determine the road surface plane corresponding to the road surface grid; determine the edge point pair that matches the road surface plane corresponding to the road surface grid; for any edge point in the edge point pair, replace the coordinates of the edge point with the merged coordinates; and determine the edge constraint based on the merged coordinates and the original coordinates of the unreplaced coordinates in the edge point pair.

[0101] In some optional implementations of this embodiment, the optimization function determination unit is further configured to: for each road surface grid in the road surface grid set, determine the forward vertex and backward vertex of each vertex of the road surface grid along the vehicle's driving direction; and determine the plane smoothness constraint based on the forward vertex and backward vertex of each vertex of each road surface grid.

[0102] In some optional implementations of this embodiment, it also includes: a hole filling preprocessing unit, which is configured to determine the quality score of each point cloud coordinate in the road surface point cloud data based on the distance between the point cloud coordinate and the vehicle driving trajectory and the density corresponding to the point cloud coordinate; determine the target point cloud coordinate from the road surface point cloud data based on the quality score of each point cloud coordinate; determine the point cloud hole filling coordinate based on the height value of the target point cloud coordinate; and update the road surface point cloud data based on the point cloud hole filling coordinate.

[0103] In some optional implementations of this embodiment, the hole filling preprocessing unit is further configured to: determine the quality score information and neighborhood height information corresponding to the point cloud hole filling coordinates; determine the target point cloud hole filling coordinates from the point cloud hole filling coordinates based on the quality score information and neighborhood height information; and update the road surface point cloud data based on the target point cloud hole filling coordinates.

[0104] In some optional implementations of this embodiment, the lane line generation unit 504 is further configured to: obtain a two-dimensional lane line coordinate set; for each two-dimensional lane line coordinate in the two-dimensional lane line coordinate set, determine the three-dimensional coordinate corresponding to the two-dimensional lane line coordinate based on the two-dimensional lane line coordinate and the grid equation corresponding to each road surface grid; and generate a three-dimensional lane line based on each three-dimensional coordinate.

[0105] It should be understood that the units 501 to 504 described in the apparatus 500 for generating map lane lines are respectively the same as those in the reference Figure 2 Therefore, the operations and features described above for the method for generating lane lines are also applicable to the device 500 and the units contained therein, and will not be repeated here.

[0106] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of information such as road point cloud data are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0107] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0108] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0110] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0111] Computing unit 601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 601 performs the various methods and processes described above, such as the method for generating map lane lines. For example, in some embodiments, the method for generating map lane lines can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the method for generating map lane lines described above can be performed. Alternatively, in other embodiments, computing unit 601 can be configured to perform the method for generating map lane lines via any other suitable means (e.g., via firmware).

[0112] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0116] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0117] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0118] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0119] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for generating map lane lines, comprising: Obtain road point cloud data; For each point cloud coordinate in the road surface point cloud data, a quality score of the point cloud coordinate is determined based on the distance between the point cloud coordinate and the vehicle's driving trajectory and the density corresponding to the point cloud coordinate; based on the quality score of each point cloud coordinate, a target point cloud coordinate is determined from the road surface point cloud data; and based on the height value of the target point cloud coordinate, a point cloud hole filling coordinate is determined; Based on the point cloud hole filling coordinates, updating the road surface point cloud data; Determine a road surface grid set based on the updated road surface point cloud data; Based on a preset optimization function, each road surface grid in the road surface grid set is optimized to obtain a grid equation corresponding to each road surface grid, wherein the optimization function is constructed based on internal features of the road surface grid and external features of the road surface grid, the internal features of the road surface grid including the density of point cloud data in the road surface grid and the distance between the point cloud data and the driving trajectory, and the external features of the road surface grid including smooth features corresponding to intersection edges of the road surface grid and smooth features of vertices of the road surface grid along the vehicle's driving direction; Lane lines are generated based on the grid equations corresponding to the road surface grids.

2. The method according to claim 1, wherein Based on the updated road surface point cloud data, determine the road surface grid set, including: Determine each road surface plane based on the updated road surface point cloud data; For each road surface plane, determine the road surface grid corresponding to the road surface plane; The road surface grid set is determined based on the road surface grids corresponding to each road surface plane.

3. The method according to claim 1, further comprising: The optimization function is generated based on at least one of the following constraints: a plane fitting constraint, an edge constraint, and a plane smoothness constraint.

4. The method according to claim 3, wherein: The plane fitting constraint is determined based on the following steps: For each road surface grid in the road surface grid set, determining a weight corresponding to the road surface grid and a height value fitting error corresponding to the road surface grid; The plane fitting constraint is obtained by performing weighted summation on the weights and height value fitting errors corresponding to each road surface grid.

5. The method according to claim 4, wherein The step of determining a weight corresponding to each road surface grid in the road surface grid set includes: For each road surface grid in the road surface grid set, determining the amount of point cloud data in the road surface grid and the distance between the road surface grid and the vehicle driving trajectory; A weight corresponding to the road surface grid is determined based on the quantity of the point cloud data and the distance of the road surface grid.

6. The method according to claim 3, wherein: The edge constraints are determined based on the following steps: For each road surface grid in the road surface grid set, determining a road surface plane corresponding to the road surface grid; Determining edge point pairs that match the road surface plane corresponding to the road surface grid; For any edge point in the edge point pair, replacing the coordinates of the edge point with the merged coordinates; The edge constraint is determined based on the merged coordinates and the original coordinates of the unreplaced coordinates in the edge point pair.

7. The method according to claim 3, wherein: The plane smoothness constraint is determined based on the following steps: For each road surface grid in the road surface grid set, determining a forward vertex and a backward vertex of each vertex of the road surface grid along a vehicle travel direction; The plane smoothness constraint is determined based on the forward vertex and the backward vertex of each vertex of each road surface grid.

8. The method according to claim 1, wherein The updating of the road surface point cloud data based on the point cloud hole filling coordinates includes: Determining quality score information and neighborhood height information corresponding to the point cloud hole filling coordinates; Determining target point cloud hole filling coordinates from the point cloud hole filling coordinates based on the quality score information and the neighborhood height information; The road surface point cloud data is updated based on the target point cloud hole filling coordinates.

9. The method according to claim 1, wherein The generating of lane lines based on the grid equations corresponding to the road surface grids includes: Get the two-dimensional lane line coordinate set; For each two-dimensional lane line coordinate in the two-dimensional lane line coordinate set, determining a three-dimensional coordinate corresponding to the two-dimensional lane line coordinate based on the two-dimensional lane line coordinate and a grid equation corresponding to each road surface grid; Generate 3D lane lines based on each 3D coordinate.

10. A device for generating lane lines on a map, comprising: a data acquisition unit configured to acquire road surface point cloud data; The hole filling preprocessing unit is configured to determine, for each point cloud coordinate in the road surface point cloud data, a quality score of the point cloud coordinate based on a distance between the point cloud coordinate and a vehicle driving trajectory and a density corresponding to the point cloud coordinate; determine a target point cloud coordinate from the road surface point cloud data based on the quality score of each point cloud coordinate; and determine a point cloud hole filling coordinate based on a height value of the target point cloud coordinate; Based on the point cloud hole filling coordinates, updating the road surface point cloud data; a set determining unit configured to determine a road surface grid set based on the updated road surface point cloud data; an equation determination unit configured to optimize each road surface grid in the road surface grid set based on a preset optimization function to obtain a grid equation corresponding to each road surface grid, wherein the optimization function is constructed based on internal features of the road surface grid and external features of the road surface grid, the internal features of the road surface grid including the density of point cloud data in the road surface grid and the distance between the point cloud data and the driving trajectory, and the external features of the road surface grid including smooth features corresponding to intersection edges of the road surface grid and smooth features of vertices of the road surface grid along the vehicle's travel direction; The lane line generating unit is configured to generate lane lines based on the grid equations corresponding to the road surface grids.

11. The device according to claim 10, wherein The set determination unit is further configured to: Determine each road surface plane based on the updated road surface point cloud data; For each road surface plane, determine the road surface grid corresponding to the road surface plane; The road surface grid set is determined based on the road surface grids corresponding to each road surface plane.

12. The apparatus according to claim 10, further comprising: The optimization function determination unit is configured to generate the optimization function based on at least one of the following constraints: a plane fitting constraint, an edge connection constraint, and a plane smoothness constraint.

13. The device according to claim 12, wherein The optimization function determination unit is further configured to: For each road surface grid in the road surface grid set, determining a weight corresponding to the road surface grid and a height value fitting error corresponding to the road surface grid; The plane fitting constraint is obtained by performing weighted summation on the weights and height value fitting errors corresponding to each road surface grid.

14. The device according to claim 13, wherein The optimization function determination unit is further configured to: For each road surface grid in the road surface grid set, determining the amount of point cloud data in the road surface grid and the distance between the road surface grid and the vehicle driving trajectory; A weight corresponding to the road surface grid is determined based on the quantity of the point cloud data and the distance of the road surface grid.

15. The device according to claim 12, wherein The optimization function determination unit is further configured to: For each road surface grid in the road surface grid set, determining a road surface plane corresponding to the road surface grid; Determining edge point pairs that match the road surface plane corresponding to the road surface grid; For any edge point in the edge point pair, replacing the coordinates of the edge point with the merged coordinates; The edge constraint is determined based on the merged coordinates and the original coordinates of the unreplaced coordinates in the edge point pair.

16. The device according to claim 12, wherein The optimization function determination unit is further configured to: For each road surface grid in the road surface grid set, determining a forward vertex and a backward vertex of each vertex of the road surface grid along a vehicle travel direction; The plane smoothness constraint is determined based on the forward vertex and the backward vertex of each vertex of each road surface grid.

17. The device according to claim 10, wherein The hole filling preprocessing unit is further configured to: Determining quality score information and neighborhood height information corresponding to the point cloud hole filling coordinates; Determining target point cloud hole filling coordinates from the point cloud hole filling coordinates based on the quality score information and the neighborhood height information; The road surface point cloud data is updated based on the target point cloud hole filling coordinates.

18. The device according to claim 10, wherein The lane line generating unit is further configured to: Get the two-dimensional lane line coordinate set; For each two-dimensional lane line coordinate in the two-dimensional lane line coordinate set, determining a three-dimensional coordinate corresponding to the two-dimensional lane line coordinate based on the two-dimensional lane line coordinate and a grid equation corresponding to each road surface grid; Generate 3D lane lines based on each 3D coordinate.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images