Lane line generation method, device, and medium

Abstract

The application discloses a lane line generation method, device and medium, the method comprises the following steps: acquiring a plurality of point cloud data of a to-be-labeled road surface and two-dimensional road images of the to-be-labeled road surface under different visual angles; determining a reference lane line of the to-be-labeled road surface according to the two-dimensional road images under the different visual angles; the reference lane line contains a plurality of labeling points; for each labeling point in the reference lane line, target point cloud data matched with the labeling point is determined from the plurality of point cloud data; and based on the target point cloud data corresponding to all the labeling points, a lane line of the to-be-labeled road surface is generated, which can improve the accuracy of the lane line.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of intelligent driving, and particularly relates to a lane line generation method, device and medium. BACKGROUND

[0002] With the development of the automobile industry, automatic driving and auxiliary driving have become a hot development direction, and lane line generation is an important link. A vehicle-mounted device can generate lane lines in real time to provide guidance for vehicle driving. For example, a driver can drive a vehicle according to lane lines generated by a vehicle-mounted device, and in a driverless scenario, automatic driving of a vehicle can also be realized according to lane lines generated by a vehicle-mounted device.

[0003] Currently, 3D point cloud data of a road surface can be obtained, and the 3D point cloud data is labeled by an artificial manner to obtain coordinates of lane lines. The lane line coordinates can also be stored in a map database, and the lane lines can be displayed based on a driving route when a user uses it later.

[0004] However, it is very difficult to manually label 3D point cloud data, and there is a large error in the labeling result. Currently, lane line generation still has the problem of low accuracy. SUMMARY

[0005] In view of the above defects or deficiencies in the prior art, it is desirable to provide a lane line generation method, device and medium, which can improve the accuracy of lane line generation.

[0006] In a first aspect, a lane line generation method is provided, comprising:

[0007] obtaining a plurality of point cloud data of a to-be-labeled road surface, and a two-dimensional road image of the to-be-labeled road surface under different perspectives;

[0008] determining a reference lane line of the to-be-labeled road surface according to the two-dimensional road image under different perspectives; the reference lane line comprises a plurality of labeling points;

[0009] for each labeling point in the reference lane line, determining target point cloud data matched with the labeling point from the plurality of point cloud data;

[0010] generating a lane line of the to-be-labeled road surface based on the target point cloud data corresponding to all the labeling points.

[0011] In a second aspect, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the method of the first aspect is implemented.

[0012] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and when the program is executed by a processor, the method of the first aspect is implemented.

[0013] Fourthly, a computer program product is provided, which includes instructions that, when executed by a processor, implement the method described in the first aspect.

[0014] The lane line generation method, device, and medium provided in this application acquire point cloud data of the road surface and two-dimensional road images of the road surface from multiple perspectives, and can generate reference lane lines based on the two-dimensional road images. Based on each annotation point included in the reference lane line, the point cloud data matching the annotation point can be determined from the point cloud data, and the lane line is generated based on the point cloud data matching all annotation points. While generating lane lines using point cloud data, the method provided in this application can also filter and fuse point cloud coordinates based on two-dimensional road images of the road surface from multiple perspectives, combining data from both modalities to generate lane lines. This retains both the advantage of the large amount of information in image data and the strong anti-interference characteristic of point cloud data, effectively improving the accuracy of lane line generation. Furthermore, compared to the technical solution of manually annotating 3D point cloud data to obtain lane lines, this greatly reduces the implementation difficulty and improves the accuracy of the annotation results.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0017] Figure 1 This is a schematic diagram of model training provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of the implementation environment of an embodiment of this application;

[0019] Figure 3 A schematic flowchart illustrating the lane line generation method provided in this application embodiment;

[0020] Figure 4 A schematic diagram of the annotation points provided in the embodiments of this application;

[0021] Figure 5 This is a schematic diagram of annotation point matching provided for an embodiment of this application;

[0022] Figure 6 A schematic diagram of the annotation points provided in the embodiments of this application;

[0023] Figure 7 This is a schematic diagram of annotation point matching provided for an embodiment of this application;

[0024] Figure 8 A coordinate fusion schematic diagram provided for an embodiment of the present application;

[0025] Figure 9 Another flow schematic diagram of a lane line generation method provided for an embodiment of the present application;

[0026] Figure 10 A structure schematic diagram of a lane line generation device provided for an embodiment of the present application;

[0027] Figure 11 A structure schematic diagram of a computer device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0028] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0029] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0030] In recent years, important progress has been made in the research of computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence. Artificial intelligence (AI) is a new science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline that involves chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks and many other technology categories. Computer vision, as an important branch of artificial intelligence, is specifically to enable machines to recognize the world. Computer vision technology generally includes face recognition, liveness detection, fingerprint recognition and anti-forgery verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, three-dimensional reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning, and other technologies. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security control, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile imaging, cloud services, smart home, wearable devices, driverless vehicles, autonomous driving, intelligent medical care, face payment, face unlocking, fingerprint unlocking, face and certificate verification, smart screen, smart TV, camera, mobile Internet, network live broadcast, beauty, makeup, medical cosmetology, intelligent temperature measurement, and other fields.

[0031] Firstly, the terms involved in the present application are explained and described.

[0032] (1) Lane line

[0033] Lane line, also known as guide lane line, is a lane marking that guides the direction. It can be used to indicate the driving direction of the vehicle. In the field of intelligent driving, lane lines can be marked on the road surface image displayed on the vehicle display screen to prompt the driving direction.

[0034] (2) Point cloud coordinates

[0035] Point cloud data refers to the data obtained by scanning a target object with signals emitted by a sensor, which can represent the three-dimensional shape of the target object. Each point cloud data can be a coordinate in a three-dimensional coordinate system. Therefore, point cloud data can include point cloud coordinates, which are three-dimensional coordinates.

[0036] (3) Model training: input a large amount of data with known classification results into an initial model, so that the algorithm inside the model learns the classification rules of these data, so that the trained model can classify unknown data.

[0037] The data of a large number of known classification results input into the model can be referred to as a training sample of the model. Taking supervised machine learning as an example, the true classification result of the training sample can be used as a label of the training sample. In the model training process, feature extraction is performed on the training sample, the extracted features are input into the classification function of the model, and the output result of the model can be obtained. Further, the loss between the output result of the model and the label of the training sample can be determined according to a loss function, and the model is iteratively trained according to the loss until the output result of the model approaches the label of the training sample. That is, the model has the ability to accurately classify data. In addition, the features obtained by the model through feature extraction on the model input can be referred to as training features of the model, which are usually key features for identifying and classifying the model input. For example, when identifying a face in a picture, the key features to be referred to can be "face contour", "eyes", "mouth", etc. The model can extract face contour features, eye features, mouth features, etc. from the input picture, and the classification function of the model can output the identification result of the model based on these features.

[0038] At present, lane generation is mainly performed by means of 3D point cloud data. First, the point cloud coordinates of the road surface can be collected, and the lane lines of the road surface can also be obtained by manually labeling the point cloud coordinates. This method has the problems of large manual labeling error and low labeling result accuracy. In order to improve the labeling efficiency, a machine learning model can also be used to realize automatic labeling of lane lines to provide users with auxiliary functions such as road information and driving direction. In the training process of the above machine learning model, 3D point cloud data of the road surface is needed, and then lane lines are labeled by manual labeling based on the 3D point cloud data to obtain training samples and labels (i.e., manually labeled lane lines). The training process is as follows: Figure 1 The training sample is input into the neural network model, and the neural network model performs feature extraction and labeling on the training sample, so as to output a lane line prediction result. The neural network model can also be iteratively trained according to the loss between the lane line predicted by the model and the lane line labeled by the manual labeling until the lane line predicted by the model approaches the lane line labeled by the manual labeling, and the ability to label lane lines based on point cloud data is learned.

[0039] Since it is very difficult to manually label 3D point cloud data, and the labeling result also has a large error. Therefore, the accuracy of the training sample is not high, thereby limiting the performance of the machine learning model, for example, the accuracy of the lane line generation result output by the machine learning model is not high.

[0040] Based on this, the application provides a lane line generation method, device and storage medium, which can obtain the coordinates of the lane line based on the point cloud coordinates and two-dimensional images of the road, generate the lane line based on the coordinates, and save the cost without manual annotation of 3D point cloud data. In addition, compared with the scheme of generating the lane line only based on the point cloud data, the coordinates of the lane line are obtained based on the point cloud coordinates and two-dimensional images, which can improve the accuracy of the lane line.

[0041] Figure 2 The intelligent driving system to which the method provided by the application is applicable. Referring to Figure 2 , the system includes a client 10 and a background server 20. In the field of intelligent driving, the client 10 is arranged on a vehicle and can be an application installed on a vehicle-mounted device or a vehicle-mounted device, or an application installed on a personal device (for example, a mobile phone) of a user. The client 10 can obtain road information related to the vehicle, such as congestion information, speed information, lane lines and the like, and output the road information to the user as auxiliary information for driving. Alternatively, the client 10 can also control other devices (for example, a line control system) of the vehicle based on the road information to realize automatic driving of the vehicle.

[0042] Among them, the server 20 supports the background implementation of various functions provided by the client 10. For example, the server 20 can send congestion information to the client 10 so that the client 10 prompts the received congestion information.

[0043] The above-mentioned background server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (content delivery network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0044] The client 202 can be a terminal or an application installed on a terminal. The terminal can be a device including but not limited to a personal computer, a platform computer, a smart phone, a vehicle-mounted terminal and the like, and the embodiments of the application do not limit this.

[0045] The embodiments of the application provide a lane line generation method, and the execution subject of the method can be the client 10 described above or the background server 20. Referring to Figure 3 , the method includes the following steps:

[0046] 301, obtaining a plurality of point cloud data of a to-be-labeled road surface and a two-dimensional road surface image of the to-be-labeled road surface under different viewing angles;

[0047] 302、determining reference lane lines of the road to be labeled according to the plurality of two-dimensional road images; the reference lane lines include a plurality of labeling points;

[0048] 303、for each labeling point in the reference lane lines, determining target point cloud data matching the labeling point from the plurality of point cloud data;

[0049] 304、generating lane lines of the road to be labeled based on the target point cloud data corresponding to all labeling points of the reference lane lines.

[0050] It should be noted that the road can be pre-collected information, and the collected information can be stored in a map database. In the process of driving, the positioning result of the client (for example, an application installed on the vehicle equipment) can be obtained, and the corresponding road information can be matched in the map database based on the positioning result of the vehicle, and intelligent driving services such as assisted driving or automatic driving can be provided based on the obtained road information. The road information can include lane line information. The method provided in the embodiments of the present application can generate lane lines with high precision based on the point cloud data and two-dimensional road images of the road. The method can be applied in the stage of creating a map database, that is, the lane lines are generated by using the method provided in the embodiments of the present application, and the information of the lane lines can be stored in the map database. In the actual driving scene, the client can determine the lane lines of the current driving road surface according to the lane line information in the map database. Alternatively, the present application can also be applied in real-time driving, that is, the method provided in the embodiments of the present application is used to generate lane lines for the current driving road surface in the process of real-time driving.

[0051] In a possible implementation of the above step S301, the road surface can be scanned by a sensor to obtain point cloud data of the road surface. The point cloud data can include a plurality of point cloud coordinates of the road to be labeled. In addition, the road to be labeled can be imaged to obtain two-dimensional road images of the road to be labeled at different angles.

[0052] For example, when imaging the road to be labeled, a plurality of image acquisition devices can be arranged at different positions (different positions of the vehicle). The angles of view of the image acquisition devices are different. After the plurality of image acquisition devices capture the road to be labeled, a plurality of two-dimensional road images of the road to be labeled can be obtained, each two-dimensional road image corresponding to a different angle of view. Alternatively, when imaging the road to be labeled, only one image acquisition device can be arranged, and the image acquisition device can be controlled to switch the angle of view. Then, two-dimensional road images of the road to be labeled at different angles of view can be obtained.

[0053] In the embodiments of the present application, the plurality of two-dimensional road images can be fused to generate the reference lane line based on the fused image, or the reference lane line can be generated based on each two-dimensional road image. Specifically, the following two methods are included.

[0054] Firstly, the plurality of two-dimensional road images are fused to obtain a fused image. The reference lane line is further generated based on the fused image. The lane line generation processing can be performed on the fused image to obtain the reference lane line of the road surface to be labeled.

[0055] In a specific implementation, the plurality of two-dimensional road images can be projected onto the same plane by using the internal and external parameters of the cameras with multiple perspectives to obtain the fused image.

[0056] Secondly, the lane line generation processing is performed on each two-dimensional road image respectively to obtain the reference lane line in each two-dimensional road image.

[0057] In a possible implementation, the lane line generation processing mentioned above can be performed on the two-dimensional road image of the road surface by manual labeling to obtain the reference lane line in each two-dimensional road image.

[0058] In another implementation, the lane line generation processing can also be implemented by a machine learning model. For example, the two-dimensional road image is input into a lane line detection model to obtain the reference lane line in the two-dimensional road image. Alternatively, the fused image can be input into the lane line detection model to obtain the reference lane line in the fused image. The lane line detection model is obtained by training a plurality of historical two-dimensional road images and the lane lines labeled in the historical two-dimensional road images.

[0059] In a possible implementation, after the two-dimensional road image of the road surface to be labeled is input into the lane line detection model, the feature extraction processing can be performed on the two-dimensional road image by using a preset feature extraction rule, and the lane line generation result in the two-dimensional road image is obtained based on the extracted features. The lane line detection model is obtained by training sample data to learn a network structure model with lane line generation capability. The input of the lane line detection model is the two-dimensional road image, and the output is the labeled lane line in the two-dimensional road image, and the lane line detection model has the ability of image recognition on the input two-dimensional road image, and is a neural network model capable of predicting the lane line in the input two-dimensional road image. The lane line detection model can include a plurality of network structures. Different layers of network structures process the data input thereto differently, and transmit the output results to the next network layer until the last network layer is processed to obtain the lane line generation result of the input image.

[0060] Optionally, the feature extraction rule refers to a feature extraction strategy pre-set according to an actual application scenario, or can be a general feature extraction algorithm, etc. As a possible implementation manner, the feature extraction algorithm can be a Scale-Invariant Feature Transform (SIFT) algorithm, or a Speeded Up Robust Features (SURF) algorithm, or an Oriented FAST and Rotated BRIEF (ORB) feature detection algorithm, etc. to obtain the lane line generation result of the input image.

[0061] It should be noted that the above implementations of feature extraction on the to-be-recognized image to obtain the reference lane line are merely examples, and the embodiments of the present application are not limited thereto.

[0062] In a possible implementation manner, the embodiments of the present application do not limit the specific implementation manner of the lane line marking point, which can be obtained by clustering or screening. For example, the lane line includes a plurality of coordinate points, the coordinate points included in the lane line can be clustered based on coordinate values, and the clustered coordinates (i.e. the cluster center) are taken as the marking point, so that a plurality of marking points of the lane line can be obtained. Alternatively, the plurality of coordinate points included in the lane line can be screened according to the curvature of the line segment of the lane line, and a plurality of coordinate points are obtained, and the line segment formed by connecting the screened coordinate points is the same as or similar to the lane line.

[0063] It should be noted that the target point cloud data for determining the marking point is the point cloud data with the highest similarity to the marking point among the plurality of point cloud data of the to-be-labeled road surface. Although the point cloud data and the marking point are data in different modalities, the point cloud data often contains height information of the scanned object, while the marking point is a point on a two-dimensional image and cannot represent the height information of the scanned object, but for the same scanned object, the point cloud data and the marking point on the two-dimensional image have similarity. When the similarity between the marking point and a certain point cloud data is large, it can be considered that the marking point and the point cloud data are imaging data of the same object, and there is a matching relationship between them.

[0064] In a possible implementation manner, the similarity between the marking point and each of the plurality of point cloud data can be determined, and the point cloud data with the largest similarity is taken as the target point cloud data matched with the marking point.

[0065] Exemplarily, the point cloud data can be dimensionally converted to obtain data of the same dimension as the labeled point. The similarity between the labeled point and the point cloud data can be calculated based on the converted point cloud data and the labeled point. For example, the point cloud data is three-dimensional data (x1, y1, z1), and the labeled point is two-dimensional data (x2, y2). The three-dimensional data (x1, y1, z1) can be converted into two-dimensional data (x0, y0), and the similarity between the labeled point and the point cloud data can be determined based on the two-dimensional data (x0, y0) and the two-dimensional data (x2, y2).

[0066] In a possible implementation, the similarity between the coordinates can be the Euclidean distance between the coordinates. The similarity between the coordinates is negatively correlated with the distance between the coordinates. The smaller the distance between the coordinates, the higher the similarity between the coordinates. For example, the point cloud data of the road to be labeled is (x1, y1, z1) and (x2, y2, z2), and the dimensionally converted data is (x3, y3) and (x4, y4). The labeled point is (w0, v0). The distances between (w0, v0) and (x3, y3) and (x4, y4) are calculated respectively. Assuming that the distance between (x4, y4) and (w0, v0) is relatively small, the point cloud data (x2, y2, z2) corresponding to (x4, y4) is taken as the target point cloud data of the labeled point.

[0067] In a possible implementation, the target point cloud data corresponding to all the labeled points under each view angle can be fused to obtain the lane line of the road to be labeled. The granularity of the fusion processing is the labeled point, that is, the target point cloud data corresponding to a single labeled point is fused, and the target point cloud data of different labeled points is not fused. The result of the fusion processing of a single labeled point is related to the characteristics of the target point cloud data corresponding to the labeled point.

[0068] Exemplarily, all the target point cloud data corresponding to each labeled point can be fused, and the lane line can be generated based on the fusion result of each labeled point.

[0069] In the method provided in the embodiments of the present application, after the point cloud data of the road surface and the two-dimensional road images of the road surface under multiple perspectives are obtained, the reference lane line can be generated based on the two-dimensional road images. Based on each labeled point contained in the reference lane line, the point cloud data matched with the labeled point can be determined from the point cloud data, and the lane line can be generated according to the point cloud data matched with all the labeled points. The method provided in the present application can generate the lane line by means of the point cloud data, and can also filter and fuse the point cloud coordinates based on the two-dimensional road images of the road surface under multiple perspectives, so as to generate the lane line in combination with the data of the two modalities. The method can not only retain the advantage of large amount of image data information, but also retain the characteristic of strong anti-interference of the point cloud data, and effectively improves the accuracy of lane line generation. In addition, compared with the technical solution of obtaining the lane line by manually labeling the 3D point cloud data, the implementation difficulty is greatly reduced, and the accuracy of the labeling result is also improved.

[0070] In another embodiment of the present application, the point cloud data of the road surface to be labeled includes three-dimensional coordinates, which can be mapped to a two-dimensional plane to obtain two-dimensional mapping coordinates. Then, the similarity between the point cloud data and the labeled point is determined based on the two-dimensional mapping coordinates of the point cloud data, that is, whether the point cloud data matches the labeled point, to determine the target point cloud data of the labeled point.

[0071] For example, the aforementioned "determining target point cloud data matched with the labeled point from multiple point cloud data" includes: converting each of the point cloud data into corresponding two-dimensional mapping coordinates, and determining the matching coordinates of the labeled point from the obtained multiple two-dimensional mapping coordinates; and taking the point cloud data corresponding to the matching coordinates in the multiple point cloud data as the target point cloud data matched with the labeled point.

[0072] In a specific implementation, the coordinate system of the sensor for collecting the point cloud coordinates can be a three-dimensional coordinate system, and therefore the point cloud coordinates in the point cloud data are often three-dimensional coordinates (which can also be referred to as 3D coordinates). A two-dimensional coordinate system can be selected, and the point cloud coordinates in the point cloud data are respectively mapped to the two-dimensional coordinate system to obtain the two-dimensional mapping coordinates corresponding to each point cloud data. After obtaining the two-dimensional mapping coordinates of the point cloud data, the two-dimensional coordinates matched with the labeled point, that is, the matching coordinates of the labeled point, can be determined from the multiple two-dimensional mapping coordinates.

[0073] In a possible implementation, the matching coordinates of the labeled point can be the two-dimensional coordinate with the highest similarity to the labeled point among the multiple two-dimensional mapping coordinates obtained by converting the multiple point cloud data. The similarity between the labeled point and the two-dimensional mapping coordinates is negatively related to the distance therebetween, and the smaller the distance between the labeled point and the two-dimensional mapping coordinates, the higher the similarity therebetween. For example, the distance between the labeled point and each two-dimensional mapping coordinate is calculated, and the two-dimensional mapping coordinate with the smallest distance is taken as the matching coordinates of the labeled point.

[0074] In another embodiment of the present application, the annotation points have matching coordinates in multiple perspectives. The two-dimensional road images in each perspective can be traversed to determine the matching coordinates of the annotation points in each perspective. For example, for each perspective, the original coordinates of the annotation points in the two-dimensional road image in the perspective are determined; further, the similarity of the original coordinates with each two-dimensional mapping coordinate is determined, and the two-dimensional mapping coordinate with the highest similarity with the original coordinates is taken as the matching coordinate of the annotation points in the perspective.

[0075] In a possible implementation, the multiple two-dimensional road images are fused, the reference lane lines are obtained based on the fused image, the reference lane lines in the fused image can be annotated to obtain multiple annotation points. Further, the original coordinates of the annotation points in the two-dimensional road images can be inversely calculated according to the annotation points, so as to determine the matching coordinates of the annotation points based on the similarity of the original coordinates and the above-mentioned two-dimensional mapping coordinates.

[0076] For example, the original coordinates of the annotation points in the fused image in each two-dimensional road image are first determined by the inverse calculation method; further, for each original coordinate corresponding to the annotation points, the similarity of the original coordinate with multiple two-dimensional mapping coordinates is calculated, and the two-dimensional mapping coordinate with the highest similarity with the original coordinate is taken as the matching coordinate of the annotation points. That is, one original coordinate of the annotation points in the two-dimensional road image in one perspective can be determined, the matching coordinate of the annotation points in one perspective can be determined according to the similarity between the original coordinate and each two-dimensional mapping coordinate, and further, the matching coordinates of the annotation points in each perspective can be obtained.

[0077] For example, the original coordinates of the annotation points in the fused image in each two-dimensional road image are first determined by the inverse calculation method; further, for each original coordinate corresponding to the annotation points, the similarity of the original coordinate with multiple two-dimensional mapping coordinates is calculated, and the two-dimensional mapping coordinate with the highest similarity with the original coordinate is taken as the matching coordinate of the annotation points. That is, one original coordinate of the annotation points in the two-dimensional road image in one perspective can be determined, the matching coordinate of the annotation points in one perspective can be determined according to the similarity between the original coordinate and each two-dimensional mapping coordinate, and further, the matching coordinates of the annotation points in each perspective can be obtained. Figure 4 For example, the two-dimensional road images of the road surface to be annotated in three perspectives are obtained, which are image 1, image 2 and image 3 respectively. The image 1, image 2 and image 3 are projected onto the same screen to obtain a fused image 4. The lane line generation processing is performed on the image 4 to obtain the reference lane lines. The reference lane lines include annotation points A1-A10.

[0078] Based on the projection relationship of image 1 and image 4, the labeled points A1-A10 are processed by inverse calculation to obtain the original coordinates a1, a2, a3…a10 corresponding to the labeled points A1-A10 in image 1 in turn; based on the projection relationship of image 2 and image 4, the labeled points A1-A10 are processed by inverse calculation to obtain the original coordinates b1, b2, b3…b10 corresponding to the labeled points A1-A10 in image 2 in turn; based on the projection relationship of image 3 and image 4, the labeled points A1-A10 are processed by inverse calculation to obtain the original coordinates c1, c2, c3…c10 corresponding to the labeled points A1-A10 in image 3 in turn. That is, the original coordinates of the labeled point A1 under different perspectives are a1, b1, c1, the original coordinates of the labeled point A2 under different perspectives are a2, b2, c2…, and the original coordinates of the labeled point A10 under different perspectives are a10, b10, c10.

[0079] Reference Figure 5 For the labeled point A1, the similarity of the original coordinate a1 and the plurality of two-dimensional mapping coordinates is calculated, and the two-dimensional mapping coordinate with the highest similarity is taken as the matching coordinate 1 of the labeled point A1 under one perspective. Similarly, the similarity of the original coordinates b1 and c1 and the plurality of two-dimensional mapping coordinates is calculated respectively, and the matching coordinate 2 and the matching coordinate 3 of the labeled point A1 under the other two perspectives can be determined. In this way, the matching coordinates of A1-A10 under three different perspectives can be determined.

[0080] In another possible implementation, the plurality of two-dimensional road images can be processed by lane line generation to obtain a plurality of reference lane lines. Each reference lane line is composed of a plurality of coordinate points, and the coordinate points can be clustered to obtain a plurality of labeled points. When the two-dimensional road images of the road surface to be labeled under a plurality of perspectives are obtained, the coordinate points included in the reference lane lines of each two-dimensional road image are clustered to obtain original labeled points (which can also be referred to as original coordinates). For the original labeled points in different two-dimensional road images, if the coordinate difference between the original labeled points is less than a preset threshold, it is considered to be the same labeled point.

[0081] For example, the two-dimensional road images of the road surface to be labeled under three perspectives are obtained, which are image 1, image 2 and image 3 as shown in the drawings. Figure 6 The coordinate points included in the reference lane lines in image 1 are clustered to obtain original labeled points a1, a2, a3…a10, the coordinate points included in the reference lane lines in image 2 are clustered to obtain original labeled points b1, b2, b3…b10, and the coordinate points included in the reference lane lines in image 3 are clustered to obtain original labeled points c1, c2, c3…c10.

[0082] The difference between the coordinate values of a1, b1, and c1 is less than a preset threshold, and it is considered as the same annotation point 1. The difference between the coordinate values of a2, b2, and c2 is less than a preset threshold, and it is considered as the same annotation point 2. In this way, the difference between the coordinate values of a10, b10, and c10 is less than a preset threshold, and it is considered as the same annotation point 10. That is, image 1, image 2, and image 3 correspond to 10 annotation points.

[0083] After determining the annotation points by clustering, the matching coordinates of the annotation points under the current view angle can also be determined according to the similarity between the original annotation points corresponding to the annotation points in a two-dimensional road image under a view angle and the two-dimensional mapping coordinates.

[0084] For example, referring to Figure 7 , the original annotation point of annotation point 1 in image 1 is (x1, y1), the original annotation point in image 2 is (x2, y2), and the original annotation point in two-dimensional road image 3 is (x3, y3). The coordinate with the highest similarity to (x1, y1) among the above-mentioned multiple two-dimensional mapping coordinates is (w1, z1), the coordinate with the highest similarity to (x2, y2) is (w2, z2), and the coordinate with the highest similarity to (x3, y3) is (w3, z3). Then, the two-dimensional mapping coordinates matched with annotation point 1 under each view angle are (w1, z1), (w2, z2), and (w3, z3). In this way, the matching coordinates of each annotation point under multiple view angles can be determined.

[0085] It should be noted that the similarity between the two-dimensional mapping coordinates and the annotation points is negatively related to the distance therebetween, and the closest two-dimensional mapping coordinates to the annotation points can be selected by calculating the distance between the two-dimensional mapping coordinates and the annotation points.

[0086] Based on Figure 4 , Figure 5 , after determining the matching coordinates of the annotation points, the target point cloud data of the annotation points can be determined based on the matching coordinates, and the point cloud data corresponding to the matching coordinates is taken as the target point cloud data of the annotation points. For example, referring to Figure 8The original coordinates of the marking point 1 are (x1, y1), (x2, y2) and (x3, y3) respectively calculated reversely from the three different perspectives. The matching coordinates corresponding to (x1, y1), (x2, y2) and (x3, y3) are (w1, z1), (w2, z2) and (w3, z3) respectively under the three different perspectives. The point cloud coordinates before conversion of (w1, z1) are (o1, p1, q1), the point cloud coordinates before conversion of (w2, z2) are (o2, p2, q2), and the point cloud coordinates before conversion of (w3, z3) are (o3, p3, q3), that is, all the point cloud coordinates corresponding to the marking point 1 are (o1, p1, q1), (o2, p2, q2) and (o3, p3, q3); similarly, all the point cloud coordinates corresponding to the marking points 2 to 10 can be determined.

[0087] Further, referring to Figure 8 The point cloud coordinates corresponding to the marking points 1 to 10 can be fused to obtain the fusion coordinates corresponding to the marking points 1 to 10. Further, the lane lines can be generated according to the fusion coordinates corresponding to the marking points 1 to 10.

[0088] In another embodiment of the present application, the two-dimensional coordinate system for converting the point cloud data can be the coordinate system of the image acquisition device. For example, the point cloud data can be converted into the imaging coordinates of the image acquisition device based on the intrinsic and extrinsic parameters of the image acquisition device; the imaging coordinates are two-dimensional coordinates.

[0089] The intrinsic parameter can be a camera intrinsic parameter, which is used to convert a certain coordinate from a camera coordinate system to a pixel coordinate system. The extrinsic parameter can be a camera extrinsic parameter, which is used to convert a certain coordinate from a world coordinate system to a camera coordinate system.

[0090] In a possible implementation manner, the sensor for collecting the point cloud coordinates and the image acquisition device can be used in association. That is, the sensor and the image acquisition device are used in association when the same object is scanned by using the sensor and the image acquisition device used in association. For example, when the lane lines of the road to be marked are determined, the laser sensor and the camera of the same vehicle scan the road to be marked.

[0091] Specifically, when the sensor and the image acquisition device are used in association, the imaging perspective of the image acquisition device and the imaging perspective of the sensor have an association relationship, and the plurality of point cloud data of the road to be marked can be converted into the imaging coordinates (two-dimensional mapping coordinates) of the image acquisition device based on the intrinsic and extrinsic parameters of the image acquisition device.

[0092] In a possible implementation manner, the point cloud coordinates (x w , y w , zw ) into imaging coordinates (u, v):

[0093]

[0094] wherein Zc is a normalization coefficient, K is the intrinsic parameter, and R, T are the extrinsic parameters.

[0095] In another embodiment of the present application, the point cloud data obtained by the sensor can also be pre-processed to improve the accuracy of the point cloud data. For example, the aforementioned obtaining of the plurality of point cloud data of the to-be-labeled road surface includes: first obtaining initial point cloud data obtained by the sensor scanning the to-be-labeled road surface, filtering out non-road surface point cloud data in the initial point cloud data to obtain candidate point cloud data; further, based on the candidate point cloud data, the to-be-labeled road surface is subjected to surface fitting processing to obtain the plurality of point cloud data of the to-be-labeled road surface.

[0096] It can be understood that the road surface height can be smooth without mutation, and therefore the height information of the point cloud coordinates in the point cloud data can be used to filter the non-road surface point cloud and remove the non-road surface point cloud data in the initial point cloud data. For example, assuming that the point cloud coordinates are (x, y, z), wherein x and y are position information in a two-dimensional plane, and z is height information, the point cloud coordinates with non-zero z coordinates can be filtered out to obtain candidate point cloud data.

[0097] After filtering the non-road surface point cloud coordinates, the point cloud coordinates become sparse, for example, the point cloud data at the road surface obstacles becomes hollow. The to-be-labeled road surface can also be subjected to surface fitting processing based on the candidate point cloud data to fill in these hollows. For example, the least square method can be used to parameterize the candidate point cloud data, and further, the results of the parameterization processing can be used for surface fitting to fill in the point cloud hollows.

[0098] In another embodiment of the present application, a specific implementation of generating a lane line based on the target point cloud data of the labeled points is also provided. For example, for each labeled point, the target point cloud data corresponding to the labeled point can be fused to obtain a fusion point cloud of the labeled point. Further, based on the fusion point clouds of all the labeled points, a lane line of the to-be-labeled road surface is generated.

[0099] For example, the fusion processing of the target point cloud data corresponding to one labeled point can be mean value processing of point cloud information of the same dimension to obtain data of the dimension in the fusion point cloud. For example, the coordinate values on the same coordinate axis can be subjected to mean value processing to obtain a fusion value corresponding to the coordinate axis. The fusion point cloud can include fusion values of various coordinate axes.

[0100] For example, for each coordinate axis of the point cloud coordinate system, the coordinate values of all point cloud coordinates corresponding to the labeled point on a coordinate axis are processed by mean value to obtain the fusion coordinate of the labeled point on the coordinate axis.

[0101] For example, the all point cloud coordinates corresponding to the labeled point 1 are (o1, p1, q1), (o2, p2, q2) and (o3, p3, q3), and the corresponding coordinate values of each coordinate value are processed by mean value to obtain the fusion coordinate [(o1+o2+o3) / 3, (p1+p2+p3) / 3, (q1+q2+q3)]. Further, the fusion coordinates of the labeled point 1 to the labeled point 10 can be connected to obtain the lane line.

[0102] The embodiment of the application further provides a lane line generation method, as shown in the method, the method comprises the following steps: Figure 9

[0103] S1, scanning the to-be-labeled road surface to obtain initial point cloud data of the to-be-labeled road surface and two-dimensional road images of the to-be-labeled road surface at different viewing angles;

[0104] Wherein, the vehicle-mounted device can control the sensor and the image acquisition device to scan the to-be-labeled road surface at the same time, the sensor can send the point cloud data of the to-be-labeled road surface to the vehicle-mounted device, and the image acquisition device can send the two-dimensional road images of the to-be-labeled road surface at different viewing angles to the vehicle-mounted device.

[0105] S2, filtering non-road surface point cloud data in the initial point cloud data to obtain candidate point cloud data;

[0106] S3, based on the candidate point cloud data, performing surface fitting to fill in point cloud holes to obtain point cloud data of the to-be-labeled road surface;

[0107] S4, based on the intrinsic parameters and extrinsic parameters of the image scanning device, converting the point cloud data of the to-be-labeled road surface into two-dimensional mapping coordinates;

[0108] S5, fusing a plurality of two-dimensional road images to generate a reference lane line based on the fused image;

[0109] Specifically, the fused image can be input into a pre-trained lane line detection model to obtain a predicted lane line output by the lane line detection model, which is the reference lane line of the to-be-labeled road surface.

[0110] S6, for each labeled point in the reference lane line, determining the original coordinates of the labeled point on the two-dimensional road image.

[0111] ​Specifically, different two-dimensional lane images correspond to one view angle, and based on the mapping relationship between the fusion image and the two-dimensional road image, the original coordinates of the labeling point in the two-dimensional road image of the view angle can be calculated. By analogy, the original coordinates of the labeling point corresponding to each view angle can be determined, that is, one labeling point corresponds to multiple original coordinates.

[0112] S7, for each labeling point, determining the matching coordinates of the labeling point under each view angle according to the original coordinates of the labeling point.

[0113] Specifically, for any view angle A, the distance between the original coordinates 1 corresponding to the labeling point under the view angle A and each two-dimensional mapping coordinates is calculated, and the two-dimensional mapping coordinates closest to the distance is taken as the matching coordinates of the labeling point under the view angle A.

[0114] Correspondingly, the matching coordinates of the labeling point under each view angle can be determined.

[0115] S8, for each labeling point, fusing all the matching coordinates of the labeling point under all the view angles to obtain the fusion coordinates corresponding to the labeling point.

[0116] Specifically, the fusion of the above matching coordinates is to perform mean value processing on the information of the same dimension to obtain the information of the dimension in the fusion coordinates.

[0117] S9, generating the lane line of the road to be labeled according to the fusion coordinates of all the labeling points.

[0118] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the lane line generation method described in the embodiment of the present application. For example, the steps of the method shown in the embodiment of the present application can be executed. Figure 3

[0119] The embodiment of the present application provides a computer program product, which contains instructions, and the instructions are executed by a processor to realize the steps of the method shown in the foregoing Figure 3

[0120] It should be noted that, although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the shown operations must be performed to achieve the desired results.

[0121] Figure 10 The block schematic diagram of the lane line generation device of one embodiment of the present application is shown in FIG. 10. Referring to Figure 10 The device includes an acquisition unit 1001, a labeling unit 1002, a matching unit 1003 and a generation unit 1004.

[0122] ​​The acquisition unit 1001 is configured to acquire a plurality of point cloud data of a to-be-labeled road surface and two-dimensional road images of the to-be-labeled road surface under different viewing angles;

[0123] The labeling unit 1002 is configured to determine a reference lane line of the to-be-labeled road surface according to the two-dimensional road images under the different viewing angles; the reference lane line includes a plurality of labeling points.

[0124] The matching unit 1003 is configured to determine, for each labeling point in the reference lane line, target point cloud data matched with the labeling point from the plurality of point cloud data.

[0125] The generation unit 1004 is configured to generate a lane line of the to-be-labeled road surface based on the target point cloud data corresponding to all the labeling points.

[0126] In one embodiment, the matching unit 1003 is specifically configured to convert each of the point cloud data into a corresponding two-dimensional mapping coordinate, and determine a matching coordinate of the labeling point from the plurality of two-dimensional mapping coordinates.

[0127] The point cloud data corresponding to the matching coordinate in the plurality of point cloud data is taken as the target point cloud data matched with the labeling point.

[0128] In one embodiment, the matching unit 1003 converts the point cloud data into a plurality of two-dimensional mapping coordinates, including:

[0129] The point cloud data is converted into imaging coordinates of an image acquisition device based on intrinsic and extrinsic parameters of the image acquisition device; the imaging coordinates are two-dimensional coordinates; the image acquisition device is a device used in association with a sensor that collects the plurality of point cloud data.

[0130] In one embodiment, the matching unit 1003 determines the matching coordinate of the labeling point from the plurality of two-dimensional mapping coordinates, including:

[0131] For each viewing angle, an original coordinate of the labeling point in the two-dimensional road image under the viewing angle is determined;

[0132] The similarity between the original coordinate and each two-dimensional mapping coordinate is determined, and the two-dimensional mapping coordinate with the highest similarity to the original coordinate is taken as the matching coordinate of the labeling point.

[0133] In one embodiment, the labeling unit 1002 is specifically configured to perform fusion processing on the plurality of two-dimensional road images to obtain a fusion image.

[0134] The reference lane line is generated based on the fusion image.

[0135] In an embodiment, the labeling unit 1002 performs a lane line generation process on the fusion image to obtain the reference lane line, including:

[0136] inputting the fusion image into a lane line detection model to obtain a reference lane line in the fusion image output by the lane line detection model; the lane line detection model is obtained based on a plurality of historical two-dimensional road images and lane lines labeled in the historical two-dimensional road images.

[0137] In an embodiment, the generating unit 1004 is specifically configured to, for each of the labeling points, perform fusion processing on target point cloud data corresponding to the labeling point under each view angle to obtain fusion point cloud corresponding to each of the labeling points.

[0138] The lane line is generated based on the fusion point cloud corresponding to all the labeling points.

[0139] In an embodiment, the obtaining unit 1001 is specifically configured to obtain initial point cloud data obtained by a sensor scanning the road surface to be labeled, filter out non-road surface point cloud data in the initial point cloud data to obtain candidate point cloud data.

[0140] Based on the candidate point cloud data, the road surface to be labeled is subjected to surface fitting processing to obtain a plurality of point cloud data of the road surface to be labeled.

[0141] The lane line generation device provided by the embodiments of the present application can generate a reference lane line based on the two-dimensional road image after obtaining the point cloud data of the road surface and the two-dimensional road image of the road surface under a plurality of view angles. Based on each labeling point contained in the reference lane line, the point cloud data matched with the labeling point can be determined from the point cloud data, and the lane line is generated according to all the point cloud data matched with the labeling points. The method provided by the present application can generate the lane line by means of the point cloud data, and can also filter and fuse the point cloud coordinates based on the two-dimensional road image of the road surface under a plurality of view angles, and generate the lane line by combining the data of the two modalities. The method can not only retain the advantage of large amount of image data information, but also retain the characteristics of strong anti-interference of point cloud data, effectively improving the accuracy of lane line generation. In addition, compared with the technical solution of obtaining the lane line by manually labeling the 3D point cloud data, the implementation difficulty is greatly reduced, and the accuracy of the labeling result is also improved.

[0142] It should be understood that the units described in the lane line generation method are described with reference to the Figure 3The various steps in the described method correspond. Thus, the operations and features described above for the method apply equally to the model training apparatus and the units contained therein, which will not be described again. The model training apparatus can be pre- implemented in a browser or other secure application of the computer device, or can be loaded into the browser or its secure application of the computer device by downloading or the like. The corresponding units in the model training apparatus can cooperate with the units in the computer device to realize the solutions of the embodiments of the present application.

[0143] In the foregoing detailed description, several modules or units are mentioned. The division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0144] It should be noted that the details of the anomaly identification apparatus and the model training apparatus of the embodiments of the present application that are not disclosed are referred to the details disclosed in the above embodiments of the present application, which will not be described again.

[0145] The following refers to Figure 11 , Figure 11 A structural schematic diagram of a computer device suitable for implementing the embodiments of the present application is shown. As Figure 11 shown, the computer system includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1102 or programs loaded from a storage portion 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for operation instructions of the system are also stored. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0146] The following components are connected to the I / O interface 1105: an input portion 1106 including a keyboard, a mouse, and the like; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 1108 including a hard disk, and the like; and a communication portion 1109 including a network interface card such as a LAN card, a modem, and the like. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 1110 as necessary, so that a computer program read therefrom is installed in the storage portion 1108 as necessary.

[0147] In particular, according to embodiments of the present application, the above reference to flowcharts Figure 3 or Figure 9 The described processes can be implemented as a computer software program. For example, embodiments of the present application include a computer program product which includes a computer program tangibly embodied on a computer readable medium, the computer program including program code for executing methods illustrated by the flowcharts. In such embodiments, the computer program includes program code for executing methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1109, and / or installed from the detachable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, the above-described functions defined in the system of the present application are executed.

[0148] It should be noted that the computer readable medium shown in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a carrier wave in a propagated data signal, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0149] The computer program product of the present application can be a computer program embodied on a non-transitory computer readable medium. When the computer program runs on a computing device, a series of operational steps will be implemented to generate a lane line. These steps are as described in the embodiments disclosed in the present application. The above described embodiments are merely given as non-limiting examples. Many variations and modifications are possible and contemplated, depending on specific needs. Therefore, the scope of the present application should be determined by the appended claims.

[0150] The units or modules described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The units or modules described can be arranged in a processor, for example, a processor can be described as comprising a first collecting module, a second collecting module, and a sending module. In some cases, the names of the units or modules do not constitute a limitation on the units or modules themselves.

[0151] As another aspect, the present application also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The computer readable storage medium stores one or more programs, and when the programs are used by one or more processors to execute the lane line generation method described in the present application.

[0152] The above description is merely given as a preferred embodiment and a description of the principles of the application. It should be understood by those skilled in the art that the disclosed scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above technical features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A lane line generation method characterized by, The method comprises the following steps: obtaining a plurality of point cloud data of a to-be-labeled road surface and a two-dimensional road image of the to-be-labeled road surface under different viewing angles; determining a reference lane line of the to-be-labeled road surface according to the two-dimensional road image under different viewing angles; the reference lane line comprises a plurality of labeled points; for each labeled point in the reference lane line, for each viewing angle, determining the original coordinates of the labeled point in the two-dimensional road image under the viewing angle; determining the similarity between the original coordinates and each two-dimensional mapping coordinates, and taking the two-dimensional mapping coordinates with the highest similarity with the original coordinates as the matching coordinates of the labeled point under the viewing angle; or, for each labeled point in the reference lane line in the fusion image of a plurality of two-dimensional road images, determining the corresponding original coordinates of the labeled point in each two-dimensional road image by a back calculation method; calculating the similarity between the original coordinates and a plurality of two-dimensional mapping coordinates, and taking the two-dimensional mapping coordinates with the highest similarity with the original coordinates as the matching coordinates of the labeled point; taking the point cloud data corresponding to the matching coordinates in a plurality of the point cloud data as the target point cloud data matched with the labeled point; for each labeled point, performing fusion processing on the target point cloud data corresponding to the labeled point under each viewing angle to obtain a fusion point cloud corresponding to each labeled point; and generating the lane line based on the fusion point clouds corresponding to all the labeled points.

2. The method of claim 1, wherein, The conversion of each point cloud data into corresponding two-dimensional mapping coordinates comprises: based on the intrinsic and extrinsic parameters of an image acquisition device, converting the point cloud data into imaging coordinates of the image acquisition device; the imaging coordinates are two-dimensional coordinates; the image acquisition device and the sensor used for collecting the plurality of point cloud data are associated devices.

3. The method according to claim 1 or 2, characterized in that, The determination of the matching coordinates of the labeled point from the plurality of two-dimensional mapping coordinates comprises: for each viewing angle, determining the original coordinates of the labeled point in the two-dimensional road image under the viewing angle; determining the similarity between the original coordinates and each two-dimensional mapping coordinates, and taking the two-dimensional mapping coordinates with the highest similarity with the original coordinates as the matching coordinates of the labeled point under the viewing angle.

4. The method of claim 1, wherein, The determination of the reference lane line of the to-be-labeled road surface according to the plurality of two-dimensional road images comprises: performing fusion processing on the plurality of two-dimensional road images to obtain a fusion image; generating the reference lane line based on the fusion image.

5. The method of claim 4, wherein, The lane line generation processing on the fusion image to obtain the reference lane line comprises: inputting the fusion image into a lane line detection model to obtain the reference lane line in the fusion image output by the lane line detection model; the lane line detection model is obtained by training based on a plurality of historical two-dimensional road images and the lane lines labeled in the historical two-dimensional road images.

6. The method of claim 1, wherein, The obtaining of the plurality of point cloud data of the to-be-labeled road surface comprises: obtaining initial point cloud data obtained by a sensor scanning the to-be-labeled road surface, filtering out non-road surface point cloud data in the initial point cloud data to obtain candidate point cloud data; performing curved surface fitting processing on the to-be-labeled road surface based on the candidate point cloud data to obtain a plurality of point cloud data of the to-be-labeled road surface.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the method of any one of claims 1-6.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the method of any one of claims 1-6.

9. A computer program product, characterised in that, The computer program product comprises instructions which, when executed, cause the method of any one of claims 1-6 to be performed.

Images