Lane line marking quality scoring method and device, equipment, storage medium and program product

Abstract

The invention discloses a lane line marking quality scoring method, device and equipment, a storage medium and a program product, and the method comprises the steps: obtaining three-dimensional lane line marking data and two-dimensional lane line data of a collection time period, performing lane line matching on the three-dimensional lane line marking data and the two-dimensional lane line data based on each acquisition moment in the acquisition time period, performing similarity scoring on matching line pairs at each acquisition moment, and aggregating similarity scores at each acquisition moment to obtain a quality evaluation score of the three-dimensional lane line marking data; according to the invention, similarity scoring is carried out on the matching line pairs after matching of the three-dimensional lane data and the two-dimensional lane data, aggregation is carried out based on similarity scores, and the quality evaluation score of the three-dimensional lane line marking data is obtained, so that the to-be-evaluated three-dimensional lane line marking data is effectively covered by quality evaluation, and the evaluation efficiency is improved. The problem of evaluation omission is effectively avoided, and the evaluation accuracy of the lane line marking data is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of lane line recognition technology, and in particular to a lane line marking quality scoring method, device, equipment, storage medium and program product. Background Art

[0002] Three-dimensional lane line recognition is one of the most important perception tasks in intelligent driving. Its function is to use the data collected by the sensors on the vehicle to infer the three-dimensional coordinates of the lane lines or road edges around the vehicle.

[0003] In recent years, computer vision-based 3D lane recognition technology has gradually become mainstream, given its cost-effectiveness and increasing accuracy. However, this progress is not without challenges. The lack of freshness of map data may lead to the loss of some lane information. At the same time, the accuracy of vehicle posture is limited by the accuracy of the positioning system. These problems may affect the reliability of recognition. Therefore, the quality of the labeled data produced by the current computer vision-based 3D lane recognition technology is often uneven. In order to ensure that the quality of the output labeled data is qualified, the quality of the labeled data needs to be evaluated. The evaluation criteria of the current labeled data quality evaluation scheme are difficult to quantify and are quite subjective. Different people may have different judgment criteria for the same labeled data, resulting in inaccurate evaluation. In addition, the evaluation requires a huge amount of manpower to check each frame of data one by one, and if a sample check is performed, it will not cover all the data. Summary of the invention

[0004] The main purpose of the present invention is to provide a lane line marking quality scoring method, device, equipment, storage medium and program product, aiming to solve the technical problems that the existing technology cannot accurately evaluate the quality of three-dimensional lane line marking data and the quality evaluation coverage is not comprehensive.

[0005] To achieve the above object, the present invention provides a lane marking quality scoring method, the method comprising the following steps:

[0006] Obtaining 3D lane line marking data and 2D lane line data within the acquisition time period;

[0007] Performing lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected during the collection time period to obtain matching line pairs corresponding to each lane image frame;

[0008] Performing similarity scoring on the matching line pairs to obtain a similarity score for each lane image frame;

[0009] The similarity scores of the lane image frames are aggregated to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0010] Optionally, the performing similarity scoring on the matching line pairs to obtain a similarity score for each lane image frame includes:

[0011] Converting a first lane line in the matching line pair into a first point set, wherein the first lane line is a labeled lane line in the three-dimensional lane line labeling data;

[0012] Converting a second lane line in the matching line pair that matches the first lane line into a second point set, wherein the second lane line is an identified lane line in the two-dimensional lane line data;

[0013] Determine a distance function and an angle function between the first lane line and the second lane line based on the first point set and the second point set;

[0014] The matching line pairs are scored for similarity according to the distance function and the angle function to obtain a similarity score for each lane image frame.

[0015] Optionally, performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain a similarity score for each lane image frame includes:

[0016] Performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain an initial score for the matching line pairs;

[0017] Determine whether the initial score of each matching line pair meets a preset similarity condition, and use the matching line pairs that meet the preset similarity condition as candidate line pairs;

[0018] Determine an I / O ratio parameter of the candidate line pair, and filter the candidate line pairs based on the I / O ratio parameter to determine a target line pair;

[0019] The similarity score of each lane image frame is determined based on the similarity scoring result of the target line pair.

[0020] Optionally, performing lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame includes:

[0021] Acquire lane image frames collected at each collection time within a collection time period;

[0022] Projecting the three-dimensional lane line annotation data at each acquisition moment onto the lane image frame corresponding to the acquisition moment to obtain a lane line projection result;

[0023] Based on each lane image frame collected within the collection time period, lane line matching is performed on the lane line projection result and the two-dimensional lane line recognition result to obtain matching line pairs corresponding to each lane image frame.

[0024] Optionally, performing lane line matching on the lane line projection result and the two-dimensional lane line recognition result based on each lane image frame collected within the collection time period to obtain a matching line pair corresponding to each lane image frame includes:

[0025] Obtaining color information and line type information of the lane line projection result and the two-dimensional lane line recognition result;

[0026] Grouping the lane line projection result and the two-dimensional lane line recognition result based on the color information and the line type information to obtain a plurality of lane line groups;

[0027] Determine a bipartite graph matching result between the lane line projection result and the two-dimensional lane line recognition result in each lane line group;

[0028] Lane line matching is performed on the lane line projection result and the two-dimensional lane line recognition result according to the bipartite graph matching result and each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame.

[0029] Optionally, aggregating the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data includes:

[0030] determining repeated line pairs among the matched line pairs of the lane image frame;

[0031] Aggregating the similarity scores of the repeated line pairs belonging to the same matching line pair to obtain the annotation quality score of each lane image frame collected during the collection time period;

[0032] Based on the acquisition time of each lane image frame, weighted aggregation is performed on the annotation quality scores of the lane image frames at the same acquisition time to obtain a single frame quality score within the acquisition time period;

[0033] The single-frame quality scores within the acquisition time period are aggregated to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0034] In addition, to achieve the above-mentioned purpose, the present invention further proposes a lane marking quality scoring device, the lane marking quality scoring device comprising:

[0035] A labeling data acquisition module is used to acquire the three-dimensional lane line labeling data and the two-dimensional lane line data within the acquisition time period;

[0036] A lane line matching module, configured to perform lane line matching between the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period, to obtain matching line pairs corresponding to each lane image frame;

[0037] A similarity scoring module is used to score the similarity of the matching line pairs to obtain a similarity score for each lane image frame;

[0038] The quality scoring module is used to aggregate the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also proposes a lane line marking quality scoring device, which includes: a memory, a processor, and a lane line marking quality scoring program stored in the memory and executable on the processor, wherein the lane line marking quality scoring program is configured to implement the steps of the lane line marking quality scoring method described above.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a lane line marking quality scoring program is stored. When the lane line marking quality scoring program is executed by a processor, the steps of the lane line marking quality scoring method described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer program product, which includes a lane line marking quality scoring program, and when the lane line marking quality scoring program is executed by a processor, the steps of the lane line marking quality scoring method described above are implemented.

[0042] The present invention acquires three-dimensional lane line annotation data and two-dimensional lane line data within a collection time period, performs lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period, obtains matching line pairs corresponding to each lane image frame, performs similarity scoring on the matching line pairs, obtains similarity scores of each lane image frame, aggregates the similarity scores of the lane image frames, and obtains a quality evaluation score of the three-dimensional lane line annotation data; because the present invention performs similarity scoring on the matching line pairs after matching the three-dimensional lane data with the two-dimensional lane data, aggregates based on the similarity scores, and obtains the quality evaluation score of the three-dimensional lane line annotation data, the quality evaluation effectively covers the three-dimensional lane line annotation data to be evaluated, effectively avoids the problem of evaluation omissions, and greatly improves the evaluation accuracy of the lane line annotation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1It is a structural diagram of a lane marking quality scoring device in a hardware operating environment involved in an embodiment of the present invention;

[0044] Figure 2 This is a flow chart of a first embodiment of a lane marking quality scoring method of the present invention;

[0045] Figure 3 This is a flow chart of a second embodiment of the lane marking quality scoring method of the present invention;

[0046] Figure 4 A schematic diagram of an erroneously matched line pair and a normally matched line pair in the second embodiment of the lane marking quality scoring method of the present invention;

[0047] Figure 5 This is a flow chart of a third embodiment of the lane marking quality scoring method of the present invention;

[0048] Figure 6 This is a structural block diagram of the first embodiment of the lane marking quality scoring device of the present invention.

[0049] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0051] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a lane marking quality scoring device in a hardware operating environment according to an embodiment of the present invention.

[0052] like Figure 1As shown, the lane marking quality scoring device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the lane marking quality scoring device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a lane marking quality scoring program.

[0055] exist Figure 1 In the lane line marking quality scoring device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the lane line marking quality scoring device of the present invention can be set in the lane line marking quality scoring device, and the lane line marking quality scoring device calls the lane line marking quality scoring program stored in the memory 1005 through the processor 1001, and executes the lane line marking quality scoring method provided by the embodiment of the present invention.

[0056] The embodiment of the present invention provides a method for scoring the quality of lane marking, referring to Figure 2 , Figure 2 Schematic diagram of the process of the first embodiment of the lane marking quality scoring method of the present invention.

[0057] In this embodiment, the lane marking quality scoring method includes the following steps:

[0058] Step S10: Acquire the three-dimensional lane line annotation data and the two-dimensional lane line data within the acquisition time period.

[0059] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes a lane marking quality scoring device (referred to as the scoring device) as an example to illustrate this embodiment and the following embodiments.

[0060] It is understandable that the acquisition time period may be a period of time for acquiring continuous images, for example, the acquisition time period may be a period of 10 seconds, 15 seconds or 20 seconds. The scoring device may acquire images based on a preset acquisition time interval through one or more cameras mounted on the same vehicle, obtain multiple continuous images within the acquisition time period, and then perform 3D lane line annotation and 2D lane line recognition on each image to obtain 3D lane line annotation data and 2D lane line data, respectively.

[0061] It should be noted that the above three-dimensional lane line marking data may be data obtained by marking the lane lines of each image within the acquisition time period in three-dimensional space. The above two-dimensional lane line data may be data obtained by identifying the lane lines of each image within the acquisition time period in two-dimensional space.

[0062] It is understandable that the scoring device can obtain continuous images collected within a collection time period, perform three-dimensional lane line annotation and two-dimensional lane line recognition on each image, and obtain three-dimensional lane line annotation data and two-dimensional lane line data.

[0063] In some embodiments, the scoring device may obtain image frames at each moment in the acquisition time period, input the image frames into a pre-built 2D lane line recognition model, process each image at each moment, and obtain two-dimensional lane line data.

[0064] Step S20: performing lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame.

[0065] It should be noted that the lane image frame may be one or more lane image frames collected within a collection time period. The collection time period may include multiple collection moments. Each collection moment corresponds to one or more lane image frames, that is, one frame or multiple frames may be collected for the same scene.

[0066] It should be understood that the matching line pair can be a matching pair between a three-dimensional lane line and a two-dimensional lane line collected at the same time. The matching line pair includes a first lane line in the three-dimensional lane line annotation data and a second lane line in the two-dimensional lane line data that matches the first lane line. The first lane line is the annotation result of the lane image after the three-dimensional lane line annotation; the second lane line is the recognition result of the lane image after the two-dimensional lane line recognition.

[0067] It can be understood that this embodiment determines the lane image frame at each moment in the acquisition time period, matches the three-dimensionally annotated lane lines in each image frame with the two-dimensionally recognized lane lines, and obtains matching line pairs in the lane image frame.

[0068] Furthermore, in order to improve the evaluation efficiency, the above step S20:

[0069] Step S21: Acquire lane image frames collected at each collection time within a collection time period.

[0070] It should be understood that the scoring device may acquire multiple lane image frames by collecting continuous images based on the collection time interval within the collection time period through one or more cameras carried on the same vehicle.

[0071] In some embodiments, the camera for capturing continuous images may be a CCD camera, a CMOS camera, or the like.

[0072] Step S22: Projecting the three-dimensional lane line annotation data at each acquisition moment onto the lane image frame corresponding to the acquisition moment to obtain a lane line projection result.

[0073] It should be noted that the lane image frames may be image frames collected at each collection moment within a collection time period.

[0074] It is understandable that the scoring device may project the three-dimensional lane line annotation data at each moment onto each image at that moment to obtain the projection result of the three-dimensional lane line.

[0075] In some embodiments, the scoring device may obtain the internal parameter matrix (i.e., intrinsic parameters) and external parameter matrix (i.e., extrinsic parameters) of the camera that captures the image, and project the three-dimensional lane line annotation data at each moment onto the lane image frame corresponding to each moment based on the internal and extrinsic parameters of the camera.

[0076] Among them, the above-mentioned internal parameters can be parameters that describe the internal properties of the camera, such as focal length, principal point (optical center) coordinates, distortion coefficient and other parameters; the above-mentioned external parameters can be parameters that describe the position and posture of the camera in the world coordinate system, such as rotation matrix and translation vector and other parameters.

[0077] Step S23: inputting the two-dimensional lane line data into a preset two-dimensional lane line recognition model for recognition to obtain a two-dimensional lane line recognition result.

[0078] It should be noted that the two-dimensional lane line recognition model can be a pre-built neural network model for identifying two-dimensional lane lines. For example, the scoring device can pre-train and iterate the original neural network model based on the training set and the test set to obtain the two-dimensional lane line recognition model.

[0079] In some embodiments, the scoring device may pre-process the two-dimensional lane line data, input the pre-processed two-dimensional lane line data into a preset two-dimensional lane line recognition model for recognition, and obtain a two-dimensional lane line recognition result, wherein the pre-processing may include scaling processing, color space conversion processing, normalization processing and / or data enhancement processing on the two-dimensional lane line data.

[0080] Step S24: performing lane line matching between the lane line projection result and the two-dimensional lane line recognition result based on each lane image frame collected within the collection time period, and obtaining matching line pairs corresponding to each lane image frame.

[0081] It should be noted that, in this embodiment, the 3D lane line annotation data at each moment can be projected onto each image frame at that moment to obtain the lane line projection result.

[0082] It can be understood that the scoring device compares the lane line projection result and the two-dimensional lane line data in each image, matches the lane lines in the two, and thus obtains the matching line pairs of the lane image frame.

[0083] Furthermore, in order to effectively match the lane lines, the above step S24 may include:

[0084] Step S241: Obtain color information and line type information of the lane line projection result and the two-dimensional lane line recognition result.

[0085] It should be noted that the color information may be the lane line color of each lane line in the lane line projection result and the two-dimensional lane line recognition result, for example, the lane line color may include white, yellow, etc. The above-mentioned linear information may be the lane line type of each lane line in the lane line projection result and the two-dimensional lane line recognition result, for example, the lane line type may include a solid line, a dotted line, a stop line, a road edge line, etc.

[0086] Step S242: grouping the lane line projection results and the two-dimensional lane line recognition results based on the color information and the line type information to obtain multiple lane line groups.

[0087] It can be understood that the scoring device can group the lane lines in the lane line projection results and the two-dimensional lane line recognition results by color based on color information; the scoring device can group the lane lines in the lane line projection results and the two-dimensional lane line recognition results by line type based on line type information, and perform comprehensive grouping based on the color grouping results and the line type grouping results to obtain multiple lane line groups.

[0088] Step S243: Determine a bipartite graph matching result between the lane line projection result and the two-dimensional lane line recognition result in each lane line group.

[0089] It is understandable that the bipartite graph matching result may be the result of calculating the maximum score bipartite graph matching between the projection result and the recognition result in each group by applying the Hungarian matching algorithm. The lane line score may be obtained based on the lane line similarity score result.

[0090] Step S244: performing lane line matching on the lane line projection result and the two-dimensional lane line recognition result according to the bipartite graph matching result and each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame.

[0091] It should be noted that, in this embodiment, lane line matching can be divided into three steps: grouping, matching, and filtering. Grouping refers to grouping the projection results and recognition results according to line type (solid line / dashed line / stop line / road edge line, etc.) and lane line color.

[0092] It is understandable that the above matching can be a maximum score bipartite graph matching between the projection results and the recognition results calculated in each group using the Hungarian matching algorithm. The lane score is calculated by the aforementioned lane similarity scoring algorithm. The above filtering is to filter out the wrong matching line pairs in the matching line pairs obtained in the previous step (the wrong matching line pairs may include matching line pairs with a low degree of similarity and / or matching line pairs with an intersection-over-union ratio parameter of 0).

[0093] Step S30: performing similarity scoring on the matching line pairs to obtain a similarity score for each lane image frame.

[0094] It should be noted that the similarity score may be a similarity score of the matching line pairs contained in the lane image frame.

[0095] It can be understood that this embodiment obtains the similarity score of the matching line pair by performing a similarity comparison between the three-dimensionally annotated lane line contained in the matching line pair and the two-dimensionally recognized lane line and scoring them based on the degree of similarity.

[0096] Step S40: Aggregating the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0097] It should be noted that the quality evaluation score may be an overall score of the annotation quality of the three-dimensional lane line annotation data to be evaluated.

[0098] It can be understood that since there can be multiple lane image frames collected within the collection time period, when there are multiple lane image frames, in order to evaluate the multiple frames, this embodiment can aggregate the similarity scores of the lane image frames into an overall score, which is the evaluation score of the 3D lane line annotation quality of this segment of data.

[0099] This embodiment obtains three-dimensional lane line annotation data and two-dimensional lane line data within a collection time period, performs lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period, obtains matching line pairs corresponding to each lane image frame, performs similarity scoring on the matching line pairs, obtains similarity scores for each lane image frame, aggregates the similarity scores of the lane image frames, and obtains a quality evaluation score for the three-dimensional lane line annotation data; since this embodiment performs similarity scoring on the matching line pairs after matching the three-dimensional lane data with the two-dimensional lane data, and aggregates based on the similarity scores to obtain the quality evaluation score of the three-dimensional lane line annotation data, the quality evaluation effectively covers the three-dimensional lane line annotation data to be evaluated, effectively avoids the problem of evaluation omissions, and greatly improves the evaluation accuracy of the lane line annotation data.

[0100] refer to Figure 3 , Figure 3 Schematic diagram of the process of the second embodiment of the lane marking quality scoring method of the present invention.

[0101] Based on the above first embodiment, in this embodiment, step S30 further includes:

[0102] Step S31: converting the first lane line in the matching line pair into a first point set.

[0103] It should be noted that the first lane line is the annotated lane line in the three-dimensional lane line annotation data. The first lane line can be a lane line and / or a road edge line. The lane line and / or the road edge line can be approximated as a straight line in the local range of the vehicle. Therefore, the similarity of the two lane lines and / or the road edge lines in the matching line pair can be characterized by the distance and angle between the two lines. The smaller the distance and the angle, the more similar they are.

[0104] Step S32: Convert the second lane line in the matching line pair that matches the first lane line into a second point set.

[0105] It should be noted that the second lane line is an identified lane line in the two-dimensional lane line data.

[0106] It can be understood that in this embodiment, the first lane line l1 can be identified based on the first point set, and the second lane line l2 can be identified based on the second point set. The first point set and the second point set refer to the following formula:

[0107]

[0108]

[0109] Step S33: Determine a distance function and an angle function between the first lane line and the second lane line based on the first point set and the second point set.

[0110] It should be noted that the distance function can represent the interval distance between the first lane line and the second lane line; the angle function can represent the angle between the first lane line and the second lane line.

[0111] It can be understood that the least squares fitting straight lines of the first lane line and the second lane line are respectively and Then the distance function d(l1, l2) between the first lane line l1 and the second lane line l2 can be defined as the distance from the point in l2 to the straight line The average distance is as follows:

[0112]

[0113] The angle function α(l1,l2) between the first lane line l1 and the second lane line l2 is defined as and The angle is as follows:

[0114]

[0115] Step S34: performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain a similarity score for each lane image frame.

[0116] It should be noted that the larger the distance and / or angle between the first lane line and the second lane line in the matching line pair, the lower the similarity between the two, the lower the similarity score of the matching line pair, and the lower the similarity score of the lane image frame; conversely, the smaller the distance and / or angle between the first lane line and the second lane line, the higher the similarity.

[0117] It can be understood that the similarity score between the first lane line l1 and the second lane line l2 is an exponentially weighted sum of a distance function and an angle function, that is, referring to the following formula, where s d and r is a fixed parameter, which means that when d(l1,l2)=r dWhen the distance score is s d ,s α and r α Similarly, the distance score weight w d and angle score weight w α Need to meet w d +w α =1.s d 、s α The value range of is (0, 1). At this time, score(l1,l2) is a monotonically decreasing function of d(l1,l2) and α(l1,l2). That is, the larger the distance and angle between the first lane line l1 and the second lane line l2, the lower the similarity score. α is the reference angle, r d is the reference distance.

[0118]

[0119] Furthermore, in order to accurately obtain the similarity score of the lane image frame, the above step S34 may include:

[0120] Step S341: performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain an initial score of the matching line pairs.

[0121] It should be noted that the initial score may be a similarity score preliminarily evaluated based on the distance and angle between the first lane line and the second lane line in the matching line pair.

[0122] Step S342: determining whether the initial score of each matching line pair satisfies a preset similarity condition, and taking the matching line pairs that satisfy the preset similarity condition as candidate line pairs.

[0123] It can be understood that the above-mentioned preset similarity condition can be used to filter out matching line pairs with lower similarity scores, that is, to filter out matching line pairs with lower similarity.

[0124] It should be understood that this embodiment can filter the matching line pairs, that is, filter out the wrong matching line pairs in the matching line pairs, and the filtering conditions may include: the similarity score is small and the intersection-and-joint ratio parameter of the minimum circumscribed rectangle of the lane line is 0.

[0125] Step S343: determining the I / O ratio parameters of the candidate line pairs, and filtering the candidate line pairs based on the I / O ratio parameters to determine the target line pairs.

[0126] It should be noted that the above-mentioned intersection over union (IoU) parameter may be the intersection over union ratio of the minimum circumscribed rectangle of the lane line.

[0127] It should be noted that filtering the candidate line pairs based on the intersection-and-joint ratio parameter can filter out special cases where the line pairs are far apart but have small line distances and angles. Figure 4 , Figure 4 Schematic diagram of incorrectly matched line pairs and normal matched line pairs. In order to accurately filter out such incorrectly matched line pairs, if the width of the minimum circumscribed rectangle of the lane line is too small, the width of the circumscribed rectangle should be widened.

[0128] Step S344: Determine the similarity score of each lane image frame based on the similarity scoring result of the target line pair.

[0129] It can be understood that the larger the angle and distance between the two lane lines in the target line pair, the lower the similarity score, and the smaller the distance, the higher the similarity score.

[0130] This embodiment converts the first lane line in the matching line pair into a first point set, where the first lane line is the labeled lane line in the three-dimensional lane line annotation data; converts the second lane line in the matching line pair that matches the first lane line into a second point set, where the second lane line is the identified lane line in the two-dimensional lane line data; determines a distance function and an angle function between the first lane line and the second lane line based on the first point set and the second point set; performs a similarity score on the matching line pair according to the distance function and the angle function, obtains a similarity score for each of the lane image frames, and converts the lane lines into point sets for similarity judgment, thereby accurately identifying the similarity between the first lane line and the second lane line, thereby improving the accuracy of the similarity score.

[0131] refer to Figure 5 , Figure 5 Schematic diagram of the process of the third embodiment of the lane marking quality scoring method of the present invention.

[0132] Based on the above embodiments, in this embodiment, in step S40, the method further includes:

[0133] Step S401: determining repeated line pairs in the matching line pairs of the lane image frame.

[0134] It should be noted that repeated line pairs can be the same three-dimensional lane line that appears repeatedly in multiple lane image frames. Since the scoring device can capture one or more lane image frames at the same acquisition time, the same lane line may be captured in multiple images, i.e., repeated line pairs appear.

[0135] Step S402: Aggregating the similarity scores of the repeated line pairs belonging to the same matching line pair to obtain the annotation quality score of each lane image frame collected within the collection time period.

[0136] It can be understood that, in this embodiment, the similarity scores of the matching line pairs of the same 3D lane line in multiple images can be aggregated into the annotation quality score of a single lane line by using arithmetic averaging.

[0137] Step S403: performing weighted aggregation on the annotation quality scores of the lane image frames at the same acquisition time based on the acquisition time of each lane image frame, to obtain a single frame quality score within the acquisition time period.

[0138] It can be understood that since the scoring device can collect one or more lane image frames at the same acquisition moment, that is, the same acquisition moment may correspond to one or more lane image frames. Therefore, in order to improve the evaluation efficiency, this embodiment can weightedly aggregate the annotation quality scores of multiple lane image frames collected at the same moment, and use weighted average to aggregate the quality scores of multiple lane lines at the same moment into a single-frame annotation quality score.

[0139] It should be noted that, in this embodiment, weighted aggregation may be performed using weighted averaging to balance the “far away is small, near is large” characteristic of camera imaging.

[0140] In some embodiments, since camera imaging has the characteristic of "small at far distance and large at near distance". Although lane lines generally extend from the rear of the vehicle to the front and rear, the same lane line includes both near and far distances. However, for lane lines that are farther away laterally, the distance between the matched line pairs after projection is smaller and the score is higher. If the arithmetic average is used to calculate the similarity score of a single frame, the score will be raised. To compensate for this phenomenon, a lower weight should be given to lane lines that are farther away laterally. In addition, under normal circumstances, there should be at least two lane lines around the vehicle. If only one pair of line pairs is matched, it means that one of the 3D lane line annotation, 2D lane line recognition results or camera parameters is incorrect. Therefore, when only one pair of lane lines is matched in a single frame, a penalty item should be added to the overall score of the single frame. Taking the above two points into consideration, the weight of the weighted average aggregation of the single frame score is determined as follows:

[0141]

[0142] Where n is the number of 3D lane line annotations that are successfully matched at that moment, and w j is the weight of the j-th 3D lane line score in the single frame score calculation, and They represent the average lateral distances from the jth and kth 3D lane lines to the origin of the vehicle coordinate system, respectively. ρ is a parameter that controls the importance of lane lines with closer lateral distances (i.e., the lane line lateral distance factor). The larger the value, the greater the weight of the score of the closer lane line in the single frame score. If the score of the jth 3D lane line is s j , then the single frame annotation quality score at this moment refers to the following formula:

[0143] s=wj ·s j

[0144] Step S404: Aggregate the single-frame quality scores within the acquisition time period to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0145] It should be understood that there are multiple acquisition moments within the acquisition time period. Therefore, this embodiment can aggregate the single-frame quality scores corresponding to each acquisition moment to obtain the overall quality score of the acquisition time period, that is, to obtain the quality evaluation score of the three-dimensional lane line annotation data.

[0146] It should be noted that the scoring device may use the percentile of the scores of all frames in the same segment of data as the annotation quality score of the entire segment of data.

[0147] It should be noted that using the weighted average of the scores to calculate the total score can only get the average level of the annotation quality of this segment of the trajectory, which cannot well reflect the annotation quality of most frames and avoid a few low-scoring frames from lowering the score of the overall trajectory. Therefore, this embodiment can use the percentile of all frame scores as the annotation quality score of the entire segment of data in this segment of the trajectory, that is, refer to the following formula:

[0148] k=1+(n-1)*p%

[0149] Score p =Score s +(Score s+1 -Score s )*(ks)

[0150] Among them, Score represents the sequence of single-frame quality scores within the acquisition time period sorted from small to large; k represents the position of the p percentile in the sequence; s represents the integer part of k, and if k is an integer, they are equal

[0151] This embodiment determines the repeated line pairs in the matching line pairs of the lane image frames, aggregates the similarity scores of the repeated line pairs belonging to the same matching line pairs, obtains the annotation quality scores of each lane image frame acquired within the acquisition time period, performs weighted aggregation on the annotation quality scores of the lane image frames at the same acquisition time based on the acquisition time of each lane image frame, obtains the single frame quality score within the acquisition time period, aggregates the single frame quality scores within the acquisition time period, and obtains the quality evaluation score of the three-dimensional lane line annotation data, thereby effectively improving the quality evaluation efficiency and achieving an overall scoring of the three-dimensional lane line data while ensuring the accuracy of the scoring.

[0152] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which a lane line marking quality scoring program is stored. When the lane line marking quality scoring program is executed by a processor, the steps of the lane line marking quality scoring method described above are implemented.

[0153] In addition, an embodiment of the present invention further provides a computer program product, including a lane line marking quality scoring program, which implements the steps of the lane line marking quality scoring method described above when executed by a processor.

[0154] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the lane marking quality scoring method described above, and will not be repeated here.

[0155] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the lane marking quality scoring device of the present invention.

[0156] like Figure 6 As shown, the lane marking quality scoring device proposed in the embodiment of the present invention includes:

[0157] The annotation data acquisition module 10 is used to acquire the three-dimensional lane line annotation data and the two-dimensional lane line data within the acquisition time period;

[0158] A lane line matching module 20, configured to perform lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected during the collection time period, to obtain matching line pairs corresponding to each lane image frame;

[0159] A similarity scoring module 30 is used to score the similarity of the matching line pairs to obtain a similarity score of each lane image frame;

[0160] The quality scoring module 40 is used to aggregate the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data.

[0161] Furthermore, the similarity scoring module 30 is also used to convert the first lane line in the matching line pair into a first point set, where the first lane line is the labeled lane line in the three-dimensional lane line annotation data; convert the second lane line in the matching line pair that matches the first lane line into a second point set, where the second lane line is the identified lane line in the two-dimensional lane line data; determine the distance function and the angle function between the first lane line and the second lane line based on the first point set and the second point set; and perform similarity scoring on the matching line pair according to the distance function and the angle function to obtain a similarity score for each of the lane image frames.

[0162] Furthermore, the similarity scoring module 30 is also used to perform similarity scoring on the matching line pairs according to the distance function and the angle function to obtain initial scores of the matching line pairs; determine whether the initial scores of each matching line pair meet a preset similarity condition; filter the matching line pairs based on the judgment result to determine candidate line pairs; determine the intersection-and-joint ratio parameters of the candidate line pairs; filter the candidate line pairs based on the intersection-and-joint ratio parameters to determine the target line pairs; and determine the similarity scores of each lane image frame based on the similarity scoring result of the target line pairs.

[0163] Furthermore, the lane line matching module 20 is also used to obtain lane image frames collected at each collection moment within a collection time period; project the three-dimensional lane line annotation data at each collection moment to the lane image frame corresponding to the collection moment to obtain a lane line projection result; and perform lane line matching on the lane line projection result and the two-dimensional lane line recognition result based on each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame.

[0164] Furthermore, the lane line matching module 20 is also used to obtain color information and line type information of the lane line projection result and the two-dimensional lane line recognition result; group the lane line projection result and the two-dimensional lane line recognition result based on the color information and the line type information to obtain multiple lane line groups; determine the bipartite graph matching result between the lane line projection result and the two-dimensional lane line recognition result in each lane line group; perform lane line matching on the lane line projection result and the two-dimensional lane line recognition result according to the bipartite graph matching result and each lane image frame collected during the collection time period to obtain matching line pairs corresponding to each lane image frame.

[0165] Furthermore, the quality scoring module 40 is also used to determine the repeated line pairs in the matching line pairs of the lane image frame; aggregate the similarity scores of the repeated line pairs belonging to the same matching line pair to obtain the annotation quality score of each lane image frame acquired within the acquisition time period; perform weighted aggregation on the annotation quality scores of the lane image frames at the same acquisition time based on the acquisition time of each lane image frame to obtain the single frame quality score within the acquisition time period; aggregate the single frame quality scores within the acquisition time period to obtain the quality evaluation score of the three-dimensional lane line annotation data.

[0166] This embodiment obtains three-dimensional lane line annotation data and two-dimensional lane line data within a collection time period, performs lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period, obtains matching line pairs corresponding to each lane image frame, performs similarity scoring on the matching line pairs, obtains similarity scores for each lane image frame, aggregates the similarity scores of the lane image frames, and obtains a quality evaluation score for the three-dimensional lane line annotation data; since this embodiment performs similarity scoring on the matching line pairs after matching the three-dimensional lane data with the two-dimensional lane data, and aggregates based on the similarity scores to obtain the quality evaluation score of the three-dimensional lane line annotation data, the quality evaluation effectively covers the three-dimensional lane line annotation data to be evaluated, effectively avoids the problem of evaluation omissions, and greatly improves the evaluation accuracy of the lane line annotation data.

[0167] Other embodiments or specific implementations of the lane marking quality scoring device of the present invention can refer to the above-mentioned method embodiments, which will not be repeated here.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0169] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0171] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A lane marking quality scoring method, characterized in that: The lane marking quality scoring method includes: Obtaining 3D lane line marking data and 2D lane line data within the acquisition time period; Performing lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected during the collection time period to obtain matching line pairs corresponding to each lane image frame; Performing similarity scoring on the matching line pairs to obtain a similarity score for each lane image frame; The similarity scores of the lane image frames are aggregated to obtain a quality evaluation score of the three-dimensional lane line annotation data.

2. The lane marking quality scoring method according to claim 1, characterized in that: The performing similarity scoring on the matching line pairs to obtain a similarity score for each lane image frame includes: Converting a first lane line in the matching line pair into a first point set, wherein the first lane line is a labeled lane line in the three-dimensional lane line labeling data; Converting a second lane line in the matching line pair that matches the first lane line into a second point set, wherein the second lane line is an identified lane line in the two-dimensional lane line data; Determine a distance function and an angle function between the first lane line and the second lane line based on the first point set and the second point set; The matching line pairs are scored for similarity according to the distance function and the angle function to obtain a similarity score for each lane image frame.

3. The lane marking quality scoring method according to claim 2, characterized in that: The performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain a similarity score for each lane image frame includes: Performing similarity scoring on the matching line pairs according to the distance function and the angle function to obtain an initial score for the matching line pairs; Determine whether the initial score of each matching line pair meets a preset similarity condition, and use the matching line pairs that meet the preset similarity condition as candidate line pairs; Determine an I / O ratio parameter of the candidate line pair, and filter the candidate line pairs based on the I / O ratio parameter to determine a target line pair; The similarity score of each lane image frame is determined based on the similarity scoring result of the target line pair.

4. The lane marking quality scoring method according to any one of claims 1 to 3, characterized in that: The performing lane line matching on the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period to obtain a matching line pair corresponding to each lane image frame includes: Acquire lane image frames collected at each collection time within a collection time period; Projecting the three-dimensional lane line annotation data at each acquisition moment onto the lane image frame corresponding to the acquisition moment to obtain a lane line projection result; Inputting the two-dimensional lane line data into a preset two-dimensional lane line recognition model for recognition to obtain a two-dimensional lane line recognition result; Based on each lane image frame collected within the collection time period, lane line matching is performed on the lane line projection result and the two-dimensional lane line recognition result to obtain matching line pairs corresponding to each lane image frame.

5. The lane marking quality scoring method according to claim 4, characterized in that: The lane line matching of the lane line projection result and the two-dimensional lane line recognition result based on each lane image frame collected within the collection time period to obtain a matching line pair corresponding to each lane image frame includes: Obtaining color information and line type information of the lane line projection result and the two-dimensional lane line recognition result; Grouping the lane line projection result and the two-dimensional lane line recognition result based on the color information and the line type information to obtain a plurality of lane line groups; Determine a bipartite graph matching result between the lane line projection result and the two-dimensional lane line recognition result in each lane line group; Lane line matching is performed on the lane line projection result and the two-dimensional lane line recognition result according to the bipartite graph matching result and each lane image frame collected within the collection time period to obtain matching line pairs corresponding to each lane image frame.

6. The lane marking quality scoring method according to any one of claims 1 to 3, characterized in that: The aggregating the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data includes: determining repeated line pairs among the matched line pairs of the lane image frame; Aggregating the similarity scores of the repeated line pairs belonging to the same matching line pair to obtain the annotation quality score of each lane image frame collected during the collection time period; Performing weighted aggregation on the annotation quality scores of the lane image frames at the same acquisition time to obtain a single frame quality score within the acquisition time period; The single-frame quality scores within the acquisition time period are aggregated to obtain a quality evaluation score of the three-dimensional lane line annotation data.

7. A lane marking quality scoring device, characterized in that: The lane marking quality scoring device comprises: A labeling data acquisition module is used to acquire the three-dimensional lane line labeling data and the two-dimensional lane line data within the acquisition time period; A lane line matching module, configured to perform lane line matching between the three-dimensional lane line annotation data and the two-dimensional lane line data based on each lane image frame collected within the collection time period, to obtain matching line pairs corresponding to each lane image frame; A similarity scoring module is used to score the similarity of the matching line pairs to obtain a similarity score for each lane image frame; The quality scoring module is used to aggregate the similarity scores of the lane image frames to obtain a quality evaluation score of the three-dimensional lane line annotation data.

8. A lane marking quality scoring device, characterized in that: The lane line marking quality scoring device includes: a memory, a processor, and a lane line marking quality scoring program stored in the memory and executable on the processor, wherein the lane line marking quality scoring program is configured to implement the lane line marking quality scoring method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a lane line marking quality scoring program, and when the lane line marking quality scoring program is executed by the processor, the lane line marking quality scoring method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a lane line marking quality scoring program, and when the lane line marking quality scoring program is executed by a processor, the steps of the lane line marking quality scoring method according to any one of claims 1 to 6 are implemented.

Images