Lane line fitting method, electronic device, and storage medium

Abstract

The application discloses a lane line fitting method, an electronic device and a storage medium. The method comprises the following steps: extracting lane line simulation point sets in a lane image obtained by shooting; mapping lane line simulation points in the lane line simulation point sets to a vehicle body coordinate system to obtain lane line simulation point sets in the vehicle body coordinate system; performing primary fitting on a lane line based on lane line simulation points in the lane line simulation point sets in the vehicle body coordinate system to obtain a primary fitting lane line in the vehicle body coordinate system; in a case where it is determined that there is abnormal fitting based on the primary fitting lane line, obtaining segmentation points based on abnormal fitting points; and performing segmented fitting on the lane line simulation point sets according to the segmentation points to obtain a final fitting lane line. According to the above scheme, whether there is abnormal fitting is determined based on the primary fitting lane line obtained after the primary fitting, and the segmentation points are obtained, so that the fitting accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a lane line fitting method, electronic device, and storage medium. Background Technology

[0002] In autonomous driving technology, lane line recognition is required through visual perception, and the accuracy of lane line recognition largely determines driving safety.

[0003] Typically, existing lane line recognition technologies fit lane lines as cubic polynomial curves. While this can accurately fit some simple lane lines, its accuracy is often lower for more complex lane lines.

[0004] Therefore, improving the accuracy of lane line fitting has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides at least one lane line fitting method, electronic device, and storage medium to improve the accuracy of lane line fitting.

[0006] The first aspect of this application provides a lane line fitting method, comprising: extracting a set of simulated lane lines from a captured lane image; mapping the simulated lane lines in the set to a vehicle coordinate system to obtain a set of simulated lane lines in the vehicle coordinate system; performing an initial fitting of the lane lines based on the simulated lane lines in the set to obtain an initial fitted lane line in the vehicle coordinate system; if an abnormal fitting is determined based on the initial fitted lane line, obtaining segmentation points based on the abnormal fitting points; and performing segmented fitting of the simulated lane lines in the set according to the segmentation points to obtain a final fitted lane line.

[0007] The above scheme obtains the initial fitted lane line by performing an initial fitting on the simulated lane line point set. When anomalies are identified based on the initial fitted lane line, segmentation points are obtained from the anomaly fitting points. The simulated lane line point set is then segmented and fitted according to the segmentation points to obtain the final fitted lane line. This method can obtain more accurate segmentation points, thereby improving the accuracy of lane line fitting.

[0008] The phrase "based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system, the lane line is initially fitted to obtain the initially fitted lane line under the vehicle coordinate system" further includes: if there is no abnormal fitting in the initial fitting, the initially fitted lane line is used as the final fitted lane line and output.

[0009] The process of "performing an initial fit on the lane lines based on the lane line simulation points in the vehicle coordinate system to obtain an initial fitted lane line in the vehicle coordinate system" further includes: obtaining the difference value of each lane line simulation point, wherein the difference value is calculated by the position coordinates of each lane line simulation point and the position coordinates of the corresponding fitted lane line simulation point, and the fitted lane line simulation point is calculated based on the lane line simulation points and the initial fitted lane line in the vehicle coordinate system; identifying lane line simulation points with difference values ​​greater than a difference threshold as outlier points and calculating the number of outlier points; and determining whether the initial fitted lane line has outlier points based on the number of outlier points.

[0010] The step of "determining whether the initially fitted lane line has abnormal fitting based on the number of abnormal fitting points" includes: determining that the initial fitting has no abnormal fitting when the number of abnormal fitting points is 0; calculating the average difference when the number of abnormal fitting points is 1, and determining the relationship between the average difference and the difference threshold; determining that the initially fitted lane line has no abnormal fitting when the average difference is less than 1 / Q of the difference threshold; and determining that the initially fitted lane line has abnormal fitting when the average difference is not less than 1 / Q of the difference threshold. Here, Q is a constant greater than 1, the average difference is calculated from the difference values ​​corresponding to M lane line simulation points before and after the abnormal fitting point, and M is a preset integer; and determining that the initially fitted lane line has abnormal fitting when the number of abnormal fitting points is greater than 1.

[0011] The phrase "in the case of determining the existence of abnormal fitting based on the initially fitted lane line, obtaining segmentation points based on the abnormal fitting points" includes: in the case of abnormal fitting in the initial fitting, at least one of the abnormal fitting points is used as a segmentation point.

[0012] Where the number of outlier fitting points is greater than 1, the step of "taking at least one of the outlier fitting points as a segmentation point" includes: traversing all outlier fitting points, finding the two outlier fitting points with the largest difference value and their coordinates in the vehicle coordinate system, to determine the positional relationship of the two outlier fitting points with the largest difference value relative to the initially fitted lane line in the vehicle coordinate system; traversing all lane line simulation points between the two outlier fitting points with the largest difference value, calculating the angular error between two adjacent lane line simulation points, obtaining the two adjacent lane line simulation points with the largest angular error, and calculating the difference value between the two adjacent lane line simulation points; when the two outlier fitting points with the largest difference value are located on the same side of the initially fitted lane line in the vehicle coordinate system, the point with the larger difference value among the two adjacent lane line simulation points is taken as the segmentation point; when the two outlier fitting points with the largest difference value are located on different sides of the initially fitted curve in the vehicle coordinate system, the point with the smaller difference value among the two adjacent lane line simulation points is taken as the segmentation point.

[0013] The step of "segmenting and fitting the simulated lane line points according to the segmentation points to obtain the final fitted lane line" includes: fitting the simulated lane line points in the vehicle coordinate system before the segmentation points based on an Nth-order curve model to obtain a first fitted lane line; establishing constraints and fitting the simulated lane line points in the vehicle coordinate system after the segmentation points based on the constraints to obtain a second fitted lane line; and using the first fitted lane line and the second fitted lane line as the final fitted lane line.

[0014] The constraints include: the coordinates of the segment points conform to the functional relationship of the first fitted lane line and the functional relationship of the second fitted lane line; the first and second derivatives of the function of the second fitted lane line and the function of the first fitted lane line at the segment points are equal.

[0015] A second aspect of this application provides an electronic device including a camera, a memory, and a processor, wherein the camera, the memory, and the processor are respectively coupled to the processor, and the processor is used to execute program instructions stored in the memory to implement the lane line fitting method described above.

[0016] A third aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the lane line fitting method described above.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the lane line fitting method of this application;

[0020] Figure 2 This is a schematic diagram of the vehicle body coordinate system of this application;

[0021] Figure 3 This is a flowchart illustrating another embodiment of the lane line fitting method of this application;

[0022] Figure 4 This is a flowchart illustrating yet another embodiment of the lane line fitting method of this application;

[0023] Figure 5 This is a flowchart illustrating yet another embodiment of the lane line fitting method of this application;

[0024] Figure 6 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0025] Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0026] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0027] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0028] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the lane line fitting method of this application.

[0030] Specifically, the lane line fitting method includes the following steps:

[0031] Step S11: Extract the set of simulated lane lines from the captured lane image;

[0032] In a specific implementation scenario, lane images can be acquired using a device with camera capabilities. The acquired images are then input into a trained deep learning semantic segmentation model. The deep learning semantic segmentation model analyzes the images to obtain a binary image of lane line semantic segmentation. The pixels in the binary image of lane line semantic segmentation are then scanned line by line to obtain a set of simulated lane line points in the lane image coordinate system.

[0033] In a specific implementation scenario, the pixels in the lane line semantic segmentation binary image obtained from the deep learning semantic segmentation model can be traversed row by row from bottom to top and from right to left to obtain lane line simulation points and lane line segmentation points, and their positions are recorded. For the first image pixel row where a lane line segmentation point appears, each lane line simulation point in that image pixel row is recorded as the initial point for different lane line simulation point sets. Since lane lines usually have a certain width, for the same lane line, there are usually multiple pixels representing the lane line in the same image pixel row. For example, in the lane line semantic segmentation binary image, white pixels represent lane lines, and black pixels represent other event types besides lane lines. Even for the same lane line, multiple white pixels will appear in the same image pixel row. To reduce the number of points in the lane line simulation point set, only one pixel is taken as the lane line simulation point for that lane line. In this implementation scenario, the white pixels representing the lane line can be equidistant from the white pixels on both sides (i.e., lane line segmentation points). White pixels serve as the lane line simulation points for this lane line; of course, lane line simulation points can also be obtained in other ways, which are only illustrative examples in this implementation scenario and are not intended to be limiting; lane line segmentation points are points on both sides of the lane line, used to represent the division between different lane lines or lane lines and other road surfaces. For example, taking a solid yellow line on the road as an example, in the same image pixel row, the rightmost and leftmost pixels of the solid yellow line can be used to represent the division points between the lane line and other road surfaces; in this implementation scenario, multiple lane line simulation point sets can be obtained by analyzing the lane line semantic segmentation binary image, and each lane line simulation point set is used to simulate one lane line.

[0034] In a specific implementation scenario, after traversing a row of image pixels to obtain the initial points in different sets of lane line simulation points, for each lane line simulation point obtained by traversing the next row of image pixels, a lateral comparison needs to be performed. If the lateral distance between a lane line simulation point in the current row and a lane line simulation point obtained in the previous row is less than the matching threshold, then the two lane line simulation points are considered to belong to the same set of lane line simulation points, that is, points on the same lane line, and the lane line simulation point is added to the corresponding set of lane line simulation points. If the lateral distance between a lane line simulation point in the current row and every lane line simulation point obtained in the previous row is greater than the matching threshold, then the lane line simulation point is considered to belong to a new set of lane line simulation points, that is, a point on a new lane line, and a new set of lane line simulation points needs to be established, with the lane line simulation point being used as the initial point of the new set.

[0035] It should be noted that the matching threshold is used to measure the continuity between lane line simulation points, that is, whether they can be classified into the same lane line simulation point set. In practical applications, any suitable value can be selected, and this application does not impose any restrictions. In addition, in actual roads, the same lane line is usually continuous in the longitudinal direction. In the lane image coordinate system, the lateral distance between lane line simulation points on different lane lines is usually large. When the lateral distance between two lane line simulation points is greater than a certain value, it can be considered that the two lane line simulation points do not belong to the same lane line.

[0036] Step S12: Map the lane line simulation points in the lane line simulation point set to the vehicle coordinate system to obtain the lane line simulation point set in the vehicle coordinate system.

[0037] In a specific implementation scenario, after obtaining the set of simulated lane lines in the lane image coordinate system, the simulated lane lines in the set are projected onto the vehicle coordinate system through the mapping relationship between the camera's internal and external participating coordinate systems, thus obtaining the set of simulated lane lines in the vehicle coordinate system.

[0038] It should be noted that in this article, please refer to... Figure 2 , Figure 2 This is a schematic diagram of the vehicle body coordinate system of this application; the vehicle body coordinate system takes the center point of the rear axle as the origin, the driving direction of the vehicle as the x-axis, and the y-axis is perpendicular to the driving direction of the vehicle.

[0039] Step S13: Perform an initial fit on the lane line based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system to obtain the initial fitted lane line under the vehicle coordinate system.

[0040] In a specific implementation scenario, after obtaining the simulated point set of the lane line in the vehicle coordinate system, in order to determine whether the lane line needs to be piecewise fitted and to find the segmentation points, an initial fitting of the lane line is required. For example, this application divides the lane line into two segments for fitting, and uses a cubic curve model as the fitting model for the lane line, that is, the fitted lane line can be expressed as:

[0041]

[0042] Where x represents the ordinate of the t-th lane line simulation point, that is, the longitudinal coordinate position of that point relative to the vehicle; y t (x) represents the y-coordinate value of the t-th lane line simulation point, i.e., the lateral coordinate position of the point relative to the vehicle; y0 represents the lateral offset between the rear axle center of the vehicle and the lane line centerline; θ represents the angle between the vehicle's driving direction and the lane line; c0 represents the curvature of the lane line; and c1 represents the rate of change of the curvature of the lane line. For example, this application uses a square-root Kalman filter to fit the lane line simulation points. In the initial fitting, all lane line simulation points in each lane line simulation point set are used, and the system noise matrix R in the square-root Kalman filter is adjusted so that nearby points have a greater influence on the fitting parameters during the fitting process, thereby facilitating the determination of the segment point positions.

[0043] Step S14: If an abnormal fit is determined based on the initially fitted lane line, segmentation points are obtained based on the abnormal fit points.

[0044] Step S15: Perform segmented fitting on the simulated lane line point set according to the segmented points to obtain the final fitted lane line.

[0045] The above scheme obtains the initial fitted lane line by performing an initial fitting on the simulated lane line point set. When anomalies are identified based on the initial fitted lane line, segmentation points are obtained from the anomaly fitting points. The simulated lane line point set is then segmented and fitted according to the segmentation points to obtain the final fitted lane line. This method can obtain more accurate segmentation points, thereby improving the accuracy of lane line fitting.

[0046] In some possible implementation scenarios, the step of "performing an initial fit on the lane line based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system to obtain the initial fitted lane line under the vehicle coordinate system" further includes: if there is no abnormal fitting in the initial fit, the initial fitted lane line is used as the final fitted lane line and output.

[0047] In some possible implementation scenarios, please refer to Figure 3 , Figure 3This is a flowchart illustrating another embodiment of the lane line fitting method of this application; specifically, after the step of "performing an initial fit on the lane line based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system to obtain the initially fitted lane line under the vehicle coordinate system" it further includes:

[0048] Step S31: Obtain the difference value of each lane line simulation point, wherein the difference value is calculated by the position coordinates of each lane line simulation point and the position coordinates of the corresponding fitted lane line simulation point, and the fitted lane line simulation point is calculated based on the lane line simulation point and the initial fitted lane line in the vehicle coordinate system.

[0049] In a specific implementation scenario, the initial difference threshold is set to δ. dis The number of outlier points is 0; after obtaining the initial fitted lane line in the vehicle coordinate system, all lane line simulation points in the set of lane line simulation points used to fit the initial fitted lane line are traversed, and the actual lateral coordinates (y, y) of each point in the vehicle coordinate system are calculated. true ) and the fitted lateral coordinates (y) calculated using the initially fitted lane lines fit The absolute value of the difference between the simulated lane line points and the initial fitted lane line is used as the difference value. Specifically, the above-mentioned calculation of the fitted lateral coordinates using the fitted lane line can be achieved by substituting the longitudinal coordinates (i.e., the x-axis coordinates in the vehicle coordinate system) of the simulated lane line points into the function representation of the initial fitted lane line, thereby obtaining the fitted lateral coordinate values. It should be noted that, in this implementation scenario, the fitted lane line simulated points can be obtained by substituting the longitudinal coordinates of the simulated lane line points into the function representation of the initial fitted lane line to calculate their lateral coordinates.

[0050] Step S32: Select the lane line simulation points with difference values ​​greater than the difference threshold as outlier fitting points, and calculate the number of outlier fitting points;

[0051] In a specific implementation scenario, after obtaining the difference values ​​of each lane line simulation point, the difference values ​​of each point are compared with the difference threshold δ. dis If the difference value of a lane line simulation point is greater than the difference threshold, then the point is regarded as an outlier and the coordinates of the outlier in the vehicle coordinate system are recorded. Finally, the coordinates of all outlier points in the vehicle coordinate system and the number of outlier points are recorded.

[0052] Step S33: Based on the number of abnormal fitting points, determine whether there is abnormal fitting in the initially fitted lane line.

[0053] In some possible implementation scenarios, "determining whether the initially fitted lane line has an abnormal fit based on the number of abnormal fit points" includes:

[0054] Step S331: In response to the fact that the number of outlier points is 0, determine that there are no outliers in the initial fitting;

[0055] In a specific implementation scenario, if the number of outlier points is 0, it is determined that there is no outlier in the initial fit, and the lane line of the initial fit is directly used as the lane line of the final fit.

[0056] Step S332: In response to the fact that the number of outlier fitting points is 1, calculate the average difference and determine the relationship between the average difference and the difference threshold; if the average difference is less than 1 / Q of the difference threshold, determine that the initially fitted lane line does not have outlier fitting; if the average difference is not less than 1 / Q of the difference threshold, determine that the initially fitted lane line has outlier fitting; where Q is a constant greater than 1, the average difference is calculated from the difference values ​​corresponding to M lane line simulation points before and after the outlier fitting point, and M is a preset integer;

[0057] In a specific implementation scenario, if there is only one outlier, then the average difference E between the M lane line simulation points before and after the outlier needs to be used. avg Determine whether the outlier point belongs to the noise point; assume that the outlier point is the Kth lane line simulation point in its lane line simulation point set, and the coordinates of each lane line simulation point in the vehicle coordinate system are (x k ,y true,k Let f(x) be the function expression for the initial fit of the lane lines. Then, the average difference can be calculated as follows:

[0058]

[0059] In this specific implementation scenario, M can be 3, that is, if E avg Less than δ dis If E = 3, then the outlier is considered a noise point, and no lane line segmentation fitting is needed; the initially fitted lane line is directly used as the final fitted lane line. avg Not less than δ dis If the value is 3, it is determined that there is an abnormal fit in the initial fitting of the lane line. This abnormal fit point is used as the segment point when fitting the lane line in segments, and subsequent segment fitting is performed.

[0060] Step S333: In response to the number of abnormal fitting points being greater than 1, it is determined that the initially fitted lane line has an abnormal fitting.

[0061] In some possible implementation scenarios, the phrase "in the case of determining the existence of abnormal fitting based on the initially fitted lane line, obtaining segmentation points based on the abnormal fitting points" includes: in the case of abnormal fitting in the initial fitting, taking at least one of the abnormal fitting points as segmentation points, performing segmented fitting, and then obtaining the final fitted lane line.

[0062] In some possible implementation scenarios, please refer to Figure 4 , Figure 4 This is a flowchart illustrating another embodiment of the lane line fitting method of this application; specifically, when the number of abnormal fitting points is greater than 1, the step of "taking at least one of the abnormal fitting points as a segmentation point" includes:

[0063] Step S41: Traverse all outlier points, find the two outlier points with the largest difference and their coordinates in the vehicle coordinate system, so as to determine the positional relationship of the two outlier points with the largest difference relative to the initially fitted lane line in the vehicle coordinate system.

[0064] In a specific implementation scenario, iterate through all recorded outlier points and find the coordinates P of the two points with the largest differences in the vehicle coordinate system. error1 (x error1 ,y error1 ) and P error2 (x error2 ,y error2 The process involves determining the positional relationship between the two points and the initially fitted lane line. Specifically, determining the positional relationship between the two points and the initially fitted lane line includes: obtaining the curve representation of the initially fitted lane line in the vehicle coordinate system through the function expression of the initially fitted lane line; and obtaining the positional relationship of the two outlier fitting points with the largest difference values ​​relative to the curve of the initially fitted lane line in the vehicle coordinate system through their coordinates, i.e., whether they are located on the same side or opposite side of the curve.

[0065] Step S42: Traverse all lane line simulation points between the two outlier fitting points with the largest difference value, calculate the angle error between two adjacent lane line simulation points, obtain the two adjacent lane line simulation points with the largest angle error, and calculate the difference value between the two adjacent lane line simulation points.

[0066] In a specific implementation scenario, all points between two outlier fitting points in the lane line simulation point set are traversed. The angular error between two adjacent lane line simulation points is calculated as follows: Assuming P1(x1,y1) and P2(x2,y2) are two adjacent points in the traversed lane line simulation points, the position of the point on the initially fitted lane line corresponding to these two points can be obtained as P1(x1,y1) based on the function expression f(x) of the initially fitted lane line. 1,fit (x1,f(x1)) and P2,fit (x2, f(x2)), then the angle error α can be calculated as:

[0067]

[0068] This yields the two adjacent lane line simulation points with the largest angle error. The difference between the two adjacent lane line simulation points is then calculated. The method for calculating the difference has been described in the above implementation scenario and will not be repeated here.

[0069] Step S43: If the two outlier fitting points with the largest difference values ​​are located on the same side of the initially fitted lane line in the vehicle coordinate system, the point with the larger difference value among the two adjacent lane line simulation points is taken as the segmentation point.

[0070] Step S44: If the two outlier fitting points with the largest difference values ​​are located on different sides of the initial fitting curve in the vehicle coordinate system, the point with the smaller difference value among the two adjacent lane line simulation points is used as the segmentation point.

[0071] In some possible implementation scenarios, please refer to Figure 5 , Figure 5 This is a flowchart illustrating another embodiment of the lane line fitting method of this application; specifically, the step of "performing segmented fitting of the lane line simulation point set according to the segmented points to obtain the final fitted lane line" includes:

[0072] Step S51: Based on the Nth degree curve model, fit the segment point and the lane line simulation points in the vehicle coordinate system before the segment point to obtain the first fitted lane line.

[0073] In a specific implementation scenario, the simulated lane lines can be fitted based on a cubic curve model. Of course, in other possible implementation scenarios, N can take other values, which are not limited here. Specifically, in this implementation scenario, the square root Kalman filter is first used to apply the above-obtained segmented points p. seg (x seg ,y seg The lane line simulation points in the vehicle coordinate system (including the segment points) are fitted to obtain the function expression of the first fitted lane line:

[0074]

[0075] Segmentation point p seg Lane line angle θ seg curvature c 0,seg And the lateral distance y relative to the vehicle seg It can be represented as:

[0076]

[0077] c 0,seg =c0+c1x seg ;

[0078]

[0079] Step S52: Establish constraints, and fit the simulated lane lines in the vehicle coordinate system after the segmentation point to obtain the second fitted lane line.

[0080] In some possible implementations, the constraints include: the coordinates of the segment points conform to the functional relationship of the first fitted lane line and the functional relationship of the second fitted lane line; the function of the second fitted lane line is equal to the first and second derivatives of the function of the first fitted lane line at the segment points.

[0081] In a specific implementation scenario, the coordinates of the segment points conform to the functional relationships of both the first and second fitted lane lines. The fact that the first and second derivatives of the function of the second fitted lane line and the function of the first fitted lane line are equal at the segment points can be expressed as follows: the coordinates of the segment points conform to the functional relationships of the two fitted lane lines, and the lane line angles and curvatures of the two fitted lane lines at the segment points are equal. Specifically, the lane line angle of the second fitted lane line can be obtained using the above constraints. and curvature And the lateral position of the vehicle body at the segmentation point The expression is:

[0082]

[0083]

[0084]

[0085] Integrating the above formula, we get:

[0086]

[0087]

[0088] Based on the above formula, the functional expression for the second fitted lane line can be obtained as follows:

[0089]

[0090]

[0091] Where y0,θ,c0,c1 are the fitting parameters of the first fitted lane line, x seg This represents the vertical distance from the segment point to the rear axle of the vehicle. To accommodate additional state variables when fitting the second fitted lane line, the same square root Kalman filter was used as the first fitted lane line. To avoid significantly interfering with the results of the first fitted lane line while maintaining the accuracy of the second fitted lane line, the update magnitude of the same fitting parameters y0, θ, c0, c1 as in the first segment was reduced, while the number of newly added state variables was increased. The update range.

[0092] Step S53: Output the first fitted lane line and the second fitted lane line as the final fitted lane line.

[0093] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the electronic device 60 of this application. The electronic device 60 includes a camera 61, a memory 62, and a processor 63. The camera 61, the memory 62, and the processor 63 are respectively coupled to the processor 63. The processor 63 is used to execute program instructions stored in the memory 62 to implement the steps in any of the lane line fitting method embodiments described above. In a specific implementation scenario, the electronic device 60 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 60 may also include mobile devices such as laptops and tablets, without limitation.

[0094] Specifically, processor 63 controls itself and memory 62 to implement the steps of any of the above-described image detection model training method embodiments, or to implement the steps of any of the above-described image detection method embodiments. Processor 63 can also be referred to as a CPU (Central Processing Unit). Processor 63 may be an integrated circuit chip with signal processing capabilities. Processor 63 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 63 can be implemented using integrated circuit chips.

[0095] The above scheme obtains the initial fitted lane line by performing an initial fitting on the simulated lane line point set. When anomalies are identified based on the initial fitted lane line, segmentation points are obtained from the anomaly fitting points. The simulated lane line point set is then segmented and fitted according to the segmentation points to obtain the final fitted lane line. This method can obtain more accurate segmentation points, thereby improving the accuracy of lane line fitting.

[0096] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 70 of this application. The computer-readable storage medium 70 stores program instructions 701 that can be executed by a processor. When the program instructions 701 are executed by the processor, they implement the steps in the above-described lane line fitting method embodiment.

[0097] The above scheme obtains the initial fitted lane line by performing an initial fitting on the simulated lane line point set. When anomalies are identified based on the initial fitted lane line, segmentation points are obtained from the anomaly fitting points. The simulated lane line point set is then segmented and fitted according to the segmentation points to obtain the final fitted lane line. This method can obtain more accurate segmentation points, thereby improving the accuracy of lane line fitting.

[0098] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0099] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0100] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A lane line fitting method, characterized in that, include: Extract the simulated lane line point set from the captured lane image; Map the lane line simulation points in the lane line simulation point set to the vehicle coordinate system to obtain the lane line simulation point set in the vehicle coordinate system. Based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system, the lane line is initially fitted to obtain the initially fitted lane line under the vehicle coordinate system. Obtain the difference value of each lane line simulation point, wherein the difference value is calculated by the position coordinates of each lane line simulation point and the position coordinates of the corresponding fitted lane line simulation point, and the fitted lane line simulation point is calculated based on the lane line simulation point and the initially fitted lane line in the vehicle coordinate system. The lane line simulation points whose difference value is greater than the difference threshold are identified as outlier fitting points, and the number of outlier fitting points is calculated. Based on the number of abnormal fitting points, determine whether the initially fitted lane line has abnormal fitting. If an abnormal fit is determined based on the initially fitted lane line, segmentation points are obtained based on the abnormal fit points. The lane line simulation point set is segmented and fitted according to the segmentation points to obtain the final fitted lane line. The phrase "when it is determined that there is an abnormal fit based on the initially fitted lane line, obtaining segmentation points based on the abnormal fit points" includes: In the event of an outlier in the initial fit, at least one of the outlier points is used as a segmentation point. When the number of outlier points is greater than 1, the step of "taking at least one of the outlier points as a segmentation point" includes: Traverse all outlier points, find the two outlier points with the largest difference and their coordinates in the vehicle coordinate system, and determine the positional relationship of the two outlier points with the largest difference relative to the initially fitted lane line in the vehicle coordinate system. Traverse all lane line simulation points between the two outlier fitting points with the largest difference value, calculate the angle error between two adjacent lane line simulation points, obtain the two adjacent lane line simulation points with the largest angle error, and calculate the difference value between the two adjacent lane line simulation points. When the two outlier fitting points with the largest difference values ​​are located on the same side of the initially fitted lane line in the vehicle coordinate system, the point with the larger difference value among the two adjacent lane line simulation points is used as the segmentation point. When the two outlier fitting points with the largest differences are located on different sides of the initial fitting curve in the vehicle coordinate system, the point with the smaller difference between the two adjacent lane line simulation points is used as the segmentation point.

2. The method according to claim 1, characterized in that, The phrase "based on the lane line simulation points in the lane line simulation point set under the vehicle coordinate system, the lane line is initially fitted to obtain the initially fitted lane line under the vehicle coordinate system" further includes: If there are no outliers in the initial fit, the initially fitted lane line is used as the final fitted lane line and output.

3. The method according to claim 1, characterized in that, The phrase "determining whether the initially fitted lane line has an abnormal fit based on the number of abnormal fitting points" includes: In response to the fact that the number of outlier points is 0, it is determined that there are no outliers in the initial fit. In response to the fact that the number of outlier fitting points is 1, the average difference is calculated, and the relationship between the average difference and the difference threshold is determined. If the average difference is less than 1 / Q of the difference threshold, it is determined that the initially fitted lane line does not have outlier fitting. If the average difference is not less than 1 / Q of the difference threshold, it is determined that the initially fitted lane line has outlier fitting. Wherein, Q is a constant greater than 1, and the average difference is calculated from the difference values ​​corresponding to M lane line simulation points before and after the outlier fitting point, where M is a preset integer. If the number of abnormal fitting points is greater than 1, it is determined that the initially fitted lane line has an abnormal fit.

4. The method according to claim 1, characterized in that, The phrase "performing segmented fitting of the lane line simulation point set according to the segmented points to obtain the final fitted lane line" includes: Based on the Nth-order curve model, the lane line simulation points in the vehicle coordinate system before the segment point are fitted to obtain the first fitted lane line. Establish constraints, and fit the simulated lane lines in the vehicle coordinate system after the segmentation point to obtain the second fitted lane line. The first fitted lane line and the second fitted lane line are output as the final fitted lane line.

5. The method according to claim 4, characterized in that, The constraints include: The coordinates of the segment points conform to the functional relationship of the first fitted lane line and the functional relationship of the second fitted lane line; The function of the second fitted lane line has the same first and second derivatives at the segmentation points as the function of the first fitted lane line.

6. An electronic device, characterized in that, The device includes a camera, a memory, and a processor, wherein the camera, the memory, and the processor are respectively coupled to the processor, and the processor is used to execute program instructions stored in the memory to implement the lane line fitting method according to any one of claims 1 to 5.

7. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the lane line fitting method according to any one of claims 1 to 5.

Images