Lane line processing method, device and equipment and computer readable storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a lane line processing method, device and equipment and a computer readable storage medium. According to the invention, the actual position distribution of the lane line is calculated and analyzed through the point sets corresponding to the plurality of line segments obtained through the recognition of the single-frame road image, and the first point sets with the incidence relation and intersection are merged. The condition that the lane lines in the actual road are fractured or shielded can be repaired through simple linear calculation based on the point sets, a second point set corresponding to each actual lane line is obtained, discrete points in the second point sets are further fitted into a curve, and the position information of the lane lines is obtained. The influence of poor lane line quality on subsequent processing is avoided, the reliability and processing efficiency of lane line processing are improved, and the requirements of continuity and real-time performance of a high-order intelligent driving function are met.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a lane line processing method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of intelligent driving technology, the accurate perception of the surrounding environment of the vehicle, especially the road structure, becomes a crucial link in the process of improving road safety, reducing driving burden and optimizing traffic efficiency.

[0003] Among them, the accuracy of lane line recognition is directly related to the safety and reliability of the intelligent driving system. At present, lane line recognition models can be relied on for lane line recognition and optimization processing.

[0004] However, lane line recognition models are too dependent on the quality of lane lines. If there are factors such as road surface wear, shadow obstruction, water mark interference, etc. that cause lane lines to break, it will directly affect the recognition effect and speed of the lane line recognition model, making it difficult to meet the needs of high-order intelligent driving function continuity and real-time. SUMMARY

[0005] In order to solve the above technical problems, the present disclosure provides a lane line processing method, device, equipment and computer readable storage medium to meet the needs of high-order intelligent driving function continuity and real-time.

[0006] In a first aspect, the embodiments of the present disclosure provide a lane line processing method, comprising:

[0007] lane line recognition is performed on a single frame of road image to obtain a line segment recognition result, the line segment recognition result comprising a first point set corresponding to a plurality of line segments respectively;

[0008] determining the association relationship between the plurality of line segments according to the first point set corresponding to the plurality of line segments respectively, to obtain at least one group of associated line segments corresponding to the plurality of line segments;

[0009] merging the first point set corresponding to the associated line segments to obtain a plurality of second point sets corresponding to the plurality of line segments;

[0010] performing curve fitting on each of the second point sets respectively to obtain a lane line fitting result corresponding to the single frame of road image.

[0011] In a second aspect, the embodiments of the present disclosure provide a lane line processing device, comprising:

[0012] The recognition module is configured to perform lane line recognition on a single frame of road image to obtain a line segment recognition result, the line segment recognition result comprising a first point set corresponding to a plurality of line segments respectively;

[0013] determining a correlation relationship between the plurality of line segments according to the first point sets corresponding to the plurality of line segments respectively, to obtain at least one group of associated line segments corresponding to the plurality of line segments;

[0014] merging the first point sets corresponding to the associated line segments to obtain a plurality of second point sets corresponding to the plurality of line segments;

[0015] fitting a curve for each of the second point sets respectively to obtain a lane line fitting result corresponding to the single-frame road image.

[0016] In a third aspect, an electronic device is provided, and the electronic device comprises:

[0017] a memory;

[0018] a processor; and

[0019] a computer program;

[0020] The computer program is stored in the memory and configured to be executed by the processor to implement the method of the first aspect.

[0021] In a fourth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program. The computer program is executed by a processor to implement the method of the first aspect.

[0022] In a fifth aspect, a computer program product is also provided, and the computer program product comprises a computer program or instructions. The computer program or instructions are executed by a processor to implement the lane line processing method.

[0023] The lane line processing method, device, equipment and computer-readable storage medium provided by the embodiments of the present disclosure can calculate and analyze the point sets corresponding to the plurality of line segments obtained by recognizing the single-frame road image, merge the first point sets having the correlation relationship and intersecting with each other, perform simple linear calculation based on the point sets to repair the lane line breakage or occlusion in the actual road, obtain the second point sets corresponding to each actual lane line respectively, further fit the discrete points in the second point sets into a curve to obtain the position information of the lane line, avoid the influence of poor lane line quality on subsequent processing, improve the reliability and processing efficiency of the lane line processing, and meet the continuity and real-time requirement of high-order intelligent driving functions. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0026] Figure 1 A lane line processing method flowchart provided by the present disclosure;

[0027] Figure 2 A structural schematic diagram of a lane line processing device provided by the present disclosure;

[0028] Figure 3 A structural schematic diagram of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0030] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, not all the embodiments.

[0031] In recent years, intelligent driving technology has developed rapidly, from basic driving assistance systems such as adaptive cruise control and lane departure warning, to higher levels of autonomous driving. The core goal of these systems is to improve road safety, reduce driving burden and optimize traffic efficiency. Whether it is L2 level lane centering, or L3, L4 level structured road autonomous driving (such as high-speed navigation piloting), or even higher order urban navigation piloting functions, all highly depend on the accurate perception of the vehicle to the surrounding environment, especially the road structure. Among them, the lane line as the most direct and authoritative visual identifier of the road boundary and direction, its accurate identification constitutes the absolute cornerstone of the lateral control (the position and direction of the vehicle in the lane) of the intelligent driving system. Whether the boundary lines of the lane and the adjacent lane where the vehicle is located and their curvature and type (such as dashed line, solid line) can be detected in real time and accurately determines whether the vehicle can safely drive in the lane, timely plan a lane changing path, and match the steering wheel angle with the road curvature. Therefore, lane line recognition is a crucial first step in the intelligent driving environment perception system, and its performance is directly related to the function implementation, safety and reliability of the system.

[0032] However, the original lane line detection results obtained by relying on cameras, laser radars or fusion perception technology often have limitations: the detected points (feature points) may have jitter (inter-frame jump), breakage (missing of part of the line segment), position deviation or irregular shape (such as too sparse or dense detected points), especially in complex scenes (such as severe light changes, lane line wear, rain cover, strong shadows, road repair) or multi-sensor data (such as front-facing cameras and surround-view cameras) fusion. If these original and imperfect detection information is directly used in vehicle control modules (such as path planning, steering actuator control), it may cause unstable vehicle trajectory, steering wheel jitter, or in the worst case, misjudgment (such as the vehicle mistakenly thinking that it is deviating from the lane and making unnecessary correction, or failing to recognize deviation from the lane). At this time, the value of lane line post-processing technology is highlighted. It is not just limited to the "detection" level, but shoulders the key tasks of "optimization", "reconstruction", "correlation" and "decision". The post-processing algorithm needs to filter noise points, fill in missing line segments, smooth the detected trajectory, connect discontinuous points into logically coherent lane lines, effectively correlate the detection results on different sensors or time sequences (temporal / spatial consistency processing), and finally output stable, smooth, and clear topological structure (such as lane center line, left and right boundary line) and semantic information (lane type, curvature, drivable area) usable lane information. It can be said that lane line post-processing is the core hub of processing "rough" perception into "finished" control, which is crucial for improving overall system performance, safety and driving experience, and is a key guarantee for realizing the application of various warning, lateral control, automatic navigation and other functions.

[0033] Therefore, overcoming the lane line breakage and missing problem in complex scenes is one of the core challenges of realizing robust perception, and is also a bottleneck for ensuring the continuity and safety of high-level intelligent driving functions. Conventional post-processing methods may have problems such as relying on complex models for time-consuming reasoning, requiring a large amount of prior information for auxiliary speculation, or the repaired results being easily disturbed to have poor shape when facing such breakage, making it difficult to balance the accuracy and processing efficiency of repair at the same time, especially in real-time systems that require high frame rate processing.

[0034] To solve the above problems, the present embodiment provides a lane line processing method, which will be described below in conjunction with specific embodiments.

[0035] Figure 1 A lane line processing method flowchart is provided for the present embodiment. The method can be applied to terminals with data processing functions, such as car machines, smartphones, palm computers, tablets, desktop computers, notebook computers, all-in-one computers, etc. It can be understood that the lane line processing method provided by the present embodiment can also be applied in other scenarios.

[0036] The following will be described in conjunction withFigure 2 The application scenario shown, to Figure 1 The lane line processing method shown is introduced, which includes the following specific steps:

[0037] S101, lane line recognition is performed on a single frame of road image to obtain a line segment recognition result, and the line segment recognition result includes a first point set corresponding to each of a plurality of line segments.

[0038] The single frame of road image refers to the road picture captured by the camera or laser radar device mounted on the vehicle at a certain moment, which can be a single image or a frame of picture in a continuously collected video stream.

[0039] The line segment recognition result is the result of preliminary lane line recognition on a single frame of road image. Due to the fact that the lane line on the actual road surface may be broken, water blocked, etc., the plurality of line segments in the line segment recognition result may belong to the same lane line, or may belong to a plurality of lane lines respectively, and needs to be determined and merged according to subsequent steps.

[0040] Among them, the lane line recognition on the single frame of road image includes: identifying according to the pixel value of the pixel in the image, determining the region with a pixel value gradient greater than a preset gradient threshold as the boundary of the line segment; or inputting the single frame of road image into the segmentation network for image segmentation to obtain the region corresponding to the line segment.

[0041] The lane line in the actual road has a certain width, and the line segment in this step is a physical representation of the actual lane line, which can be the left boundary line, the right boundary line or the center line of the actual lane line, or a line segment fitted according to a predetermined rule, which is not limited by the embodiments of the present disclosure.

[0042] Further, the pixel point information in the image is converted according to the internal parameters of the image acquisition device and the position of the image acquisition point (such as the installation position of the image acquisition device), and the points in the image are mapped to the actual space to obtain a plurality of line segments and a first point set corresponding to each of the plurality of line segments.

[0043] Among them, each line segment has its own first point set, which includes a plurality of sampling points belonging to the line segment, and each sampling point has its own position information, such as the coordinate information (x1, y1) of the sampling point in the vehicle coordinate system, wherein x1 represents the longitudinal distance, which is the distance of the sampling point from the vehicle in the driving direction of the vehicle, and y1 represents the transverse distance, which is the distance of the sampling point from the vehicle in the vertical direction of the driving direction of the vehicle.

[0044] S102, determining the association relationship between the plurality of line segments according to the first point set corresponding to each of the plurality of line segments to obtain at least one group of associated line segments corresponding to the plurality of line segments.

[0045] The association relationship refers to a spatial topological connection relationship between a plurality of line segments that can belong to the same lane line. The plurality of line segments having the association relationship are a group of associated line segments, and correspond to the same lane line on the actual road.

[0046] For a line segment, the position information of each sampling point in the corresponding first point set reflects the position and direction of the line segment. According to the distribution of the plurality of line segments in the spatial position corresponding to the first point set of the plurality of line segments, if there are at least two line segments that meet the preset condition, it is determined that they have an association relationship, and correspond to the same lane line on the actual road.

[0047] S103, merging the first point set corresponding to the associated line segments to obtain a plurality of second point sets.

[0048] The second point set is a merged line segment point set. Each second point set corresponds to a different lane line in the actual road.

[0049] Specifically, if there is no intersection between the plurality of first point sets corresponding to each group of associated line segments, the plurality of first point sets corresponding to each group of associated line segments are merged into a second point set; if there is an intersection between the plurality of first point sets corresponding to at least two groups of associated line segments, the plurality of first point sets corresponding to the at least two groups of associated line segments are merged into a second point set.

[0050] Optionally, each first point set is assigned a unique identifier, referred to as a first identifier.

[0051] If there is no same first identifier in the plurality of first point sets corresponding to each group of associated line segments, it is determined that there is no intersection between the plurality of first point sets corresponding to each group of associated line segments, and the plurality of first point sets corresponding to the same group of associated line segments are merged into a second point set.

[0052] If there is a same first identifier in the plurality of first point sets corresponding to at least two groups of associated line segments, it is determined that there is an intersection between the plurality of first point sets corresponding to the at least two groups of associated line segments, and there is a head-to-tail connection between the at least two groups of associated line segments. The plurality of first point sets corresponding to the at least two groups of associated line segments are merged into a second point set.

[0053] Optionally, each second point set also has a unique identifier for identification, referred to as a second identifier. The second identifier can be the first identifier of the first point set in each second point set, or a new identifier newly assigned.

[0054] S104, performing curve fitting on each second point set respectively to obtain a lane line fitting result corresponding to a single-frame road image.

[0055] The second point set includes a plurality of sampling points and corresponds to a same lane line in an actual road. Curve fitting is performed based on the single second point set to convert the discrete sampling points in the second point set into a continuous curve function, which can represent position information of the lane line in the actual road.

[0056] Curve fitting is performed on each second point set respectively to obtain position information of a plurality of lane lines corresponding to the single frame of road image.

[0057] The plurality of line segments obtained by recognizing the single frame of road image are used to calculate and analyze the actual position distribution of the lane line, the first point sets having a correlation and intersection are merged, and linear calculation based on the point sets can repair the lane line breakage or occlusion in the actual road to obtain the second point set corresponding to each actual lane line. The discrete points in the second point set are further fitted into a curve to obtain the position information of the lane line, which avoids the influence of poor lane line quality on subsequent processing, improves the reliability and processing efficiency of the lane line processing, and meets the continuity and real-time requirements of high-order intelligent driving functions.

[0058] In addition, the lane line repair and merging can be realized by using the single frame of road image, which avoids complex and time-consuming calculation of cross-frame data fusion and frame data jitter, and further improves the real-time and accuracy of the lane line processing.

[0059] In some embodiments, the correlation between the plurality of line segments is determined according to the first point set corresponding to each line segment to obtain at least one group of correlated line segments, including: calculating the angle and distance between the plurality of line segments according to the first point set corresponding to each line segment; if the angle between two line segments is greater than a preset angle threshold and the distance between the two line segments is less than a preset distance threshold, it is determined that the two line segments have a correlation, and the two line segments are a group of correlated line segments.

[0060] The angle is used to measure the direction continuity between the two line segments, and the distance is used to measure the spatial deviation between the two line segments. If the angle between the two line segments is determined to be parallel and the deviation is small, it is determined that the two line segments have a correlation.

[0061] The angle and distance between the plurality of line segments are calculated according to the first point set corresponding to the plurality of line segments, including: determining the start point and end point of a first line segment in the plurality of line segments and the start point of a second line segment in the plurality of line segments according to the first point set corresponding to the plurality of line segments; calculating the included angle of a first vector and a second vector to obtain the angle between the first line segment and the second line segment, the first vector being from the end point of the first line segment to the start point of the first line segment, and the second vector being from the end point of the first line segment to the start point of the second line segment; and calculating the perpendicular distance from the start point of the second line segment to the first line segment to obtain the distance between the first line segment and the second line segment.

[0062] The first line segment and the second line segment are any two line segments in the plurality of line segments. According to the first point set corresponding to the first line segment, the start point and the end point corresponding to the first line segment can be determined; and according to the first point set corresponding to the second line segment, the start point and the end point corresponding to the second line segment can be determined.

[0063] For example, the start point a (x1, y1) and the end point b (x2, y2) of the first line segment, and the start point c (x3, y3) of the second line segment are constructed into a first vector and a second vector

[0064]

[0065] The dot product of the first vector and the second vector is calculated:

[0066]

[0067] The lengths of the first vector and the second vector are calculated:

[0068]

[0069] The included angle of the first vector and the second vector is calculated according to the dot product and the length:

[0070]

[0071] The angle can represent the angle between the first line segment and the second line segment.

[0072] Based on the above method, the angle between each two line segments in the plurality of line segments is calculated.

[0073] The perpendicular distance d from the start point of the second line segment to the first line segment is calculated as follows:

[0074]

[0075] The perpendicular distance can represent the distance between the first line segment and the second line segment.

[0076] Based on the above method, the distance between each two line segments in the plurality of line segments is calculated.

[0077] If the angle between the first line segment and the second line segment is greater than the preset angle threshold, it represents that the first line segment and the second line segment are close to parallel; if the distance between the first line segment and the second line segment is less than the preset distance threshold, it represents that the deviation between the first line segment and the second line segment is small; if the above conditions are met at the same time, it represents that the first line segment and the second line segment have a greater probability of corresponding to the same lane line.

[0078] Optionally, the angle between each two line segments in the plurality of line segments is calculated first, and a line segment group greater than the preset angle threshold is screened out; the distance between two line segments in each line segment group is further calculated, and a line segment group with a distance less than the preset threshold is determined as a group of associated line segments.

[0079] Alternatively, the distance between each two line segments in the plurality of line segments is calculated first, and a line segment group less than the preset distance threshold is screened out; the angle between two line segments in each line segment group is further calculated, and a line segment group with an angle greater than the preset angle threshold is determined as a group of associated line segments.

[0080] Optionally, before the start point and the end point of the first line segment in the plurality of line segments and the start point of the second line segment in the plurality of line segments are determined according to the first point set corresponding to the plurality of line segments, the method further comprises: determining the distance between the plurality of line segments and the sampling point according to the sampling point closest to the image acquisition point of the single frame road image in each first point set; sorting the plurality of line segments according to the distance between the plurality of line segments and the sampling point to obtain the sorted plurality of line segments; and selecting two line segments with an order difference less than a preset difference value in the sorted plurality of line segments as the first line segment and the second line segment.

[0081] The sampling point closest to the image acquisition point of the single frame road image in each first point set is the sampling point with the minimum x coordinate value in the first point set.

[0082] The plurality of line segments are sorted according to the distance between the plurality of line segments and the sampling point, and in the sorted plurality of line segments, two line segments with an order difference less than a preset difference value are closer in actual space, and have a greater probability of corresponding to the same lane line, so only two line segments with an order difference less than a preset difference value are selected for angle and distance calculation, thereby reducing the selection range of line segments from all line segments to a few line segments with an order difference within a preset difference value, thereby reducing the calculation amount and improving the calculation efficiency.

[0083] For two line segments with an order difference greater than or equal to a preset difference value, if they can meet the conditions of the preset angle threshold and the preset distance threshold, they can also be merged into the same second point set through the point set merging process, which will not affect the accuracy of lane line processing.

[0084] In addition, the lane line breaking area is intelligently positioned and repaired by using the geometric characteristics of the point set itself, so that the lane line misjudgment caused by shadows, wear and tear and the like can be reduced, and the accuracy of lane line processing can be improved.

[0085] On the basis of any of the above embodiments, the single-frame road image is obtained by image acquisition of a road on which the vehicle travels during vehicle driving, and curve fitting is performed on each second point set to obtain a lane line fitting result corresponding to the single-frame road image, including: for each second point set, a longitudinal distance matrix is constructed, the number of rows of the longitudinal distance matrix is the number of sampling points in the second point set, and the elements in each row are powers of the longitudinal distances of the sampling points, the longitudinal distance being the distance of the sampling point from the vehicle in the driving direction of the vehicle; a transverse distance vector is constructed according to the transverse distances of the sampling points in the second point set, the longitudinal distance being the distance of the sampling point from the vehicle in the direction perpendicular to the driving direction of the vehicle; vector solving is performed based on the longitudinal distance matrix and the transverse distance vector to obtain a coefficient vector, the number of elements in the coefficient vector being the same as the number of elements in each row of the longitudinal distance matrix; the elements in the coefficient vector are taken as the coefficients of the curve equation to obtain the curve equation corresponding to the second point set.

[0086] The number of columns in the longitudinal distance matrix depends on the number of terms required for the curve equation, the more the number of terms of the curve equation, the more accurate the description of the curve, but at the same time the calculation amount will be increased; the fewer the number of terms of the curve equation, the lower the calculation amount, but the accuracy of the curve description will decrease.

[0087] According to the requirements for the accuracy of curve description and the calculation speed in actual situations, the number of terms of the curve equation and the number of columns in the longitudinal distance matrix are determined.

[0088] Taking the number of terms as 4, the longitudinal distance matrix can be represented as:

[0089]

[0090] Where x represents the longitudinal distance of the sampling point in the second point set, and n is the number of sampling points in the second point set.

[0091] The transverse distance vector is represented as:

[0092]

[0093] The coefficient vector is represented as:

[0094] a=[a0,a1,a2,a3] T

[0095] Where a0, a1, a2, a3 all represent the coefficients of the curve equation.

[0096] According to the least square method, the normal equation is constructed based on the longitudinal distance matrix, the transverse distance vector and the coefficient vector:

[0097] A T Aa=A T B

[0098] Solving the normal equation can obtain the value of each element in the coefficient vector, that is, the coefficients of the curve equation.

[0099] Wherein:

[0100]

[0101] a=(A T A) -1 A T B=VΣ + U + B

[0102] Wherein, U, V are orthogonal matrices, Σ is a singular value diagonal matrix, Σ + is the pseudo-inverse matrix of the singular value diagonal matrix, which is obtained by taking the reciprocal of the non-zero singular values in the singular value diagonal matrix.

[0103] The non-diagonal elements in the singular value diagonal matrix are all 0, and the diagonal elements are singular values, which are the square roots of the eigenvalues of A T A, used to optimize the structure of A T A and simplify the calculation process.

[0104] The solved coefficient vector is represented as:

[0105] a=[c0,c1,c3,c3] T

[0106] Wherein c0, c1, c2, c3 are the values of the coefficients of the curve equation, and the curve equation corresponding to the second point set is represented as:

[0107] y=c0+c1x+c2x 2 +c3x 3

[0108] Further, the curve equation is sampled according to the preset interval distance to obtain the lane line point set corresponding to the second point set.

[0109] The preset interval distance is determined based on the requirements of lane line maintenance, path planning and other functions, so that each function can determine the position of the lane line in the road based on the finally obtained lane line point set.

[0110] According to the preset interval distance, a plurality of values of x in the curve equation are determined, and a plurality of corresponding y values are obtained, that is, the position coordinates of each lane line point in the lane line point set are determined.

[0111] The second point set is fitted into a curve equation capable of describing the position and direction of the lane line through a geometric calculation method, and a point is taken on the curve equation to obtain a lane line point set according to the requirement of the vehicle control function, thereby providing the vehicle control function downstream with geometrically accurate and continuous lane structure information, and guaranteeing the reliability of the intelligent driving function.

[0112] Figure 2 A structural schematic diagram of a lane line processing apparatus provided by the embodiments of the present disclosure is provided. The lane line processing apparatus can be a terminal with a data processing function as described in the above embodiments, or the lane line processing apparatus can be a component or assembly in the terminal. The lane line processing apparatus provided by the embodiments of the present disclosure can execute the processing procedure provided by the lane line processing method embodiments, as shown in Figure 2 The lane line processing apparatus 20 includes an identification module 21, a determination module 22, a merging module 23, and a fitting module 24. The identification module 21 is configured to perform lane line identification on a single frame of road image to obtain a line segment identification result, wherein the line segment identification result includes a first point set corresponding to each of a plurality of line segments. The determination module 22 is configured to determine a correlation relationship between the plurality of line segments according to the first point set corresponding to each of the plurality of line segments, to obtain at least one group of correlated line segments corresponding to the plurality of line segments. The merging module 23 is configured to merge the first point set corresponding to the correlated line segments to obtain a plurality of second point sets corresponding to the plurality of line segments. The fitting module 24 is configured to perform curve fitting on each of the second point sets, to obtain a lane line fitting result corresponding to the single frame of road image.

[0113] Optionally, the determination module 22 includes a calculation unit and a first determination unit. The calculation unit is configured to calculate an angle and a distance between the plurality of line segments according to the first point set corresponding to each of the plurality of line segments. The first determination unit is configured to determine that there is a correlation relationship between two line segments if the angle between the two line segments is greater than a preset angle threshold and the distance between the two line segments is less than a preset distance threshold, and the two line segments are a group of correlated line segments.

[0114] Optionally, the calculation unit is specifically configured to determine a start point and an end point of a first line segment in the plurality of line segments and a start point of a second line segment in the plurality of line segments according to the first point set corresponding to each of the plurality of line segments, calculate an included angle between a first vector and a second vector to obtain an angle between the first line segment and the second line segment, wherein the first vector points from the end point of the first line segment to the start point of the first line segment, and the second vector points from the end point of the first line segment to the start point of the second line segment, and calculate a perpendicular distance from the start point of the second line segment to the first line segment to obtain a distance between the first line segment and the second line segment.

[0115] Optionally, the determining module 22 further comprises a second determining unit configured to determine distances between the plurality of line segments and sampling points closest to the image acquisition points of the single-frame road image in each of the first point sets; sort the plurality of line segments according to the distances between the plurality of line segments and the sampling points to obtain sorted plurality of line segments; and select two line segments with an order difference less than a preset difference value from the sorted plurality of line segments as the first line segment and the second line segment.

[0116] Optionally, the merging module 23 is specifically configured to merge the plurality of first point sets corresponding to each group of associated line segments into second point sets respectively if there is no intersection between the plurality of first point sets corresponding to each group of associated line segments; and merge the plurality of first point sets corresponding to at least two groups of associated line segments into a second point set if there is an intersection between the plurality of first point sets corresponding to the at least two groups of associated line segments.

[0117] Optionally, the single-frame road image is obtained by image acquisition of a road on which a vehicle travels during vehicle driving. The fitting module 24 comprises a first constructing unit, a second constructing unit, a solving unit and an obtaining unit. The first constructing unit is configured to construct a longitudinal distance matrix for each of the second point sets, the number of rows of the longitudinal distance matrix being the number of sampling points in the second point set, and the elements in each row being the longitudinal distance of the sampling point raised to a power, the longitudinal distance being the distance of the sampling point from the vehicle in the driving direction of the vehicle. The second constructing unit is configured to construct a transverse distance vector according to the transverse distance of the sampling points in the second point set, the transverse distance being the distance of the sampling point from the vehicle in a direction perpendicular to the driving direction of the vehicle. The solving unit is configured to perform vector solving based on the longitudinal distance matrix and the transverse distance vector to obtain a coefficient vector, the number of elements in the coefficient vector being the same as the number of elements in each row of the longitudinal distance matrix. The obtaining unit is configured to obtain the elements in the coefficient vector as coefficients of a curve equation to obtain a curve equation corresponding to the second point set.

[0118] Optionally, the lane line processing apparatus 20 further comprises a sampling module configured to sample the curve equation according to a preset interval distance to obtain a lane line point set corresponding to the second point set.

[0119] Figure 2 The lane line processing apparatus of the illustrated embodiment can be used to execute the technical solutions of the method embodiments described above, and has similar implementation principles and technical effects, which will not be described herein again.

[0120] Figure 3A structural schematic diagram of an electronic device is provided in the embodiments of the present disclosure. The electronic device can be a terminal with a data processing function as described in the above embodiments. The electronic device provided in the embodiments of the present disclosure can execute the processing procedure provided in the lane line processing method embodiments, as shown in Figure 3 The electronic device 30 includes a memory 31, a processor 32, a computer program and a communication interface 33, as shown in the figure. The computer program is stored in the memory 31 and is configured to be executed by the processor 32 to implement the lane line processing method as described above.

[0121] In addition, the embodiments of the present disclosure also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the lane line processing method described in the above embodiments.

[0122] In addition, the embodiments of the present disclosure also provide a computer program product, which includes a computer program or instructions. The computer program or instructions are executed by a processor to implement the lane line processing method as described above.

[0123] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element preceded by "comprises... " does not, without more limitations, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the recited element.

[0124] The above description is merely one specific implementation of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A lane marking processing method, characterized in that, The method includes: Lane line recognition is performed on a single frame of road image to obtain line segment recognition results, which include a first set of points corresponding to multiple line segments respectively; The association relationship between the multiple line segments is determined based on the first set of points corresponding to each of the multiple line segments, thereby obtaining at least one set of associated line segments corresponding to the multiple line segments; The first set of points corresponding to the associated line segments is merged to obtain multiple second set of points corresponding to the multiple line segments; For each of the second point sets, curve fitting is performed to obtain the lane line fitting result corresponding to the single-frame road image.

2. The method according to claim 1, characterized in that, The step of determining the association relationship between the multiple line segments based on the first set of points corresponding to each of the multiple line segments, and obtaining at least one set of associated line segments corresponding to the multiple line segments, includes: Calculate the angles and distances between the multiple line segments based on the first set of points corresponding to each line segment; If the angle between two line segments is greater than a preset angle threshold and the distance is less than a preset distance threshold, then it is determined that there is an association between the two line segments, and the two line segments are a set of associated line segments.

3. The method according to claim 2, characterized in that, The step of calculating the angles and distances between the multiple line segments based on the first set of points corresponding to each line segment includes: Based on the first set of points corresponding to the multiple line segments, determine the starting point and ending point of the first line segment and the starting point of the second line segment among the multiple line segments; Calculate the angle between the first vector and the second vector to obtain the angle between the first line segment and the second line segment. The first vector points from the end point of the first line segment to the beginning point of the first line segment, and the second vector points from the end point of the first line segment to the beginning point of the second line segment. Calculate the perpendicular distance from the starting point of the second line segment to the first line segment to obtain the distance between the first line segment and the second line segment.

4. The method according to claim 3, characterized in that, Before determining the starting point and ending point of the first line segment and the starting point of the second line segment based on the first set of points corresponding to the multiple line segments, the method further includes: Based on the sampling point that is closest to the image acquisition point of the single-frame road image in each of the first point sets, the distance between the multiple line segments and the sampling point is determined; Based on the distance between the multiple line segments and the sampling points, the multiple line segments are sorted to obtain sorted multiple line segments; Two line segments whose order difference is less than a preset difference are selected from the sorted line segments as the first line segment and the second line segment.

5. The method according to claim 1, characterized in that, The step of merging the first set of points corresponding to the associated line segments to obtain multiple second set of points corresponding to the multiple line segments includes: If there is no intersection between the multiple first point sets corresponding to each group of associated line segments, then the multiple first point sets corresponding to each group of associated line segments are merged into a second point set. If there is an intersection between multiple sets of first points corresponding to at least two sets of the associated line segments, then the multiple sets of first points corresponding to at least two sets of the associated line segments are merged into a second set of points.

6. The method according to claim 1, characterized in that, The single-frame road image is obtained by capturing images of the road where the vehicle is located during vehicle travel. The process of performing curve fitting for each of the second point sets to obtain the lane line fitting result corresponding to the single-frame road image includes: For each of the second point sets, a longitudinal distance matrix is ​​constructed, wherein the number of rows in the longitudinal distance matrix is ​​the number of sampling points in the second point set, and the elements in each row are powers of the longitudinal distance of the sampling point, wherein the longitudinal distance is the distance between the sampling point and the vehicle in the direction of the vehicle's travel. A lateral distance vector is constructed based on the lateral distance of the second set of sampling points, wherein the longitudinal distance is the distance between the sampling point and the vehicle in the direction perpendicular to the vehicle's driving direction; Based on the vertical distance matrix and the horizontal distance vector, a vector solution is performed to obtain a coefficient vector, wherein the number of elements in the coefficient vector is the same as the number of elements in each row of the vertical distance matrix; The elements in the coefficient vector are used as coefficients of the curve equation to obtain the curve equation corresponding to the second point set.

7. The method according to claim 6, characterized in that, The method further includes: The curve equation is sampled according to a preset interval distance to obtain the lane line point set corresponding to the second point set.

8. A lane marking processing device, characterized in that, include: The recognition module is used to identify lane lines in a single frame of road image and obtain line segment recognition results, wherein the line segment recognition results include a first set of points corresponding to multiple line segments respectively; The determining module is used to determine the association relationship between the multiple line segments based on the first set of points corresponding to the multiple line segments respectively, and to obtain at least one set of associated line segments corresponding to the multiple line segments; The merging module is used to merge the first set of points corresponding to the associated line segments to obtain multiple second set of points corresponding to the multiple line segments; The fitting module is used to perform curve fitting for each of the second point sets to obtain the lane line fitting result corresponding to the single-frame road image.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images