Lane line detection method, device and storage medium

Abstract

The application discloses a lane line detection method and device and a storage medium, and comprises the following steps: performing lane line detection on a pretreated road image to obtain each lane line instance; performing processing on an original road image based on each lane line instance to obtain a connected domain of each lane line; calculating the attribute of the connected domain of each lane line; counting the number of connected domains satisfying an attribute threshold condition in each lane line according to the attribute of the connected domain of each lane line; determining the lane line type according to the number of connected domains satisfying the attribute threshold condition in each lane line; extracting the centroid of the connected domain satisfying the attribute threshold condition in a dashed lane line, and sorting each connected domain according to the coordinate size of the centroid; and screening a plurality of connected domains which are continuous in space according to the centroid of two adjacent connected domains to realize positioning of each dashed block in the dashed lane line. The application realizes lane line type discrimination and accurate positioning of dashed blocks on the basis of lane line detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent transportation, and particularly relates to a lane line detection method, device and storage medium. BACKGROUND

[0002] In an intelligent traffic monitoring system, high-precision lane line detection, as a core link of road network digital perception, not only provides a spatial reference for vehicle trajectory tracking and behavior prediction, but also is a key technical prerequisite for realizing judicial evidence of illegal behavior (such as illegal lane changing and parking on the road). The traditional monitoring system generally adopts a static lane template matching scheme to realize the detection function by manually presetting the lane line position parameters in the image coordinate system. However, when the deployment object is a ball camera (PTZ Camera) with pan-tilt-zoom control function, this scheme faces the severe challenge of dynamic view adaptability: the camera actively adjusts the monitoring view angle by horizontal rotation (Pan), vertical tilt (Tilt) and optical zoom (Zoom) during operation, resulting in dynamic transformation of the projection relationship between the imaging plane and the road plane. Specifically:

[0003] (1) Spatial mapping misalignment: PTZ operation changes the camera extrinsic matrix and intrinsic matrix, causing nonlinear changes in the projection equation from the world coordinate system to the image coordinate system, which directly causes the lane line position to shift in the image.

[0004] (2) Topological structure distortion: under dynamic view angle, the topological properties of lane lines such as curvature and bifurcation points show significant differences in image space, for example, wide-angle end (Zoom-out) will compress the lane line curvature, while telephoto end (Zoom-in) will enlarge the local deformation.

[0005] The above characteristics make traditional detection methods that rely on fixed regions of interest (Region of Interest, ROI) or offline calibration perspective priors likely to fail due to changes in camera parameters. Therefore, developing a lane line detection algorithm with view angle adaptability has become an inevitable technical path for intelligent traffic monitoring systems to meet the dynamic monitoring needs of PTZ devices.

[0006] From the algorithm implementation principle, lane line detection algorithms can be divided into two categories: traditional methods and deep learning methods. Traditional rule-based lane line detection algorithms (such as color threshold segmentation, Canny edge detection combined with Hough transform) rely on artificially designed feature extraction rules, and their performance degrades significantly in complex scenarios such as sudden changes in light, worn-out markings, or vehicle occlusion. With the development of deep learning technology, detection algorithms based on segmentation (such as LaneNet model), line classification (such as UFAST (Ultra-Fast-Lane-Detection, Ultra-Fast-Lane-Detection)) and parameterized modeling (such as PolyLaneNet) gradually replace traditional methods. However, existing deep learning solutions mainly focus on the near-field perception scenarios of autonomous driving, and there are two major limitations in their design:

[0007] (1) Insufficient spatial coverage: The detection range is usually limited to 3-5 lane lines around the current vehicle, making it difficult to meet the panoramic detection needs of traffic monitoring.

[0008] (2) Lack of semantic understanding: Most algorithms only output lane line positions, lacking the ability to distinguish lane line types (such as solid and dashed lines) and accurately label the positions of special areas (such as dashed line blocks and guide areas).

[0009] Although some studies attempt to combine classification networks to identify different types of lane lines, they rely on large-scale datasets containing fine-grained annotations for training. In actual engineering practice, the differences in road marking styles in different regions require frequent re-labeling and re-training of models, resulting in high adaptation costs.

[0010] For example, the patent document with publication number CN117612126A discloses a lane line detection method, which involves performing connected component analysis on each lane line binary image based on type labels. If the shape of the connected component in the lane line binary image matches a pre-set target shape, the connected component in the intersection position of the corresponding lane line binary image is disconnected. Then, based on the lane line binary image after the connected component is disconnected at the intersection position, feature points are sampled at a pre-set interval to obtain lane line feature points corresponding to each type of lane line. Finally, based on the lane line feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated. This method relies on fine-grained lane line type label data for lane line type identification, which poses a problem of high adaptation costs due to differences in road marking styles in different regions in actual engineering practice. Additionally, this method fails to distinguish intersecting lane line instances based on a pre-set target shape (Y / V shape), which is ineffective in complex topologies such as crossroads and T-shaped intersections. It also lacks robustness in complex environments such as occlusion and wear. SUMMARY

[0011] The purpose of this invention is to provide a lane line detection method, device, and storage medium to solve the problems that existing deep learning methods cannot meet the requirements of panoramic detection, lack the ability to distinguish lane line types, and lack accurate location calibration of special areas.

[0012] This invention solves the above-mentioned technical problems through the following technical solution: a lane line detection method, comprising:

[0013] Lane line detection is performed on the preprocessed road image to obtain lane line instances;

[0014] Based on each lane line instance, the original road image is processed to obtain the connected components of each lane line;

[0015] Calculate the properties of the connected components for each lane line;

[0016] The number of connected components in each lane that meet the attribute threshold conditions is counted based on the attributes of the connected components of each lane.

[0017] The lane type is determined based on the number of connected components in each lane that satisfy the attribute threshold condition;

[0018] Extract the centroids of connected components in the dashed lane lines that satisfy the attribute threshold conditions, and sort the connected components according to the coordinates of their centroids.

[0019] By selecting multiple consecutive connected regions in the space based on the centroids of two adjacent connected regions, the positioning of each dashed line block in the dashed lane line can be achieved.

[0020] Furthermore, the lane line detection of the preprocessed road image specifically includes:

[0021] A pre-trained deep learning model is used to detect lane lines in the pre-processed road image, resulting in a binary segmentation map of lane lines and background and a pixel embedding feature vector map.

[0022] The lane line and background binary segmentation map are used as a spatial mask, and a bitwise AND operation is performed with the pixel embedding feature vector map to filter out the lane line pixel embedding feature vectors. The lane line pixel embedding feature vectors are then clustered using a density clustering algorithm to obtain each lane line instance.

[0023] Furthermore, based on each lane line instance, the original road image is processed, specifically including:

[0024] Based on each lane line instance, construct a binary segmentation map of each lane line and the background;

[0025] For each binary segmentation image of lane lines and background, perform a morphological dilation operation on it;

[0026] The binary segmentation map of each lane line after the dilation operation and the background is used as a spatial mask, and a bitwise AND operation is performed with the original road image to extract the region of each lane line from the original road image.

[0027] The region of each lane line is converted to grayscale, filtered, and edge detected to obtain the edge of each lane line;

[0028] Perform a morphological closing operation on the edge of each lane line to obtain the closed region of each lane line;

[0029] Fill the closed region of each lane line to obtain the connected region of each lane line.

[0030] Furthermore, the properties of the connected region of each lane line include area, rectangularity, and aspect ratio, and the formula for calculating the rectangularity of the connected region is:

[0031] ;

[0032] in, Represents the rectangularity of the connected components; Represents the area of ​​the connected region; This represents the area of ​​the smallest bounding rectangle of the connected region.

[0033] Furthermore, multiple consecutive connected components in space are selected based on the centroids of two adjacent connected components, specifically including:

[0034] Calculate the centroid distance between two adjacent connected components;

[0035] Calculate the length of the longest side of the minimum bounding rectangle of one of two adjacent connected components;

[0036] Based on the centroid spacing and the length of the long side, it is determined whether the two dashed blocks corresponding to the two adjacent connected regions are spatially adjacent, thereby filtering out multiple spatially continuous connected regions.

[0037] Furthermore, the detection method also includes flow guidance area detection, specifically including:

[0038] All lane lines are sorted from left to right according to their position in the original road image;

[0039] For each of two adjacent lane lines, sort the vertical coordinates of all its pixels in descending order to obtain the common interval of the two adjacent lane lines on the vertical coordinate axis.

[0040] Calculate the rate of change of the spacing between two adjacent lane lines;

[0041] The existence of a diversion area is determined based on the rate of change of the spacing between two adjacent lane lines, and boundary key points are extracted from the common interval of the two adjacent lane lines with diversion areas on the vertical coordinate axis. The boundary of the diversion area is determined based on the boundary key points.

[0042] Furthermore, based on the position of each lane line in the original road image, all lane lines are sorted from left to right, specifically including:

[0043] For each lane line, extract the vertical coordinates of its bottommost pixel;

[0044] Sort the vertical coordinates of the bottommost pixels of all lane lines in descending order and generate a vertical hierarchy sequence;

[0045] Construct a first horizontal reference line at the first vertical coordinate of the vertical hierarchy sequence, calculate the first intersection point of the first horizontal reference line with all lane lines, and sort all lane lines in ascending order of the horizontal coordinates of the first intersection points to achieve the initial sorting of lane lines from left to right.

[0046] For unsorted lane lines, process them sequentially in descending order of their vertical coordinates as follows:

[0047] Construct a second horizontal reference line at the vertical coordinate of the bottom pixel of the lane line, calculate the second intersection point of the second horizontal reference line and the sorted lane lines, and insert the lane line to the left or right of the sorted lane lines according to the horizontal coordinate of the bottom pixel of the lane line and the horizontal coordinate of the second intersection point, so as to sort all lane lines from left to right.

[0048] Furthermore, the rate of change of spacing between two adjacent lane lines is calculated, specifically including:

[0049] Within the shared interval of two adjacent lane lines on the vertical coordinate axis, sample the pixel set of the left lane line along the vertical coordinate axis at certain intervals. For each sampled point of the left lane line, search for a suitable pixel in the right lane line among the two adjacent lane lines. And the pixel closest to the sampling point of the left lane line This forms a pair of matching points;

[0050] The lateral spacing of each matching point pair is calculated using the following formula:

[0051] ;

[0052] in, This represents the i-th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy pixels, Indicates the spacing threshold; This represents the i-th matching point pair between two adjacent lane lines. , The horizontal spacing between them;

[0053] The rate of change of spacing between adjacent vertical positions is calculated based on the lateral spacing of each matching point pair. The specific calculation formula is as follows:

[0054] ;

[0055] in, This represents the rate of change of the spacing between the i-th adjacent vertical positions; Indicates matching point pairs , Horizontal spacing between them This represents the (i+1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels; Indicates matching point pairs , Horizontal spacing between them This represents the (i-1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels;

[0056] The rate of change of the spacing between the i-th adjacent vertical positions is normalized.

[0057] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the lane line detection method as described above.

[0058] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the lane line detection method as described above.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] This invention achieves lane line instance-level detection by performing pixel detection and clustering on road images. Then, it classifies lane lines as real or virtual by statistically analyzing the attributes of the connected components of each lane line. Finally, it locates the dashed line blocks within the dashed lane lines by extracting the centroids of the connected components. This method of lane line classification and dashed line block localization based on the geometric features of connected components eliminates the need for training on large-scale datasets with fine-grained annotations, thus avoiding the increased adaptation costs caused by frequent re-annotation and retraining required by traditional lane line classification methods.

[0061] The present invention also sorts the detected lane lines and detects the guiding area by calculating the rate of change of the spacing between adjacent lane lines. It can sensitively capture the emission or convergence characteristics of lane lines, thereby accurately identifying the guiding area.

[0062] Compared to existing technologies (i.e., patent document with publication number CN117612126A), this invention designs an adaptive classification and localization strategy for virtual and real lane lines based on the geometric feature differences of the connected domains of virtual and real lane lines. This strategy does not rely on lane line type label data, solving the problem of high adaptation costs caused by differences in road marking styles in different regions in actual engineering practice. In addition, this invention generates a binary classification map of lane lines and background and high-dimensional pixel embedding features through a deep learning model. Through a density clustering algorithm, the feature distance of pixels belonging to the same lane line is close, while the feature distance of pixels belonging to different lane lines is far, automatically completing the separation of lane line instances. The pixel belonging is determined by the feature space distance, without the need for preset shape rules. It has high generalization performance in complex environments and does not require additional annotation costs, maintaining effectiveness in complex topologies. Attached Figure Description

[0063] To more clearly illustrate the technical solution of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of the lane line detection method in an embodiment of the present invention;

[0065] Figure 2 This is a lane line label diagram in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the regions of each lane line extracted in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the connected regions of each lane line extracted in an embodiment of the present invention;

[0068] Figure 5 This is a diagram illustrating the classification of lane lines and the centroid positioning of dashed line blocks in an embodiment of the present invention.

[0069] Figure 6 This is a schematic diagram of the common section of two adjacent lane lines on the vertical coordinate axis in an embodiment of the present invention;

[0070] Figure 7 This is a diagram showing the detection effect of the flow guiding area in an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0073] Example 1

[0074] Traditional lane detection methods rely on manually designed feature extraction rules, which are easily affected by the external environment, leading to a significant decrease in detection accuracy in complex scenes. While deep learning methods overcome these limitations, they still heavily depend on labeled data containing relevant features when performing functions such as classification and localization.

[0075] To address the aforementioned technical problems, this invention provides a lane line detection method that combines deep learning feature extraction with traditional image analysis. This method retains the ability of deep learning to express complex features while incorporating the interpretability of traditional image analysis. It can not only detect and classify lane lines but also accurately locate special regions.

[0076] Figure 1 A flowchart of the lane line detection method provided by an embodiment of the present invention is shown. Figure 1 As shown, the lane line detection method includes the following steps:

[0077] Step 1: Perform lane line detection on the preprocessed road image to obtain lane line instances.

[0078] The road image captured by the image acquisition device is read, and the road image (i.e., the original road image) is preprocessed to obtain the preprocessed road image. In this embodiment, the preprocessing includes scaling to a fixed size and numerical normalization.

[0079] In a specific embodiment of the present invention, lane line detection is performed on the preprocessed road image, specifically including:

[0080] Step 1.1: Use a pre-trained deep learning model to detect lane lines in the preprocessed road image, and obtain a binary segmentation map of lane lines and background and a pixel embedding feature vector map.

[0081] In this embodiment, the deep learning model is a neural network, and the pre-training process of the deep learning model is as follows:

[0082] Construct a sample dataset, where each sample is a preprocessed road image and its corresponding lane label map (e.g., ...). Figure 2 (As shown); load the deep learning model, and train the deep learning model using the sample dataset to obtain the pre-trained deep learning model.

[0083] In the binary segmentation map of lane lines and background, 0 represents the background and 1 represents the lane line; the pixel embedding feature vector map is a multi-channel feature map (usually with an embedding dimension of 4). Each pixel position (including background pixels) in the pixel embedding feature vector map corresponds to a low-dimensional embedding feature vector, which is used to distinguish different lane line instances.

[0084] Step 1.2: Use the binary segmentation map of lane lines and background as a spatial mask, perform a bitwise AND operation with the pixel embedding feature vector map, filter out the pixel embedding feature vectors of lane lines (i.e., the pixel embedding feature vectors classified as lane lines), and use the density clustering algorithm to cluster the pixel embedding feature vectors of lane lines to obtain each lane line instance.

[0085] Density clustering algorithms use a clustering loss function to ensure that pixels belonging to the same lane are close together, while pixels belonging to different lanes are far apart, thus achieving lane detection.

[0086] Since the number of clusters does not need to be determined before clustering, any number of lane lines can be detected without predefining the number of lane lines. Furthermore, this invention enables full-image analysis of road images, solving the problem of difficulty in meeting panoramic detection requirements.

[0087] Step 2: Based on each lane line instance, process the original road image to obtain the connected components of each lane line.

[0088] In a specific embodiment of the present invention, the original road image is processed based on each lane line instance, specifically including:

[0089] Step 2.1: Based on each lane line instance, construct a binary segmentation map of each lane line and the background; for each lane line and background binary segmentation map, perform a morphological dilation operation on it;

[0090] Step 2.2: Using the binary segmentation image of each lane line and the background after the dilation operation as a spatial mask, perform a bitwise AND operation with the original road image to extract the region of each lane line from the original road image, such as... Figure 3 As shown.

[0091] Step 2.3: Perform grayscale conversion, filtering, and edge detection on the region of each lane line to obtain the edge of each lane line;

[0092] Step 2.4: Perform a morphological closing operation on the edge of each lane line to obtain the closed region of each lane line;

[0093] Step 2.5: Fill the closed region of each lane line to obtain the connected region of each lane line.

[0094] Considering the fundamental shape differences between dashed and solid lane lines, grayscale conversion, Gaussian filtering, and Canny edge detection are performed on the region of each lane line to obtain its edges. Then, a morphological closing operation (i.e., dilation followed by erosion) is performed to form closed regions at the lane line edges. Finally, these closed regions are filled, resulting in the connected components of each lane line. Figure 4 As shown.

[0095] Step 3: Calculate the properties of the connected components for each lane line.

[0096] In this embodiment, the properties of the connected region of each lane line include area, rectangularity, and aspect ratio. The formula for calculating rectangularity is:

[0097] (1)

[0098] in, Represents the rectangularity of the connected components; Represents the area of ​​the connected region; This represents the area of ​​the smallest bounding rectangle of the connected component. The closer the rectangularity value is to 1, the closer the shape of the connected component is to a rectangle.

[0099] Step 4: Count the number of connected components in each lane that meet the attribute threshold conditions based on the attributes of the connected components of each lane.

[0100] Generally, compared to solid lane lines, dashed lane lines have more connected components with smaller areas, and each connected component is a regular rectangular block. This characteristic allows for the distinction between solid and dashed lane lines. Therefore, for each attribute of a connected component, the corresponding attribute threshold condition is:

[0101] (2)

[0102] in, and Let represent the minimum and maximum areas of the connected components, respectively. and These represent the minimum and maximum values ​​of the rectangularity of the connected components, respectively. and These represent the minimum and maximum aspect ratios of the connected components, respectively. This represents the aspect ratio of the connected components. In this embodiment, Set to 30, Set to 2000. Set it to 0.45. Set to 1, Set it to 3.5. Set it to 20.

[0103] Step 5: Determine the lane type based on the number of connected components in each lane that meet the attribute threshold conditions.

[0104] When the number of connected components satisfying the attribute threshold condition in each lane exceeds the quantity threshold, the lane is determined to be a dashed lane; when the number of connected components satisfying the attribute threshold condition in each lane does not exceed the quantity threshold, the lane is determined to be a solid lane. In this embodiment, the quantity threshold is set to 3.

[0105] Step 6: Extract the centroids of connected components in the dashed lane lines that meet the attribute threshold conditions, and sort each connected component according to the coordinates of its centroid.

[0106] To further locate the dashed line blocks within the dashed lane lines, the centroids of connected components that satisfy attribute threshold conditions are extracted as the location points for the corresponding dashed line blocks. The formula for calculating the centroid coordinates is:

[0107] , (3)

[0108] (4)

[0109] (5)

[0110] (6)

[0111] in, The centroid of the connected component is represented by its coordinates, which are based on the image coordinate system. The zeroth moment represents the contour of a connected region, which is the sum of all pixel values ​​within the connected region; Represents coordinates within a connected region Pixel value at; This represents the first moment, i.e., all x-coordinates within the connected domain. The weighted sum, with the weighting coefficients on the x-axis. The corresponding pixel values ​​describe the distribution of pixel values ​​on the horizontal axis; This represents the first moment, i.e., all ordinates within the connected domain. The weighted sum, with the weighting coefficients on the ordinate. The corresponding pixel values ​​describe the distribution of pixel values ​​on the vertical axis. In this embodiment, the image coordinate system is constructed with the top left corner of the image as the origin, the horizontal axis pointing to the right as the positive direction (i.e., the positive direction of the horizontal coordinate axis), and the vertical axis pointing downwards as the positive direction (i.e., the positive direction of the vertical coordinate axis).

[0112] In this embodiment, each connected component is sorted according to the ordinate of its centroid to obtain the sorted connected components. Any two adjacent connected components are grouped together. For example, connected components numbered 1 and 2 are grouped together, connected components numbered 2 and 3 are grouped together, connected components numbered 3 and 4 are grouped together, and so on, to obtain all groups.

[0113] Step 7: Based on the centroids of two adjacent connected components, select multiple consecutive connected components in space to locate each dashed block in the dashed lane line.

[0114] During the localization of dashed line blocks, if a vehicle obstructs the view, the obstructed portion of the dashed line block will not be detected, and the detected centroids cannot be guaranteed to be adjacent. To address this issue, three methods are used for judgment:

[0115] The first method: Typically, the length of a dashed line block on a road is 6 meters, and the distance between dashed line blocks is 9 meters. Based on this common knowledge, calculate the centroid distance between two adjacent connected regions in the image. ; Calculate the length of the longest side of the minimum bounding rectangle of one of two adjacent connected components. If the distance between the centroids With the length of the longer side If formula (7) is satisfied, it is determined that the two dashed blocks corresponding to the two adjacent connected regions are spatially adjacent; otherwise, they are not adjacent. This allows for the selection of multiple spatially continuous connected regions, each of which corresponds to a dashed block, i.e., the positioning of each dashed block in the dashed lane line.

[0116] (7)

[0117] in, This represents the proportional threshold. According to the rules for drawing dashed lines, the length of a typical dashed line block is 6 meters, and the interval between two adjacent dashed line blocks is 9 meters. Therefore, the distance between the center points of two adjacent dashed line blocks is 15 meters. On the image... and The above numerical relationships must be satisfied. Therefore, this embodiment... Set it to 3.

[0118] The second method: Due to perspective effects, lane lines in road images exhibit a "nearer, smaller farther away" characteristic. Therefore, the centroids of two adjacent dashed line blocks closer to the image acquisition device are larger, while the centroids of two adjacent dashed line blocks farther from the image acquisition device are smaller. Thus, the distance between the centroids of two adjacent connected components in different groups can be used to roughly determine whether two corresponding dashed line blocks are adjacent.

[0119] The third method: Since the distance between two adjacent dashed blocks in the real physical world is fixed, the centroid coordinates of two adjacent connected domains are transformed to the world coordinate system using a perspective transformation matrix. Then, the physical distance between the two adjacent connected domains is calculated based on their centroid coordinates in the world coordinate system. If the physical distance is close to the actual distance between the two adjacent dashed blocks (close means the physical distance is within the range of the actual distance ± deviation), it indicates that the two dashed blocks corresponding to the two adjacent connected domains are adjacent; otherwise, it indicates that the two dashed blocks corresponding to the two adjacent connected domains are not adjacent.

[0120] Only spatially adjacent connected regions are selected as dashed line blocks to complete the positioning of each dashed line block in the dashed lane line. Figure 5 This diagram shows the effect of lane line classification and centroid positioning of dashed line blocks. Figure 5 It can be seen that, based on the detection of each lane line, the system also classifies the lane lines as solid and dashed and locates each dashed block in the dashed lane line.

[0121] The detection method of the present invention further includes flow guidance region detection. In a specific embodiment of the present invention, flow guidance region detection specifically includes:

[0122] Step 8: Sort all lane lines from left to right according to their position in the original road image.

[0123] To calculate the spacing between two adjacent lane lines, the lane lines need to be sorted from left to right according to their position in the original road image. In a specific embodiment of the present invention, sorting all lane lines from left to right according to their position in the original road image specifically includes:

[0124] Step 8.1: For each lane line, extract the vertical coordinate of its bottommost pixel, i.e., the ordinate.

[0125] The bottommost pixel of the lane line refers to the pixel of the lane line closest to the bottom of the image.

[0126] Step 8.2: Sort the vertical coordinates of the bottommost pixels of all lane lines in descending order and generate a vertical hierarchy sequence.

[0127] The vertical hierarchy sequence can be represented as , This represents the vertical coordinate of the bottommost pixel of the i-th lane line, and n represents the number of lane lines.

[0128] Step 8.3: Construct the first horizontal reference line at the first vertical coordinate of the vertical hierarchy sequence, calculate the first intersection point of the first horizontal reference line with all lane lines, and sort all lane lines in ascending order of the horizontal coordinate (i.e., x-coordinate) of the first intersection point to achieve the initial sorting of lane lines from left to right.

[0129] Step 8.4: For unsorted lane lines, process them sequentially in descending order of their vertical coordinates as follows:

[0130] Construct a second horizontal reference line at the vertical coordinate of the bottom pixel of the lane line, calculate the second intersection point between the second horizontal reference line and the sorted lane lines, and insert the lane line to the left or right of the sorted lane lines according to the horizontal coordinate of the bottom pixel of the lane line and the horizontal coordinate of the second intersection point, thus completing the sorting of all lane lines.

[0131] If the horizontal coordinate of the bottom pixel of an unsorted lane line is less than the horizontal coordinate of the second intersection point, the unsorted lane line is inserted to the left of the sorted lane line corresponding to the second intersection point; if the horizontal coordinate of the bottom pixel of an unsorted lane line is greater than the horizontal coordinate of the second intersection point, the unsorted lane line is inserted to the right of the sorted lane line corresponding to the second intersection point.

[0132] Step 9: For each lane line in two adjacent lane lines, sort the vertical coordinates of all its pixels in descending order to obtain the common interval of the two adjacent lane lines on the vertical coordinate axis.

[0133] For two adjacent lane lines (denoted as left lane line L and right lane line R), in the image coordinate system, the vertical coordinates of each pixel of the left lane line (or right lane line) are sorted in descending order to obtain the common interval of the two adjacent lane lines on the vertical coordinate axis. , ].like Figure 6 As shown, let the minimum vertical coordinate of the left lane line be... The maximum vertical coordinate is The minimum vertical coordinate of the right lane line is The maximum vertical coordinate is Common interval [ , ]middle, , .

[0134] Step 10: Calculate the rate of change of the spacing between two adjacent lane lines.

[0135] Within the common interval of two adjacent lane lines on the vertical coordinate axis, sample the pixel set of the left lane line along the vertical coordinate axis at certain intervals. For each sampling point of the left lane line... Search for the lane lines on the right that meet the requirements. nearest neighbor pixel This forms matching point pairs. For each matching point pair, the horizontal spacing is calculated:

[0136] (8)

[0137] in, This represents the i-th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy pixels, Indicates the spacing threshold; This represents the i-th matching point pair between two adjacent lane lines. , The horizontal spacing between them. In this embodiment, the spacing threshold is set to 5.

[0138] The rate of change of spacing between adjacent vertical positions was calculated using the numerical difference method:

[0139] (9)

[0140] in, This represents the rate of change of the spacing between the i-th adjacent vertical positions; Indicates matching point pairs , Horizontal spacing between them This represents the (i+1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels; Indicates matching point pairs , Horizontal spacing between them This represents the (i-1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels.

[0141] Finally, Normalize to a fixed range to obtain the normalized value of the rate of change of the spacing between the i-th adjacent vertical positions.

[0142] Step 11: Determine whether a diversion area exists based on the rate of change of the spacing between two adjacent lane lines, and extract the boundary key points from the common interval of the two adjacent lane lines with diversion areas on the vertical coordinate axis, and determine the boundary of the diversion area based on the boundary key points.

[0143] A guiding zone is a special area in road traffic used to guide vehicles to diverge, merge, or change lanes, and its geometric characteristics differ significantly from those of regular lanes. Specifically, guiding zones are usually accompanied by the divergence or convergence of lane lines, resulting in a significant increase or decrease in the distance between adjacent lane lines. Based on this characteristic, if the rate of change of the distance between two adjacent lane lines at M consecutive vertical positions... The signs are consistent (i.e., all signs are positive or all signs are negative; positive signs indicate divergence, and negative signs indicate convergence), and The absolute value exceeds the rate of change threshold If so, then a flow guiding region is considered to exist. In this embodiment, The value is 0.6, M = 80% × T, where T represents the number of pixels in the left lane line.

[0144] From the common interval of two adjacent lane lines with a guiding area on the vertical coordinate axis [ , Extract boundary key points from [the data], and determine the boundary of the diversion area based on these key points. Boundary key points include the left lane line. Point A at ( , ), the right lane line is Point B at ( , ), the left lane line is Point C at ( , ), the right lane line is Point D at ( , ),like Figure 6 As shown. Figure 7 The diagram shows the detection effect of the flow guidance area.

[0145] This invention overcomes the high dependence of end-to-end deep learning models on labeled data by using a cascaded architecture of "deep learning feature extraction + traditional image analysis". Based on lane line detection, and taking into account the essential differences between different types of lane lines, an adaptive classification and localization strategy is designed to achieve integrated detection, classification and localization.

[0146] The lane line classification of this invention analyzes the essential geometric differences between real and dashed lane lines, designs decision rules based on the geometric features (area, aspect ratio, and rectangularity) of connected domains, and uses an unsupervised classification mechanism to classify real and dashed lane lines. Then, it extracts the centroid of the connected domain of dashed lane lines and uses the centroid of the connected domain to locate the dashed blocks in the dashed lane lines, without relying on labeled data.

[0147] This invention combines deep learning feature extraction with traditional image analysis, retaining the ability of deep learning to express complex features while incorporating the interpretability of traditional image analysis. It employs an unsupervised density clustering algorithm to achieve instance-level detection of any number of lane lines, overcoming the limitations of traditional preset lane line counts. By combining morphological enhancement and multi-attribute threshold conditional judgment, and through joint analysis of the area, rectangularity, and aspect ratio of connected components, it can accurately distinguish between real and virtual lane lines even under noise interference, and locate the dashed line blocks within dashed lane lines. Based on dynamic analysis of the rate of change of the distance between adjacent vertical positions of pixels in adjacent lane lines, it can sensitively capture the divergence / convergence characteristics of lane lines, thereby accurately identifying the guiding area.

[0148] Example 2

[0149] This invention also provides an electronic device, which includes a memory, a processor, and a computer program / instructions stored in the memory. The processor executes the computer program / instructions to implement the lane detection method of this invention.

[0150] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) or loaded from a storage portion into random access memory (RAM). The processor can be a multi-core processor or may contain multiple processors. In some embodiments, the processor may include a general-purpose main processor and one or more specialized coprocessors, such as a central processing unit, graphics processing unit (GPU), neural network processor (NPU), digital signal processor (DSP), etc. Various programs and data required for device operation are also stored in RAM. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0151] The processor and memory described above are used together to execute programs / instructions stored in the memory. When the program / instructions are executed by the computer, they can implement the methods, steps, or functions described in the above embodiments.

[0152] Although not shown, embodiments of the present invention also provide a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the lane line detection method of the present invention.

[0153] Readable storage media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0154] The above description only discloses specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or modifications that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A lane line detection method, characterized in that, The detection method includes: Lane line detection is performed on the preprocessed road image to obtain lane line instances; Based on each lane line instance, the original road image is processed to obtain the connected components of each lane line; Calculate the properties of the connected components for each lane line; The number of connected components in each lane that meet the attribute threshold conditions is counted based on the attributes of the connected components of each lane. The lane type is determined based on the number of connected components in each lane that satisfy the attribute threshold condition; Extract the centroids of connected components in the dashed lane lines that satisfy the attribute threshold conditions, and sort the connected components according to the coordinates of their centroids. By selecting multiple consecutive connected regions in the space based on the centroids of two adjacent connected regions, the positioning of each dashed line block in the dashed lane line can be achieved.

2. The lane line detection method according to claim 1, characterized in that, The process of detecting lane lines in the preprocessed road image specifically includes: A pre-trained deep learning model is used to detect lane lines in the pre-processed road image, resulting in a binary segmentation map of lane lines and background and a pixel embedding feature vector map. The lane line and background binary segmentation map are used as a spatial mask, and a bitwise AND operation is performed with the pixel embedding feature vector map to filter out the lane line pixel embedding feature vectors. The lane line pixel embedding feature vectors are then clustered using a density clustering algorithm to obtain each lane line instance.

3. The lane line detection method according to claim 1, characterized in that, Based on lane line instances, the original road image is processed, specifically including: Based on each lane line instance, construct a binary segmentation map of each lane line and the background; For each binary segmentation image of lane lines and background, perform a morphological dilation operation on it; The binary segmentation map of each lane line after the dilation operation and the background is used as a spatial mask, and a bitwise AND operation is performed with the original road image to extract the region of each lane line from the original road image. The region of each lane line is converted to grayscale, filtered, and edge detected to obtain the edge of each lane line; Perform a morphological closing operation on the edge of each lane line to obtain the closed region of each lane line; Fill the closed region of each lane line to obtain the connected region of each lane line.

4. The lane line detection method according to claim 1, characterized in that, The properties of the connected region of each lane line include area, rectangularity, and aspect ratio. The formula for calculating the rectangularity of the connected region is: ； in, Represents the rectangularity of the connected components; Represents the area of ​​the connected region; This represents the area of ​​the smallest bounding rectangle of the connected region.

5. The lane line detection method according to claim 1, characterized in that, Selecting multiple consecutive connected components in space based on the centroids of two adjacent connected components, specifically including: Calculate the centroid distance between two adjacent connected components; Calculate the length of the longest side of the minimum bounding rectangle of one of two adjacent connected components; Based on the centroid spacing and the length of the long side, it is determined whether the two dashed blocks corresponding to the two adjacent connected regions are spatially adjacent, thereby filtering out multiple spatially continuous connected regions.

6. The lane line detection method according to any one of claims 1 to 5, characterized in that, The detection method also includes flow guidance area detection, specifically including: All lane lines are sorted from left to right according to their position in the original road image; For each of two adjacent lane lines, sort the vertical coordinates of all its pixels in descending order to obtain the common interval of the two adjacent lane lines on the vertical coordinate axis. Calculate the rate of change of the spacing between two adjacent lane lines; The existence of a diversion area is determined based on the rate of change of the spacing between two adjacent lane lines, and boundary key points are extracted from the common interval of the two adjacent lane lines with diversion areas on the vertical coordinate axis. The boundary of the diversion area is determined based on the boundary key points.

7. The lane line detection method according to claim 6, characterized in that, Based on their position in the original road image, all lane lines are sorted from left to right, specifically including: For each lane line, extract the vertical coordinates of its bottommost pixel; Sort the vertical coordinates of the bottommost pixels of all lane lines in descending order and generate a vertical hierarchy sequence; Construct a first horizontal reference line at the first vertical coordinate of the vertical hierarchy sequence, calculate the first intersection point of the first horizontal reference line with all lane lines, and sort all lane lines in ascending order of the horizontal coordinates of the first intersection points to achieve the initial sorting of lane lines from left to right. For unsorted lane lines, process them sequentially in descending order of their vertical coordinates as follows: Construct a second horizontal reference line at the vertical coordinate of the bottom pixel of the lane line, calculate the second intersection point of the second horizontal reference line and the sorted lane lines, and insert the lane line to the left or right of the sorted lane lines according to the horizontal coordinate of the bottom pixel of the lane line and the horizontal coordinate of the second intersection point, so as to sort all lane lines from left to right.

8. The lane line detection method according to claim 6, characterized in that, Calculating the rate of change of spacing between two adjacent lane lines specifically includes: Within the shared interval of two adjacent lane lines on the vertical coordinate axis, sample the pixel set of the left lane line along the vertical coordinate axis at certain intervals. For each sampled point of the left lane line, search for a suitable pixel in the right lane line among the two adjacent lane lines. And the pixel closest to the sampling point of the left lane line This forms a pair of matching points; The lateral spacing of each matching point pair is calculated using the following formula: ； in, This represents the i-th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy pixels, Indicates the spacing threshold; This represents the i-th matching point pair between two adjacent lane lines. , The horizontal spacing between them; The rate of change of spacing between adjacent vertical positions is calculated based on the lateral spacing of each matching point pair. The specific calculation formula is as follows: ； in, This represents the rate of change of the spacing between the i-th adjacent vertical positions; Indicates matching point pairs , Horizontal spacing between them This represents the (i+1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels; Indicates matching point pairs , Horizontal spacing between them This represents the (i-1)th sampling point in the left lane line; Indicates the middle of the right lane line and the sampling point nearest neighbor and satisfy The pixels; The rate of change of the spacing between the i-th adjacent vertical positions is normalized.

9. An electronic device comprising a memory, a processor, and a computer program / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the lane detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the lane line detection method as described in any one of claims 1 to 8.

Images