Lane line recognition method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses a lane line identification method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a current road image and performing lane line segmentation to obtain a lane line segmentation result of the current road image; using a preset block strategy to block the lane line segmentation result to obtain lane line segmentation results of a plurality of block regions, wherein the plurality of block regions comprise a first block region and a second block region; determining a lane line fitting result of the first block region according to the lane line segmentation result of the first block region; and using a preset iterative optimization strategy to iteratively optimize the lane line fitting result of the first block region according to the lane line segmentation result of the second block region to obtain a lane line identification result of the current road image. The application iteratively optimizes the lane line fitting result, reduces the influence of inaccurate lane line instance segmentation results on lane line identification accuracy, and improves lane line identification accuracy and stability.

Description

Technical Field

[0001] This application relates to the field of lane line recognition technology, and in particular to a lane line recognition method and apparatus, electronic device and computer-readable storage medium. Background Technology

[0002] In the field of autonomous driving, lane line detection of images of the road ahead captured by onboard cameras is an essential part of autonomous driving perception capabilities. The detected lane lines can be used to realize autonomous driving functions such as visual-assisted positioning and lane keeping.

[0003] The goal of lane detection is to output the curve equation of the lanes contained in the image in 3D space. Existing lane detection schemes mainly use a pre-trained lane semantic segmentation model to perform semantic segmentation on the lanes in the image, then determine each lane instance through clustering, and finally obtain the curve equation of each lane instance in 3D space through curve fitting.

[0004] However, in real-world scenarios, lane line semantic segmentation results and lane line instance clustering results are prone to flaws or even obvious errors, especially for dashed lane lines, where a dashed lane line may be missegmented into two lane lines, resulting in low lane line recognition accuracy. Summary of the Invention

[0005] This application provides a lane line recognition method and apparatus, electronic device and storage medium to improve lane line recognition accuracy.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a lane line recognition method, wherein the method includes:

[0008] Acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image;

[0009] Using a preset segmentation strategy, the lane line segmentation result of the current road image is segmented into blocks to obtain lane line segmentation results corresponding to multiple block regions, including a first block region and a second block region.

[0010] Based on the lane line segmentation results corresponding to the first segmented region, determine the lane line fitting results corresponding to the first segmented region;

[0011] Based on the lane line segmentation results corresponding to the second segmented region, the lane line fitting results corresponding to the first segmented region are iteratively optimized using a preset iterative optimization strategy to obtain the lane line recognition results of the current road image.

[0012] Optionally, the step of acquiring the current road image and performing lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image includes:

[0013] The current road image is semantically segmented using a preset lane line semantic segmentation model to obtain the lane line semantic segmentation result of the current road image;

[0014] The lane line semantic segmentation results are clustered at the pixel level using a preset clustering algorithm to obtain the lane line instance segmentation results of the current road image.

[0015] Optionally, the lane line segmentation result includes lane line instances, and determining the lane line fitting result corresponding to the first segmented region based on the lane line segmentation result corresponding to the first segmented region includes:

[0016] Based on the pixels of the lane line instance corresponding to the first segmented region, determine the fitting candidate points corresponding to the lane line instance in the first segmented region.

[0017] The lane line fitting algorithm is used to fit the candidate points corresponding to the lane line instances in the first segment region to obtain the lane line fitting result corresponding to the first segment region.

[0018] Optionally, the lane line fitting result includes a lane line fitting curve, and the step of iteratively optimizing the lane line fitting result corresponding to the first block region using a preset iterative optimization strategy based on the lane line segmentation result corresponding to the second block region to obtain the lane line recognition result of the current road image includes:

[0019] Based on the lane line fitting curve corresponding to the first segmented region, sampling is performed using a preset sampling strategy to obtain lane line sampling points.

[0020] Based on the transformation relationship between the image coordinate system and the camera coordinate system, the lane line sampling points are projected onto the current road image to obtain the lane line projection points corresponding to the second block region.

[0021] Based on the lane line projection points corresponding to the second segmented region and the lane line segmentation results corresponding to the second segmented region, the lane line fitting curve corresponding to the first segmented region is iteratively optimized to obtain the lane line fitting results corresponding to the second segmented region.

[0022] The lane line recognition result of the current road image is determined based on the lane line fitting result corresponding to the second segment region.

[0023] Optionally, the lane line segmentation result includes lane line instances. The step of iteratively optimizing the lane line fitting curve corresponding to the first block region based on the lane line projection points corresponding to the second block region and the lane line segmentation result corresponding to the second block region, to obtain the lane line fitting result corresponding to the second block region, includes:

[0024] Based on the pixels of the lane line instance corresponding to the second block region, determine the fitting candidate points corresponding to the lane line instance in the second block region;

[0025] Based on the fitting candidate points corresponding to the lane line instances in the second block region and the lane line projection points corresponding to the second block region, the lane line instances in the second block region are matched with the lane line instances in the first block region.

[0026] Based on the matching results, the lane line fitting curve corresponding to the first segment region is iteratively optimized to obtain the second lane line fitting result corresponding to the second segment region.

[0027] Optionally, the step of iteratively optimizing the lane line fitting curve corresponding to the first segmented region based on the matching result to obtain the second lane line fitting result corresponding to the second segmented region includes:

[0028] Obtain the fitting candidate points corresponding to the lane line instances in the first segment region;

[0029] Based on the matching results, the fitting candidate points corresponding to the lane line instances in the first block region and the fitting candidate points corresponding to the lane line instances in the second block region are merged.

[0030] The merged candidate points are fitted using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second block region.

[0031] Optionally, the step of fitting the merged candidate points using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second block region includes:

[0032] If the matching result is a successful match, it is determined that the lane line instance in the second block region corresponds to the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instances of the same lane line are fitted.

[0033] If the matching result is a failure, it is determined that the lane line instance in the second block region is not the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instance in the second block region are fitted.

[0034] Secondly, embodiments of this application also provide a lane line recognition device, wherein the device includes:

[0035] The lane line segmentation unit is used to acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image;

[0036] The segmentation unit is used to segment the lane line segmentation result of the current road image into blocks using a preset segmentation strategy, so as to obtain lane line segmentation results corresponding to multiple segmentation regions, wherein the multiple segmentation regions include a first segmentation region and a second segmentation region.

[0037] The determining unit is used to determine the lane line fitting result corresponding to the first segmented region based on the lane line segmentation result corresponding to the first segmented region.

[0038] The optimization unit is used to iteratively optimize the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the second block region using a preset iterative optimization strategy, so as to obtain the lane line recognition result of the current road image.

[0039] Thirdly, embodiments of this application also provide an electronic device, including:

[0040] Processor; and

[0041] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0042] Fourthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0043] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The lane line recognition method of this application embodiment first acquires the current road image and performs lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image; then, using a preset block strategy, the lane line segmentation result of the current road image is divided into blocks to obtain lane line segmentation results corresponding to multiple block regions, including a first block region and a second block region; then, based on the lane line segmentation result corresponding to the first block region, the lane line fitting result corresponding to the first block region is determined; finally, based on the lane line segmentation result corresponding to the second block region, a preset iterative optimization strategy is used to iteratively optimize the lane line fitting result corresponding to the first block region to obtain the lane line recognition result of the current road image. The lane line recognition method of this application embodiment reduces the impact of inaccurate lane line instance segmentation results on the final lane line recognition accuracy by adopting a preset block strategy to divide the lane line segmentation result into blocks and using a preset iterative optimization strategy to iteratively optimize the lane line fitting result, thereby improving the lane line recognition accuracy and stability, especially for the recognition of dashed lane lines. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 This is a schematic diagram of a lane line recognition process in the prior art;

[0046] Figure 2 This is a schematic diagram of a lane line semantic segmentation result in the prior art;

[0047] Figure 3 This is a schematic diagram of a lane line instance segmentation result in the prior art;

[0048] Figure 4 This is a schematic diagram of another lane line instance segmentation result in the prior art;

[0049] Figure 5 This is a flowchart illustrating a lane line recognition method according to an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of a block region division result in an embodiment of this application;

[0051] Figure 7 This is a schematic diagram of a lane line recognition process in an embodiment of this application;

[0052] Figure 8This is a schematic diagram of the structure of a lane line recognition device according to an embodiment of this application;

[0053] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0056] For ease of understanding of the various embodiments of this application, such as Figure 1 The diagram illustrates a lane line recognition process in the prior art. First, a lane line semantic segmentation model is used to perform lane line semantic segmentation, segmenting all pixels in the image predicted as lane lines, but without distinguishing specific instances of each lane line. Figure 2 As shown, a schematic diagram of lane line semantic segmentation results in the prior art is provided, where the white part is the lane line, but the lane lines are not distinguished from each other.

[0057] Then, based on the obtained lane line semantic segmentation results, the lane line pixels are clustered to distinguish lane line instances. For example... Figure 3 As shown, a schematic diagram of lane line instance segmentation results in the prior art is provided. It can be seen that the two lane lines in the image are separated and marked as black and white, respectively.

[0058] Then, based on the lane line instance segmentation results, a point is taken for each lane line in each row. These points are transformed into the camera coordinate system (i.e., the 3D coordinate system) through camera intrinsic and extrinsic parameters, and the fitting candidate points for each lane line can be obtained. The camera intrinsic and extrinsic parameters can be obtained in advance through camera calibration.

[0059] Finally, for each lane line, the candidate points for fitting are taken and curve fitting is performed to obtain the fitted curve for each lane line.

[0060] However, in real-world scenarios, both lane line semantic segmentation results and lane line instance segmentation results may exhibit flaws or even obvious errors, especially for dashed lane lines, such as... Figure 4As shown, another lane line instance segmentation result diagram in the prior art is provided. It can be seen that the same dashed lane line within the white rectangle on the right is mistakenly segmented into two different lane lines, which causes the final lane line recognition result to produce problems such as false detection, jumps and large drifts, thereby affecting the realization of autonomous driving functions that rely on lane recognition results.

[0061] Based on this, embodiments of this application provide a lane line recognition method, such as... Figure 5 The diagram shows a flowchart of a lane line recognition method according to an embodiment of this application. The method includes at least the following steps S510 to S540:

[0062] Step S510: Obtain the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image.

[0063] In this embodiment of the application, when performing lane line recognition, it is necessary to first acquire the current road image captured by the vehicle-mounted camera, and then use a certain lane line segmentation algorithm to segment the current road image to obtain the lane line segmentation result of the current road image. The lane line segmentation algorithm can be obtained based on the existing lane line semantic segmentation model and clustering algorithm, and specifically, it can segment the pixels of each lane line instance in the image.

[0064] Step S520: Using a preset segmentation strategy, the lane line segmentation result of the current road image is segmented into blocks to obtain lane line segmentation results corresponding to multiple segmented regions, including a first segmented region and a second segmented region.

[0065] The lane line segmentation result of the current road image obtained by the aforementioned steps can be regarded as the initial lane line segmentation result. In order to further improve the accuracy of lane line recognition, certain strategies need to be adopted for optimization. The first strategy to be adopted here is to preset the block strategy. "Blocking" can be understood as dividing the current road image containing the lane line segmentation result into multiple regions to obtain the lane line segmentation results corresponding to multiple block regions.

[0066] Since lane lines captured from a vehicle-mounted camera typically extend along the vertical direction of the road image, the aforementioned pre-defined segmentation strategy can divide the current road image containing lane line segmentation results into multiple regions along the horizontal direction of the image at certain intervals. The interval size can be flexibly adjusted according to actual needs; for example, it can be equal intervals, or it can be based on the principle of near objects appearing larger and farther objects smaller, dividing from the bottom of the image upwards with increasingly smaller intervals. Figure 6As shown, a schematic diagram of the block area division result in an embodiment of this application is provided. It can be seen that the current road image containing the lane line segmentation result is divided into four block areas: N01, N02, N03, and N04.

[0067] After obtaining the aforementioned multiple segmented regions, it is also necessary to determine the current first and second segmented regions. It should be noted that in this embodiment, "first" and "second" are relative concepts, and can be considered as two horizontally adjacent segmented regions. During initial processing, based on the principle of near-large and far-small, the first segmented region can be considered as the first segmented region at the bottom of the image. Figure 6 In the N01 region, the second block region can be the N02 region. Since the optimization strategy adopted in the subsequent steps of this application is iterative optimization, after the N01 and N02 regions are processed, the N03 region will be processed based on the processing results of the corresponding N01 and N02 regions. At this time, the N03 region is the current second block region, and so on, until all block regions have been iterated and completed.

[0068] Step S530: Determine the lane line fitting result corresponding to the first segmented region based on the lane line segmentation result corresponding to the first segmented region.

[0069] After determining the current first block region, it is necessary to determine the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the first block region, such as the pixel points of each lane line instance segmented in the first block region. Since the lane line fitting result corresponds to the first block region, it can be regarded as an initial or intermediate lane line fitting result, which serves as the basis for subsequent iterative optimization.

[0070] Step S540: Based on the lane line segmentation result corresponding to the second segmented region, the lane line fitting result corresponding to the first segmented region is iteratively optimized using a preset iterative optimization strategy to obtain the lane line recognition result of the current road image.

[0071] Since the lane line fitting result of the first block region obtained in the aforementioned steps is based on the lane line segmentation result of the first block region, considering the longitudinal distribution characteristics of lane lines in the image, the lane line segmentation result corresponding to the second block region and the preset iterative optimization strategy can be used to iteratively optimize the lane line fitting result of the first block region until all block regions have been iteratively processed, thereby obtaining the final lane line recognition result.

[0072] The lane line recognition method of this application adopts a preset segmentation strategy to segment the lane line segmentation results into blocks, and uses a preset iterative optimization strategy to iteratively optimize the lane line fitting results. This reduces the impact of inaccurate lane line instance segmentation results on the final lane line recognition accuracy, and improves the lane line recognition accuracy and stability. In particular, it has high recognition accuracy for dashed lane lines.

[0073] In some embodiments of this application, the step of acquiring the current road image and performing lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image includes: performing semantic segmentation on the current road image using a preset lane line semantic segmentation model to obtain the lane line semantic segmentation result of the current road image; and performing pixel-level clustering on the lane line semantic segmentation result using a preset clustering algorithm to obtain the lane line instance segmentation result of the current road image.

[0074] When performing lane line segmentation on the current road image, a pre-trained lane line semantic segmentation model can be used to perform lane line semantic segmentation on the current road image, that is, to segment all lane line pixels from the image. Then, a preset clustering algorithm can be used to cluster all lane line pixels to obtain each lane line instance contained in the current road image.

[0075] The aforementioned lane line semantic segmentation model can be trained based on existing convolutional neural networks such as U-net, and the preset clustering algorithm can be implemented based on existing algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise). Of course, those skilled in the art can flexibly choose how to perform lane line semantic segmentation and clustering in combination with existing technologies, and no specific limitations are made here.

[0076] In some embodiments of this application, the lane line segmentation result includes lane line instances. Determining the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the first block region includes: determining the fitting candidate points corresponding to the lane line instances in the first block region based on the pixel points of the lane line instances in the first block region; and fitting the fitting candidate points corresponding to the lane line instances in the first block region using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the first block region.

[0077] Since lane lines have a certain width, each pixel row of the segmented lane line instance corresponds to multiple lane line pixels. In order to ensure the fitting effect, it is not necessary to fit all the pixels of all lane line instances. Therefore, candidate points for lane line fitting can be determined from all the pixels of the lane line instances corresponding to the first block region. For example, for each lane line instance, the middle pixel of each corresponding pixel row can be selected for lane line fitting.

[0078] Furthermore, since the selected intermediate pixels are located in the image coordinate system, while the lane line fitting is a fitting of 3D points in the camera coordinate system, the intermediate pixels selected in each row can be transformed into the camera coordinate system based on the pre-calibrated camera intrinsic and extrinsic parameters, and used as fitting candidate points for each lane line instance.

[0079] Finally, a specific lane line fitting algorithm is used to fit the candidate points corresponding to each lane line instance, thereby obtaining the lane line fitting curve for each lane line instance. The lane line fitting algorithm can be flexibly selected based on existing technologies, such as B-spline, cubic spline interpolation, RANSAC (RANdom Sampling Consensus), least squares method, etc., without specific limitations.

[0080] In some embodiments of this application, the lane line fitting result includes a lane line fitting curve. The step of iteratively optimizing the lane line fitting result corresponding to the first segmented region using a preset iterative optimization strategy based on the lane line segmentation result corresponding to the second segmented region to obtain the lane line recognition result of the current road image includes: sampling based on the lane line fitting curve corresponding to the first segmented region using a preset sampling strategy to obtain lane line sampling points; projecting the lane line sampling points onto the current road image based on the transformation relationship between the image coordinate system and the camera coordinate system to obtain lane line projection points corresponding to the second segmented region; iteratively optimizing the lane line fitting curve corresponding to the first segmented region based on the lane line projection points and the lane line segmentation result corresponding to the second segmented region to obtain the lane line fitting result corresponding to the second segmented region; and determining the lane line recognition result of the current road image based on the lane line fitting result corresponding to the second segmented region.

[0081] Taking the first segment of the current road image, region N01, as an example, after obtaining the lane line fitting curve corresponding to region N01, a certain sampling strategy can be used to sample lane line points. Then, based on the camera's intrinsic and extrinsic parameters, the sampled points are projected onto the current road image to obtain the lane line projection points. The principle of the sampling strategy is to ensure that each lane line instance has a projection point in each pixel row of the image, or at least in each pixel row of the second segment of the image, region N02.

[0082] The purpose of sampling and projecting lane line points based on the lane line fitting curve of region N01 is to predict the position of the lane line instance corresponding to region N01 in region N02. Then, combined with the lane line instance originally segmented from region N02, the relationship between the lane line instances in region N01 and region N02 can be determined. In this way, the lane line fitting curve of region N01 is iteratively optimized until all blocks are iterated and the final lane line recognition result is obtained. Through the above iterative optimization process, the problem of the same dashed lane line being missegmented into different lane lines can be avoided, thus improving the lane line recognition accuracy.

[0083] In some embodiments of this application, the lane line segmentation result includes lane line instances. The step of iteratively optimizing the lane line fitting curve corresponding to the first block region based on the lane line projection points corresponding to the second block region and the lane line segmentation result corresponding to the second block region to obtain the lane line fitting result corresponding to the second block region includes: determining the fitting candidate points corresponding to the lane line instances in the second block region based on the pixel points of the lane line instances in the second block region; matching the lane line instances in the second block region with the lane line instances in the first block region based on the fitting candidate points and the lane line projection points corresponding to the second block region; and iteratively optimizing the lane line fitting curve corresponding to the first block region based on the matching result to obtain the second lane line fitting result corresponding to the second block region.

[0084] The lane line segmentation result corresponding to region N02 contains all the pixels of the lane line instances segmented from region N02. Therefore, candidate fitting points can be selected from them first. For example, the middle pixel of each pixel row corresponding to each lane line instance can be selected as a candidate fitting point. Then, a certain matching strategy is used to compare the distance between the candidate fitting points corresponding to the lane line instances in region N02 and the projection points of the lane line sampling points in region N01 in region N02, thereby determining the matching relationship between the lane line instances in region N01 and region N02, such as whether they are the same lane line. Then, the lane line fitting curve corresponding to region N01 can be iteratively optimized based on the matching relationship between the lane line instances in region N01 and region N02.

[0085] The matching strategy described above can be set as follows: if the pixel distance between the fitted candidate point corresponding to the lane line instance in the second block region and the corresponding lane line projection point is less than a preset pixel distance threshold, and the proportion of fitted candidate points that meet this requirement reaches a first preset number threshold, then the matching is considered successful; if the pixel distance between the fitted candidate point corresponding to the lane line instance in the second block region and the corresponding lane line projection point is not less than the preset pixel distance threshold, and the proportion of fitted candidate points that do not meet the above requirement reaches a second preset number threshold, then the matching is considered unsuccessful.

[0086] In some embodiments of this application, the step of iteratively optimizing the lane line fitting curve corresponding to the first segmented region based on the matching result to obtain the second lane line fitting result corresponding to the second segmented region includes: obtaining fitting candidate points corresponding to lane line instances in the first segmented region; merging the fitting candidate points corresponding to lane line instances in the first segmented region and the fitting candidate points corresponding to lane line instances in the second segmented region based on the matching result; and fitting the merged fitting candidate points using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second segmented region.

[0087] The matching result between the lane line instances in the second block region and the lane line instances in the first block region reflects whether the lane line instances in these two regions correspond to the same lane line. Based on this matching result, the fitting candidate points corresponding to the lane line instances in the first block region and the fitting candidate points corresponding to the lane line instances in the second block region can be merged. The merged fitting candidate points may include fitting candidate points belonging to the same lane line or fitting candidate points not belonging to the same lane line. Therefore, lane line fitting can be performed according to different merging results to obtain the lane line fitting result corresponding to the second block region.

[0088] In some embodiments of this application, the step of fitting the merged candidate points using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second block region includes: if the matching result is a successful match, then it is determined that the lane line instance in the second block region corresponds to the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instance of the same lane line are fitted; if the matching result is a failed match, then it is determined that the lane line instance in the second block region does not correspond to the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instance in the second block region are fitted.

[0089] For example, if the leftmost lane line instance in the second block region matches the leftmost lane line instance in the first block region, it means that the lane line instances in these two blocks correspond to the same lane line. In this case, the merged fitting candidate points will simultaneously include the fitting candidate points of the leftmost lane line instance in the second block region and the fitting candidate points of the leftmost lane line instance in the first block region. These merged fitting candidate points are then fitted again to obtain the lane line fitting result after one iteration of optimization.

[0090] If the rightmost lane line instance in the second block region does not match any lane line instance in the first block region, it means that the rightmost lane line instance may have been segmented in the second block region. Therefore, the fitting candidate points corresponding to the rightmost lane line instance in the second block region can be fitted separately to obtain the lane line fitting result of the rightmost lane line instance in the second block region. This process is equivalent to progressively re-clustering the fitting candidate points of the lane lines.

[0091] It should be noted that the above process is only illustrated using two segmented regions as an example. The processing logic for subsequent segmented regions is similar to the aforementioned process. For example, after obtaining the lane line fitting result for region N02, the lane line points can be sampled and projected by combining all the lane line fitting curves corresponding to regions N01 and N02. The purpose is to obtain the lane line projection points of each lane line instance in regions N01 and N02 in region N03. Then, the distance between these projection points and the fitting candidate points of the lane line instances segmented from region N03 is compared to determine the matching relationship between each lane line instance segmented from region N03 and each lane line instance in regions N01 and N02. The lane line fitting results corresponding to regions N01 and N02 are updated accordingly. This process is repeated until all segmented regions have been iterated, resulting in the final lane line recognition result.

[0092] For ease of understanding of the above embodiments of this application, as follows: Figure 7The diagram illustrates a lane line recognition process according to an embodiment of this application. First, the current road image is input into a pre-trained lane line semantic segmentation model to obtain lane line semantic segmentation results. A preset clustering algorithm is then used to cluster these results, yielding lane line instance segmentation results for the current road image. Next, a preset block-based strategy is used to divide the lane line instance segmentation results for the current road image into blocks, resulting in multiple block regions. Then, lane line fitting candidate points are determined based on the pixels of the lane line instances in the first block region, and lane line fitting is performed to obtain the lane line fitting curve for the first block region. The lane line fitting curve for the first block region is sampled to obtain lane line sampling points, which are then projected onto the current road image through coordinate transformation to obtain corresponding lane line projection points for the second block region. The lane line projection points for the second block region are compared with the fitting candidate points of the lane line instances in the second block region, enabling re-clustering of the fitting candidate points and updating the lane line fitting results. This process is iterated until all block iterations are completed, and the final lane line recognition result is output.

[0093] This application also provides a lane line recognition device 800, such as... Figure 8 The diagram shows a schematic representation of a lane line recognition device according to an embodiment of this application. The device 800 includes at least: a lane line segmentation unit 810, a block division unit 820, a determination unit 830, and an optimization unit 840, wherein:

[0094] Lane segmentation unit 810 is used to acquire the current road image and perform lane segmentation on the current road image to obtain the lane segmentation result of the current road image;

[0095] The segmentation unit 820 is used to segment the lane line segmentation result of the current road image into blocks using a preset segmentation strategy, so as to obtain lane line segmentation results corresponding to multiple segmentation regions, wherein the multiple segmentation regions include a first segmentation region and a second segmentation region.

[0096] The determining unit 830 is used to determine the lane line fitting result corresponding to the first segmented region based on the lane line segmentation result corresponding to the first segmented region.

[0097] The optimization unit 840 is used to iteratively optimize the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the second block region using a preset iterative optimization strategy, so as to obtain the lane line recognition result of the current road image.

[0098] In some embodiments of this application, the lane line segmentation unit 810 is specifically used to: perform semantic segmentation on the current road image using a preset lane line semantic segmentation model to obtain lane line semantic segmentation results of the current road image; and perform pixel-level clustering on the lane line semantic segmentation results using a preset clustering algorithm to obtain lane line instance segmentation results of the current road image.

[0099] In some embodiments of this application, the lane line segmentation result includes lane line instances, and the determining unit 830 is specifically used to: determine the fitting candidate points corresponding to the lane line instances in the first block region based on the pixel points of the lane line instances corresponding to the first block region; and fit the fitting candidate points corresponding to the lane line instances in the first block region using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the first block region.

[0100] In some embodiments of this application, the lane line fitting result includes a lane line fitting curve, and the optimization unit 840 is specifically used for: sampling based on the lane line fitting curve corresponding to the first segmented region using a preset sampling strategy to obtain lane line sampling points; projecting the lane line sampling points onto the current road image based on the transformation relationship between the image coordinate system and the camera coordinate system to obtain lane line projection points corresponding to the second segmented region; iteratively optimizing the lane line fitting curve corresponding to the first segmented region according to the lane line projection points corresponding to the second segmented region and the lane line segmentation result corresponding to the second segmented region to obtain the lane line fitting result corresponding to the second segmented region; and determining the lane line recognition result of the current road image according to the lane line fitting result corresponding to the second segmented region.

[0101] In some embodiments of this application, the lane line segmentation result includes lane line instances, and the optimization unit 840 is specifically used to: determine the fitting candidate points corresponding to the lane line instances in the second block region based on the pixel points of the lane line instances corresponding to the second block region; match the lane line instances in the second block region with the lane line instances in the first block region based on the fitting candidate points corresponding to the lane line instances in the second block region and the lane line projection points corresponding to the second block region; and iteratively optimize the lane line fitting curve corresponding to the first block region based on the matching result to obtain the second lane line fitting result corresponding to the second block region.

[0102] In some embodiments of this application, the optimization unit 840 is specifically used to: obtain fitting candidate points corresponding to lane line instances in the first segmented region; merge the fitting candidate points corresponding to lane line instances in the first segmented region and the fitting candidate points corresponding to lane line instances in the second segmented region according to the matching result; and fit the merged fitting candidate points using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second segmented region.

[0103] In some embodiments of this application, the optimization unit 840 is specifically used to: if the matching result is a successful match, determine that the lane line instance of the second block region corresponds to the same lane line as the lane line instance of the first block region, and fit the fitting candidate points corresponding to the lane line instances of the same lane line; if the matching result is a failed match, determine that the lane line instance of the second block region corresponds to the same lane line as the lane line instance of the first block region, and fit the fitting candidate points corresponding to the lane line instance of the second block region.

[0104] It is understood that the lane line recognition device described above can implement each step of the lane line recognition method provided in the foregoing embodiments. The relevant explanations of the lane line recognition method are applicable to the lane line recognition device and will not be repeated here.

[0105] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 9 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0106] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0107] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0108] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a lane line recognition device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0109] Acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image;

[0110] Using a preset segmentation strategy, the lane line segmentation result of the current road image is segmented into blocks to obtain lane line segmentation results corresponding to multiple block regions, including a first block region and a second block region.

[0111] Based on the lane line segmentation results corresponding to the first segmented region, determine the lane line fitting results corresponding to the first segmented region;

[0112] Based on the lane line segmentation results corresponding to the second segmented region, the lane line fitting results corresponding to the first segmented region are iteratively optimized using a preset iterative optimization strategy to obtain the lane line recognition results of the current road image.

[0113] The above is as stated in this application. Figure 1The lane line recognition device method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0114] The electronic device can also perform Figure 1 The method for executing the lane line recognition device, and the implementation of the lane line recognition device in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0115] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the lane line recognition device in the illustrated embodiment is specifically used to perform:

[0116] Acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image;

[0117] Using a preset segmentation strategy, the lane line segmentation result of the current road image is segmented into blocks to obtain lane line segmentation results corresponding to multiple block regions, including a first block region and a second block region.

[0118] Based on the lane line segmentation results corresponding to the first segmented region, determine the lane line fitting results corresponding to the first segmented region;

[0119] Based on the lane line segmentation results corresponding to the second segmented region, the lane line fitting results corresponding to the first segmented region are iteratively optimized using a preset iterative optimization strategy to obtain the lane line recognition results of the current road image.

[0120] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0126] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0127] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0128] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A lane line recognition method, wherein, The method includes: Acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image; Using a preset segmentation strategy, the lane line segmentation result of the current road image is segmented into blocks to obtain lane line segmentation results corresponding to multiple block regions, including a first block region and a second block region. Based on the lane line segmentation results corresponding to the first segmented region, determine the lane line fitting results corresponding to the first segmented region; Based on the lane line segmentation results corresponding to the second block region, the lane line fitting results corresponding to the first block region are iteratively optimized using a preset iterative optimization strategy to obtain the lane line recognition results of the current road image. The lane line fitting result includes a lane line fitting curve. The process of iteratively optimizing the lane line fitting result corresponding to the first block region using a preset iterative optimization strategy based on the lane line segmentation result corresponding to the second block region to obtain the lane line recognition result of the current road image includes: Based on the lane line fitting curve corresponding to the first segmented region, sampling is performed using a preset sampling strategy to obtain lane line sampling points. Based on the transformation relationship between the image coordinate system and the camera coordinate system, the lane line sampling points are projected onto the current road image to obtain the lane line projection points corresponding to the second block region. Based on the lane line projection points corresponding to the second segmented region and the lane line segmentation results corresponding to the second segmented region, the lane line fitting curve corresponding to the first segmented region is iteratively optimized to obtain the lane line fitting results corresponding to the second segmented region. The lane line recognition result of the current road image is determined based on the lane line fitting result corresponding to the second segment region.

2. The method as described in claim 1, wherein, The process of acquiring the current road image and performing lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image includes: The current road image is semantically segmented using a preset lane line semantic segmentation model to obtain the lane line semantic segmentation result of the current road image; The lane line semantic segmentation results are clustered at the pixel level using a preset clustering algorithm to obtain the lane line instance segmentation results of the current road image.

3. The method as described in claim 1, wherein, The lane line segmentation result includes lane line instances, and determining the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the first block region includes: Based on the pixels of the lane line instance corresponding to the first segmented region, determine the fitting candidate points corresponding to the lane line instance in the first segmented region. The lane line fitting algorithm is used to fit the candidate points corresponding to the lane line instances in the first segment region to obtain the lane line fitting result corresponding to the first segment region.

4. The method as described in claim 1, wherein, The lane line segmentation result includes lane line instances. The iterative optimization of the lane line fitting curve corresponding to the first block region based on the lane line projection points corresponding to the second block region and the lane line segmentation result corresponding to the second block region, to obtain the lane line fitting result corresponding to the second block region, includes: Based on the pixels of the lane line instance corresponding to the second block region, determine the fitting candidate points corresponding to the lane line instance in the second block region; Based on the fitting candidate points corresponding to the lane line instances in the second block region and the lane line projection points corresponding to the second block region, the lane line instances in the second block region are matched with the lane line instances in the first block region. Based on the matching results, the lane line fitting curve corresponding to the first segment region is iteratively optimized to obtain the second lane line fitting result corresponding to the second segment region.

5. The method as described in claim 4, wherein, The step of iteratively optimizing the lane line fitting curve corresponding to the first segmented region based on the matching result to obtain the second lane line fitting result corresponding to the second segmented region includes: Obtain the fitting candidate points corresponding to the lane line instances in the first segment region; Based on the matching results, the fitting candidate points corresponding to the lane line instances in the first block region and the fitting candidate points corresponding to the lane line instances in the second block region are merged. The merged candidate points are fitted using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second block region.

6. The method of claim 5, wherein, The process of fitting the merged candidate points using a preset lane line fitting algorithm to obtain the lane line fitting result corresponding to the second block region includes: If the matching result is a successful match, it is determined that the lane line instance in the second block region corresponds to the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instances of the same lane line are fitted. If the matching result is a failure, it is determined that the lane line instance in the second block region is not the same lane line as the lane line instance in the first block region, and the fitting candidate points corresponding to the lane line instance in the second block region are fitted.

7. A lane line recognition device, wherein, The device includes: The lane line segmentation unit is used to acquire the current road image and perform lane line segmentation on the current road image to obtain the lane line segmentation result of the current road image; The segmentation unit is used to segment the lane line segmentation result of the current road image into blocks using a preset segmentation strategy, so as to obtain lane line segmentation results corresponding to multiple segmentation regions, wherein the multiple segmentation regions include a first segmentation region and a second segmentation region. The determining unit is used to determine the lane line fitting result corresponding to the first segmented region based on the lane line segmentation result corresponding to the first segmented region. The optimization unit is used to iteratively optimize the lane line fitting result corresponding to the first block region based on the lane line segmentation result corresponding to the second block region using a preset iterative optimization strategy, so as to obtain the lane line recognition result of the current road image. The lane line fitting result includes a lane line fitting curve, and the optimization unit is specifically used for: Based on the lane line fitting curve corresponding to the first segmented region, sampling is performed using a preset sampling strategy to obtain lane line sampling points. Based on the transformation relationship between the image coordinate system and the camera coordinate system, the lane line sampling points are projected onto the current road image to obtain the lane line projection points corresponding to the second block region. Based on the lane line projection points corresponding to the second segmented region and the lane line segmentation results corresponding to the second segmented region, the lane line fitting curve corresponding to the first segmented region is iteratively optimized to obtain the lane line fitting results corresponding to the second segmented region. The lane line recognition result of the current road image is determined based on the lane line fitting result corresponding to the second segment region.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 6.

Images