Lane information determination methods, equipment, media, program products and vehicles

Abstract

This application relates to a method, device, medium, program product, and vehicle for determining lane line information, including determining lane line detection data and edge detection data corresponding to a lane line image; and determining lane line information corresponding to the lane line image based on the lane line detection data and the edge detection data. The present invention aims to improve the accuracy of lane line information.

Description

Technical Field

[0001] This application relates to the field of lane line recognition, and in particular to a method, device, medium, program product and vehicle for determining lane line information. Background Technology

[0002] Lane line recognition is a technology in the field of computer vision used to detect and measure lane lines on roads, and it is of great significance for intelligent driving, road traffic planning, and monitoring systems. Deep learning models are generally used for lane line recognition, but these models have limitations and cannot accurately obtain lane line information. Summary of the Invention

[0003] This application provides a method, device, medium, program product, and vehicle for determining lane line information, which improves the accuracy of lane line information and at least partially solves the above-mentioned technical problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a method for determining lane line information is provided, the method comprising:

[0005] Determine the lane line detection data and edge detection data corresponding to the lane line image;

[0006] Based on the lane line detection data and the edge detection data, the lane line information corresponding to the lane line image is determined.

[0007] Optionally, determining the lane line information corresponding to the lane line image based on the lane line detection data and the edge detection data includes:

[0008] Based on the lane line detection data and the edge detection data, multiple target lane line edge points are determined;

[0009] The lane line information is determined based on the multiple target lane line edge points.

[0010] Optionally, determining multiple target lane line edge points based on the lane line detection data and the edge detection data includes:

[0011] The target image is determined based on the lane line detection data and the edge detection data;

[0012] The edge points of the plurality of target lane lines are determined based on the target image.

[0013] Optionally, the target image includes multiple lane line points determined based on the lane line detection data, and determining the multiple target lane line edge points based on the target image includes:

[0014] Determine the target lane edge point corresponding to each lane line point in the target image to obtain multiple target lane edge points.

[0015] Optionally, the method further includes:

[0016] Based on the direction of the lane lines, determine the target lane line edge point corresponding to each lane line point in the target image.

[0017] Optionally, the target image includes multiple edge points determined based on the edge detection data, and determining the target lane line edge point corresponding to each lane line point in the target image includes:

[0018] Obtain the first edge point among the plurality of edge points that is located within the preset area where the lane line point is located;

[0019] The target lane line edge point corresponding to the lane line point is determined based on the first edge point.

[0020] Optionally, determining the target lane line edge point corresponding to the lane line point based on the first edge point includes:

[0021] Determine the edge cluster centers corresponding to the lane line points;

[0022] Based on the edge cluster center and the first edge point, determine the target lane line edge point corresponding to the lane line point.

[0023] Optionally, determining the target lane line edge point corresponding to the lane line point based on the edge cluster center and the first edge point includes:

[0024] The second edge point is determined from the first edge point based on the edge cluster center;

[0025] The edge cluster center is updated based on the second edge point, and the process of determining the second edge point from the first edge point based on the updated edge cluster center is repeated until the preset iteration condition is met.

[0026] If the preset iteration conditions are met, the latest edge cluster center is determined as the target lane line edge point corresponding to the lane line point.

[0027] Optionally, the preset iteration conditions include the number of updates to the edge cluster centers reaching a preset threshold, and / or all the first edge points being determined as the second edge points.

[0028] Optionally, determining the second edge point from the first edge point based on the edge clustering center includes:

[0029] The edge point that is closest to the edge cluster center among the first edge points is determined as the second edge point.

[0030] Optionally, updating the edge clustering center based on the second edge point includes:

[0031] The edge cluster centers are updated by merging the second edge point and the edge cluster center.

[0032] Optionally, the edge clustering centers include at least two edge clustering centers of the lane line, and the fusion based on the second edge point and the edge clustering centers includes:

[0033] Among the at least two edge cluster centers, the target edge cluster center that belongs to the same side as the second edge point is determined;

[0034] The second edge point is merged into the target edge cluster center.

[0035] Optionally, determining the edge cluster center corresponding to the lane line point includes:

[0036] Determine the target tangent based on the lane line points and lane line direction;

[0037] The edge cluster centers of the lane line points are determined based on the target tangent.

[0038] Optionally, determining the edge clustering center of the lane line points based on the target tangent includes:

[0039] The edge cluster center is determined from the first edge point based on the target tangent.

[0040] Optionally, the plurality of target lane line edge points include target lane line edge points on the left and right sides of the lane line, and determining the lane line information based on the plurality of target lane line edge points includes:

[0041] Determine the distance difference between the edge point of the target lane line on the left and the edge point of the target lane line on the right.

[0042] The lane width of the lane line image is determined based on the distance difference and a preset linear relationship to obtain the lane line information.

[0043] Optionally, the method further includes:

[0044] The edge points of the multiple target lane lines are transformed according to the top view perspective;

[0045] Based on the converted target lane edge points, the distance difference between the target lane edge points on the left and the target lane edge points on the right is determined.

[0046] Optionally, the determination of the lane line detection data and edge detection data corresponding to the lane line image includes:

[0047] The lane line detection data is obtained by performing lane line detection on the lane line image using a deep learning algorithm.

[0048] Optionally, the determination of the lane line detection data and edge detection data corresponding to the lane line image includes:

[0049] The lane line image is convolved using a preset convolution operator to obtain a feature image;

[0050] The edge detection data is determined based on the feature image.

[0051] Optionally, the preset convolution operator includes at least one of a horizontal convolution operator, a vertical convolution operator, and a diagonal convolution operator.

[0052] Optionally, determining the edge detection data based on the feature image includes:

[0053] Based on the gradient intensity of each pixel in the feature image and the edge gradient intensity threshold, the edge pixels in the feature image are determined to determine the edge detection data.

[0054] Optionally, the edge gradient intensity threshold includes a first threshold and a second threshold. The determination of edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold, to determine the edge detection data, includes:

[0055] If the gradient intensity of the pixel is greater than or equal to the first threshold, the pixel is determined to be an edge pixel; or,

[0056] If the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, and the neighboring pixels of the pixel are edge pixels, then the pixel is determined to be an edge pixel.

[0057] Optionally, the method further includes:

[0058] The lane line image is preprocessed, including grayscale conversion and / or Gaussian filtering.

[0059] Based on the preprocessed lane line image, a feature image is obtained by performing convolution processing on the lane line image using a preset convolution operator.

[0060] Optionally, the method further includes:

[0061] Based on the gradient intensity and gradient direction of each pixel in the feature image, non-maximum suppression processing is performed on the feature image. Based on the feature image after non-maximum suppression processing, the edge detection data is determined according to the pixel intensity of each pixel in the feature image.

[0062] According to a second aspect of this application, an electronic device is also provided, including a processor connected to a memory storing a computer program, the processor being configured to run the computer program in the memory to perform any of the methods provided in this embodiment.

[0063] According to a third aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods provided in this embodiment.

[0064] According to a fourth aspect of this application, a computer program product is provided, comprising a computer program that is executed by a processor to implement any of the methods provided in this embodiment.

[0065] According to a fifth aspect of this application, a vehicle is provided that performs any of the methods described in the embodiments of this application, or includes the electronic equipment described above.

[0066] In summary, the embodiments of this application, through the above technical solutions, determine the lane line detection data and edge detection data corresponding to the lane line image, and combine the lane line detection data and edge detection data to determine the lane line information corresponding to the lane line image, which can improve the accuracy of lane line information identified from lane line images.

[0067] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0070] Figure 1 This is a flowchart illustrating one embodiment of the lane line information determination method provided in this invention.

[0071] Figure 2 This is a schematic diagram of a target image provided in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of lane line sampling provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of an edge clustering center provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of a lane line image provided in an embodiment of the present invention;

[0075] Figure 6 This is a schematic diagram of lane line detection data corresponding to a lane line image provided in an embodiment of the present invention;

[0076] Figure 7 This is a schematic diagram of edge detection data corresponding to a lane line image provided in an embodiment of the present invention;

[0077] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0078] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0079] Based on the problems mentioned in the background, deep learning models are generally used for lane line recognition in related technologies. However, lane line recognition using deep learning models has certain limitations and cannot accurately obtain lane line information. These limitations are mainly manifested in the inability to accurately identify the left and right edge lines, resulting in relatively poor lane line recognition accuracy. There are also schemes that use feature extraction for lane line recognition, but this method lacks generalization ability, is sensitive to image changes, and struggles to obtain high-precision lane line information.

[0080] To address the aforementioned issues, this application proposes a method, device, medium, program product, and vehicle for determining lane line information. This application determines lane line information corresponding to a lane line image based on lane line detection data and edge detection data, thereby improving the accuracy of lane line information identified from lane line images.

[0081] Specifically, the lane line information determination method in this application can be applied to electronic devices, which can be vehicles, devices installed on vehicles, or devices connected to vehicles, such as servers or mobile terminals. The following description uses an electronic device as an example to illustrate the various embodiments.

[0082] This application provides a method for determining lane line information. Please refer to [link / reference]. Figure 1 The lane line information determination method provided in this application includes steps S10-S20, which will be described in detail below.

[0083] S10. Determine the lane line detection data and edge detection data corresponding to the lane line image;

[0084] In this embodiment, a lane line image refers to an image containing lane lines captured by a camera or other imaging device in a road scene. Lane lines are important traffic signs and markings on the road used to divide lanes, indicate driving directions, and guide vehicle movement. By performing lane line detection on the lane line image, lane line detection data belonging to lane lines can be extracted from the lane line image. Lane line detection data can be used to characterize the position and direction of lane lines, but it cannot characterize or cannot accurately characterize the width of lane lines. Generally, lane detection data includes multiple lane line points or a single lane line curve; therefore, lane line detection data is not accurate or comprehensive enough in determining lane line information. By performing edge detection on the lane line image, edge detection data can be extracted from the lane line image to characterize the contour and boundary information in lane line image processing.

[0085] S20. Based on the lane line detection data and the edge detection data, determine the lane line information corresponding to the lane line image.

[0086] In this embodiment, by combining lane line detection information and edge detection information from the lane line image, the lane line edge can be quickly determined, thereby accurately determining comprehensive and accurate lane line information and improving the accuracy of lane line information.

[0087] In one embodiment, determining the lane line information corresponding to the lane line image based on the lane line detection data and the edge detection data includes:

[0088] Based on the lane line detection data and the edge detection data, multiple target lane line edge points are determined;

[0089] The lane line information is determined based on the multiple target lane line edge points.

[0090] In this embodiment, multiple target lane edge points can be determined by combining lane line detection data and edge detection data. These target lane edge points can be used to indicate the position of the lane line edge in the lane line image and characterize the outline of the lane line region in the lane line image. Multiple lane line edge points can form one or more curves to characterize the edge line of the lane line, thereby accurately determining lane line information based on multiple target lane line edge points.

[0091] In one embodiment, determining multiple target lane line edge points based on the lane line detection data and the edge detection data includes:

[0092] The target image is determined based on the lane line detection data and the edge detection data;

[0093] The edge points of the plurality of target lane lines are determined based on the target image.

[0094] In this embodiment, the lane line detection data and edge detection data come from the lane line image. For the same lane line recognition scenario, the lane line detection data and edge detection data can be plotted on the same binary image to obtain the target image. Based on the target image, multiple target lane line edge lines can be determined to obtain accurate lane line information.

[0095] In one embodiment, the target image includes a plurality of lane line points determined based on the lane line detection data, and the determination of the plurality of target lane line edge points based on the target image includes:

[0096] Determine the target lane edge point corresponding to each lane line point in the target image to obtain multiple target lane edge points.

[0097] In this embodiment, the target image includes multiple lane line points determined based on lane line detection data, such as... Figure 2 The pink lane line points shown can be represented in some embodiments as lane line detection data, which may include multiple lane line points obtained by performing lane line detection on the lane line image. For example, they may be lane line points detected by the UFLDv2 neural network. When generating the target image, the multiple lane line points of the lane line detection data can be synthesized into the target image so that the target image also includes these lane line points.

[0098] In this embodiment, the lane lines actually have width. Most of the lane line points determined based on the lane line detection data have fallen into the lane line area, but cannot completely represent the lane line area in the image. Based on the lane line points, their corresponding target lane line edge points can be determined. One lane line point can correspond to one or more target lane line edge points. The target lane line edge points can be located on the opposite side or both sides of the lane point.

[0099] In some embodiments, the lane line point is located within the center area of ​​the lane line. Based on the lane line point, two target lane line edge points located on both sides can be determined. In some embodiments, the lane line point is located on the edge of the lane line. Based on the lane line point, a target lane line edge point located on the opposite side can be determined. The lane line point can also be determined as the target lane line edge on the other side. Alternatively, two target lane line edge points located on both sides of the lane line can be determined based on the lane line point.

[0100] For each lane line point, one or more target lane line edge points can be determined. For multiple lane line points, multiple target lane line edge points can be determined, thus representing a more complete lane line and obtaining more accurate lane line information.

[0101] In one embodiment, the method further includes:

[0102] Based on the direction of the lane lines, determine the target lane line edge point corresponding to each lane line point in the target image.

[0103] In this embodiment, the target lane line edge points corresponding to each lane line point in the target image are determined sequentially according to the lane line direction. The lane line direction can be preset or obtained by recognizing the lane line image. Selecting lane line points according to the lane line direction to determine the corresponding target lane line edge points can cover the extraction of lane line information of the complete lane line and improve the robustness of the selected lane line edge points.

[0104] In one example, such as Figure 3 As shown, the lane line direction is from top to bottom in the target image. Therefore, the sampling is performed in regions from top to bottom. By selecting lane line points from top to bottom, the edge points of the target lane lines at different distances from far to near are filtered out in real-world conditions.

[0105] In one embodiment, the target image includes multiple edge points determined based on the edge detection data, and the step of determining the target lane line edge points corresponding to the lane line points based on the target image includes:

[0106] Obtain the first edge point among the plurality of edge points that is located within the preset area where the lane line point is located;

[0107] The target lane line edge point corresponding to the lane line point is determined based on the first edge point.

[0108] In this embodiment, the target image includes multiple edge points determined based on edge detection data, such as... Figure 2The white edge points shown can be used in some embodiments to represent edge detection data that includes multiple edge points obtained by edge detection of the lane line image. When generating the target image, these multiple edge points from the lane line detection data can be synthesized into the target image, ensuring that the target image also includes these edge points. For each lane line point, a preset region can be defined in the target image. If an edge point falls within the preset region of a lane line point, it is considered the first edge point corresponding to that lane line point. Figure 4 As shown, by identifying the first edge point within the preset area where the lane line point is located from among multiple edge points, the target lane edge point corresponding to the lane line point can be accurately determined, thereby further improving the accuracy of lane line information.

[0109] In one embodiment, determining the target lane line edge point corresponding to the lane line point based on the first edge point includes:

[0110] Determine the edge cluster centers corresponding to the lane line points;

[0111] Based on the edge cluster center and the first edge point, determine the target lane line edge point corresponding to the lane line point.

[0112] In this embodiment, the edge cluster center corresponding to the lane line point is determined. The edge cluster center can be used to characterize the location of the lane line edge point corresponding to the lane line point. Based on the first edge point in the preset area where the lane line point is located, and with the edge cluster center as the center, cluster optimization is performed to determine the target lane line edge point corresponding to the lane line point accurately, thereby further improving the accuracy of lane line information.

[0113] In one embodiment, determining the target lane line edge point corresponding to the lane line point based on the edge cluster center and the first edge point includes:

[0114] The second edge point is determined from the first edge point based on the edge cluster center;

[0115] The edge cluster center is updated based on the second edge point, and the process of determining the second edge point from the first edge point based on the updated edge cluster center is repeated until the preset iteration condition is met.

[0116] If the preset iteration conditions are met, the latest edge cluster center is determined as the target lane line edge point corresponding to the lane line point.

[0117] In this embodiment, all first edge points are traversed. Based on the edge cluster center, a first edge point is selected from all first edge points to be determined as a second edge point. The edge cluster center is then updated based on this second edge point to more accurately represent the lane line edge corresponding to the lane line. Then, based on the updated edge cluster center, the process of determining the second edge point from the first edge points is repeated. It should be noted that if a first edge point is determined as a second edge point, it will not be re-determined as a second edge point. This iterative optimization of the edge cluster center using different first edge points continues until a preset iteration condition is met. When the preset iteration condition is met, the latest edge cluster center is determined as the target lane line edge point corresponding to the lane line point, making the target lane line edge point more accurate and further improving the accuracy of lane line information.

[0118] In one embodiment, the preset iteration conditions include the number of updates to the edge cluster centers reaching a preset threshold, and / or all the first edge points being determined as the second edge points.

[0119] In this embodiment, during the iterative update of the edge cluster center through the first edge point, when it is detected that the number of iterations of the edge cluster center reaches a preset threshold, or when it is detected that all the first edge points are determined to be second edge points, it indicates that the edge cluster center has reached the required accuracy, and the iterative update of the edge cluster center can be ended.

[0120] In one embodiment, determining the second edge point from the first edge point based on the edge cluster center includes:

[0121] The edge point that is closest to the edge cluster center among the first edge points is determined as the second edge point.

[0122] In this embodiment, when determining the second edge point from the first edge point, the edge point closest to the edge cluster center among the first edge points is determined as the second edge point, so as to update from the nearest point of the edge cluster center.

[0123] In some embodiments, there are at least two edge cluster centers, such as two edge cluster centers on the left and right sides. Then, the edge point with the smallest sum of distances to each edge cluster center among the first edge points can be determined, that is, the edge point closest to the edge cluster center, and this edge point can be determined as the second edge point.

[0124] In one embodiment, updating the edge cluster center based on the second edge point includes:

[0125] The edge cluster centers are updated by merging the second edge point and the edge cluster center.

[0126] In this embodiment, the second edge points are fused to the edge cluster center to update the second edge cluster center. By fusing multiple real second edge points within the preset area of ​​the lane line, the target lane line edge points corresponding to the lane line points are obtained. Each lane line point is iteratively updated to obtain the target lane line edge points, which makes the curves connecting multiple target lane line edge points smoother. This not only represents the real lane lines more accurately but also makes it easier to extract lane line information, thereby further improving the accuracy of lane line information.

[0127] In one embodiment, the edge clustering centers include at least two edge clustering centers of the lane line, and the fusion based on the second edge point and the edge clustering centers includes:

[0128] Among the at least two edge cluster centers, the target edge cluster center that belongs to the same side as the second edge point is determined;

[0129] The second edge point is merged into the target edge cluster center.

[0130] In this embodiment, the edge clustering center includes at least two edge clustering centers of the lane line, such as edge clustering centers located on the left and right sides of the lane line. If there are at least two clustering centers, it is necessary to determine the affiliation of the second edge point, determine which side of the lane line point the second edge point belongs to, determine the target edge clustering center on the same side as the second edge point among the at least two edge clustering centers, and merge the second edge point into the target edge clustering center to update the edge clustering center corresponding to the lane line point.

[0131] In one embodiment, determining the edge cluster center corresponding to the lane line point includes:

[0132] Determine the target tangent based on the lane line points and lane line direction;

[0133] The edge cluster centers of the lane line points are determined based on the target tangent.

[0134] In this embodiment, the initial edge cluster center corresponding to the lane line point can be determined by drawing a straight line or line segment through the lane line point based on the lane line direction as the target tangent. The target tangent can represent a line perpendicular to the real lane line. Due to the influence of the shooting angle, it may not be perpendicular to the lane line direction in the target image. Based on the target tangent, the edge cluster center is determined to the opposite side or both sides of the lane line point, thereby determining the accurate edge cluster center.

[0135] In one embodiment, determining the edge clustering center of the lane line points based on the target tangent includes:

[0136] The edge cluster center is determined from the first edge point based on the target tangent.

[0137] In this embodiment, for each lane line edge corresponding to the lane line point, the first edge point closest to the target tangent among the first edge points corresponding to that lane line edge is determined and identified as the edge cluster center. Using the first edge point as the initial edge cluster center can enable the edge cluster center to more realistically and accurately represent the lane line edge, thereby further improving the accuracy of the lane line edge.

[0138] In one example, such as Figure 4 As shown, for a certain lane line point, which belongs to the lane line area, take the lane line point as the center point and search for all first edge points within the radius R of the center point in the binary image. Take the lane line point as the center and draw horizontal tangents to find the initial edge cluster centers on the left and right sides of the lane line point. Traverse all first edge points and iteratively update the edge cluster centers on the same side starting from the nearest first edge of the edge cluster centers on the left and right sides. Through multiple iterations, output the optimal edge cluster centers on the left and right sides of the lane line point.

[0139] In one embodiment, the plurality of target lane line edge points include target lane line edge points on the left and right sides of the lane line, and determining the lane line information based on the plurality of target lane line edge points includes:

[0140] Determine the distance difference between the edge points of the target lane line on the left and the edge points of the target lane line on the right.

[0141] The lane width of the lane line image is determined based on the distance difference and a preset linear relationship to obtain the lane line information.

[0142] In this embodiment, the determined multiple target lane line edge points may include the target lane line edge points on the left and right sides of the lane line. The distance difference between the left and right target lane line edge points is determined; this distance difference is the distance difference between points in the target image and needs to be converted into the actual lane line width in the lane line image. A preset linear relationship is pre-defined as the conversion basis. Based on the preset linear relationship, the distance difference between the target lane line edge points on the left and right sides is converted to determine the actual lane line width corresponding to the lane line image, thereby obtaining accurate lane line information.

[0143] In one embodiment, the method further includes:

[0144] The edge points of the multiple target lane lines are transformed according to the top view perspective;

[0145] Based on the transformed target lane edge points, the distance difference between the target lane edge points on the left and right is determined.

[0146] In this embodiment, the lane line image is generally a front-view image. The actual parallel lines are not horizontal in the front-view image. Therefore, the distance difference between the target lane line edge point and the right-side target lane line edge point varies depending on the lane line point. To reduce computational overhead, the extrinsic parameter mapping matrix from image point (x, y) to top-view projection point (x_h, y_h) can be calculated for multiple target lane line edge points according to the top-view perspective. The conversion formula is as follows:

[0147]

[0148] This yields multiple target lane edge points from a top-down view. The curves connecting the left and right target lane edge points can be parallel. The distance difference between the left and right target lane edge points can represent the actual lane width. A pre-calibrated linear relationship can be used to convert the distance difference between the two target lane edge points, thus quickly and accurately obtaining the lane width.

[0149] In some embodiments, when the lane line image is not a top-down view, the lane line detection data and edge detection data can be converted according to the top-down view first, and then the subsequent process of determining lane line information can be performed based on the converted lane line detection data and edge detection data, thereby further improving the robustness of the algorithm.

[0150] In one embodiment, the determination of lane line detection data and edge detection data corresponding to the lane line image includes:

[0151] The lane line detection data is obtained by performing lane line detection on the lane line image using a deep learning algorithm.

[0152] In this embodiment, for such Figure 5 The lane line image shown can be input into deep learning algorithms such as UFLDv2. These algorithms perform lane line detection on the image, yielding accurate lane line detection data. Figure 6 As shown.

[0153] In one embodiment, the determination of lane line detection data and edge detection data corresponding to the lane line image includes:

[0154] The lane line image is convolved using a preset convolution operator to obtain a feature image;

[0155] The edge detection data is determined based on the feature image.

[0156] In this embodiment, edge detection is performed. During the edge detection process, a preset convolution operator can be used to convolve the lane line image to obtain a feature image. Then, edge detection data can be obtained based on the feature image. The obtained edge detection data can be as follows: Figure 7 The black-and-white binary image shown represents the edge regions, with white areas. The preset convolution operator can be the Sobel operator. However, a scene-specific preset convolution operator is determined by combining the lane line information in this embodiment to improve the robustness of edge detection and the accuracy of lane line information.

[0157] In one embodiment, the preset convolution operator includes at least one of a horizontal convolution operator, a vertical convolution operator, and a diagonal convolution operator.

[0158] In this embodiment, the preset convolution operator includes at least one of a horizontal convolution operator, a vertical convolution operator, and a diagonal convolution operator, wherein the horizontal convolution operator is:

[0159]

[0160] The vertical convolution operator is:

[0161]

[0162] The oblique steering operator can include a first oblique steering operator and a second oblique steering operator, wherein the first oblique steering operator is:

[0163]

[0164] The second oblique steering operator is:

[0165]

[0166] In one embodiment, determining the edge detection data based on the feature image includes:

[0167] Based on the gradient intensity of each pixel in the feature image and the edge gradient intensity threshold, the edge pixels in the feature image are determined to determine the edge detection data.

[0168] In this embodiment, the gradient intensity of the edge region in the feature image can be clearly distinguished from the gradient intensity of the non-edge region. The gradient intensity of each pixel in the feature image is filtered by a preset edge gradient intensity threshold to select edge pixels in the feature image. The feature image scene is consistent with the lane line image, and the selected edge pixels in the feature image can be used as edge detection data of the lane line image.

[0169] In one embodiment, the edge gradient intensity threshold includes a first threshold and a second threshold. Determining edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold to determine the edge detection data includes:

[0170] If the gradient intensity of the pixel is greater than or equal to the first threshold, the pixel is determined to be an edge pixel; or,

[0171] If the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, and the neighboring pixels of the pixel are edge pixels, then the pixel is determined to be an edge pixel.

[0172] In this embodiment, a dual edge gradient intensity threshold is used to filter edges. The defined edge gradient intensity threshold includes a first threshold and a second threshold, which are respectively a high gradient intensity threshold and a low gradient intensity threshold. Using the dual edge gradient intensity threshold, pixels can be divided into strong edge pixels, weak edge pixels, and non-edge pixels. If the gradient intensity of a pixel is greater than or equal to the first threshold, the pixel is a high edge pixel. If the gradient intensity of a pixel is less than the first threshold, the pixel is a non-edge pixel. If the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, the pixel is a weak edge pixel. By examining the neighborhood of strong edge pixels, weak edge pixels are connected to strong edge pixels and updated to strong edge pixels to obtain more complete edges. That is, if the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, and the pixel's neighboring pixels are edge pixels, the pixel can be regarded as a strong edge pixel. Finally, all strong edge pixels can be used as the final edge pixels for output, improving the accuracy of edge detection data.

[0173] In one embodiment, the method further includes:

[0174] The lane line image is preprocessed, including grayscale conversion and / or Gaussian filtering.

[0175] Based on the preprocessed lane line image, a feature image is obtained by performing convolution processing on the lane line image using a preset convolution operator.

[0176] In this embodiment, after obtaining the original lane line image, it will be converted to grayscale to convert the color image to a grayscale image. The formula for converting an RGB image to a grayscale image is as follows;

[0177]

[0178] Then, Gaussian filtering is applied to achieve Gaussian blur of the image, which is a weighted average of the current pixel and its surrounding pixels, thus suppressing noisy pixels. The Gaussian formula is:

[0179]

[0180] In this embodiment, considering the lane line recognition scenario, a 3×3 Gaussian kernel can be used for Gaussian filtering:

[0181]

[0182] In one embodiment, the method further includes:

[0183] Based on the gradient intensity and gradient direction of each pixel in the feature image, non-maximum suppression processing is performed on the feature image. Based on the feature image after non-maximum suppression processing, the edge detection data is determined according to the pixel intensity of each pixel in the feature image.

[0184] In this embodiment, for the preprocessed lane line image, the gradient intensity of each pixel in the lane line image is obtained by taking the square root of the sum of squares, and the gradient direction of each pixel is calculated using the arctangent. Based on the gradient intensity and gradient direction of each pixel, non-maximum suppression processing is applied to the lane line image to refine edges and suppress responses on non-edge surfaces. For each pixel, only the pixel with the maximum gradient value is retained, while other pixels are suppressed, thus improving edge detection accuracy.

[0185] Accordingly, embodiments of this application also provide an electronic device, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1100 further includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0186] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions and processes data of the electronic device 1100, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a processor (Central Processing Unit, CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0187] In this embodiment, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as:

[0188] Determine the lane line detection data and edge detection data corresponding to the lane line image;

[0189] Based on the lane line detection data and the edge detection data, the lane line information corresponding to the lane line image is determined.

[0190] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0191] Optional, such as Figure 8 As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0192] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be used as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0193] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with networked medical devices or other electronic devices, and to transmit and receive signals with networked medical devices or other electronic devices.

[0194] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts the collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0195] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0196] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management device, thereby enabling functions such as charging, discharging, and power consumption management through the power management device. Power supply 1107 may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0197] although Figure 8 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0198] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0199] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0200] Therefore, embodiments of this application provide a computer-readable storage medium storing multiple computer programs. These computer programs can be loaded by a processor to execute any of the lane line information determination methods provided in this application. The computer program can execute the steps of the following lane line information determination method:

[0201] Determine the lane line detection data and edge detection data corresponding to the lane line image;

[0202] Based on the lane line detection data and the edge detection data, the lane line information corresponding to the lane line image is determined.

[0203] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0204] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0205] Since the computer-readable storage medium contains a computer program that can implement any of the lane line information determination methods provided in the embodiments of this application, and can execute any of the lane line information determination methods provided in the embodiments of this application, the effects are detailed in the preceding embodiments and will not be repeated here.

[0206] This application also provides a computer program product that can be loaded by a processor to execute any of the lane line information determination methods provided in this application. Specific implementations of each operation of this lane line information determination method can be found in the preceding embodiments and will not be repeated here.

[0207] Since this computer program can execute any of the lane line information determination methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the lane line information determination methods provided in the embodiments of this application can achieve. Therefore, its beneficial effects are detailed in the preceding embodiments and will not be repeated here.

[0208] This application also provides a vehicle that includes any of the above-described electronic devices, computer-readable storage media, computer program products, or performs any of the above-described methods.

[0209] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0210] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0211] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0212] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method for determining lane line information, characterized in that, include: The lane line detection data and edge detection data corresponding to the lane line image are determined. The lane line detection data is obtained by performing lane line detection on the lane line image and is used to characterize the position and direction of the lane line. The edge detection data is obtained by performing edge detection on the lane line image and is used to characterize the contour and boundary information in the lane line image processing. A target image is determined based on the lane line detection data and the edge detection data. The target image includes multiple lane line points determined based on the lane line detection data and multiple edge points determined based on the edge detection data. Determine the target lane edge point corresponding to each lane line point in the target image to obtain multiple target lane edge points; Determining the target lane line edge point corresponding to each lane line point in the target image includes: Obtain the first edge point among the plurality of edge points that is located within the preset area where the lane line point is located; Determine the target tangent based on the lane line points and lane line direction; Based on the target tangent, the edge cluster center of the lane line point is determined from the first edge point. The edge cluster center is used to characterize the location of the lane line edge point corresponding to the lane line point. Clustering optimization is performed based on the first edge point within a preset area where the lane line point is located, using the edge cluster center as the center, to determine the target lane line edge point corresponding to the lane line point, including: The edge point that is closest to the edge cluster center among the first edge points is determined as the second edge point; The second edge point and the edge cluster center are fused together to update the edge cluster center. Based on the updated edge cluster center, the process of determining the second edge point from the first edge point based on the edge cluster center is repeated until the preset iteration condition is met. Under the condition that the preset iteration condition is met, the latest edge cluster center is determined as the target lane line edge point corresponding to the lane line point; Based on the multiple target lane line edge points, determine the lane line information corresponding to the lane line image.

2. The method as described in claim 1, characterized in that, The method further includes: Based on the direction of the lane lines, determine the target lane line edge point corresponding to each lane line point in the target image.

3. The method as described in claim 1, characterized in that, The preset iteration conditions include the number of updates to the edge cluster centers reaching a preset threshold, and / or all the first edge points being determined as the second edge points.

4. The method as described in claim 1, characterized in that, The edge clustering centers include at least two edge clustering centers of the lane line, and the fusion based on the second edge point and the edge clustering centers includes: Among the at least two edge cluster centers, the target edge cluster center that belongs to the same side as the second edge point is determined; The second edge point is merged into the target edge cluster center.

5. The method as described in claim 1, characterized in that, The plurality of target lane line edge points include target lane line edge points on the left and right sides of the lane line. Determining the lane line information corresponding to the lane line image based on the plurality of target lane line edge points includes: Determine the distance difference between the edge point of the target lane line on the left and the edge point of the target lane line on the right. The lane width of the lane line image is determined based on the distance difference and a preset linear relationship to obtain the lane line information.

6. The method as described in claim 5, characterized in that, The method further includes: The edge points of the multiple target lane lines are transformed according to the top view perspective; Based on the converted target lane edge points, the distance difference between the target lane edge points on the left and the target lane edge points on the right is determined.

7. The method according to any one of claims 1-6, characterized in that, The lane line detection data and edge detection data corresponding to the determined lane line image include: The lane line detection data is obtained by performing lane line detection on the lane line image using a deep learning algorithm.

8. The method according to any one of claims 1-6, characterized in that, The lane line detection data and edge detection data corresponding to the determined lane line image include: The lane line image is convolved using a preset convolution operator to obtain a feature image; The edge detection data is determined based on the feature image.

9. The method as described in claim 8, characterized in that, The preset convolution operator includes at least one of the following: horizontal convolution operator, vertical convolution operator, and diagonal convolution operator.

10. The method as described in claim 8, characterized in that, Determining the edge detection data based on the feature image includes: Based on the gradient intensity of each pixel in the feature image and the edge gradient intensity threshold, the edge pixels in the feature image are determined to determine the edge detection data.

11. The method as described in claim 10, characterized in that, The edge gradient intensity threshold includes a first threshold and a second threshold. The determination of edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold, to determine the edge detection data, includes: If the gradient intensity of the pixel is greater than or equal to the first threshold, the pixel is determined to be an edge pixel; or, If the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, and the neighboring pixels of the pixel are edge pixels, then the pixel is determined to be an edge pixel.

12. The method as described in claim 8, characterized in that, The method further includes: The lane line image is preprocessed, including grayscale conversion and / or Gaussian filtering. Based on the preprocessed lane line image, a feature image is obtained by performing convolution processing on the lane line image using a preset convolution operator.

13. The method as described in claim 8, characterized in that, The method further includes: Based on the gradient intensity and gradient direction of each pixel in the feature image, non-maximum suppression processing is performed on the feature image. Based on the feature image after non-maximum suppression processing, the edge detection data is determined according to the pixel intensity of each pixel in the feature image.

14. An electronic device, characterized in that, The device includes a processor connected to a memory storing a computer program, the processor being configured to run the computer program in the memory to perform the lane line information determination method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the lane line information determination method according to any one of claims 1 to 13.

16. A computer program product, characterized in that, It includes a computer program, which is executed by a processor to implement the lane line information determination method according to any one of claims 1 to 13.

17. A vehicle, characterized in that, The vehicle performs the lane line information determination method as described in any one of claims 1-13, or includes the electronic device as described in claim 14.

Images