Lane line information determination method and device, medium, program product and vehicle

Abstract

The invention relates to a lane line information determination method and device, a medium, a program product and a vehicle. The method comprises the following steps: determining lane line detection data and edge detection data corresponding to a lane line image; and determining lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data. The invention aims to improve the accuracy of lane line information.

Description

Technical Field

[0001] This application relates to the field of lane line recognition, and in particular, to a method, device, medium, program product, and vehicle for determining lane line information. Background Art

[0002] Lane line recognition is a technology in the field of computer vision for detecting and measuring lane lines on roads, which is of great significance for intelligent driving, road traffic planning, and monitoring systems. In related technologies, deep learning models are generally used for lane line recognition, but the lane line recognition performed by deep learning models has certain limitations and cannot accurately obtain lane line information. Summary of the Invention

[0003] Embodiments of this application provide a method, device, medium, program product, and vehicle for determining lane line information, which improve the accuracy of lane line information to at least partially solve the above technical problems.

[0004] To achieve the above objective, according to the first aspect of this application, a method for determining lane line information is provided. The method for determining lane line information includes: Determine lane line detection data and edge detection data corresponding to a lane line image; Determine lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data.

[0005] Optionally, the determining lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data includes: Determine a plurality of target lane line edge points according to the lane line detection data and the edge detection data; Determine the lane line information according to the plurality of target lane line edge points.

[0006] Optionally, the determining a plurality of target lane line edge points according to the lane line detection data and the edge detection data includes: Determine a target image based on the lane line detection data and the edge detection data; Determine the plurality of target lane line edge points based on the target image.

[0007] Optionally, the target image includes a plurality of lane line points determined based on the lane line detection data. The determining the plurality of target lane line edge points based on the target image includes: Determine a target lane line edge point corresponding to each lane line point in the target image to obtain a plurality of target lane edge points.

[0008] Optionally, the method further includes: Determine the target lane line edge points corresponding to each lane line point in the target image according to the lane line direction.

[0009] Optionally, the target image includes a plurality of edge points determined based on the edge detection data. Determining the target lane line edge points corresponding to each lane line point in the target image includes: Obtain a first edge point within a preset area where the lane line point is located among the plurality of edge points; Determine the target lane line edge point corresponding to the lane line point according to the first edge point.

[0010] Optionally, the determining the target lane line edge point corresponding to the lane line point according to the first edge point includes: Determine the edge clustering center corresponding to the lane line point; Determine the target lane line edge point corresponding to the lane line point according to the edge clustering center and the first edge point.

[0011] Optionally, the determining the target lane line edge point corresponding to the lane line point according to the edge clustering center and the first edge point includes: Determine a second edge point from the first edge points based on the edge clustering center; Update the edge clustering center based on the second edge point, and re - execute determining the second edge point from the first edge points based on the updated edge clustering center until a preset iteration condition is met; When the preset iteration condition is met, determine the latest edge clustering center as the target lane line edge point corresponding to the lane line point.

[0012] Optionally, the preset iteration condition includes that the number of updates of the edge clustering center reaches a preset threshold, and / or all the first edge points are determined as the second edge points.

[0013] Optionally, the determining the second edge point from the first edge points based on the edge clustering center includes: Determine the edge point closest to the edge clustering center among the first edge points as the second edge point.

[0014] Optionally, the updating the edge clustering center based on the second edge point includes: Fuse the second edge point and the edge clustering center to update the edge clustering center.

[0015] Optionally, the edge clustering centers include at least two side edge clustering centers of the lane line, and the fusion based on the second edge points and the edge clustering centers includes: Determine a target edge clustering center that belongs to the same side as the second edge point among the at least two side edge clustering centers; Fuse the second edge point into the target edge clustering center.

[0016] Optionally, the determination of the edge clustering center corresponding to the lane line point includes: Determine a target tangent line based on the lane line point and the lane line direction; Determine the edge clustering center of the lane line point based on the target tangent line.

[0017] Optionally, the determination of the edge clustering center of the lane line point based on the target tangent line includes: Determine the edge clustering center from the first edge points based on the target tangent line.

[0018] Optionally, the multiple target lane line edge points include target lane line edge points on the left side and the right side of the lane line, and the determination of the lane line information based on the multiple target lane line edge points includes: Determine the distance difference between the target lane line edge points on the left side and the target lane line edge points on the right side; Determine the lane line width of the lane line image according to the distance difference and a preset linear relationship to obtain the lane line information.

[0019] Optionally, the method further includes: Convert the multiple target lane line edge points according to the perspective of a top view; Based on the converted multiple target lane line edge points, perform the determination of the distance difference between the target lane line edge points on the left side and the target lane line edge points on the right side.

[0020] Optionally, the determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Perform lane line detection on the lane line image through a deep learning algorithm to obtain the lane line detection data.

[0021] Optionally, the determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Perform convolution processing on the lane line image using a preset convolution operator to obtain a feature image; Determine the edge detection data according to the feature image.

[0022] Optionally, the preset convolution operator includes at least one of a horizontal convolution operator, a vertical convolution operator, and an oblique convolution operator.

[0023] Optionally, determining the edge detection data according to the feature image includes: Determining edge pixels in the feature image based on the gradient intensity of each pixel in the feature image and an edge gradient intensity threshold to determine the edge detection data.

[0024] Optionally, the edge gradient intensity threshold includes a first threshold and a second threshold. Determining edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold to determine the edge detection data includes: When the gradient intensity of the pixel is greater than or equal to the first threshold, determining the pixel as an edge pixel; or, When the gradient intensity of the pixel is less than the first threshold or greater than the second threshold, and an adjacent pixel of the pixel is an edge pixel, determining the pixel as an edge pixel.

[0025] Optionally, the method further includes: Performing preprocessing on the lane line image, where the preprocessing includes grayscale conversion processing and / or Gaussian filtering processing; Performing convolution processing on the lane line image by using a preset convolution operator based on the preprocessed lane line image to obtain a feature image.

[0026] Optionally, the method further includes: Performing non-maximum suppression processing on the feature image based on the gradient intensity and gradient direction of each pixel in the feature image, and determining the edge detection data according to the pixel intensity of each pixel in the feature image based on the feature image after non-maximum suppression processing.

[0027] According to a second aspect of the present application, there is also provided an electronic device, including a processor, the processor is connected to a memory, the memory stores a computer program, and the processor is configured to run the computer program in the memory to execute any method provided in this embodiment.

[0028] According to a third aspect of the present application, there is provided a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any method provided in this embodiment.

[0029] According to a fourth aspect of the present application, there is provided a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements any method provided in this embodiment.

[0030] According to a fifth aspect of the present application, there is provided a vehicle that executes any method of the embodiments of the present application, or includes the electronic device as described above.

[0031] In summary, in the embodiments of the present application, through the above technical solutions, lane line detection data and edge detection data corresponding to a lane line image are determined, and lane line information corresponding to the lane line image is determined by combining the lane line detection data and the edge detection data, which can improve the accuracy of the lane line information recognized for the lane line image.

[0032] Other features and advantages of the present application will be described in detail in the subsequent specific implementation section. Description of the Drawings

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative efforts.

[0034] In order to more fully understand the present application and its beneficial effects, the following description will be made in conjunction with the drawings, where the same reference numerals represent the same parts in the following description.

[0035] Figure 1 is a schematic flowchart of an embodiment of a method for determining lane line information provided in an embodiment of the present invention; Figure 2 is a schematic diagram of a target image provided in an embodiment of the present invention; Figure 3 is a schematic diagram of lane line sampling provided in an embodiment of the present invention; Figure 4 is a schematic diagram of an edge clustering center provided in an embodiment of the present invention; Figure 5 is a schematic diagram of a lane line image provided in an embodiment of the present invention; Figure 6 is a schematic diagram of lane line detection data corresponding to a lane line image provided in an embodiment of the present invention; Figure 7 is a schematic diagram of edge detection data corresponding to a lane line image provided in an embodiment of the present invention; Figure 8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Description of the Embodiments

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0037] Based on the problems mentioned in the foregoing background art, in the related art, a deep learning model is generally used for lane line recognition. However, the lane line recognition performed by the deep learning model has certain limitations and cannot accurately obtain lane line information. This limitation is mainly manifested in the inability to accurately identify the left and right edge lines, and actually the recognition accuracy of the lane lines is poor. There is also a solution that uses feature extraction for lane line recognition, but this method lacks generalization ability, is sensitive to image changes, and it is also difficult to obtain high-precision lane line information.

[0038] To solve the above problems, the embodiments of the present application propose a lane line information determination method, device, medium, program product, and vehicle. The embodiments of the present application determine the lane line information corresponding to the lane line image according to the lane line detection data and edge detection data corresponding to the lane line image, which can improve the accuracy of the lane line information recognized for the lane line image.

[0039] Specifically, the lane line information determination method in the present application can be applied to an electronic device, which can also be a vehicle, or is set on a vehicle, or is a device connected to the vehicle, such as a server, a mobile terminal, etc. Subsequently, taking the execution subject of the lane line information determination method as an electronic device as an example, each embodiment will be described in detail.

[0040] The present application provides a lane line information determination method. Please refer to Figure 1 , the lane line information determination method provided by the embodiments of the present application includes steps S10 - step S20, which will be introduced in detail below.

[0041] S10. Determine the lane line detection data and edge detection data corresponding to the lane line image; In this embodiment, the lane line image refers to an image containing lane lines captured by a camera or other imaging device in a road scene. Lane lines are important traffic markings on the road used to divide lanes, indicate driving directions, and guide vehicle travel. By performing lane line detection on the lane line image, lane line detection data belonging to the lane lines in the lane line image can be extracted. The lane line detection data can be used to characterize the position, orientation, etc. of the lane lines, but cannot characterize or accurately characterize the lane line width. Generally, the lane detection data includes multiple lane line points or a lane line curve. Therefore, it is not accurate and comprehensive enough to determine lane line information based on the lane line detection data. By performing edge detection on the lane line image, edge detection data in the lane line image can be extracted to characterize the information of the contours and boundaries in the lane line image processing.

[0042] S20. Determine the lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data.

[0043] In this embodiment, by combining the lane line detection information and the edge detection information of the lane line image, the lane line edge can be quickly determined, so as to accurately determine comprehensive and accurate lane line information and improve the accuracy of the lane line information.

[0044] In one embodiment, the determining the lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data includes: Determine a plurality of target lane line edge points according to the lane line detection data and the edge detection data; Determine the lane line information according to the plurality of target lane line edge points.

[0045] In this embodiment, according to the lane line detection data and the edge detection data, a plurality of target lane edge points can be comprehensively determined. The target lane edge points can be used to indicate the lane line edge positions in the lane line image and characterize the contours of the lane line regions in the lane line image. A plurality of lane line edge points can form one or more curves to characterize the edge lines of the lane lines. Therefore, based on the plurality of target lane line edge points, the lane line information can be accurately determined.

[0046] In one embodiment, the determining a plurality of target lane line edge points according to the lane line detection data and the edge detection data includes: Determine a target image based on the lane line detection data and the edge detection data; Determine the plurality of target lane line edge points based on the target image.

[0047] In this embodiment, the lane line detection data and the edge detection data are from a lane line image. For a specific same lane line recognition scenario, the lane line detection data and the edge detection data can be drawn on the same binary image to obtain a target image. Based on the target image, multiple target lane line edge lines can be determined to obtain accurate lane line information.

[0048] In one embodiment, the target image includes multiple lane line points determined based on the lane line detection data. Determining the multiple target lane line edge points based on the target image includes: Determining the target lane line edge point corresponding to each lane line point in the target image to obtain multiple target lane edge points.

[0049] In this embodiment, the target image includes multiple lane line points determined based on the lane line detection data, such as Figure 2 the pink lane line points shown. In some embodiments, the lane line detection data may include multiple lane line points obtained by performing lane line detection on a lane line image. For example, they may be lane line points detected by the neural network UFLDv2. When generating the target image, the multiple lane line points of the lane line detection data can be synthesized into the target image, so that the target image also includes these lane line points.

[0050] In this embodiment, the lane line actually has a width. Most of the multiple lane line points determined based on the lane line detection data have fallen within the lane line area, but they cannot fully represent the lane line area in the image. Based on the lane line points, the corresponding target lane line edge points can be determined. One lane line point can correspond to one or more target lane line edge points, and the target lane line edge points can be located on the opposite side or both sides of the lane point.

[0051] In some embodiments, the lane line points are located within the center area of the lane line. Based on the lane line points, two target lane line edge points located on both sides of them can be determined. In some embodiments, the lane line points are located on the edge of the lane line. Based on the lane line points, one target lane line edge point located on the opposite side of them can be determined, and the lane line point can also be determined as the target lane line edge on the other side. Two target lane line edge points located on both sides of the lane line can also be determined based on the lane line points.

[0052] For each lane line point, one or more target lane line edge points can be determined. For multiple lane line points, multiple target lane line edge points can be determined, so as to represent a more complete lane line and obtain more accurate lane line information.

[0053] In one embodiment, the method further includes: Determining the target lane line edge point corresponding to each lane line point in the target image according to the lane line direction.

[0054] In this embodiment, according to the lane line direction, the target lane line edge points corresponding to each lane line point in the target image are determined in sequence. The lane line direction can be preset or obtained by recognizing the lane line image. Selecting the lane line points according to the lane line direction to determine the corresponding target lane line edge points can cover the extraction of the lane line information of the complete lane line and improve the robustness of the screening of the selected lane line edge points.

[0055] In one example, as Figure 3 shown, the lane line direction is from top to bottom in the target image. Therefore, region sampling is performed from top to bottom. By selecting lane line points from top to bottom, the screening of the target lane line edge points at different distances from far to near in the actual situation is completed.

[0056] In one embodiment, the target image includes a plurality of edge points determined based on the edge detection data. Determining the target lane line edge points corresponding to the lane line points based on the target image includes: Obtaining a first edge point in the preset region where the lane line point is located among the plurality of edge points; Determining the target lane line edge point corresponding to the lane line point according to the first edge point.

[0057] In this embodiment, the target image includes a plurality of edge points determined based on the edge detection data, such as Figure 2 the white edge points shown. In some embodiments, the edge detection data may include a plurality of edge points obtained by performing edge detection on the lane line image. When generating the target image, the plurality of edge points of the lane line detection data can be synthesized into the target image, so that the target image also includes these edge points. For each lane line point, a preset region can be correspondingly divided in the target image. If an edge point falls into the preset region where a certain lane line point is located, it is used as the first edge point corresponding to the lane line point. As Figure 4 shown, determining the first edge point in the preset region where the lane line point is located among the plurality of edge points can accurately determine the target lane edge point corresponding to the lane line point, so as to further improve the accuracy of the lane line information.

[0058] In one embodiment, the determining the target lane line edge point corresponding to the lane line point according to the first edge point includes: Determining the edge clustering center corresponding to the lane line point; Determining the target lane line edge point corresponding to the lane line point according to the edge clustering center and the first edge point.

[0059] In this embodiment, the edge clustering center corresponding to the lane line points is determined. The edge clustering center can be used to represent the position where the lane line edge points corresponding to the lane line points are located. According to the first edge points within the preset area where the lane line points are located, and performing clustering optimization with the edge clustering center as the center, the target lane line edge points corresponding to the lane line points can be accurately determined, further improving the accuracy of the lane line information.

[0060] In one embodiment, the determining the target lane line edge points corresponding to the lane line points according to the edge clustering center and the first edge points includes: Determining second edge points from the first edge points based on the edge clustering center; Updating the edge clustering center based on the second edge points, and re - executing determining second edge points from the first edge points based on the updated edge clustering center until a preset iteration condition is met; When the preset iteration condition is met, determining the latest edge clustering center as the target lane line edge points corresponding to the lane line points.

[0061] In this embodiment, all the first edge points are traversed. The first edge points selected from all the first edge points based on the edge clustering center are determined as the second edge points, so as to update the edge clustering center based on the second edge points, making it more accurately represent the lane line edge corresponding to the lane line. Then, re - execute determining the second edge points from the first edge points based on the updated edge clustering center. It should be noted that if a first edge point is determined as a second edge point, it will not be re - determined as a second edge point as a first edge point, so as to iteratively optimize the edge clustering center through different first edge points until the preset iteration condition is met. When the preset iteration condition is met, determining the latest edge clustering center as the target lane line edge points corresponding to the lane line points can make the target lane line edge points more accurate and further improve the accuracy of the lane line information.

[0062] In one embodiment, the preset iteration condition includes that the number of updates of the edge clustering center reaches a preset threshold, and / or all the first edge points are determined as the second edge points.

[0063] In this embodiment, during the process of iteratively updating the edge clustering center through the first edge points, when it is detected that the number of iterative updates of the edge clustering center reaches the preset threshold, or when it is detected that all the first edge points are determined as the second edge points, it indicates that the edge clustering center has reached the required accuracy, and the iterative update of the edge clustering center can be ended.

[0064] In one embodiment, the determining the second edge points from the first edge points based on the edge clustering center includes: Determine the edge point among the first edge points that is closest to the edge clustering center as the second edge point.

[0065] In this embodiment, when determining the second edge point from the first edge points, the edge point among the first edge points that is closest to the edge clustering center is determined as the second edge point for updating from the closest point to the edge clustering center.

[0066] In some embodiments, there are at least two edge clustering centers, such as two edge clustering centers on the left and right sides. Then, the edge point among the first edge points with the smallest sum of distances to each edge clustering center, that is, the edge point closest to the edge clustering center, can be determined and used as the second edge point.

[0067] In one embodiment, updating the edge clustering center based on the second edge point includes: Fuse the second edge point and the edge clustering center to update the edge clustering center.

[0068] In this embodiment, the second edge point is fused into the edge clustering center to update the second edge clustering center. By fusing multiple real second edge points within the preset area of the lane line, the target lane line edge points corresponding to the lane line points are obtained. Each lane line point is iteratively updated to obtain the target lane line edge points, which can make the curve formed by multiple target lane line edge points smoother. This can not only more accurately represent the real lane line but also facilitate the extraction of lane line information, thereby further improving the accuracy of lane line information.

[0069] In one embodiment, the edge clustering center includes at least two edge clustering centers on both sides of the lane line. The fusing according to the second edge point and the edge clustering center includes: Determine the target edge clustering center on the same side as the second edge point among the at least two edge clustering centers on both sides; Fuse the second edge point into the target edge clustering center.

[0070] In this embodiment, the edge clustering center includes at least two edge clustering centers on both sides of the lane line, such as the edge clustering centers on the left and right sides of the lane line. If there are at least two clustering centers, it is necessary to determine the attribution of the second edge point to determine which side of the lane line point it belongs to. Determine the target edge clustering center on the same side as the second edge point among the at least two edge clustering centers on both sides, and fuse the second edge point into the target edge clustering center to update the edge clustering center corresponding to the lane line point.

[0071] In one embodiment, determining the edge clustering center corresponding to the lane line point includes: Determine a target tangent line based on the lane line points and the lane line direction; Determine the edge clustering center of the lane line points based on the target tangent line.

[0072] In this embodiment, for the initial edge clustering center corresponding to the lane line points, a straight line or line segment can be drawn through the lane line points based on the lane line direction as the target tangent line. This target tangent line can represent a line perpendicular to the true lane line. Affected by the shooting perspective, it may not be perpendicular to the lane line direction in the target image. Based on the target tangent line, determine the edge clustering center on the opposite side or both sides of the lane line points, so as to determine an accurate edge clustering center.

[0073] In one embodiment, the determining the edge clustering center of the lane line points based on the target tangent line includes: Determine the edge clustering center from the first edge points based on the target tangent line.

[0074] In this embodiment, for each side of the lane line edge corresponding to the lane line points, among the first edge points corresponding to the lane line edge on that side, the first edge point closest to the target tangent line is determined as the edge clustering center. Using the first edge point as the initial edge clustering center can enable the edge clustering center to more truly and accurately represent the lane line edge, thereby further improving the accuracy of the lane line edge.

[0075] In one example, as Figure 4 shown, for a certain lane line point, which belongs to the points within the lane line area, taking this lane line point as the center point, search for all first edge points within a radius R of the center point in the binary image. Taking the lane line point as the center, draw a horizontal tangent line to find the edge clustering centers on the left and right sides of the initial lane line point. Traverse all the first edge points, and start iteratively updating the edge clustering centers on the same side from the first edge points closest to the edge clustering centers on the left and right sides. Through multiple rounds of iteration, output the optimal edge clustering centers on the left and right sides of this lane line point.

[0076] In one embodiment, the multiple target lane line edge points include the target lane line edge points on the left side and the right side of the lane line. The determining the lane line information according to the multiple target lane line edge points includes: Determine the distance difference between the target lane line edge points on the left side and the right side; Determine the lane line width of the lane line image according to the distance difference and a preset linear relationship to obtain the lane line information.

[0077] In this embodiment, the determined multiple target lane line edge points may include the target lane line edge points on the left side of the lane line and the target lane line edge points on the right side of the lane line. The distance difference between the target lane line edge points on the left and right sides is determined. This distance difference is the distance difference between points in the target image and needs to be converted into the true lane line width of the lane line in the lane line image. A preset linear relationship is pre-calibrated as the conversion basis, and the distance difference between the target lane line edge points on the left and right sides is converted based on the preset linear relationship, so as to determine the true lane line width corresponding to the lane line image and obtain accurate lane line information.

[0078] In one embodiment, the method further includes: Converting the multiple target lane line edge points according to the perspective of the top view; Based on the converted multiple target lane line edge points, determine the distance difference between the target lane line edge points on the left and the target lane line edge points on the right.

[0079] In this embodiment, the lane line image is generally a front view image, and the real parallel lines are not in a horizontal relationship in the front view image. Therefore, the distance differences between the target lane line edge points corresponding to different lane line points and the target lane line edge points on the right are different. To reduce the computational consumption, the multiple target lane line edge points can be calculated and converted according to the perspective of the top view using the external parameter mapping matrix from the image point (x, y) to the top view projection point (x_h, y_h). The conversion formula is as follows:

[0080] In this way, multiple target lane line edge points in the perspective of the top view are obtained. The curves formed by connecting the target lane line edge points on the left and right sides respectively can be parallel, and the distance difference between the target lane line edge points on the left and the target lane line edge points on the right can represent the true lane line width. The preset linear relationship pre-calibrated can be used to convert the distance difference between the target lane line edge points on both sides to quickly and accurately obtain the lane line width.

[0081] In some embodiments, when the lane line image does not belong to the top view angle, the lane line detection data and the edge detection data can be first converted according to the perspective of the top view, and then the subsequent process of determining the lane line information is performed based on the converted lane line detection data and edge detection data, thereby further improving the robustness of the algorithm.

[0082] In one embodiment, the determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Performing lane line detection on the lane line image through a deep learning algorithm to obtain the lane line detection data.

[0083] In this embodiment, for the lane line image as shown in Figure 5 , the lane line image can be input into deep learning algorithms such as UFLDv2. Through the deep learning algorithm, lane line detection can be performed on the lane line image, and accurate lane line detection data can be obtained, as shown in Figure 6 .

[0084] In one embodiment, the determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Performing convolution processing on the lane line image using a preset convolution operator to obtain a feature image; Determining the edge detection data according to the feature image.

[0085] In this embodiment, for edge detection, during the edge detection process, convolution processing can be first performed on the lane line image using a preset convolution operator to obtain a feature image, and then edge detection data can be obtained based on the feature image. The obtained edge detection data can be a black-and-white binary image as shown in Figure 7 . The white areas in the figure are the edge regions. The preset convolution operator can be a sobel operator. Combining the lane line information in this embodiment, a preset convolution operator dedicated to scene design is determined to improve the robustness of edge detection and the accuracy of lane line information.

[0086] In one embodiment, the preset convolution operator includes at least one of a horizontal direction convolution operator, a vertical direction convolution operator, and an oblique direction convolution operator.

[0087] In this embodiment, the preset convolution operator includes at least one of a horizontal direction convolution operator, a vertical direction convolution operator, and an oblique direction convolution operator. Among them, the horizontal direction convolution operator is:

[0088] Among them, the vertical direction convolution operator is:

[0089] Among them, the oblique direction turning operator can include a first oblique direction turning operator and a second oblique direction turning operator. The first oblique direction turning operator is:

[0090] The second oblique direction turning operator is:

[0091] In one embodiment, the determining the edge detection data according to the feature image includes: Based on the gradient intensity of each pixel in the feature image and the edge gradient intensity threshold, determining the edge pixels in the feature image to determine the edge detection data.

[0092] In this embodiment, the gradient intensity of the edge region in the feature image can be significantly distinguished from that of the non-edge region. The gradient intensity of each pixel in the feature image is filtered using a preset edge gradient intensity threshold, and the edge pixels in the feature image are screened out. The feature image scene is consistent with the lane line image, and the edge pixels screened out from the feature image can be used as the edge detection data of the lane line image.

[0093] In one embodiment, the edge gradient intensity threshold includes a first threshold and a second threshold. Determining the edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold to determine the edge detection data includes: When the gradient intensity of the pixel is greater than or equal to the first threshold, determining that the pixel is an edge pixel; or, When the gradient intensity of the pixel is less than the first threshold or greater than the second threshold, and the adjacent pixel of the pixel is an edge pixel, determining that the pixel is an edge pixel.

[0094] In this embodiment, a dual edge gradient intensity threshold is used to screen edges. The defined edge gradient intensity threshold includes a first threshold and a second threshold, which are a high gradient intensity threshold and a low gradient intensity threshold respectively. Using the dual edge gradient intensity threshold, pixels can be divided into strong edge pixels, weak edge pixels, and non-edge pixels. When the gradient intensity of a pixel is greater than or equal to the first threshold, the pixel is a high edge pixel. When the gradient intensity of a pixel is less than the first threshold, the pixel is a non-edge pixel. When the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, the pixel is a weak edge pixel. By checking the neighborhood of the strong edge pixels, the weak edge pixels are connected to the strong edge pixels and updated to strong edge pixels to obtain a more complete edge. That is, when the gradient intensity of a pixel is less than the first threshold or greater than the second threshold, and the adjacent pixel of the pixel is an edge pixel, the pixel can be used as a strong edge pixel. Finally, all the strong edge pixels can be output as the final edge pixels, improving the accuracy of the edge detection data.

[0095] In one embodiment, the method further includes: Preprocessing the lane line image, where the preprocessing includes grayscale conversion processing and / or Gaussian filtering processing; Based on the preprocessed lane line image, performing convolution processing on the lane line image using a preset convolution operator to obtain a feature image.

[0096] In this embodiment, after obtaining the original lane line image, it will be subjected to grayscale conversion to convert the color image into a grayscale image. The formula for converting an RGB image to a grayscale image is as follows;

[0097] Then, Gaussian filtering is adopted to complete the Gaussian blur of the image, that is, the weighted average of the current pixel and the surrounding pixels, to complete the suppression of noise pixels. The Gaussian formula is:

[0098] In this embodiment, combined with the lane line recognition scenario in this embodiment, the following 3×3 Gaussian kernel can be used for Gaussian filtering processing:

[0099] In one embodiment, the method further includes: Based on the gradient intensity and gradient direction of each pixel in the feature image, non-maximum suppression processing is performed on the feature image, and based on the feature image after non-maximum suppression processing, the edge detection data is determined according to the pixel intensity of each pixel in the feature image.

[0100] In this embodiment, for the preprocessed lane line image, the gradient intensity of each pixel in the lane line image is obtained by using the method of summing squares and taking the square root, and the gradient direction of each pixel is calculated using the arctangent. Non-maximum value suppression processing is performed on the lane line image according to the gradient intensity and gradient direction of each pixel to refine the edge and suppress the response on non-edges. For each pixel, only the pixel with the maximum gradient value is retained, while other pixels are suppressed to improve the edge detection accuracy.

[0101] Correspondingly, an embodiment of the present application also provides an electronic device, as Figure 8 shown, Figure 8 is a schematic structural diagram of the electronic device provided by the embodiment of the present application. The electronic device 1100 further includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. Among them, the processor 1101 is electrically connected to the memory 1102. Those skilled in the art can understand that the structural diagram of the electronic device shown in the figure does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0102] The processor 1101 is the control center of the electronic device 1100, connecting various parts of the entire electronic device 1100 through various interfaces and circuits. By running or loading software programs and / or units stored in the memory 1102, and by calling the data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby monitoring the entire electronic device 1100. The processor 1101 can be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Network Processor (NP), etc., and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application.

[0103] In the embodiments of the present application, the processor 1101 in the electronic device 1100 will load the instructions corresponding to the processes of one or more application programs into the memory 1102 according to the following steps, and the processor 1101 will run the application programs stored in the memory 1102 to implement various functions, such as: Determine the lane line detection data and edge detection data corresponding to the lane line image; According to the lane line detection data and the edge detection data, determine the lane line information corresponding to the lane line image.

[0104] For the specific implementation of the above operations, reference can be made to the previous embodiments, which will not be elaborated here.

[0105] Optionally, as Figure 8 shown, the electronic device 1100 further includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. Among them, the processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107 respectively. Those skilled in the art can understand that Figure 8 the structure of the electronic device shown in

[0106] The touch display screen 1103 can be used to display a graphical user interface and receive operation instructions generated by a user acting on the graphical user interface. The touch display screen 1103 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. The touch panel can be used to collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute the corresponding program. Optionally, the touch panel can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1101, and can receive and execute the commands sent by the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides a corresponding visual output on the display panel according to the type of touch event. In the embodiment of the present invention, the touch panel and the display panel can be integrated into the touch display screen 1103 to implement input and output functions. However, in some embodiments, the touch panel and the touch panel can be implemented as two independent components to implement input and output functions. That is, the touch display screen 1103 can also be used as a part of the input unit 1106 to implement the input function.

[0107] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network medical devices or other electronic devices through wireless communication, and transmit and receive signals with network medical devices or other electronic devices.

[0108] The audio circuit 1105 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 1105 can convert the received audio data into an electrical signal and transmit it to the speaker, which converts it into a sound signal for output. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1105, converted into audio data, and then the audio data is output to the processor 1101 for processing. After that, it is sent to another electronic device through the radio frequency circuit 1104, or the audio data is output to the memory 1102 for further processing. The audio circuit 1105 may also include an earphone jack to provide communication between the peripheral earphone and the electronic device.

[0109] The input unit 1106 can be used to receive input digital, character information or user feature information (such as fingerprint, iris, face information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0110] The power supply 1107 is used to supply power to each component of the electronic device 1100. Optionally, the power supply 1107 can be logically connected to the processor 1101 through a power management device, so as to realize functions such as management of charging, discharging, and power consumption management through the power management device. The power supply 1107 may also include any components such as one or more DC or AC power supplies, a recharge device, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0111] Although Figure 8 not shown in the figure, the electronic device 1100 may also include a camera, a sensor, a Wi-Fi module, a Bluetooth module, etc., which will not be elaborated here.

[0112] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] Those of ordinary skill in the art can understand that all or part of the steps in the above various methods can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by the processor.

[0114] Therefore, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by the processor to execute any lane line information determination method provided by the embodiments of the present application. The computer programs can execute the steps of the following lane line information determination method: Determine the lane line detection data and edge detection data corresponding to the lane line image; Determine the lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data.

[0115] For the specific implementation of each of the above operations, reference may be made to the previous embodiments and will not be elaborated herein.

[0116] Among them, the computer-readable storage medium may include: Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disc, etc.

[0117] Since a computer program capable of implementing the beneficial storage achievable by any of the lane line information determination methods provided in the embodiments of the present application can be implemented in the computer-readable storage medium and can execute any of the lane line information determination methods provided in the embodiments of the present application, for the effects, reference may be made to the previous embodiments and will not be elaborated herein.

[0118] The embodiments of the present application further provide a computer program product, which can be loaded by a processor to execute any of the lane line information determination methods provided in the embodiments of the present application. For the specific implementation of each operation of the lane line information determination method, reference may be made to the previous embodiments and will not be elaborated herein.

[0119] Since the computer program can execute any of the lane line information determination methods provided in the embodiments of the present application and can achieve the beneficial effects achievable by any of the lane line information determination methods provided in the embodiments of the present application, for its beneficial effects, reference may be made to the previous embodiments and will not be elaborated herein.

[0120] The embodiments of the present application further provide a vehicle, which includes any of the above electronic devices, computer-readable storage media, computer program products, or executes any of the above methods.

[0121] In the description of the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, "a plurality of" means two or more unless otherwise specifically defined.

[0122] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not elaborated in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0123] Among the embodiments, embodiments, and related technical features of the present application, they can be combined and replaced with each other without conflict.

[0124] The above are only the preferred embodiments of the present application, and do not impose any formal restrictions on the present application. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still fall within the scope of the technical solution of the present application.

Claims

1. A lane line information determination method, characterized in that, Including: Determine lane line detection data and edge detection data corresponding to a lane line image; Determine lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data.

2. The method according to claim 1, wherein The determining the lane line information corresponding to the lane line image according to the lane line detection data and the edge detection data includes: Determine a plurality of target lane line edge points according to the lane line detection data and the edge detection data; Determine the lane line information according to the plurality of target lane line edge points.

3. The method according to claim 2, wherein The determining a plurality of target lane line edge points according to the lane line detection data and the edge detection data includes: Determine a target image based on the lane line detection data and the edge detection data; Determine the plurality of target lane line edge points based on the target image.

4. The method according to claim 3, characterized in that, The target image includes a plurality of lane line points determined based on the lane line detection data, and the determining the plurality of target lane line edge points based on the target image includes: Determine a target lane line edge point corresponding to each lane line point in the target image to obtain a plurality of target lane line edge points.

5. The method according to claim 4, wherein The method further includes: Determine a target lane line edge point corresponding to each lane line point in the target image according to the lane line direction.

6. The method according to claim 4, wherein The target image includes a plurality of edge points determined based on the edge detection data, and the determining a target lane line edge point corresponding to each lane line point in the target image includes: Obtain a first edge point in the plurality of edge points that is within a preset area where the lane line point is located; Determine a target lane line edge point corresponding to the lane line point according to the first edge point.

7. The method according to claim 6, wherein The determining a target lane line edge point corresponding to the lane line point according to the first edge point includes: Determine an edge clustering center corresponding to the lane line point; Determine a target lane line edge point corresponding to the lane line point according to the edge clustering center and the first edge point.

8. The method according to claim 7, wherein The determining a target lane line edge point corresponding to the lane line point according to the edge clustering center and the first edge point includes: Determine a second edge point from the first edge points based on the edge clustering center; Update the edge clustering center based on the second edge point, and re - execute determining the second edge point from the first edge points based on the updated edge clustering center until a preset iteration condition is met; When the preset iteration condition is met, determine the latest edge clustering center as the target lane line edge point corresponding to the lane line point.

9. The method according to claim 8, wherein The preset iteration condition includes that the update times of the edge clustering center reach a preset threshold, and / or all the first edge points are determined as the second edge points.

10. The method according to claim 8, wherein The determining a second edge point from the first edge points based on the edge clustering center includes: Determine the edge point closest to the edge clustering center among the first edge points as the second edge point.

11. The method according to claim 8, characterized in that The updating the edge clustering center based on the second edge point includes: Fuse the second edge point and the edge clustering center to update the edge clustering center.

12. The method according to claim 11, wherein The edge clustering centers include at least two side edge clustering centers of the lane line, and the fusion based on the second edge points and the edge clustering centers includes: Determine a target edge clustering center that belongs to the same side as the second edge point among the at least two side edge clustering centers; Fuse the second edge point into the target edge clustering center.

13. The method according to claim 7, characterized in that The determination of the edge clustering center corresponding to the lane line point includes: Determine a target tangent based on the lane line point and the lane line direction; Determine the edge clustering center of the lane line point based on the target tangent.

14. The method according to claim 13, wherein The determination of the edge clustering center of the lane line point based on the target tangent includes: Determine the edge clustering center from the first edge points based on the target tangent.

15. The method according to claim 2, wherein The multiple target lane line edge points include target lane line edge points on the left side and the right side of the lane line. The determination of the lane line information based on the multiple target lane line edge points includes: Determine the distance difference between the target lane line edge points on the left side and the target lane line edge points on the right side; Determine the lane line width of the lane line image according to the distance difference and a preset linear relationship to obtain the lane line information.

16. The method according to claim 15, wherein The method further includes: Convert the multiple target lane line edge points according to the perspective of a top view; Based on the converted multiple target lane line edge points, perform the determination of the distance difference between the target lane line edge points on the left side and the target lane line edge points on the right side.

17. The method according to any one of claims 1 to 16, characterized in that, The determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Perform lane line detection on the lane line image through a deep learning algorithm to obtain the lane line detection data.

18. The method according to any one of claims 1-16, characterized in that, The determination of the lane line detection data and the edge detection data corresponding to the lane line image includes: Perform convolution processing on the lane line image using a preset convolution operator to obtain a feature image; Determine the edge detection data according to the feature image.

19. The method according to claim 18, wherein The preset convolution operator includes at least one of a horizontal direction convolution operator, a vertical direction convolution operator, and an oblique direction convolution operator.

20. The method according to claim 18, wherein The determination of the edge detection data according to the feature image includes: Based on the gradient intensity of each pixel in the feature image and an edge gradient intensity threshold, determine the edge pixels in the feature image to determine the edge detection data.

21. The method according to claim 20, wherein The edge gradient intensity threshold includes a first threshold and a second threshold. The determination of the edge pixels in the feature image based on the gradient intensity of each pixel and the edge gradient intensity threshold to determine the edge detection data includes: When the gradient intensity of the pixel is greater than or equal to the first threshold, determine that the pixel is an edge pixel; or, When the gradient intensity of the pixel is less than the first threshold or greater than the second threshold, and the adjacent pixel of the pixel is an edge pixel, determine that the pixel is an edge pixel.

22. The method according to claim 18, wherein The method further includes: Perform preprocessing on the lane line image, and the preprocessing includes grayscale conversion processing and / or Gaussian filtering processing; Based on the preprocessed lane line image, perform convolution processing on the lane line image using a preset convolution operator to obtain a feature image.

23. The method according to claim 18, wherein The method further includes: Based on the gradient intensity and gradient direction of each pixel in the feature image, perform non-maximum suppression processing on the feature image, and based on the feature image after non-maximum suppression processing, determine the edge detection data according to the pixel intensity of each pixel in the feature image.

24. An electronic device, characterized in that, It includes a processor, the processor is connected to a memory, the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the lane line information determination method according to any one of claims 1 to 23.

25. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the lane line information determination method according to any one of claims 1 to 23.

26. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the lane line information determination method according to any one of claims 1 to 23.

27. A vehicle, characterized in that, The vehicle executes the lane line information determination method according to any one of claims 1-23, or includes the electronic device according to claim 24.

Images