Lane detection methods, devices, electronic equipment and storage media

Abstract

This application discloses a lane line detection method, apparatus, electronic device, and storage medium. The method is applied to an in-vehicle terminal and includes: acquiring an environmental image of the vehicle; performing edge feature point recognition on the environmental image to obtain edge feature points of each lane marker contained in the environmental image, the edge feature points including multiple first edge feature points and multiple second edge feature points; pairing the multiple first edge feature points and multiple second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane marker; selecting the lane marker closest to the vehicle as the target lane line based on the multiple target feature point pairing groups corresponding to each lane marker; and fitting the target lane line according to the multiple target feature point pairing groups corresponding to the target lane line to obtain the fitting result corresponding to the target lane line. Implementing this application embodiment can improve the accuracy of lane line detection.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a lane line detection method, device, electronic device and storage medium. Background Technology

[0002] For intelligent vehicles, lane detection is a crucial component of autonomous driving technology. It helps vehicles identify lane markings on the road, enabling autonomous driving. Lane departure warning, lane changing, lane keeping, and automatic parking are all functions that rely on lane detection. In real-world driving scenarios, lane markings can become difficult to detect due to road wear, obstructions, and changes in lighting. Therefore, accurately detecting lane markings has become a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application discloses a lane line detection method, apparatus, electronic device, and storage medium, which can improve the accuracy of lane line detection.

[0004] This application discloses a lane line detection method applied to an in-vehicle terminal; the method includes:

[0005] Acquire environmental images of the vehicle;

[0006] Edge feature point recognition is performed on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image. The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature point belonging to the same lane sign and the vehicle is greater than the distance between the second edge feature point and the vehicle.

[0007] Pair the plurality of first edge feature points and the plurality of second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane mark;

[0008] Based on the pairing groups of multiple target feature points corresponding to each lane sign, the lane sign closest to the vehicle is selected as the target lane line, and the target lane line is fitted according to the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0009] This application embodiment obtains the edge feature points of each lane sign by performing edge feature point recognition on the environmental image, thereby accurately extracting the edge feature points belonging to the lane sign from the vehicle's surrounding environment. Furthermore, it pairs multiple first edge feature points with multiple second edge feature points to effectively filter out invalid edge feature points, thus accurately determining multiple target feature point pairings corresponding to each lane sign. It can also select the lane sign closest to the vehicle as the target lane line, improving the accuracy of lane line detection and avoiding the problem of not being able to determine the target lane line when multiple lane lines are detected, thus avoiding the need to fit redundant lane lines.

[0010] In one embodiment, the step of performing edge feature point recognition on the environmental image to obtain the edge feature points of each lane marker contained in the environmental image includes:

[0011] The environmental image is convolved by an edge detection operator to obtain the gradient value of each pixel in the environmental image. The edge detection operator includes a convolution kernel with a kernel size of 1*N, where N is an odd number greater than 1. The convolution kernel includes N elements. In the convolution kernel, the element at the Yth position is the opposite of the element at the N-Y+1th position. Y is a positive integer less than or equal to N, and Y is not equal to (N+1) / 2.

[0012] Based on the gradient value and gradient threshold of each pixel, the edge feature points of each lane marker contained in the environmental image are determined from each pixel.

[0013] Based on the shape features of lane markings such as lane lines, this application embodiment only needs to consider the grayscale changes in the X direction of the environmental image. By performing convolution processing on the environmental image using a convolution kernel with a kernel size of 1*N, the extraction of edge feature points of lane markings becomes more effective and accurate.

[0014] In one embodiment, determining the edge feature points of each lane marker contained in the environment image from the pixels based on the gradient values ​​and gradient thresholds of each pixel includes:

[0015] Pixels with gradient values ​​greater than a first gradient threshold are identified as first edge feature points of each lane identifier, and pixels with gradient values ​​less than a second gradient threshold are identified as second edge feature points of each lane identifier. The first gradient threshold and the second gradient threshold are opposites of each other, and the first gradient threshold is greater than the second gradient threshold.

[0016] This application embodiment uses a first gradient threshold and a second gradient threshold, which are opposites of each other, to extract the first edge feature points and the second edge feature points on each lane marking from each pixel of the environmental image more effectively and accurately.

[0017] In one embodiment, pairing the plurality of first edge feature points and the plurality of second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane identifier includes:

[0018] Based on a first distance threshold, multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle are paired to obtain multiple first feature point pairing groups corresponding to the same side of the vehicle.

[0019] Based on the second distance threshold, the first edge feature points and the second edge feature points contained in the multiple first feature point pairing groups corresponding to the same side of the vehicle are re-paired, and some first edge feature points and some second edge feature points are merged to obtain multiple target feature point pairing groups corresponding to the lane signs located on the same side of the vehicle.

[0020] This application embodiment pairs multiple first edge feature points and multiple second edge feature points according to a first distance threshold, thereby effectively filtering out invalid edge feature points and accurately determining multiple target feature point pairing groups corresponding to each lane sign. Furthermore, after re-pairing and finely merging the multiple first feature point pairing groups located on the same side of the vehicle according to the second distance threshold, multiple more effective and accurate target feature point pairing groups are obtained. The lane sign closest to the vehicle is selected as the target lane line based on the target feature point pairing groups corresponding to each lane sign, which improves the accuracy of lane line detection and avoids the problem of not being able to determine the target lane line when multiple lane lines are detected, thus fitting the redundant lane lines.

[0021] In one embodiment, pairing multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to a first distance threshold includes:

[0022] From a plurality of second edge feature points belonging to the same side of the vehicle, determine the second edge feature point that has the smallest distance to the target first edge feature point and whose distance to the target first edge feature point is less than the first distance threshold, and use it as the second edge feature point paired with the target first edge feature point. The target first edge feature point is any one of the plurality of first edge feature points belonging to the same side of the vehicle.

[0023] This application embodiment determines a second edge feature point that is paired with a target first edge feature point from a plurality of second edge feature points belonging to the same side of the vehicle. It not only filters out second edge feature points whose distance meets the first distance threshold according to the first distance threshold, but also selects the second edge feature point that is closest to the target first edge feature point from the second edge feature points whose distance meets the first distance threshold as the second edge feature point paired with the target first edge feature point, which is beneficial to improving the accuracy and efficiency of feature point pairing.

[0024] In one embodiment, determining the second edge feature point that has the smallest distance to the target first edge feature point and a distance to the target first edge feature point that is less than the first distance threshold from among a plurality of second edge feature points belonging to the same side of the vehicle, as the second edge feature point paired with the target first edge feature point, includes:

[0025] According to the order of distance of the first edge feature points from the vehicle from farthest to closest, each first edge feature point belonging to the same side of the vehicle is paired one by one;

[0026] For each first edge feature point, according to the order of distance of the second edge feature points from the vehicle from farthest to closest, determine the first distance between the current first edge feature point and the current second edge feature point on the same side of the vehicle. Determine whether the first distance is less than the first distance threshold. If so, determine that the current second edge feature point is a second edge feature point paired with the current first edge feature point. If not, take the next unpaired second edge feature point as the new current second edge feature point, and re-execute the step of determining the first distance between the current first edge feature point and the current second edge feature point on the same side of the vehicle.

[0027] This application embodiment pairs the first edge feature points and the second edge feature points based on a first distance threshold and the order of the first edge feature points from the vehicle from farthest to closest, and the order of the second edge feature points from the vehicle from farthest to closest. This can further filter out invalid first edge feature points and second edge feature points, thereby efficiently and accurately obtaining multiple first feature point pairing groups located on the same side of the vehicle, improving the efficiency and accuracy of lane line detection.

[0028] In one embodiment, the step of re-pairing the first edge feature points and second edge feature points included in the multiple first feature point pairing groups corresponding to the same side of the vehicle according to a second distance threshold, and merging some of the first edge feature points and some of the second edge feature points to obtain multiple target feature point pairing groups corresponding to the lane markings located on the same side of the vehicle, includes:

[0029] According to the order of the second edge feature points from farthest to nearth of the vehicle and the order of the first edge feature points from nearthest to farthest of the vehicle, determine the second distance between the current second edge feature point and the current first edge feature point in each pairing group of first feature points on the same side of the vehicle, and determine whether the second distance is less than the second distance threshold.

[0030] If so, the current first edge feature point is determined to be a first edge feature point that is re-paired with the current second edge feature point, and the current second edge feature point, the first edge feature point paired with the current second edge feature point in the plurality of first feature point pairing groups, the current first edge feature point, and the second edge feature point paired with the current first edge feature point in the plurality of first feature point pairing groups are merged to obtain a target feature point pairing group. Then, the next unmerged second edge feature point is taken as the new current second edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed.

[0031] If not, then the next first edge feature point is taken as the new current first edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each pair of first feature points corresponding to the same side of the vehicle is repeated.

[0032] Based on a second distance threshold, this application re-pairs multiple first edge feature points and multiple second edge feature points contained in multiple first feature point pairing groups located on the same side of the vehicle according to the order of second edge feature points from farthest to nearest distance from the vehicle and the order of first edge feature points from nearest to farthest distance from the vehicle. Through fine merging processing, multiple more effective and accurate target feature point pairing groups are obtained, which helps to improve the accuracy and efficiency of lane line detection.

[0033] This application discloses a lane line detection device applied to an in-vehicle terminal; the device includes:

[0034] The acquisition module is used to acquire environmental images of the vehicle;

[0035] The recognition module is used to perform edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image. The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature point belonging to the same lane sign and the vehicle is greater than the distance between the second edge feature point and the vehicle.

[0036] The pairing module is used to pair the plurality of first edge feature points and the plurality of second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane sign;

[0037] The fitting module is used to select the lane sign closest to the vehicle as the target lane line based on the pairing groups of multiple target feature points corresponding to each lane sign, and to fit the target lane line based on the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0038] This application discloses an electronic device, including:

[0039] A memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the method described in any of the above embodiments.

[0040] This application discloses a computer-readable storage medium storing a computer program, wherein when executed by a processor, the computer program causes the processor to perform the methods described in any of the above embodiments.

[0041] The lane line detection method, apparatus, electronic device, and storage medium disclosed in this application are used to identify feature points in the environmental image of a vehicle to obtain multiple first edge feature points and multiple second edge feature points of each lane sign contained in the environmental image. Multiple first edge feature points and multiple second edge feature points belonging to the same lane sign are paired to obtain multiple feature point pairing groups corresponding to each lane sign. The lane sign closest to the vehicle is taken as the target lane line, and the target lane line is fitted according to the multiple target feature point pairing groups corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0042] Therefore, this embodiment of the application performs edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign, thereby accurately extracting the edge feature points belonging to the lane sign from the vehicle's surrounding environment. Furthermore, it pairs multiple first edge feature points and multiple second edge feature points to effectively filter out invalid edge feature points, thereby accurately determining multiple target feature point pairing groups corresponding to each lane sign. It can also select the lane sign closest to the vehicle as the target lane line, improving the accuracy of lane line detection and avoiding the problem of not being able to determine the target lane line when multiple lane lines are detected, thus fitting redundant lane lines. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram illustrating an application scenario of a lane line detection method disclosed in an embodiment of this application;

[0045] Figure 2 This is a schematic flowchart of a lane line detection method disclosed in an embodiment of this application;

[0046] Figure 3a This is a schematic diagram of a vehicle body coordinate system disclosed in an embodiment of this application;

[0047] Figure 3b This is a schematic diagram of a region of interest detected by a front camera of a vehicle, as disclosed in an embodiment of this application.

[0048] Figure 3c This is a schematic diagram of a region of interest detected by a rear camera of a vehicle, as disclosed in an embodiment of this application.

[0049] Figure 3d This is a schematic diagram of a region of interest detected by a left-side camera of a vehicle, as disclosed in an embodiment of this application.

[0050] Figure 3e This is a schematic diagram of a region of interest detected by a camera on the right side of a vehicle, as disclosed in an embodiment of this application.

[0051] Figure 3f This is a schematic diagram of a region of interest mapped from a bird's-eye view image, as disclosed in an embodiment of this application.

[0052] Figure 4 This is a schematic diagram of the edge feature points of each lane marking contained in an environmental image disclosed in an embodiment of this application;

[0053] Figure 5 This is a schematic flowchart of another lane line detection method disclosed in an embodiment of this application;

[0054] Figure 6a This is a schematic diagram of a convolution kernel scanning method and convolution processing result disclosed in an embodiment of this application;

[0055] Figure 6b This is a schematic diagram of a 1*5 convolution kernel disclosed in an embodiment of this application;

[0056] Figure 7This is a schematic flowchart of another lane line detection method disclosed in an embodiment of this application;

[0057] Figure 8a This is a schematic diagram of a method for pairing multiple first edge feature points and multiple second edge feature points belonging to the same side of a vehicle, as disclosed in an embodiment of this application.

[0058] Figure 8b This is a schematic diagram of a method disclosed in this application for re-pairing and merging each first edge feature point and each second edge feature point contained in a pairing group of multiple first feature points corresponding to the same side of a vehicle;

[0059] Figure 9 This is a schematic diagram of the structure of a lane line detection device disclosed in an embodiment of this application;

[0060] Figure 10 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0062] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0063] In related technologies, lane detection methods can be categorized into several approaches. One approach is to use histograms to filter candidate feature points belonging to lane lines and then perform least-squares fitting. However, when multiple lane lines appear on a road, it is difficult to accurately select the required lane line data. Another approach is to use Hough transform-based line detection algorithms based on the voting concept. However, this method has poor real-time performance, and the detected lines in complex environments have certain slope errors, resulting in low accuracy and making it difficult to apply to scenarios such as online calibration. Yet another approach is to use semantic segmentation-based lane detection algorithms based on deep learning. Since these algorithms predict lane lines pixel by pixel, they are time-consuming and inefficient. A third approach is to use anchor box-based object detection methods based on deep learning. However, these methods perform poorly when vehicles are changing lanes or driving on curved roads.

[0064] For intelligent vehicles, onboard cameras are the foundation for many advanced driver assistance functions (ADAS) such as automatic emergency braking, lane departure warning, lane keeping assist, and automatic parking. These all rely on image recognition to detect lane lines, obstacles, and pedestrians. Since camera calibration plays a crucial role in ADAS, it calculates the camera's intrinsic and extrinsic parameters to convert pixel coordinates in an image into physical coordinates in the real world, ensuring that the camera can accurately identify and locate objects and scenes around the vehicle.

[0065] Currently, most camera calibration methods involve calibrating a stationary vehicle using a calibration board before it leaves the factory. This method calculates the camera's intrinsic and extrinsic parameters based on the board's imaging position on the image plane. This method has high requirements for the calibration environment, and the calibration data can change during vehicle use due to various factors such as lens repair, reduced tire pressure, and physical collisions. Recalibrating the vehicle in these situations would be extremely inconvenient. Therefore, online calibration methods are particularly convenient and important. Online calibration can solve the problem of calibrating and repairing any camera after the vehicle leaves the factory, allowing camera calibration to be completed while the vehicle is in motion, reducing the requirements for the calibration site. However, considering the potential shift in camera position during vehicle operation or when encountering bumps, effective online calibration requires detecting road surface data such as lane markings in the surrounding environment during vehicle movement.

[0066] This application discloses a lane line detection method, apparatus, electronic device, and storage medium, which can improve the accuracy of lane line detection. A detailed description will follow with reference to the accompanying drawings.

[0067] Please see Figure 1 , Figure 1This is a schematic diagram illustrating an application scenario of a lane line detection method disclosed in an embodiment of this application. The vehicle 10 includes an on-board terminal 20 and a camera 30.

[0068] The vehicle terminal 20 may include vehicle electronic devices such as vehicle controllers, but this application embodiment does not limit the scope. The vehicle terminal 20 may be installed inside the vehicle 10.

[0069] Camera 30 may include, but is not limited to, wide-angle cameras, fisheye cameras, etc., and there is no specific limitation.

[0070] During the driving of vehicle 10, camera 30 can capture environmental images of vehicle 10; the environmental images may include lane markings 40 around vehicle 10, and lane markings 40 may include, but are not limited to, lane lines, road arrows, parking space lines, stop lines, and other traffic markings. Vehicle terminal 20 can capture environmental images of vehicle 10 through camera 30 to detect lane lines based on the environmental images.

[0071] Please refer to further information. Figure 2 , Figure 2 This is a schematic flowchart of a lane line detection method disclosed in an embodiment of this application. Wherein, Figure 2 The described lane detection method is applicable to on-board terminals. For example... Figure 2 As shown, the lane line detection method may include the following steps:

[0072] 201. Obtain environmental images of the vehicle.

[0073] The vehicle-mounted terminal can acquire environmental images of the vehicle through a camera.

[0074] In some embodiments, the number of cameras can be multiple, and these cameras can be arranged around the vehicle body, allowing the on-board terminal to acquire environmental images of the vehicle captured by each camera from different angles. For example... Figure 1 As shown, the four cameras can be positioned at the front, rear, left, and right sides of the vehicle. Optionally, the cameras on the left and right sides of the vehicle can be mounted on the vehicle's side mirrors or side doors.

[0075] Lane markings, such as lane lines, exhibit significant distortion and appear curved in images directly captured by wide-angle or fisheye cameras, hindering lane line detection. In some embodiments, the onboard terminal can stitch together images captured by various cameras to obtain a bird's-eye view (BEV) image, serving as the vehicle's environmental image. The bird's-eye view image generates a virtual view from above the vehicle, providing a complete 360-degree view of the vehicle's surroundings. Therefore, using the stitched bird's-eye view image as the vehicle's environmental image reduces distortion of lane markings and improves lane detection accuracy.

[0076] 202. Perform edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image.

[0077] Extracting feature points from a complete environmental image (such as a complete bird's-eye view) by an in-vehicle terminal is extremely time-consuming and cannot meet real-time requirements. Furthermore, the number of extracted feature points would be enormous, hindering further feature point filtering. Therefore, in some embodiments, before performing edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained within the image, the in-vehicle terminal can determine a Region of Interest (ROI) in the environmental image. Feature point recognition is then performed within the ROI to obtain the edge feature points of each lane sign contained within the ROI.

[0078] like Figure 3a As shown, Figure 3a This is a schematic diagram of a vehicle body coordinate system disclosed in an embodiment of this application. The vehicle body coordinate system has its origin at the center of the rear wheel axle, with the X-axis parallel to the front-rear direction of the vehicle body, the Y-axis parallel to the rear wheel axle, and the Z-axis perpendicular to the ground downwards. Depending on the needs of the actual application scenario, the vehicle-mounted terminal can determine a region at a certain distance from the vehicle body as a region of interest within the vehicle body coordinate system. For example... Figures 3b-3e As shown, Figure 3b The region of interest 301 detected by the front camera of the vehicle is shown in the vehicle body coordinate system; Figure 3c The region of interest 302 detected by the rear camera of the vehicle is shown in the vehicle body coordinate system; Figure 3d The region of interest 303 detected by the left-side camera of the vehicle is shown in the vehicle body coordinate system; Figure 3e The region of interest 304 detected by the right-side camera of the vehicle is shown in the vehicle body coordinate system.

[0079] As an optional implementation, after determining the region of interest (ROI) in the vehicle coordinate system, the vehicle-mounted terminal can, based on the camera's imaging principle, utilize parameters such as camera extrinsic parameters (translation, rotation) and camera intrinsic parameters (focal length, eccentricity, distortion) to map the ROI from the vehicle coordinate system onto the environmental image, thereby determining the ROI within the environmental image. For example, the vehicle-mounted terminal can map the ROI from the vehicle coordinate system to a bird's-eye view image, such as... Figure 3f As shown, Figure 3f This is a schematic diagram of a region of interest mapped from a bird's-eye view image, as disclosed in an embodiment of this application. Figure 3f The bird's-eye view image includes region of interest 301 detected by the front camera of vehicle 10, region of interest 302 detected by the rear camera of vehicle 10, region of interest 303 detected by the left camera of vehicle 10, and region of interest 304 detected by the right camera of vehicle 10. It is evident that by dividing the region of interest, the number of feature points processed is greatly reduced, the detection speed is improved, thus meeting the requirements of real-time detection, and this is highly beneficial for subsequent feature point filtering operations.

[0080] In some embodiments, the vehicle terminal performs edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image. This may include the following steps: performing convolution processing on the environmental image using an edge detection operator to obtain the gradient value of each pixel value in the environmental image; determining the edge feature points of each lane sign contained in the environmental image from each pixel based on the gradient value of each pixel and a gradient threshold. The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature points belonging to the same lane sign and the vehicle is greater than the distance between the second edge feature points and the vehicle.

[0081] Edge detection operators can slide a convolution kernel as a window across the environment image, calculating the gradient value of each pixel in the environment image through convolution operations. The convolution kernel is used to extract feature points from the image by performing convolution operations on each pixel. The convolution kernel can be a two-dimensional matrix or an array. The convolution operation can be performed by multiplying the convolution kernel element-wise with the corresponding pixels in the environment image within the window, and then summing the results to obtain the convolution operation result. Edge detection operators may include, but are not limited to, first-order difference operators such as the Sobel operator, Roberts operator, and Prewitt operator, as well as second-order difference operators such as the Laplacian operator.

[0082] After calculating the gradient value corresponding to each pixel, an appropriate gradient threshold can be selected to determine the edge feature points of each lane sign contained in the environmental image from each pixel. In some embodiments, the vehicle terminal determines the edge feature points of each lane sign contained in the environmental image from each pixel based on the gradient value of each pixel and the gradient threshold, which may include the following steps:

[0083] Pixels with gradient values ​​greater than a first gradient threshold are identified as first edge feature points of each lane identifier, and pixels with gradient values ​​less than a second gradient threshold are identified as second edge feature points of each lane identifier, with the first and second gradient thresholds having opposite signs.

[0084] The absolute values ​​of the first and second gradient thresholds can be similar or equal. For example, the first gradient threshold can be 60 and the second gradient threshold can be -50; or, the first gradient threshold can be 60 and the second gradient threshold can be -60.

[0085] It should be noted that within the same lane marking, the gradient values ​​of the first and second edge feature points have different signs. This is because the edge detection operator's response to an edge is determined by the direction of grayscale change. The first edge feature point is the side of the lane marking farther from the vehicle, while the second edge feature point is the side of the lane marking closer to the vehicle. Therefore, they are on opposite sides of the same lane marking. When the convolution kernel slides across the environment image, since the grayscale changes of the first and second edge feature points are in opposite directions, their corresponding gradient directions are opposite, and thus, their gradient values ​​have opposite signs. Therefore, the signs of the first and second gradient thresholds are opposite. Which one is positive and which one is negative is determined by the defined positive direction and is not specified here.

[0086] Please see Figure 4 , Figure 4 This is a schematic diagram of the edge feature points of various lane markings contained in an environmental image disclosed in an embodiment of this application. For example... Figure 4 As shown, the edge feature points include multiple first edge feature points 401 and multiple second edge feature points 402. The distance between the first edge feature point 401 belonging to the same lane marking and the vehicle 10 is greater than the distance between the second edge feature point 402 and the vehicle 10.

[0087] It should be noted that the distances between the first edge feature point and the second edge feature point and the vehicle in this embodiment can refer to pixel distances in the image. For example, the distance between the first edge feature point and the vehicle can be the distance between the first edge feature point and the center pixel of the vehicle in the environmental image.

[0088] 203. Pair multiple first edge feature points and multiple second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane sign.

[0089] In some embodiments, the vehicle terminal can pair multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to a first distance threshold to obtain multiple target feature point pairing groups corresponding to each lane sign.

[0090] It should be noted that "same side of the vehicle" can refer to the left, right, front, and rear of the vehicle, etc. For example, "same side of the vehicle" can refer to... Figure 3d The region of interest on the left side of the vehicle shown can also refer to... Figure 3e The region of interest (ROI) on the right side of the vehicle is shown. ROI on the same side of the vehicle can include various lane markings such as lane lines, stop lines, parking space lines, and road arrows. Pairing multiple first edge feature points and multiple second edge feature points on the same side of the vehicle can effectively improve the efficiency and accuracy of edge feature point pairing.

[0091] Since the actual width of lane lines on real roads is generally between 15 cm and 30 cm, assuming that the horizontal width of each pixel in the environmental image corresponds to an actual width of approximately 2.5 cm in the real world, in some embodiments, the first distance threshold can be the width of 5 to 15 pixels, thus corresponding to an actual width of 5*2.5 cm to 15*2.5 cm in the real world. Based on this first distance threshold, the vehicle terminal can match first and second edge feature points belonging to the same side of the vehicle under the condition of satisfying the first distance threshold, thereby filtering out invalid edge feature points. For example, the vehicle terminal can pair first and second edge feature points belonging to the same side of the vehicle that are less than the first distance threshold apart, obtaining multiple target feature point pairing groups corresponding to each lane marking.

[0092] This not only filters out edge feature points that do not belong to lane markings, but also filters out some edge feature points of lane markings that do not conform to the actual width of the lane lines by determining the first distance threshold based on the actual width of the lane lines on the real road. After filtering out invalid edge feature points, it is more conducive to improving the efficiency and accuracy of lane line detection.

[0093] 204. Based on the pairing groups of multiple target feature points corresponding to each lane sign, select the lane sign closest to the vehicle as the target lane line, and fit the target lane line according to the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0094] In real-world road scenarios, multiple lane markings often appear simultaneously. However, it's not always necessary to detect all of them. For example, during online camera calibration, detecting too many lane lines can actually decrease calibration efficiency. Therefore, when multiple lane markings appear simultaneously, the lane marking closest to the vehicle can be retained as the target lane line. The target lane line is then fitted only based on paired groups of target feature points corresponding to it, thus avoiding fitting unnecessary lane lines and improving both the efficiency and accuracy of lane line detection.

[0095] It should be noted that the distance calculated by the vehicle-mounted terminal between the vehicle and the lane marking can refer to the pixel distance in the image. For example, the vehicle-mounted terminal can calculate the average distance between each first edge feature point and each second edge feature point corresponding to the lane marking and the center pixel point corresponding to the vehicle, as the distance between the lane marking and the vehicle.

[0096] In some embodiments, the method of fitting the target lane line may include, but is not limited to, the Random Sample Consensus (RANSAC) algorithm, the least squares method, etc.

[0097] The method of this application embodiment can be applied to the application scenario of online calibration of vehicle cameras and other sensors. Due to the special calibration environment on the road, there are various factors such as light intensity and reflection. Traditional lane line detection methods cannot accurately and effectively detect lane line information. The lane line detection method proposed in this application embodiment can greatly improve the accuracy and efficiency of lane line detection.

[0098] This application embodiment obtains the edge feature points of each lane sign by performing edge feature point recognition on the environmental image, thereby accurately extracting the edge feature points belonging to the lane sign from the vehicle's surrounding environment. Furthermore, it pairs multiple first edge feature points with multiple second edge feature points to effectively filter out invalid edge feature points, thus accurately determining multiple target feature point pairings corresponding to each lane sign. It can also select the lane sign closest to the vehicle as the target lane line, improving the accuracy of lane line detection and avoiding the problem of not being able to determine the target lane line when multiple lane lines are detected, thus avoiding the need to fit redundant lane lines.

[0099] Please refer to further information. Figure 5 , Figure 5 This is a flowchart illustrating another lane detection method disclosed in an embodiment of this application. This lane detection method can be applied to an in-vehicle terminal and may include the following steps:

[0100] 501. Obtain environmental images of the vehicle.

[0101] 502. Perform convolution processing on the environment image using the edge detection operator to obtain the gradient values ​​of each pixel in the environment image.

[0102] In the embodiments of this application, the edge detection operator includes a convolution kernel with a kernel size of 1*N, where N is an odd number greater than 1, and the convolution kernel includes N elements; in the convolution kernel, the element arranged at the Yth position is the opposite of the element arranged at the N-Y+1th position, where Y is a positive integer less than or equal to N, and Y is not equal to (N+1) / 2.

[0103] Traditional edge detection operators (such as the Robert operator and the Sobel operator) use 3*3 convolution kernels for edge extraction and detection. These traditional edge detection operators are suitable for situations where the grayscale changes in the X and Y directions of the image are relatively gentle. In complex road environments, these traditional edge detection operators have difficulty accurately extracting the edge feature points of lane lines, which brings trouble to the further edge feature point selection work.

[0104] Therefore, this application proposes a convolution kernel with a kernel size of 1*N, which is more suitable for the shape features of lane markings such as lane lines. It only needs to consider the grayscale changes in the X direction of the environmental image, making the extraction of edge feature points of lane markings more effective and accurate, and facilitating subsequent screening of edge feature points. For example, such as Figure 4 As shown, the lane marking 40 includes lane lines, which extend vertically (Y direction) in the environmental image. The grayscale of the horizontal sides of the lane lines is significantly different from that of the surrounding pixels. Based on this difference, edge feature points can be effectively extracted by only considering the grayscale changes in the horizontal direction (i.e., the X direction). Furthermore, the edge feature points can be identified as the first edge feature point 401 and the second edge feature point 402 based on the direction of the grayscale changes.

[0105] Specifically, the environmental image can be scanned line by line and pixel by pixel using a 1*N convolution kernel, such as... Figure 6a As shown, Figure 6a This is a schematic diagram of a convolution kernel scanning method and convolution processing result disclosed in an embodiment of this application. As can be seen, the convolution kernel scans the environmental image line by line, and scans each pixel in each row in order from left to right. After convolution processing of each pixel, a peak value of the gradient appears on the side of the lane marking away from the vehicle, and a valley value of the gradient appears on the side of the lane marking closer to the vehicle.

[0106] In some embodiments, the edge detection operator can be a convolutional kernel with a kernel size of 1*5, wherein the five elements of the convolutional kernel are -1, -1, 0, 1, and 1, respectively. For example... Figure 6b As shown, Figure 6b This is a schematic diagram of a 1*5 convolution kernel disclosed in an embodiment of this application. As can be seen, in this convolution kernel, the first element is the opposite of the fifth element; the second element is the opposite of the fourth element; and the third element is located at the center of the five elements and has no corresponding element.

[0107] Since the length of the convolution kernel corresponds to a real-world distance value that is generally 0.5 to 1 times the actual width of the lane line, it is appropriate to consider the following: Assuming the size of an environmental image (such as a bird's-eye view) is 230 mm wide, and this 230 mm width corresponds to a real-world distance of 5 meters, then the real-world width corresponding to each pixel in the horizontal direction of this environmental image is 5000 / 230 = 21.7 mm, approximately 2.2 cm. The size of the elements in the convolution kernel is the same as the size of the pixels in the environmental image, therefore the real-world width corresponding to the elements in the convolution kernel is also 2.2 cm. On real roads, the actual width of lane lines is generally between 10 cm and 25 cm, so 0.5 to 1 times the actual width of the lane line is roughly between 5 cm and 25 cm.

[0108] Therefore, the kernel size is set to 1*5, meaning the kernel length is 5*2.2 = 11 cm. This kernel length corresponds to 0.5 to 1 times the actual width of a lane line in the real world. Thus, the kernel size is 1*N, where the value of N can be determined based on the width of each pixel in the environment image along the row direction, corresponding to the width in the real world.

[0109] Building upon the example above, suppose the width of the environment image (e.g., a bird's-eye view) becomes k times, i.e., (230*k) millimeters. The actual distance in the real world corresponding to the changed width of the environment image remains 5 meters. Then, the value of N in the convolution kernel becomes (5-1)*k+1. Therefore, the length of the convolution kernel can be adjusted according to the width of the real world corresponding to each pixel along the row direction of the environment image, improving the flexibility of the convolution kernel in performing convolution operations on the environment image.

[0110] The above embodiments take into account the actual width of lane markings such as lane lines in the real world, and creatively propose a convolution kernel with a kernel size of 1*5 based on the width of each pixel in the environmental image along the row direction in the real world. This kernel is better suited to the shape features of lane markings such as lane lines, enabling the convolution kernel to extract the edge feature points of lane markings very efficiently and accurately, which facilitates the subsequent screening of edge feature points.

[0111] 503. Based on the gradient value and gradient threshold of each pixel, determine the edge feature points of each lane sign contained in the environmental image.

[0112] The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature point belonging to the same lane marking and the vehicle is greater than the distance between the second edge feature point and the vehicle.

[0113] In some embodiments, the vehicle terminal determines pixels with gradient values ​​greater than a first gradient threshold as first edge feature points of each lane identifier, and determines pixels with gradient values ​​less than a second gradient threshold as second edge feature points of each lane identifier. The first gradient threshold and the second gradient threshold are opposites of each other, and the first gradient threshold is greater than the second gradient threshold.

[0114] For example, the first gradient threshold can be 60, and the second gradient threshold can be -60. After the edge detection operator scans the environmental image line by line and pixel by pixel through a 1*N convolution kernel, pixels with gradient values ​​greater than 60 can be used as the first edge feature points of the lane marking, and pixels with gradient values ​​less than -60 can be used as the second edge feature points of the lane marking.

[0115] In the above embodiments, the absolute values ​​of the first gradient threshold and the second gradient threshold are the same, but they are opposites of each other, which can extract the first edge feature points and the second edge feature points on each lane mark more effectively and accurately.

[0116] 504. Pair multiple first edge feature points and multiple second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane sign.

[0117] 505. Based on the pairing groups of multiple target feature points corresponding to each lane sign, select the lane sign closest to the vehicle as the target lane line, and fit the target lane line according to the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0118] This application embodiment is based on the shape features of lane markings such as lane lines. It only needs to consider the grayscale changes in the X direction in the environmental image. It performs convolution processing on the environmental image using a convolution kernel with a kernel size of 1*N, making the extraction of edge feature points of lane markings more effective and accurate. Furthermore, it pairs multiple first edge feature points with multiple second edge feature points, thereby effectively filtering out invalid edge feature points. This accurately determines multiple target feature point pairings corresponding to each lane marking, and can select the lane marking closest to the vehicle as the target lane line. This improves the accuracy of lane line detection and avoids the problem of not being able to determine the target lane line when multiple lane lines are detected, thus avoiding the problem of fitting redundant lane lines.

[0119] Please refer to further information. Figure 7 , Figure 7 This is a flowchart illustrating another lane detection method disclosed in an embodiment of this application. This lane detection method can be applied to an in-vehicle terminal and may include the following steps:

[0120] 701. Obtain environmental images of the vehicle.

[0121] 702. Perform edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image.

[0122] The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature point belonging to the same lane marking and the vehicle is greater than the distance between the second edge feature point and the vehicle.

[0123] 703. Based on the first distance threshold, pair multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle to obtain multiple first feature point pairing groups corresponding to the same side of the vehicle.

[0124] In some embodiments, the vehicle terminal pairs multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to a first distance threshold, which may include the following steps:

[0125] From multiple second edge feature points belonging to the same side of the vehicle, determine the second edge feature point with the smallest distance to the target first edge feature point and a distance to the target first edge feature point that is less than a first distance threshold, and use it as the second edge feature point paired with the target first edge feature point. The target first edge feature point is any one of the multiple first edge feature points belonging to the same side of the vehicle.

[0126] It should be noted that the above pairing process can be performed on each row of edge feature points in the environmental image. An environmental image typically consists of multiple rows and columns of pixels, some of which are identified as edge feature points (first edge feature points or second edge feature points) after edge feature point identification. The vehicle-mounted terminal can pair multiple first edge feature points and multiple second edge feature points in each row of the environmental image to obtain multiple target feature point pairing groups corresponding to each lane marking. Generally, the direction of the lane lines is perpendicular to the row direction of each row of edge feature points in the environmental image.

[0127] For example, multiple first edge feature points belonging to the same side of the vehicle are A0, B0, C0, and D0, and multiple second edge feature points belonging to the same side of the vehicle are E0, F0, G0, and H0. The target first edge feature point is any one of the multiple first edge feature points belonging to the same side of the vehicle. Let's assume the target first edge feature point is B0. If the second edge feature points that are less than a first distance threshold with respect to B0 are G0 and H0, then there are two second edge feature points that meet the first distance threshold condition with respect to B0. Therefore, we need to select the second edge feature point that is paired with B0. Thus, the second edge feature point that is closest to B0 between G0 and H0 can be used as the second edge feature point paired with B0. The above embodiment determines the second edge feature point that is paired with the target first edge feature point from multiple second edge feature points belonging to the same side of the vehicle. It not only filters out the second edge feature points whose distance meets the first distance threshold according to the first distance threshold, but also selects the second edge feature point that is closest to the target first edge feature point from the second edge feature points whose distance meets the first distance threshold as the second edge feature point to be paired with the target first edge feature point, which helps to improve the accuracy and efficiency of feature point pairing.

[0128] Since the actual width of lane lines on real roads is generally between 15 cm and 30 cm, the first distance threshold can be determined based on the actual width of the lane lines. For example, the first distance threshold can be 30 cm or 40 cm.

[0129] Furthermore, in some embodiments, the vehicle terminal determines, from a plurality of second edge feature points belonging to the same side of the vehicle, the second edge feature point with the smallest distance to the target first edge feature point and a distance to the target first edge feature point less than a first distance threshold, as the second edge feature point paired with the target first edge feature point. This may include the following steps:

[0130] Following the order of distance from the vehicle to the first edge feature points, each first edge feature point belonging to the same side of the vehicle is paired one by one. For each first edge feature point, following the order of distance from the vehicle to the second edge feature points, a first distance is determined between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle. It is then determined whether the first distance is less than a first distance threshold. If so, the current second edge feature point is determined as the second edge feature point paired with the current first edge feature point. If not, the next unpaired second edge feature point is taken as the new current second edge feature point, and the step of determining the first distance between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle is repeated. Here, the first distance is the distance between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle.

[0131] The above embodiments pair the first edge feature points and the second edge feature points based on a first distance threshold and the order of the first edge feature points from the vehicle from farthest to closest, and the order of the second edge feature points from the vehicle from farthest to closest. This can further filter out invalid first edge feature points and second edge feature points, thereby efficiently and accurately obtaining multiple pairs of first feature points located on the same side of the vehicle, improving the efficiency and accuracy of lane line detection.

[0132] The following is combined with Figure 8a The above embodiments will be described in detail. Figure 8a This is a schematic diagram of a method for pairing multiple first edge feature points and multiple second edge feature points belonging to the same side of a vehicle, as disclosed in an embodiment of this application. Figure 8a The image includes edge feature points on the left side of the vehicle, comprising multiple first edge feature points 401 and multiple second edge feature points 402. The multiple first edge feature points 401 include feature point X, feature point Y, and feature point Z, and the multiple second edge feature points 402 include feature point M and feature point N. Feature points X, Y, Z, M, and N are all located in the same row in the environmental image.

[0133] Therefore, following the order of distance from the vehicle to the first edge feature points, each first edge feature point belonging to the same side of the vehicle is paired up one by one. Since the vehicle is located... Figure 8a The right side of the edge feature point in the image is the first edge feature point. Therefore, the feature points X, Y and Z are arranged in order of distance from the vehicle from the first edge feature point to the vehicle. So, feature point X is paired first.

[0134] First, feature point X is taken as the current first edge feature point. Then, feature point M is taken as the current second edge feature point according to the order of distance from the vehicle from farthest to closest. The first distance between feature point M and feature point X is determined. If the first distance is less than the first distance threshold, feature point M and feature point X are paired. If the first distance is greater than the first distance threshold, the next unpaired second edge feature point is determined as feature point N, which is taken as the new current second edge feature point. The step of determining the first distance between the current first edge feature point and the current second edge feature point on the same side of the vehicle is then repeated.

[0135] If feature point M and feature point X are successfully paired, then feature point Y is selected as the current first edge feature point according to the order of distance from the vehicle, from farthest to closest. The next unpaired second edge feature point (feature point N) is selected as the current second edge feature point. If the first distance between feature point Y and feature point N is less than a first distance threshold, then feature point Y and feature point N are considered successfully paired. At this point, for the remaining first edge feature point (feature point Z), no second edge feature point can be found for matching, therefore feature point Z fails to pair.

[0136] In some embodiments, the vehicle terminal may also perform the following steps: if there is a first edge feature point that fails to pair and / or a second edge feature point that fails to pair, then the first edge feature point that fails to pair and / or the second edge feature point that fails to pair is removed from the plurality of first edge feature points and the plurality of second edge feature points belonging to the same side of the vehicle.

[0137] The above embodiments can improve the efficiency of pairing first and second edge feature points by removing first and / or second edge feature points that fail to pair, while avoiding interference from invalid edge feature points. This allows for efficient and accurate acquisition of multiple first feature point pairing groups located on the same side of the vehicle, thereby improving the efficiency and accuracy of lane line detection.

[0138] The failure to pair a first edge feature point may include: for the current first edge feature point, no next unpaired second edge feature point can be found; or, the current first edge feature point cannot satisfy the requirement that the first distance between it and any second edge feature point is less than a first distance threshold. The failure to pair a second edge feature point may include: when all first edge feature points have been successfully paired, there is a second edge feature point that has not yet been paired; or the requirement that the first distance between it and any first edge feature point is less than a first distance threshold.

[0139] It should be noted that, Figure 8aThe pairing process shown can be the process of pairing edge feature points in any row of an environment image. For each row of edge feature points in the environment image, the pairing process can be performed as follows: Figure 8a The pairing process is shown.

[0140] 704. Based on the second distance threshold, re-pair the first edge feature points and the second edge feature points contained in the multiple first feature point pairing groups corresponding to the same side of the vehicle, and merge some of the first edge feature points and some of the second edge feature points to obtain multiple target feature point pairing groups corresponding to the lane signs on the same side of the vehicle.

[0141] In some embodiments, the vehicle terminal can re-pair the first edge feature points and second edge feature points contained in multiple first feature point pairing groups located on the same side of the vehicle. This can be achieved by pairing first and second edge feature points whose distance to each other is less than a second distance threshold, which can be less than the first distance threshold. By setting different distance thresholds for re-pairing, the results of the previous pairing can be verified and corrected, thereby obtaining multiple target feature point pairing groups and improving the accuracy of edge feature point pairing. Furthermore, the re-pairing process is performed within the same side of the vehicle, improving the efficiency of edge feature point pairing.

[0142] Since pairing multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to the first distance threshold results in multiple first feature point pairing groups corresponding to the same side of the vehicle, an excessive number of first feature point pairing groups is detrimental to the efficiency and accuracy of lane line detection. Therefore, more refined pairing and merging processing of multiple first feature point pairing groups is required. In other embodiments, the vehicle terminal re-pairs the various first edge feature points and various second edge feature points contained in the multiple first feature point pairing groups corresponding to the same side of the vehicle according to the second distance threshold, and merges some of the first edge feature points and some of the second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane mark located on the same side of the vehicle. This may include the following steps:

[0143] Based on the order of the second edge feature points from farthest to closest to the vehicle and the order of the first edge feature points from closest to farthest to the vehicle, determine the second distance between the current second edge feature point and the current first edge feature point in each pair of first feature points on the same side of the vehicle, and determine whether the second distance is less than the second distance threshold.

[0144] If so, the current first edge feature point is determined as the first edge feature point to be re-paired with the current second edge feature point, and the current second edge feature point, the first edge feature point paired with the current second edge feature point in multiple first feature point pairing groups, the current first edge feature point, and the second edge feature point paired with the current first edge feature point in multiple first feature point pairing groups are merged to obtain the target feature point pairing group; and the next unmerged second edge feature point is taken as the new current second edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed;

[0145] If not, the next first edge feature point is taken as the new current first edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each pair of first feature points on the same side of the vehicle is repeated. The second distance is the distance between the current first edge feature point and the current second edge feature point in each pair of first feature points on the same side of the vehicle.

[0146] The following is combined with Figure 8b The above embodiments will be described in detail. Figure 8b This is a schematic diagram of a method disclosed in this application for re-pairing and merging each first edge feature point and each second edge feature point contained in a pairing group of multiple first feature points corresponding to the same side of a vehicle. Figure 8b This includes multiple first edge feature points and multiple second edge feature points located on the right side of the vehicle. Figure 8b The first and second edge feature points shown can be edge feature points of a certain row in the environmental image. Assuming... Figure 8b The included edge feature points are located in row 149 of the environment image.

[0147] The multiple second edge feature points are feature points A1, B1, C1, D1, and E1, respectively, and are located in columns 425, 449, 469, 493, and 525 of row 149 of the environmental image. The multiple first edge feature points are feature points A2, B2, C2, D2, and E2, respectively, and are located in columns 444, 462, 477, 498, and 530 of row 149 of the environmental image.

[0148] After pairing multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to the first distance threshold, feature point A1 and feature point A2 are in the same first feature point pairing group, feature point B1 and feature point B2 are in the same first feature point pairing group, feature point C1 and feature point C2 are in the same first feature point pairing group, feature point D1 and feature point D2 are in the same first feature point pairing group, and feature point E1 and feature point E2 are in the same first feature point pairing group.

[0149] Therefore, to re-pair the first edge feature points and second edge feature points within a pairing group of multiple first feature points located on the same side of the vehicle, they can be arranged in order of distance from the second edge feature points to the vehicle (from farthest to closest) and in order of distance from the first edge feature points to the vehicle (from closest to farthest). Figure 8b As shown, first from Figure 8b The calculation starts from the leftmost second edge feature point (feature point A1), and A1 is taken as the current second edge feature point. The rightmost first edge feature point (feature point E2), the second rightmost first edge feature point (feature point D2), ..., are taken in sequence until the first edge feature point (feature point A2) closest to A1 is taken as the current first edge feature point. The second distance between the current second edge feature point and the current first edge feature point is calculated. In the above traversal calculation process, the current second edge feature point and the current first edge feature point whose second distance between them is less than the second distance threshold for the first time can be re-paired.

[0150] For example, if the second distance threshold is 40, the distance between feature point E2 and feature point A1 is 105, the distance between feature point D2 and feature point A1 is 73, the distance between feature point C2 and feature point A1 is 52, and the distance between feature point B2 and feature point A1 is 37. Therefore, feature point B2 is the first edge feature point that appears for the first time and whose second distance to feature point A1 is less than the second distance threshold of 40. Therefore, feature point B2 is re-paired with feature point A1, and feature point A1, feature point A2 that was originally paired with feature point A1 in the first feature point pairing group, feature point B2, and feature point B1 that was originally paired with feature point B2 in the first feature point pairing group are merged to obtain a target feature point pairing group.

[0151] Furthermore, the next unmerged second edge feature point (feature point C1) is taken as the new current second edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in the multiple first feature point pairing groups located on the same side of the vehicle is repeated. If the second distance between feature point E2 and feature point C1 is 61 and the second distance between feature point D2 and feature point C1 is 20, which is less than the second distance threshold of 40, then feature point D2 and feature point C1 are re-paired, and feature point D2, feature point C2, feature point D1 and feature point C1 are merged to obtain a target feature point pairing group.

[0152] The second distance between the remaining feature points E1 and E2 is less than the second distance threshold. Therefore, feature points E1 and E2 are re-paired and merged to obtain a target feature point pairing group.

[0153] Based on the second distance threshold, the above embodiments re-pair multiple first edge feature points and multiple second edge feature points contained in multiple first feature point pairing groups located on the same side of the vehicle according to the order of the second edge feature points from farthest to nearest of the vehicle and the order of the first edge feature points from nearest to farthest of the vehicle, and perform fine merging processing to obtain multiple more effective and accurate target feature point pairing groups, which helps to improve the accuracy and efficiency of lane line detection.

[0154] Traditional lane detection methods often suffer from numerous interference points and false detections due to image noise and other factors such as lighting, significantly affecting the accuracy of calibration results. This patent employs a strategy of feature point pairing, merging, and filtering to effectively filter out invalid edge feature points, improving the accuracy and efficiency of lane detection. This approach is highly suitable for scenarios involving online calibration of vehicle cameras and other sensors.

[0155] 705. Based on the pairing groups of multiple target feature points corresponding to each lane sign, select the lane sign closest to the vehicle as the target lane line, and fit the target lane line according to the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0156] This application embodiment performs edge feature point recognition on environmental images to obtain edge feature points of each lane sign, thereby accurately extracting edge feature points belonging to lane signs from the vehicle's surrounding environment. Multiple first edge feature points and multiple second edge feature points are paired according to a first distance threshold, effectively filtering out invalid edge feature points. This accurately determines multiple target feature point pairing groups corresponding to each lane sign. Furthermore, based on a second distance threshold, multiple first feature point pairing groups containing multiple first and second edge feature points located on the same side of the vehicle undergo re-pairing and fine merging processing to obtain multiple more effective and accurate target feature point pairing groups. The lane sign closest to the vehicle is selected as the target lane line based on the target feature point pairing groups corresponding to each lane sign, improving the accuracy of lane line detection and avoiding the problem of not being able to determine the target lane line when multiple lane lines are detected, thus requiring fitting of redundant lane lines.

[0157] like Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of a lane line detection device disclosed in an embodiment of this application. The lane line detection device 900 includes: an acquisition module 910, an identification module 920, a pairing module 930, and a fitting module 940.

[0158] The acquisition module 910 is used to acquire environmental images of the vehicle;

[0159] The recognition module 920 is used to perform edge feature point recognition on the environmental image to obtain the edge feature points of each lane sign contained in the environmental image. The edge feature points include multiple first edge feature points and multiple second edge feature points. The distance between the first edge feature point belonging to the same lane sign and the vehicle is greater than the distance between the second edge feature point and the vehicle.

[0160] The pairing module 930 is used to pair multiple first edge feature points and multiple second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane mark;

[0161] The fitting module 940 is used to select the lane sign closest to the vehicle as the target lane line based on the pairing groups of multiple target feature points corresponding to each lane sign, and to fit the target lane line based on the pairing groups of multiple target feature points corresponding to the target lane line to obtain the fitting result corresponding to the target lane line.

[0162] In one embodiment, the recognition module 920 is further configured to perform convolution processing on the environmental image using an edge detection operator to obtain the gradient value of each pixel in the environmental image. The edge detection operator includes a convolution kernel with a kernel size of 1*N, where N is an odd number greater than 1. The convolution kernel includes N elements, where the element at the Yth position in the convolution kernel is the opposite of the element at the N-Y+1th position. Y is a positive integer less than or equal to N, and Y is not equal to (N+1) / 2. Based on the gradient value of each pixel and the gradient threshold, the edge feature points of each lane marking contained in the environmental image are determined from each pixel.

[0163] In one embodiment, the identification module 920 is further configured to identify pixels with gradient values ​​greater than a first gradient threshold as first edge feature points of each lane sign, and to identify pixels with gradient values ​​less than a second gradient threshold as second edge feature points of each lane sign; the first gradient threshold and the second gradient threshold are opposites of each other, and the first gradient threshold is greater than the second gradient threshold.

[0164] In one embodiment, the pairing module 930 is further configured to pair multiple first edge feature points and multiple second edge feature points belonging to the same side of the vehicle according to a first distance threshold, to obtain multiple first feature point pairing groups corresponding to the same side of the vehicle; and to re-pair each first edge feature point and each second edge feature point included in the multiple first feature point pairing groups corresponding to the same side of the vehicle according to a second distance threshold, and to merge some first edge feature points and some second edge feature points to obtain multiple target feature point pairing groups corresponding to each lane sign located on the same side of the vehicle.

[0165] In one embodiment, the pairing module 930 is further configured to determine, from among a plurality of second edge feature points belonging to the same side of the vehicle, a second edge feature point that has the smallest distance to the target first edge feature point and whose distance to the target first edge feature point is less than a first distance threshold, as the second edge feature point to be paired with the target first edge feature point; the target first edge feature point is any one of the plurality of first edge feature points belonging to the same side of the vehicle.

[0166] In one embodiment, the pairing module 930 is further configured to pair each of the first edge feature points belonging to the same side of the vehicle one by one in order of distance from the first edge feature points to the vehicle; for each first edge feature point, in order of distance from the second edge feature points to the vehicle, determine the first distance between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle, and determine whether the first distance is less than the first distance threshold. If so, determine that the current second edge feature point is the second edge feature point paired with the current first edge feature point. If not, take the next unpaired second edge feature point as the new current second edge feature point, and re-execute the step of determining the first distance between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle.

[0167] In one embodiment, the pairing module 930 is further configured to determine the second distance between the current second edge feature point and the current first edge feature point in each pairing group of first feature points on the same side of the vehicle, according to the order of the second edge feature points from farthest to nearth of the vehicle and the order of the first edge feature points from nearthest to farthth of the vehicle, and to determine whether the second distance is less than the second distance threshold.

[0168] If yes, then the current first edge feature point is determined as the first edge feature point to be re-paired with the current second edge feature point, and the current second edge feature point, the first edge feature point paired with the current second edge feature point in multiple first feature point pairing groups, the current first edge feature point, and the second edge feature point paired with the current first edge feature point in multiple first feature point pairing groups are merged to obtain the target feature point pairing group; and the next unmerged second edge feature point is taken as the new current second edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed; if no, the next first edge feature point is taken as the new current first edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed.

[0169] like Figure 10 As shown, in one embodiment, an electronic device is provided, which may include: a memory 1010 storing executable program code; a processor 1020 coupled to the memory 1010; the processor 1020 calls the executable program code stored in the memory 1010 to implement the lane line detection method provided in the above embodiments.

[0170] This application discloses a computer-readable storage medium storing a computer program that causes a computer to perform the methods described in the above embodiments.

[0171] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0172] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply a sequential order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs. In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several requests to cause a computer device (which may be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0173] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0174] The above provides a detailed description of a lane line detection method, apparatus, electronic device, and storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A lane line detection method characterized by, The application is applied to a vehicle terminal, and the method comprises the following steps: An environment image of a vehicle is acquired; An edge detection operator is used to perform convolution processing on the environment image to obtain gradient values of each pixel point in the environment image, wherein the edge detection operator comprises a convolution kernel with a kernel size of 1*N, N is an odd number greater than 1, the convolution kernel comprises N elements, and the element arranged at the Yth position in the convolution kernel is a negative number of the element arranged at the N-Y+1th position, Y is a positive integer less than or equal to N, and Y is not equal to (N+1) / 2; Edge feature points of each lane mark contained in the environment image are determined from each pixel point according to the gradient values of each pixel point and a gradient threshold, wherein the edge feature points comprise a plurality of first edge feature points and a plurality of second edge feature points, the distance between the first edge feature points belonging to the same lane mark and the vehicle is greater than the distance between the second edge feature points and the vehicle; The plurality of first edge feature points and the plurality of second edge feature points are paired to obtain a plurality of target feature point pairing groups corresponding to each lane mark; The lane mark closest to the vehicle is obtained as a target lane line according to the plurality of target feature point pairing groups corresponding to each lane mark, and the target lane line is fitted according to the plurality of target feature point pairing groups corresponding to the target lane line to obtain a fitting result corresponding to the target lane line.

2. The method of claim 1, wherein, The edge feature points of each lane mark contained in the environment image are determined from each pixel point according to the gradient values of each pixel point and a gradient threshold, and the method comprises the following steps: The pixel points with gradient values greater than a first gradient threshold in each pixel point are determined as the first edge feature points, and the pixel points with gradient values less than a second gradient threshold are determined as the second edge feature points, the first gradient threshold and the second gradient threshold are negative numbers of each other, and the first gradient threshold is greater than the second gradient threshold.

3. The method of claim 1, wherein, The plurality of first edge feature points and the plurality of second edge feature points are paired to obtain a plurality of target feature point pairing groups corresponding to each lane mark, and the method comprises the following steps: The plurality of first edge feature points and the plurality of second edge feature points belonging to the same side of the vehicle are paired according to a first distance threshold to obtain a plurality of first feature point pairing groups corresponding to the same side of the vehicle; Each first edge feature point and each second edge feature point contained in the plurality of first feature point pairing groups corresponding to the same side of the vehicle are re-paired according to a second distance threshold, and part of the first edge feature points and part of the second edge feature points are merged to obtain a plurality of target feature point pairing groups corresponding to each lane mark located on the same side of the vehicle.

4. The method of claim 3, wherein, The plurality of first edge feature points and the plurality of second edge feature points belonging to the same side of the vehicle are paired according to a first distance threshold, and the method comprises the following steps: From the multiple second edge feature points belonging to the same side of the vehicle, a second edge feature point with the smallest distance to a target first edge feature point and a distance smaller than the first distance threshold to the target first edge feature point is determined as a second edge feature point paired with the target first edge feature point, the target first edge feature point being any one of the multiple first edge feature points belonging to the same side of the vehicle.

5. The method of claim 4, wherein, The determining, from the multiple second edge feature points belonging to the same side of the vehicle, a second edge feature point with the smallest distance to a target first edge feature point and a distance smaller than the first distance threshold to the target first edge feature point as a second edge feature point paired with the target first edge feature point, comprises: pairing each first edge feature point belonging to the same side of the vehicle in turn according to the distance of the first edge feature point to the vehicle from far to near; for each first edge feature point, determining a first distance between the current first edge feature point and a current second edge feature point belonging to the same side of the vehicle according to the distance of the second edge feature point to the vehicle from far to near, judging whether the first distance is smaller than the first distance threshold, if yes, determining the current second edge feature point as a second edge feature point paired with the current first edge feature point, if not, taking a next unpaired second edge feature point as a new current second edge feature point and re-executing the step of determining the first distance between the current first edge feature point and the current second edge feature point belonging to the same side of the vehicle.

6. The method of claim 3, wherein, The re-pairing and merging, according to the second distance threshold, between each first edge feature point and each second edge feature point included in the multiple first feature point pairing groups corresponding to the same side of the vehicle, to obtain multiple target feature point pairing groups corresponding to each lane mark on the same side of the vehicle, comprises: determining a second distance between a current second edge feature point and a current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle according to the distance of the second edge feature point to the vehicle from far to near and according to the distance of the first edge feature point to the vehicle from near to far, and judging whether the second distance is smaller than the second distance threshold; If yes, it is determined that the current first edge feature point is a first edge feature point that is re-paired with the current second edge feature point, and the current second edge feature point, the first edge feature point paired with the current second edge feature point in the plurality of first feature point pairing groups, the current first edge feature point, and the second edge feature point paired with the current first edge feature point in the plurality of first feature point pairing groups are merged to obtain a target feature point pairing group, and the next unmerged second edge feature point is taken as a new current second edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed. If no, the next first edge feature point is taken as a new current first edge feature point, and the step of determining the second distance between the current second edge feature point and the current first edge feature point in each first feature point pairing group corresponding to the same side of the vehicle is re-executed.

7. A lane line detection device characterized by comprising: The application is applied to a vehicle terminal; the device comprises: An acquisition module is configured to acquire an environment image of a vehicle. An identification module is configured to perform convolution processing on the environment image by using an edge detection operator to obtain gradient values of each pixel point in the environment image, wherein the edge detection operator comprises a convolution kernel with a kernel size of 1*N, N is an odd number greater than 1, the convolution kernel comprises N elements, and the element arranged at the Yth position in the convolution kernel is a negative number of the element arranged at the (N-Y+1)th position, Y is a positive integer less than or equal to N, and Y is not equal to (N+1) / 2; and each lane mark edge feature point contained in the environment image is determined from each pixel point according to the gradient values of the pixel points and a gradient threshold value; the edge feature points comprise a plurality of first edge feature points and a plurality of second edge feature points, and the distance between the first edge feature points belonging to the same lane mark and the vehicle is greater than the distance between the second edge feature points and the vehicle. A pairing module is configured to pair the plurality of first edge feature points and the plurality of second edge feature points to obtain a plurality of target feature point pairing groups corresponding to each lane mark. A fitting module is configured to screen a lane mark closest to the vehicle as a target lane line according to the plurality of target feature point pairing groups corresponding to each lane mark, and fit the target lane line according to the plurality of target feature point pairing groups corresponding to the target lane line to obtain a fitting result corresponding to the target lane line.

8. An electronic device, comprising: The application comprises: A memory storing executable program codes; A processor coupled with the memory; The processor invokes the executable program codes stored in the memory to execute the method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to execute the method in any one of claims 1 to 6. The computer readable storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to execute the method in any one of claims 1 to 6.

Images