Lane departure warning method and device

Abstract

The application provides a lane departure warning method and device, acquires a set of images to be detected, the set of images to be detected includes continuous multiple frames of images to be detected and vehicle lateral distance, lane line inclination angle and pixel lateral distance under the bird's eye view IPM corresponding to each frame of images to be detected; whether the vehicle lateral distance corresponding to a single frame of images to be detected is greater than a first threshold value is judged, whether the lane line inclination angle corresponding to the current frame of images to be detected is less than a second threshold value is judged, or whether the pixel lateral distance under the IPM corresponding to the current frame of images to be detected is greater than a third threshold value is judged; if the vehicle lateral distance is greater than the first threshold value, or the lane line inclination angle is less than the second threshold value, or the pixel lateral distance under the IPM is greater than the third threshold value, then it is determined that the corresponding image to be detected is that the lane line does not deviate. The application can effectively solve the problem that the result of lateral distance is discontinuous and extremely unstable, is suitable for more complex scenes, and has excellent robustness.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety technology, and in particular to a lane departure warning method and device. Background Technology

[0002] Lane departure warning (LDW) is an important component of advanced driver assistance systems (ADAS), specifically referring to the behavior of a vehicle gradually deviating from its original lane during driving. Most current LDW technical standards use the time before the vehicle is about to cross the lane line (TLC) to determine whether the vehicle has touched the lane boundary and issue a lane departure warning.

[0003] The basic process of lane departure warning (LDW) based on TLC is as follows: Obtain the current vehicle's position within the lane; obtain the vehicle's distance from the lane lines; calculate the vehicle's lateral speed; calculate the time remaining before the vehicle crosses the lane lines; and determine if the calculated time before crossing exceeds a set threshold. Regardless of the LDW alarm standard, known road and vehicle information is required, including lane line positions, vehicle position, and multi-frame position information. With this information already obtained, different LDW alarm mechanisms can be specified according to different standards.

[0004] However, the TLC lane departure warning mechanism has the following main problems: 1) it heavily relies on the accuracy of vehicle lateral distance calculation; 2) it heavily relies on the continuity of vehicle lateral distance across multiple frames; 3) it cannot continue to issue warnings after the vehicle has crossed the lane line; 4) it continuously issues false alarms when the vehicle is at the boundary of lane departure. Furthermore, the current lane departure warning mechanism cannot be applied to the entire process of a vehicle moving from its current lane to another lane or returning to its current lane. False alarms are easily generated when lane position calculation is inaccurate or when the continuity of position across multiple frames is insufficient, especially in poor road conditions such as bumpy scenarios. It also cannot handle LDW alarms when there is only a single lane line. Summary of the Invention

[0005] This invention provides a lane departure warning method and device to address the shortcomings of poor sensitivity in existing lane line warning systems. It can avoid false alarms caused by factors such as vehicle lateral distance and bumpy conditions, thereby improving the accuracy of lane line warnings.

[0006] This invention provides a lane departure warning method, comprising: acquiring a set of images to be detected, the set of images to be detected including multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle, and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of the images to be detected; determining whether the vehicle lateral distance corresponding to a single frame of the images to be detected is greater than a first threshold, determining whether the lane line tilt angle corresponding to the current frame of the images to be detected is less than a second threshold, or determining whether the pixel lateral distance under the IPM corresponding to the current frame of the images to be detected is greater than a third threshold; if the vehicle lateral distance is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under the IPM is greater than the third threshold, then determining that the corresponding image to be detected is a lane line that has not deviated.

[0007] According to a lane departure warning method provided by the present invention, after determining that the lane line of the corresponding image to be detected has not deviated, the method includes: grouping the images to be detected that have been determined to have no lane line deviation by a first preset frame number to obtain a plurality of first image sets to be detected; the first image sets to be detected include images to be detected that meet the first preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; obtaining, according to each first image set to be detected, a first mean and a first variance of the lane line tilt angle corresponding to each first image set to be detected, and obtaining a first count value that the lane line tilt angle of the current frame image to be detected in each first image set to be detected is greater than the lane line tilt angle of the previous frame image to be detected; determining whether the first mean of each first image set to be detected is greater than a fourth threshold, whether the first variance is greater than a fifth threshold, and whether the first count value is greater than a sixth threshold, and determining that the lane line of the image to be detected in the corresponding first image set has deviated based on the first mean being greater than the fourth threshold, the first variance being greater than the fifth threshold, and the first count value being greater than the sixth threshold within the first preset frame number; otherwise, determining that the lane line of the image to be detected in the corresponding first image set has not deviated.

[0008] According to a lane departure warning method provided by the present invention, after determining that the lane line of the corresponding image to be detected has not deviated, the method includes: obtaining the vehicle lateral distance outlier, lane line tilt angle outlier, and pixel lateral distance outlier under IPM of the image to be detected that has been determined to have no lane line deviation, based on the absolute median difference (MAD) algorithm; and filtering out the vehicle lateral distance outlier, the lane line tilt angle outlier, and the pixel lateral distance outlier under IPM, respectively.

[0009] According to a lane departure warning method provided by the present invention, after determining that the lane lines of the images to be detected in the corresponding first image set have not deviated, the method includes: regrouping the first image sets to be detected according to a second preset frame number to obtain a second image set to be detected; the second image set to be detected includes images to be detected that meet the second preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; obtaining a second mean and a second variance of the pixel lateral distance under IPM for each of the second image sets to be detected, and obtaining a second count value for the lane line tilt angle of the current frame image to be detected in each of the second image sets to be detected that is greater than that of the previous frame; determining whether the second mean of each of the second image sets to be detected is less than a seventh threshold, whether the second variance is greater than an eighth threshold, and whether the second count value is greater than a ninth threshold, and determining that the lane lines of the images to be detected in the corresponding second image set have deviated based on the second mean being less than the seventh threshold, the second variance being greater than the eighth threshold, and the second count value being greater than the ninth threshold; otherwise, determining that the lane lines of the images to be detected in the corresponding second image set have not deviated.

[0010] According to a lane departure warning method provided by the present invention, after determining that the lane lines of the images to be detected in the corresponding second image set have not deviated, the method includes: regrouping the second image sets to be detected according to a third preset frame number to obtain a third image set to be detected; the third image set to be detected includes images to be detected that meet the third preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; according to each of the third image sets to be detected, obtaining a third variance of the vehicle lateral distance corresponding to each of the third image sets to be detected, obtaining a third count value that the vehicle lateral distance of the current frame image to be detected in each of the third image sets to be detected is less than the previous frame, and obtaining a fourth variance of the last three frames image to be detected in each of the third image sets to be detected; based on the fact that the lateral distance of the current frame image to be detected in each of the third image sets to be detected is less than a first distance preset value, and the third variance of the corresponding third image set to be detected is greater than a first variance preset threshold, the method further includes: If the third count value of the detected image set is greater than the first count preset threshold and meets the first set value, the fourth variance of the corresponding third image set to be detected is greater than the second variance preset threshold, and the indices of the last three frames of the image set to be detected are consecutive frames, then it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is about to cross the line and change lanes at high speed; otherwise, based on the fact that the lateral distance of the current frame of the image to be detected in each of the third image sets to be detected is less than the second distance preset value, the third variance of the corresponding third image set to be detected is greater than the third variance preset threshold, the third count value of the corresponding third image set to be detected is greater than the second count preset threshold, the fourth variance of the corresponding third image set to be detected is greater than the fourth variance preset threshold, and the indices of the last three frames of the image set to be detected are consecutive frames, then it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is at the edge of crossing the line and changing lanes at low speed; otherwise, it is determined that the lane line of the image to be detected in the corresponding third image set has not deviated.

[0011] According to a lane departure warning method provided by the present invention, obtaining a third image set to be detected includes: regrouping each second image set to be detected, which is determined to be free of lane deviation, by a third preset number of frames to obtain an initial third image set to be detected; and filtering out vehicles whose lateral distances in each frame of the initial third image set to be detected are greater than the third preset distance value based on a third preset distance value to obtain a third image set to be detected.

[0012] According to a lane departure warning method provided by the present invention, when obtaining the third variance of the vehicle lateral distance corresponding to each of the third image sets to be detected, a third mean of the vehicle lateral distance corresponding to each of the third image sets to be detected is also obtained; when obtaining the fourth variance of the last three frames of the image set to be detected in each of the third image sets to be detected, a fourth mean of the last three frames of the image set to be detected is also obtained.

[0013] After determining that the lane lines of the images to be detected within the corresponding third image set have not deviated, the process includes: determining, based on each of the third image sets to be detected where the lane lines have not deviated, the number of frames in the corresponding third image set where the lateral distance between the vehicle and the vehicle is less than a fourth preset distance value; if the number of frames is greater than a first preset number of frames and the third count value of the corresponding third image set is greater than a third preset threshold, then it is determined that the lane lines of the images to be detected within the corresponding third image set have deviated, indicating that the vehicle is currently crossing the lane lines; otherwise,

[0014] If the third mean of the third set of images to be detected, which is determined to be without lane line deviation, is less than the first mean preset threshold, the third count value of the corresponding third set of images to be detected is greater than the fourth count preset threshold, and the lateral distance of the current frame of the image to be detected in the corresponding third set of images to be detected is less than the fifth distance preset value, then it is determined that the lane line of the image to be detected in the corresponding third set of images to be detected has deviated, indicating that the vehicle is on the edge of the lane line and is about to cross it; otherwise,

[0015] If the third mean of the third set of images to be detected, which is determined to be without lane line deviation, is less than the preset threshold of the third mean, the third count value of the corresponding third set of images to be detected is greater than the preset threshold of the fifth count and meets the second set value, the fourth mean of the corresponding third set of images to be detected is less than the preset threshold of the fourth mean, the indices of the last three frames of images to be detected in the corresponding third set of images to be detected are consecutive frames, and the lateral distance of the current frame of images to be detected in the corresponding third set of images to be detected is less than the preset value of the sixth distance, then it is determined that the lane line of the image to be detected in the corresponding third set of images to be detected has deviated, indicating that the lateral distance of the vehicle is changing rapidly and there is a possibility of crossing the line; otherwise...

[0016] If the third mean of the third set of images to be detected, which is determined to be without lane line deviation, is less than the fifth mean preset threshold, the third count value of the corresponding third set of images to be detected is greater than the sixth count preset threshold and meets the third set value, the fourth mean of the corresponding third set of images to be detected is less than the sixth mean preset threshold, and the difference in lateral distance between the current frame of the image to be detected and the previous frame of the image to be detected in the corresponding third set of images to be detected is greater than a preset difference threshold, then it is determined that the lane line of the image to be detected in the corresponding third set of images to be detected has deviated, indicating that the lateral distance of the vehicle is changing at a low speed and there is a possibility of crossing the line; otherwise, it is determined that the lane line of the image to be detected in the corresponding third set of images to be detected has not deviated.

[0017] The present invention also provides a lane departure warning device, comprising: a data acquisition module for acquiring a set of images to be detected, the set of images to be detected including multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of the images to be detected; a deviation judgment module for judging whether the vehicle lateral distance corresponding to a single frame of the images to be detected is greater than a first threshold, judging whether the lane line tilt angle corresponding to the current frame of the images to be detected is less than a second threshold, or judging whether the pixel lateral distance under the IPM corresponding to the current frame of the images to be detected is greater than a third threshold; and a result determination module for judging that if the vehicle lateral distance is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under the IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane line that has not deviated.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the lane departure warning methods described above.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the lane departure warning methods described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the lane departure warning methods described above.

[0021] The lane departure warning method and device provided by this invention judge whether the lane line deviates by measuring the vehicle's lateral distance, the lane line tilt angle, and the pixel lateral distance under the bird's-eye view IPM. This facilitates lane departure warning based on the deviation judgment result. In addition, by combining the combined features of variance, mean, and the number of lateral distance decreases, it effectively solves the problem of unstable lateral distance results due to bumps and discontinuities. It can also simultaneously handle alarms for the entire process of a vehicle moving from the current lane to another lane or returning to its original lane. It is suitable for more varied and complex scenarios and has excellent robustness. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic flowchart of the lane departure warning method provided by the present invention;

[0024] Figure 2 This is a schematic diagram of the lane departure warning device provided by the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Figure 1 A schematic flowchart of a lane departure warning method according to the present invention is shown, the method comprising:

[0028] S11, Obtain the image set to be detected. The image set to be detected includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of images to be detected.

[0029] S12, determine whether the lateral distance of the vehicle corresponding to the single frame image to be detected is greater than the first threshold, determine whether the lane line tilt angle corresponding to the current frame image to be detected is less than the second threshold, or determine whether the pixel lateral distance under the IPM corresponding to the current frame image to be detected is greater than the third threshold.

[0030] S13. If the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the lateral distance of the pixels under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane line that has not deviated.

[0031] It should be noted that S1N in this specification does not represent the order of the lane departure warning methods. The lane departure warning method of the present invention will be described in detail below.

[0032] Step S11: Obtain the image set to be detected. The image set to be detected includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of images to be detected.

[0033] In this embodiment, obtaining the set of images to be detected includes: obtaining the images to be detected; detecting the images to be detected to obtain the position of the vehicle relative to the current lane; and obtaining the lateral distance of the vehicle, the lane line tilt angle, and the pixel lateral distance under the bird's-eye view IPM based on the position of the vehicle relative to the current lane.

[0034] Step S12: Determine whether the lateral distance of the vehicle corresponding to a single frame of the image to be detected is greater than the first threshold, determine whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than the second threshold, or determine whether the pixel lateral distance under IPM corresponding to the current frame of the image to be detected is greater than the third threshold.

[0035] It should be noted that by comparing the vehicle's lateral distance, lane line tilt angle, or pixel lateral distance under IPM with the corresponding thresholds, it is possible to predict and issue warnings about lane line deviation based on whether the vehicle's lateral distance is greater than the first threshold, whether the lane line tilt angle is less than the second threshold, or whether the pixel lateral distance under IPM is greater than the third threshold.

[0036] Step S13: If the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane line that has not deviated.

[0037] It should be added that if the lateral distance of the vehicle is not greater than the first threshold, the lane line tilt angle is not less than the second threshold, and / or the pixel lateral distance under IPM is not greater than the third threshold, then the corresponding image to be detected is determined to have deviated from the lane line. Furthermore, the first threshold can be 2m, the second threshold can be 65°, and the third threshold can be 10. These can be set according to actual design and usage requirements to improve the accuracy of lane departure warnings.

[0038] In this embodiment, after determining that the corresponding image to be detected is a lane line that has not deviated, the process includes:

[0039] Sa1, based on the images to be detected that are determined not to deviate from the lane line, groups them according to a first preset frame number to obtain multiple first image sets to be detected. Each first image set to be detected includes images that meet the first preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM. It should be noted that the first preset frame number can be set according to actual design requirements. For example, when the first preset frame number is 8, the images to be detected that have not deviated from the lane line are divided into multiple first image sets according to time sequence, and each first image set contains 8 frames.

[0040] Sa2, based on each first image set to be detected, obtains the first mean m1 and the first variance v1 of the lane line tilt angle corresponding to each first image set to be detected, and obtains the first count value n1 of the lane line tilt angle of the current frame image to be detected in each first image set to be detected being greater than the lane line tilt angle of the previous frame image to be detected.

[0041] Sa3 determines whether the first mean m1 of each first image set to be detected is greater than the fourth threshold, whether the first variance v1 is greater than the fifth threshold, and whether the first count value n1 is greater than the sixth threshold. Based on the first mean m1 being greater than the fourth threshold, the first variance v1 being greater than the fifth threshold, and the first count value m1 being greater than the sixth threshold within a first preset number of frames, it is determined that the lane line of the image to be detected in the corresponding first image set has deviated; otherwise, it is determined that the lane line of the image to be detected in the corresponding first image set has not deviated.

[0042] It should be added that the fourth threshold can be 60°, the fifth threshold can be 2, and the sixth threshold can be 3. The specific values ​​can be set according to the actual design and usage requirements. Accordingly, when m1 > 60°, v1 > 2, and n1 > 3, a lane departure warning (LDW) is determined to have occurred.

[0043] In an optional embodiment, after determining that the corresponding image to be detected is one where the lane lines have not deviated, the process includes: obtaining outlier values ​​for vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM for the image to be detected where the lane lines have not deviated, based on the Absolute Median Difference (MAD) algorithm; and filtering out these outlier values. It should be noted that filtering out these outlier values ​​avoids their impact on the lane departure warning result, reduces false alarms, decreases computational load, and improves the efficiency of lane departure warning.

[0044] In addition, to further improve the accuracy of lane departure warning, after determining that the lane lines in the corresponding first set of images to be detected have not deviated, the following steps are included:

[0045] Sb1, based on the first set of images to be detected that are determined not to deviate from the lane line, they are regrouped according to a second preset frame number to obtain a second set of images to be detected. The second set of images to be detected includes the images to be detected that meet the second preset frame number, along with their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM. It should be noted that the second preset frame number can be set according to actual design requirements; the first preset frame number can be used as a reference. It should also be noted that when the second preset frame number is the same as the first preset frame number, regrouping is not necessary; in this case, the first set of images to be detected that are determined not to deviate from the lane line can be directly used as the second set of images to be detected.

[0046] Sb2, based on each second image set to be detected, obtains the second mean m2 and the second variance v2 of the pixel lateral distance under IPM for each second image set to be detected, and obtains the second count value n2 of the lane line tilt angle of the current frame image to be detected in each second image set to be detected being greater than that of the previous frame.

[0047] Sb3 determines whether the second mean m2 of each second image set to be detected is less than the seventh threshold, whether the second variance v2 is greater than the eighth threshold, and whether the second count value n2 is greater than the ninth threshold. Based on the second mean m2 being less than the seventh threshold, the second variance v2 being greater than the eighth threshold, and the second count value n2 being greater than the ninth threshold, it is determined that the lane line of the image to be detected in the corresponding second image set has deviated; otherwise, it is determined that the lane line of the image to be detected in the corresponding second image set has not deviated.

[0048] It should be added that the seventh threshold can be 8, the eighth threshold can be 1, and the ninth threshold can be 2. The specific values ​​can be set according to the actual design and usage requirements. Accordingly, when m2 < 8, v2 > 1, and n2 > 2, a lane departure warning (LDW) is determined to have occurred.

[0049] In an optional embodiment, after determining that the lane lines of the images to be detected within the corresponding second set of images to be detected have not deviated, the process includes:

[0050] Sc1, based on the second set of images to be detected that are determined to have no deviation from the lane line, regroups them according to the third preset frame number to obtain the third set of images to be detected; the third set of images to be detected includes the images to be detected that meet the third preset frame number and their corresponding vehicle lateral distance, lane line tilt angle and pixel lateral distance under IPM.

[0051] It should be noted that obtaining the third set of images to be detected includes: regrouping each of the second sets of images to be detected, which are determined to have no lane line deviation, according to a third preset frame number, to obtain an initial third set of images to be detected; and filtering out vehicles with lateral distances greater than the third preset distance in each frame of the initial third set of images to be detected, based on a third preset distance value, to obtain the third set of images to be detected. In this embodiment, the third preset distance value and the third preset distance value can be set according to actual design requirements. For example, the third preset distance value can be 8, and the third preset distance value can be 2, thereby filtering out images with lateral distances greater than 2 in eight consecutive frames of images to be detected, so as to avoid data interference and affect the efficiency and accuracy of subsequent lane line warnings.

[0052] In addition, the third preset frame number can be set according to actual design requirements; refer to the second preset frame number for details. It should be noted that when the third preset frame number is the same as the second preset frame number, regrouping is not necessary. In this case, the second set of images determined to be without lane line deviation is directly used as the third set of images to be detected.

[0053] Sc2, based on each third image set to be detected, obtain the third variance v3 of the vehicle lateral distance corresponding to each third image set to be detected, obtain the third count value n3 of the vehicle lateral distance of the current frame image to be detected in each third image set to be detected being less than the previous frame, and obtain the fourth variance v4 of the last three frames image to be detected in each third image set to be detected.

[0054] Sc3: Based on the fact that the lateral distance of the current frame of the image to be detected in each third image set is less than the first distance preset value, the third variance v3 of the corresponding third image set is greater than the first variance preset threshold, the third count value n3 of the corresponding third image set is greater than the first count preset threshold and meets the first set value, the fourth variance v4 of the corresponding third image set is greater than the second variance preset threshold, and the index of the last three frames of the image to be detected in the corresponding third image set is a consecutive frame, it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is about to cross the line and change lanes at high speed; otherwise, it is determined that the lane line of the image to be detected in the corresponding third image set has not deviated.

[0055] It should be added that the first distance preset value can be 1.05m, the first variance preset threshold can be 0.02, the first count preset threshold can be 5, the first setting value can be 5, and the second variance preset threshold can be 0.01. The specific settings can be made according to the actual design and usage requirements.

[0056] Furthermore, when it is determined that the lane line of the image to be detected in the corresponding third image set has not deviated, the method further includes: based on the fact that the lateral distance of the current frame image to be detected in each third image set is less than the second distance preset value, the third difference v3 of the corresponding third image set is greater than the third difference preset threshold, the third count value n3 of the corresponding third image set is greater than the second count preset threshold, the fourth variance v4 of the corresponding third image set is greater than the fourth variance preset threshold, and the index of the last three frames of the image to be detected in the corresponding third image set is a consecutive frame, it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is on the edge of the line and changing lanes at low speed; otherwise, it is determined that the lane line of the image to be detected in the corresponding third image set has not deviated.

[0057] It should be noted that the second distance preset value can be 1m, the third difference preset threshold can be 3, the second count preset threshold can be 5, and the fourth variance preset threshold can be 0.0075. The specific settings can be made according to the actual design and usage requirements.

[0058] It should be noted that the first preset distance value is not exactly the same as the second preset distance value, the third preset difference threshold is not exactly the same as the first preset variance threshold, the second preset count threshold is not exactly the same as the first preset count threshold, and the fourth preset variance threshold is not exactly the same as the second preset variance threshold. These can be set according to actual design requirements. By using the above-mentioned lane line deviation judgment method, which combines the combined characteristics of variance, mean, and the number of lateral distance decreases, the problem of unstable and discontinuous lateral distance results due to bumps can be effectively solved. Furthermore, it can simultaneously handle alarms for the entire process of a lane changing from the current lane to another lane or returning to the same lane.

[0059] In an optional embodiment, when obtaining the third variance v3 of the vehicle lateral distance corresponding to each third image set to be detected, the third mean m3 of the vehicle lateral distance corresponding to each third image set to be detected is also obtained; and when obtaining the fourth variance v4 of the last three frames of the image set to be detected in each third image set to be detected, the fourth mean m4 of the last three frames of the image set to be detected is also obtained.

[0060] Accordingly, based on the above judgment, after determining that the lane lines of the images to be detected within the corresponding third set of images to be detected have not deviated, the process includes:

[0061] Sd1, based on each third set of images determined to have no lane line deviation, determines the number of frames f of the images in the corresponding third set of images where the lateral distance between the vehicle and the target vehicle is less than a fourth distance preset value. In this embodiment, the fourth distance preset value can be set according to actual design requirements, for example, it can be 0.9m.

[0062] Sd2, based on the frame number f being greater than the first preset frame number and the third count value n3 corresponding to the third image set being detected being greater than the third preset threshold, determines that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is in a state of crossing the line; otherwise, it determines that the lane line of the image to be detected in the corresponding third image set has not deviated. This method can address the problem of insufficient accuracy in lateral distance prediction, solve the lane departure warning problem for single lane lines, improve the accuracy of lane departure warnings, and reduce false alarms.

[0063] It should be added that the first preset frame count and the third preset threshold can be set according to actual design requirements. For example, the first preset frame count can be 6 and the third preset threshold can be 5.

[0064] Furthermore, based on the above judgment, after determining that the lane lines of the detected images in the corresponding third image set have not deviated, the method further includes: if the third mean m3 of the third image set for which lane lines have not deviated is less than the first mean preset threshold, the third count value n3 of the corresponding third image set for which lane lines have not deviated is greater than the fourth count preset threshold, and the lateral distance of the current frame of the detected images in the corresponding third image set for which lane lines have not deviated is less than the fifth distance preset value, then it is determined that the lane lines of the detected images in the corresponding third image set have deviated, indicating that the vehicle is on the edge of the line and is about to cross the line; otherwise, it is determined that the lane lines of the detected images in the corresponding third image set have not deviated.

[0065] It should be added that the first average preset threshold can be 1.05m, the fourth count preset threshold can be 5, and the fifth distance preset value can be 0.85m. At this time, the prediction result of whether the lane line has deviated is more accurate.

[0066] Further, based on the above judgment, after determining that the lane lines of the detected images in the corresponding third image set have not deviated, the method further includes: if the third mean m3 of the third image set for which lane lines have not deviated is less than a preset threshold for the third mean, the third count value n3 of the corresponding third image set is greater than a preset threshold for the fifth count and meets the second set value, the fourth mean m4 of the corresponding third image set is less than a preset threshold for the fourth mean, the indices of the last three frames of the detected images in the corresponding third image set are consecutive frames, and the lateral distance of the current frame of the detected images in the corresponding third image set is less than a preset value for the sixth distance, then it is determined that the lane lines of the detected images in the corresponding third image set have deviated, indicating that the lateral distance of the vehicle is changing rapidly and there is a possibility of crossing the line; otherwise, it is determined that the lane lines of the detected images in the corresponding third image set have not deviated.

[0067] It should be added that the third average preset threshold can be 1m, the fourth average preset threshold can be 0.8m, the fifth count preset threshold can be 4, and the sixth distance preset value can be 0.9m. Additionally, the second setting value can be set with reference to the first setting value, for example, it can be 5; no further limitations are made here.

[0068] Further, based on the above judgment, after determining that the lane lines of the detected images in the corresponding third image set have not deviated, the method further includes: if the third mean m3 of the third image set for which lane lines have not deviated is less than the fifth mean preset threshold, the third count value n3 of the corresponding third image set is greater than the sixth count preset threshold and meets the third set value, the fourth mean m4 of the corresponding third image set is less than the sixth mean preset threshold, and the difference in lateral distance between the current frame of the detected image and the previous frame of the detected image in the corresponding third image set is greater than the preset difference threshold, then it is determined that the lane lines of the detected images in the corresponding third image set have deviated, indicating that the lateral distance of the vehicle is changing at a low speed and there is a possibility of crossing the line; otherwise, it is determined that the lane lines of the detected images in the corresponding third image set have not deviated.

[0069] It should be added that the preset threshold for the fifth mean can be 0.95m, the preset threshold for the sixth mean can be 0.75m, the preset threshold for the sixth count can be 4, and the preset difference threshold can be 0.1m. These can be set according to actual design and usage requirements. Additionally, the third setting can be set with reference to the first setting, for example, it can be 5; no further limitations are made here.

[0070] In summary, the embodiments of the present invention determine whether the lane line deviates by using the vehicle's lateral distance, the lane line tilt angle, and the pixel lateral distance under the bird's-eye view IPM, thereby facilitating deviation warnings based on the deviation judgment results. In addition, by combining the combined features of variance, mean, and the number of lateral distance decreases, the invention effectively solves the problem of unstable lateral distance results due to bumps and discontinuities. Furthermore, it can simultaneously handle alarms for the entire process of a vehicle moving from the current lane to another lane or returning to its original lane, making it suitable for more varied and complex scenarios and exhibiting excellent robustness.

[0071] The lane departure warning device provided by the present invention is described below. The lane departure warning device described below and the lane departure warning method described above can be referred to in correspondence.

[0072] Figure 2 A schematic diagram of a lane departure warning device is shown, the device comprising:

[0073] Data acquisition module 21 acquires the image set to be detected, which includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of images to be detected.

[0074] The deviation judgment module 22 determines whether the lateral distance of the vehicle corresponding to a single frame of the image to be detected is greater than the first threshold, whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than the second threshold, or whether the pixel lateral distance under IPM corresponding to the current frame of the image to be detected is greater than the third threshold.

[0075] If the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the lateral distance of the pixels under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be an image where the lane line has not deviated.

[0076] In this embodiment, the data acquisition module 21 includes: an image acquisition unit for acquiring an image to be detected; a detection unit for detecting the image to be detected to obtain the position of the vehicle relative to the current lane; and a parameter acquisition unit for obtaining the lateral distance of the vehicle, the lane line tilt angle, and the pixel lateral distance under the bird's-eye view IPM based on the position of the vehicle relative to the current lane.

[0077] The deviation judgment module 22 includes: a first deviation judgment unit, which judges whether the lateral distance of the vehicle corresponding to a single frame of the image to be detected is greater than a first threshold; a second deviation judgment unit, which judges whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than a second threshold; and a third deviation judgment unit, which judges whether the pixel lateral distance under IPM corresponding to the current frame of the image to be detected is greater than a third threshold.

[0078] Result determination module 23 determines that if the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane line that has not deviated; otherwise, the corresponding image to be detected is determined to be a lane line that has deviated.

[0079] Furthermore, based on the determination that the corresponding image to be detected is a lane line without deviation, the result determination module 23 includes: a first image set acquisition unit, which groups the images to be detected that are determined to be lane lines without deviation according to a first preset frame number to obtain multiple first image sets to be detected; the first image sets to be detected include images to be detected that meet the first preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; and a first data acquisition unit, which obtains the first mean and first variance of the lane line tilt angle corresponding to each first image set to be detected, and obtains the first mean and first variance of the lane line tilt angle of each first image set to be detected. A first count value is used to determine whether the lane line tilt angle of the current frame of the image to be detected in a set of images to be detected is greater than the lane line tilt angle of the previous frame of the image to be detected. The first judgment unit determines whether the first mean of each set of images to be detected is greater than a fourth threshold, whether the first variance is greater than a fifth threshold, and whether the first count value is greater than a sixth threshold. Based on the first mean being greater than the fourth threshold, the first variance being greater than the fifth threshold, and the first count value being greater than the sixth threshold within a first preset number of frames, the unit determines that the lane line of the image to be detected in the corresponding set of images to be detected has deviated. Otherwise, the unit determines that the lane line of the image to be detected in the corresponding set of images to be detected has not deviated.

[0080] In an optional embodiment, the result determination module 23, based on the determination that the corresponding image to be detected is a lane line without deviation, further includes: an outlier determination unit, which obtains the vehicle lateral distance outlier, lane line tilt angle outlier, and pixel lateral distance outlier under IPM for the image to be detected that is determined to be a lane line without deviation, according to the absolute median difference (MAD) algorithm; and a filtering unit, which filters out the vehicle lateral distance outlier, lane line tilt angle outlier, and pixel lateral distance outlier under IPM respectively.

[0081] In an optional embodiment, to further improve the accuracy of lane departure warning, based on the determination that the lane lines in the corresponding first image set to be detected have not deviated, the result determination module 23 further includes: a second image set acquisition unit, which regroups each of the first image sets to be detected that has been determined to have no lane line deviation according to a second preset frame number to obtain a second image set to be detected; the second image set to be detected includes the images to be detected that meet the second preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; and a second data acquisition unit, which obtains the corresponding second image set to be detected according to each second image set to be detected. The second judgment unit determines whether the second mean and second variance of the pixel lateral distance under IPM of the measured image set, and whether the lane line tilt angle of the current frame of the image to be detected in each second image set is greater than that of the previous frame; the second judgment unit determines whether the second mean of each second image set is less than a seventh threshold, whether the second variance is greater than an eighth threshold, and whether the second count value is greater than a ninth threshold, and determines that the lane line of the image to be detected in the corresponding second image set has deviated based on the second mean being less than the seventh threshold, the second variance being greater than the eighth threshold, and the second count value being greater than the ninth threshold; otherwise, it determines that the lane line of the image to be detected in the corresponding second image set has not deviated.

[0082] In an optional embodiment, based on the determination that the lane lines of the images to be detected in the corresponding second image set have not deviated, the result determination module 23 further includes: a third image set acquisition unit, which regroups each of the second image sets to be detected that has been determined to have no lane line deviation by a third preset frame number to obtain a third image set to be detected; the third image set to be detected includes images to be detected that meet the third preset frame number and their corresponding vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM; and a third data acquisition unit, which obtains a third difference of the vehicle lateral distance corresponding to each third image set to be detected, and obtains a third count value in which the vehicle lateral distance of the current frame image to be detected in each third image set is less than the previous frame. The third judgment unit determines that the lane line of the image to be detected in the third image to be detected is deviated, indicating that the vehicle is about to cross the line and change lanes at high speed; otherwise, it determines that the lane line of the image to be detected in the third image to be detected is not deviated. The third judgment unit determines that the lane line of the image to be detected in the third image to be detected is not deviated, based on the fact that the lateral distance of the current frame image to be detected in the third image to be detected is less than the first distance preset value, the third third difference of the corresponding third image to be detected is greater than the first variance preset threshold, the third count value of the corresponding third image to be detected is greater than the first count preset threshold and meets the first set value, the fourth difference of the corresponding third image to be detected is greater than the second variance preset threshold, and the index of the last three frames of the image to be detected in the third image to be detected is a continuous frame.

[0083] It should be added that the third image set acquisition unit includes: a grouping subunit, which regroups each of the second images to be detected that are determined to have not deviated from the lane line according to a third preset number of frames to obtain an initial third image set to be detected; and an image set acquisition subunit, which filters out the lateral distance of vehicles in each frame of the initial third image set that is greater than the third preset distance value based on a third preset distance value to obtain a third image set to be detected.

[0084] Furthermore, based on the above determination that the lane lines of the images to be detected in the third set of images to be detected have not deviated, the result determination module 23 further includes: a first lane line deviation judgment unit, which determines that the lane lines of the images to be detected in the third set of images to be detected have deviated, indicating that the vehicle is on the edge of the lane line and changing lanes at low speed; otherwise, it determines that the lane lines of the images to be detected in the third set of images to be detected have not deviated.

[0085] In an optional embodiment, the third data acquisition unit, when obtaining the third variance of the vehicle lateral distance corresponding to each third image set to be detected, also obtains the third mean of the vehicle lateral distance corresponding to each third image set to be detected; and, when obtaining the fourth variance of the last three frames of images to be detected in each third image set to be detected, also obtains the fourth mean of the last three frames of images to be detected in each third image set to be detected.

[0086] Accordingly, based on the above determination that the lane lines of the images to be detected in the corresponding third image set have not deviated, the result determination module 23 further includes: a frame number determination unit, which determines the number of frames of the images to be detected in the corresponding third image set where the lateral distance of the vehicle is less than a fourth distance preset value, according to each third image set where the lane lines have not deviated; and a second lane line deviation judgment unit, which determines that the lane lines of the images to be detected in the corresponding third image set have deviated, and indicates that the vehicle is in a state of crossing the line, based on the frame number being greater than a first preset frame number and the third count value of the corresponding third image set being greater than a third preset threshold; otherwise, it determines that the lane lines of the images to be detected in the corresponding third image set have not deviated.

[0087] Furthermore, based on the above determination that the lane lines of the images to be detected in the third image set to be detected have not deviated, the result determination module 23 further includes: a third lane line deviation judgment unit, which determines that the lane lines of the third image set to be detected, where the third mean value is less than the first mean preset threshold, the third count value of the third image set to be detected is greater than the fourth count preset threshold, and the lateral distance of the current frame image to be detected in the third image set to be detected is less than the fifth distance preset value, and determines that the lane lines of the images to be detected in the corresponding third image set to be detected have deviated, indicating that the vehicle is on the edge of the line and is about to cross the line; otherwise, it determines that the lane lines of the images to be detected in the corresponding third image set to be detected have not deviated.

[0088] Furthermore, based on the above determination that the lane lines of the images to be detected in the third set of images to be detected have not deviated, the result determination module 23 further includes: a fourth lane line deviation judgment unit, which determines that the lane lines of the third set of images to be detected, where the lane lines have not deviated, are less than a third average preset threshold, the third count value of the third set of images to be detected is greater than a fifth count preset threshold and meets a second set value, the fourth average value of the third set of images to be detected is less than a fourth average preset threshold, the indices of the last three frames of images to be detected in the third set of images to be detected are consecutive frames, and the lateral distance of the current frame of images to be detected in the third set of images to be detected is less than a sixth distance preset value, and determines that the lane lines of the images to be detected in the corresponding third set of images to be detected have deviated, indicating that the lateral distance of the vehicle is changing rapidly and there is a possibility of crossing the line; otherwise, it determines that the lane lines of the images to be detected in the corresponding third set of images to be detected have not deviated.

[0089] Furthermore, based on the above determination that the lane lines of the images to be detected in the third set of images to be detected have not deviated, the result determination module 23 further includes: if the third mean of the third set of images to be detected that the lane lines have not deviated is less than the fifth mean preset threshold, the third count value of the third set of images to be detected is greater than the sixth count preset threshold and meets the third set value, the fourth mean of the third set of images to be detected is less than the sixth mean preset threshold, and the difference in lateral distance between the current frame of the image to be detected and the previous frame of the image to be detected in the third set of images to be detected is greater than the preset difference threshold, then it is determined that the lane lines of the images to be detected in the corresponding third set of images to be detected have deviated, indicating that the lateral distance of the vehicle is changing at a low speed and there is a possibility of crossing the line; otherwise, it is determined that the lane lines of the images to be detected in the corresponding third set of images to be detected have not deviated.

[0090] In summary, this embodiment of the invention uses a deviation judgment module to determine whether a lane line has deviated based on the vehicle's lateral distance, lane line tilt angle, and pixel lateral distance under the bird's-eye view IPM obtained by the data acquisition module. This facilitates the result determination module in issuing deviation warnings based on the deviation judgment results. Furthermore, by combining the combined features of variance, mean, and the number of lateral distance decreases, the invention effectively addresses the problem of unstable lateral distance results due to bumps and discontinuities. It can also simultaneously handle alarms for the entire process of a vehicle moving from its current lane to another lane or returning to its original lane, making it suitable for more varied and complex scenarios and exhibiting excellent robustness.

[0091] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 31, a communication interface 32, a memory 33, and a communication bus 34. The processor 31, communication interface 32, and memory 33 communicate with each other via the communication bus 34. The processor 31 can call logical instructions in the memory 33 to execute a lane departure warning method. This method includes: acquiring a set of images to be detected, which includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle, and pixel lateral distance under the bird's-eye view IPM corresponding to each frame; determining whether the vehicle lateral distance corresponding to a single frame of the image to be detected is greater than a first threshold; determining whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than a second threshold; or determining whether the pixel lateral distance under the IPM corresponding to the current frame of the image to be detected is greater than a third threshold; if the vehicle lateral distance is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under the IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane departure warning image.

[0092] Furthermore, the logical instructions in the aforementioned memory 33 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the lane departure warning method provided by the above methods. The method includes: acquiring a set of images to be detected, the set of images to be detected including multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of images to be detected; determining whether the vehicle lateral distance corresponding to a single frame of images to be detected is greater than a first threshold, determining whether the lane line tilt angle corresponding to the current frame of images to be detected is less than a second threshold, or determining whether the pixel lateral distance under the IPM corresponding to the current frame of images to be detected is greater than a third threshold; if the vehicle lateral distance is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under the IPM is greater than the third threshold, then determining that the corresponding image to be detected is a lane line that has not deviated.

[0094] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the lane departure warning method provided by the above methods. The method includes: acquiring a set of images to be detected, the set of images to be detected including multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle, and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of images to be detected; determining whether the vehicle lateral distance corresponding to a single frame of images to be detected is greater than a first threshold, determining whether the lane line tilt angle corresponding to the current frame of images to be detected is less than a second threshold, or determining whether the pixel lateral distance under the IPM corresponding to the current frame of images to be detected is greater than a third threshold; if the vehicle lateral distance is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under the IPM is greater than the third threshold, then determining that the corresponding image to be detected is a lane line that has not deviated.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lane departure warning method, characterized in that, include: Acquire a set of images to be detected, which includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of the images to be detected; Determine whether the lateral distance of the vehicle corresponding to a single frame of the image to be detected is greater than a first threshold, determine whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than a second threshold, or determine whether the pixel lateral distance under IPM corresponding to the current frame of the image to be detected is greater than a third threshold. If the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the pixel lateral distance under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be a lane line that has not deviated. After determining that the corresponding image to be detected is a lane line that has not deviated, the process includes: Based on the images to be detected that are determined to have no deviation from the lane line, they are grouped according to a first preset number of frames to obtain multiple sets of first images to be detected. The first set of images to be detected includes images to be detected that meet the first preset frame number and their corresponding vehicle lateral distance, lane line tilt angle and pixel lateral distance under IPM; Based on each of the first image sets to be detected, the first mean and first variance of the lane line tilt angle corresponding to each of the first image sets to be detected are obtained, and the first count value is obtained in which the lane line tilt angle of the current frame image to be detected in each of the first image sets is greater than the lane line tilt angle of the previous frame image to be detected. Determine whether the first mean of each of the first image sets to be detected is greater than a fourth threshold, whether the first variance is greater than a fifth threshold, and whether the first count value is greater than a sixth threshold. Based on the first mean being greater than the fourth threshold, the first variance being greater than the fifth threshold, and the first count value being greater than the sixth threshold within the first preset number of frames, determine that the lane line of the image to be detected in the corresponding first image set has deviated; otherwise, determine that the lane line of the image to be detected in the corresponding first image set has not deviated.

2. The lane departure warning method according to claim 1, characterized in that, After determining that the corresponding image to be detected is a lane line that has not deviated, the process includes: Based on the absolute median difference (MAD) algorithm, the outlier values ​​of vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM are obtained for the detected image that is determined to have no lane line deviation. Outlier values ​​for vehicle lateral distance, lane line tilt angle, and pixel lateral distance under IPM are filtered out respectively.

3. The lane departure warning method according to claim 1, characterized in that, After determining that the lane lines of the images to be detected within the first set of images to be detected have not deviated, the process includes: Based on the first set of images to be detected that are determined to have no deviation from the lane line, they are regrouped according to the second preset frame number to obtain the second set of images to be detected. The second set of images to be detected includes images to be detected that meet the second preset frame number and their corresponding vehicle lateral distance, lane line tilt angle and pixel lateral distance under IPM. Based on each of the second image sets to be detected, the second mean and the second variance of the pixel lateral distance under IPM for each of the second image sets to be detected are obtained, and the second count value of the lane line tilt angle of the current frame image to be detected in each of the second image sets to be detected is obtained, which is greater than that of the previous frame. Determine whether the second mean of each second image set to be detected is less than the seventh threshold, whether the second variance is greater than the eighth threshold, and whether the second count value is greater than the ninth threshold. Based on the second mean being less than the seventh threshold, the second variance being greater than the eighth threshold, and the second count value being greater than the ninth threshold, determine that the lane line of the image to be detected in the corresponding second image set has deviated; otherwise, determine that the lane line of the image to be detected in the corresponding second image set has not deviated.

4. The lane departure warning method according to claim 3, characterized in that, After determining that the lane lines of the images to be detected within the second set of images to be detected have not deviated, the process includes: Based on the second set of images to be detected that are determined to have no deviation from the lane line, they are regrouped according to a third preset frame number to obtain a third set of images to be detected; the third set of images to be detected includes images to be detected that meet the third preset frame number and their corresponding vehicle lateral distance, lane line tilt angle and pixel lateral distance under IPM. Based on each of the third image sets to be detected, the third variance of the vehicle lateral distance corresponding to each of the third image sets to be detected is obtained, the third count value of the vehicle lateral distance of the current frame image to be detected in each of the third image sets to be detected being less than the previous frame is obtained, and the fourth variance of the last three frames image to be detected in each of the third image sets to be detected is obtained. If the lateral distance of the current frame of the image to be detected in each of the third image sets is less than a first preset distance value, the third variance of the corresponding third image set is greater than a first preset variance threshold, the third count value of the corresponding third image set is greater than a first preset count threshold and meets a first set value, the fourth variance of the corresponding third image set is greater than a second preset variance threshold, and the indices of the last three frames of the image to be detected in the corresponding third image set are consecutive frames, then it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is about to cross the line and change lanes at high speed; otherwise, If the lateral distance of the current frame of the image to be detected in each of the third image sets is less than the second distance preset value, the third difference of the corresponding third image set is greater than the third difference preset threshold, the third count value of the corresponding third image set is greater than the second count preset threshold, the fourth variance of the corresponding third image set is greater than the fourth variance preset threshold, and the index of the last three frames of the image to be detected in the corresponding third image set is a consecutive frame, then it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is on the edge of the line and changing lanes at low speed; otherwise, it is determined that the lane line of the image to be detected in the corresponding third image set has not deviated.

5. The lane departure warning method according to claim 4, characterized in that, The obtained third set of images to be detected includes: Based on the second set of images to be detected that are determined to have not deviated from the lane line, they are regrouped according to the third preset frame number to obtain the initial third set of images to be detected. Based on the third distance preset value, the vehicle lateral distances in each frame of the initial third image set to be detected that are greater than the third distance preset value are filtered out to obtain the third image set to be detected.

6. The lane departure warning method according to claim 4, characterized in that, When obtaining the third difference of the vehicle lateral distance for each of the third image sets to be detected, a third mean of the vehicle lateral distance for each of the third image sets to be detected is also obtained. When obtaining the fourth variance of the last three frames of the third image set to be detected, the fourth mean of the last three frames of the third image set to be detected is also obtained. After determining that the lane lines of the images to be detected within the third set of images to be detected have not deviated, the process includes: Based on each third image set that is determined to have no lane line deviation, determine the number of frames of the corresponding third image set where the vehicle's lateral distance is less than the fourth distance preset value. If the number of frames is greater than the first preset number of frames, and the third count value of the corresponding third image set to be detected is greater than the third preset threshold, then it is determined that the lane line of the image to be detected in the corresponding third image set has deviated, indicating that the vehicle is in a state of crossing the line; otherwise, If the third mean of the third set of images to be detected, which is determined to be without lane line deviation, is less than a preset threshold for the first mean, the third count value of the corresponding third set of images to be detected is greater than a preset threshold for the fourth count, and the lateral distance of the current frame of the image to be detected in the corresponding third set of images to be detected is less than a preset value for the fifth distance, then it is determined that the lane line of the image to be detected in the corresponding third set of images to be deviated, indicating that the vehicle is on the edge of the lane line and is about to cross it; otherwise, If the third mean of the third image set to be detected, which is determined to be without lane line deviation, is less than a preset threshold for the third mean, the third count value of the corresponding third image set to be detected is greater than a preset threshold for the fifth count and meets a second set value, the fourth mean of the corresponding third image set to be detected is less than a preset threshold for the fourth mean, the indices of the last three frames of the corresponding third image set to be detected are consecutive frames, and the lateral distance of the current frame of the corresponding third image set to be detected is less than a preset value for the sixth distance, then it is determined that the lane line of the corresponding image set to be detected has deviated, indicating that the lateral distance of the vehicle is changing rapidly and there is a possibility of crossing the line; otherwise, If the third mean of the third set of images to be detected, which is determined to be without lane line deviation, is less than the fifth mean preset threshold, the third count value of the corresponding third set of images to be detected is greater than the sixth count preset threshold and meets the third set value, the fourth mean of the corresponding third set of images to be detected is less than the sixth mean preset threshold, and the difference in lateral distance between the current frame of the image to be detected and the previous frame of the image to be detected in the corresponding third set of images to be detected is greater than the preset difference threshold, then it is determined that the lane line of the image to be detected in the corresponding third set of images to be detected has deviated, indicating that the lateral distance of the vehicle is changing at a low speed and there is a possibility of crossing the line; otherwise, It was determined that the lane lines in the corresponding third set of images to be detected did not deviate.

7. A lane departure warning device, characterized in that, include: The data acquisition module acquires a set of images to be detected, which includes multiple consecutive frames of images to be detected and the vehicle lateral distance, lane line tilt angle, and pixel lateral distance under the bird's-eye view IPM corresponding to each frame of the images to be detected. The deviation judgment module determines whether the lateral distance of the vehicle corresponding to a single frame of the image to be detected is greater than a first threshold, whether the lane line tilt angle corresponding to the current frame of the image to be detected is less than a second threshold, or whether the pixel lateral distance under IPM corresponding to the current frame of the image to be detected is greater than a third threshold. If the lateral distance of the vehicle is greater than the first threshold, or the lane line tilt angle is less than the second threshold, or the lateral distance of the pixels under IPM is greater than the third threshold, then the corresponding image to be detected is determined to be an image where the lane line has not deviated. After determining that the corresponding image to be detected is a lane line that has not deviated, the result determination module further includes: The first image set acquisition unit groups the images to be detected that are determined to be without lane line deviation according to a first preset frame number to obtain multiple first image sets to be detected; the first image sets to be detected include images to be detected that meet the first preset frame number and their corresponding vehicle lateral distance, lane line tilt angle and pixel lateral distance under IPM. The first data acquisition unit obtains, based on each of the first image sets to be detected, the first mean and the first variance of the lane line tilt angle corresponding to each of the first image sets to be detected, and obtains a first count value in which the lane line tilt angle of the current frame image to be detected in each of the first image sets to be detected is greater than the lane line tilt angle of the previous frame image to be detected. The first judgment unit determines whether the first mean of each first image set to be detected is greater than a fourth threshold, whether the first variance is greater than a fifth threshold, and whether the first count value is greater than a sixth threshold. Based on the first mean being greater than the fourth threshold, the first variance being greater than the fifth threshold, and the first count value being greater than the sixth threshold within the first preset number of frames, it determines that the lane line of the image to be detected in the corresponding first image set has deviated; otherwise, it determines that the lane line of the image to be detected in the corresponding first image set has not deviated.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the lane departure warning method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the lane departure warning method as described in any one of claims 1 to 6.

Images