Lane departure early warning method and device and electronic equipment

Abstract

The invention provides a lane departure early warning method, a lane departure early warning device and electronic equipment. The method comprises the steps of acquiring a road condition video image in front of a running vehicle in real time; sending the obtained image into the trained lane line detection model to obtain a coordinate set of left and right lane line points, and fitting to obtain a parameter equation; calculating intersection points of left and right lane lines of a lane where the vehicle is located and image bottom edges; judging whether the vehicle deviates from the lane or not according to the interaction ratio of the left and right lane line bottom intersection line segment to a preset line segment; when it is judged that the vehicle reaches the deviation threshold value, alarm service logic judgment is conducted; and if lane departure is determined, alarming and prompting. Only a camera needs to be used, a tedious calibration process does not need to be carried out, other sensors are not needed, or information such as the yaw angle of the vehicle speed is not read, so that the device can be conveniently and rapidly installed on the vehicle, meanwhile, the cost is low, and it is guaranteed that an alarm can be given in time when a driver deviates from driving.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving assistance technology, and in particular to a lane departure warning method, device and electronic equipment. Background Art

[0002] Lane Departure Warning LDW is a lane assist ADAS function, which is the basis for higher-level ADAS functions such as Lane Departure Prevention LDP, Lane Centering Control LCC, and Emergency Lane Keeping ELK. By real-time monitoring of the vehicle's driving status in the lane, when it is judged that a lane departure that is not intended by the driver has occurred or is about to occur, the driver is warned through sound, vibration, etc., to improve road safety and provide the driver with a better driving experience.

[0003] At present, lane departure warning methods are mainly divided into methods based on camera image analysis and methods based on high-precision map analysis. The high-precision map analysis method relies on high-precision maps, vehicle positioning and cameras. Most of them are integrated with the vehicle system when leaving the factory. The algorithm accuracy is high, but this method requires multiple sensors to implement, and the cost and hardware requirements are high. For example, patent CN113538919A determines lane departure information through high-precision maps, target navigation data and current vehicle position. For example, patent CN110329253A relies on maps, cameras and other sensors to predict lane departure by comparing the predicted driving road direction and driving route.

[0004] The lane departure warning method based on camera image analysis is to monitor the lane line of the road ahead and then analyze the position relationship between the vehicle and the lane line to determine whether a deviation has occurred. This method does not need to rely on high-precision maps or other advanced systems, so it can be installed on the vehicle later, with low cost and accuracy that can meet the requirements in most usage scenarios. Therefore, many independent driving assistance system hardware products adopt this method. The current common method is to first accurately calibrate the internal and external parameters and the world coordinate system position of the camera, then use the lane line detection algorithm to obtain the pixel coordinates of the lane line in the image, and convert it into the world coordinate system coordinates, and finally determine whether the vehicle deviates from the lane by judging whether the lateral distance between the vehicle and the lane line is less than the set threshold, or whether the lateral distance between the center line of the vehicle and the center line of the lane is greater than the set threshold. However, this method requires accurate calibration of the internal and external parameters of the camera and the world coordinate system. Another commonly used method is a lane departure warning method based on TTLC (time across the lane). For example, patent CN111660928A uses a deep learning segmentation model and a connected domain detection algorithm to detect lane lines, and combines vehicle speed information to calculate TTLC. When TTLC is less than the set threshold, the system will issue a warning. However, this method requires real-time acquisition of other information such as vehicle speed and yaw angle, and assumes that the current vehicle speed remains unchanged to calculate TTLC. However, the actual motion model of the vehicle when it deviates is very complex, which makes it difficult to accurately calculate TTLC, which may lead to a high false alarm rate. Summary of the invention

[0005] The purpose of the present invention is to provide a lane departure warning method, device and electronic device to solve the above-mentioned defects. The method provided by the present invention only needs to use a camera, does not require a tedious calibration process, and does not rely on other sensors or read information such as vehicle speed and yaw angle. Therefore, it can be easily and quickly installed on the vehicle, and the cost is low.

[0006] In a first aspect, the present invention provides a lane departure warning method, comprising:

[0007] Real-time acquisition of video images of road conditions ahead of the moving vehicle;

[0008] The acquired image is sent to the trained lane line detection model to obtain the coordinate set of the left and right lane line points, and the parametric equation is fitted;

[0009] Calculate the intersection point between the left and right lane lines of the lane where the vehicle is located and the bottom edge of the image;

[0010] Determine whether the vehicle has deviated from the lane based on the interaction ratio between the intersection line segment of the bottom edge of the left and right lane lines and the preset line segment;

[0011] When it is determined that the vehicle has reached the deviation threshold, an alarm business logic judgment is performed;

[0012] If it is determined that the vehicle has left the lane, an alarm will be issued.

[0013] Furthermore, real-time acquisition of video images of road conditions ahead of the moving vehicle includes:

[0014] Acquire the video stream of the road in front of the vehicle in real time, and take frames of the video to obtain image frames;

[0015] The image frame is preprocessed to obtain a preprocessed image, wherein the preprocessing operation includes row color format conversion, image scaling, normalization and cropping.

[0016] Furthermore, the preprocessed image is fed into the trained lane detection model to obtain the coordinate set of the left and right lane points, and the parametric equation is fitted, including:

[0017] The preprocessed image is fed into the trained UFLD lane line detection model, and the model outputs the coordinate set of the left and right lane line points of the lane where the vehicle is located;

[0018] For the coordinates of the left and right lane line points (x1, y1), (x2, y2), ...., (x n ,y n ), and perform quadratic polynomial fitting according to the following formula to obtain the parameter equations of the left and right lane lines;

[0019] x=a0+a1y+a2y 2

[0020] Among them, y is the independent variable, that is, the y-axis coordinate on the image, x is the dependent variable, that is, the x-axis coordinate on the image, a0, a1, a2 are the parameters to be fitted in the equation, which are based on the data points (x1, y1), (x2, y2), ...., (x n ,y n ) is obtained by least squares estimation.

[0021] Further, the intersection point of the left and right lane lines of the lane where the vehicle is located with the bottom edge of the image is calculated, including: using the left and right lane line parameter equations, calculating the value of x when y=h according to the following formula, and calculating the intersection point X of the left and right lane lines with the bottom edge of the image L , X R ;

[0022]

[0023]

[0024] Among them, h is the image height and w is the image width.

[0025] Further, judging whether the vehicle deviates from the lane according to the interaction ratio of the intersection line segment of the bottom edges of the left and right lane lines and the preset line segment includes:

[0026] After the vehicle reaches the middle of the lane, the user connects the intersection point P of the bottom edge of the left and right lane lines in the figure. L P R The line segment is the preset line segment, and the preset area is located according to the preset line segment;

[0027] Calculate the intersection point X of the bottom edge of the left and right lane lines L X R Line segment and preset line segment P L P R The IOU is calculated as follows:

[0028]

[0029] Where X L X R ∩P L P R represents the intersection area of ​​two line segments, X L X R ∪P L P R represents the area of ​​the union of two line segments;

[0030] When the IOU is less than the threshold T, it is judged that lane departure has occurred.

[0031] Furthermore, the threshold T is 0.3-0.6.

[0032] Furthermore, the specific steps of alarm business logic judgment include:

[0033] S1. When the algorithm is initialized, the variable event_mark is set to 0 to record new alarm events;

[0034] S2. After the lane departure decision module is executed, abnormal detection and judgment are performed; specifically, when the intersection point X L X R When the line length is less than 500 pixels, X R -X L <500, event_mark=0 and jump to the image acquisition module again; when the intersection point X L = 0 and the intersection point X of the bottom edge of the right lane line R =0, that is, X L = = 0 and X R ==w, event_mark=0 and jump to the image acquisition module again. If no abnormality is detected, proceed to the next step, which is used for filtering lane line detection and fitting abnormal results to reduce false alarms;

[0035] S3. When the abnormality in step S2 is judged to be normal, determine whether the lane departure decision result is less than the threshold value T. If it is less than T, proceed to the next step; otherwise, event_mark is accumulated by 1 and jumps to the image acquisition module again;

[0036] When the condition of step S3 is met, it is determined whether event_mark is greater than 8. If so, the alarm prompt model will sound an alarm and event_mark=0; otherwise, event_mark=0 and the process jumps back to the image acquisition module.

[0037] In a second aspect, the present invention provides a lane departure warning device, comprising:

[0038] An image acquisition module is used to acquire a video image of the road condition ahead of the moving vehicle in real time;

[0039] The lane detection module is used to send the acquired image to the trained lane detection model, obtain the coordinate set of the left and right lane points, fit the parameter equation, and calculate the intersection point of the left and right lane lines of the lane where the vehicle is located with the bottom edge of the image;

[0040] Parameter configuration module, used for configuring preset line segments;

[0041] The lane departure decision module is used to determine whether the vehicle has deviated from the lane based on the interaction ratio between the intersection line segment of the bottom edge of the left and right lane lines and the preset line segment;

[0042] The alarm logic judgment module is used to make alarm business logic judgment when it is judged that the vehicle reaches the deviation threshold;

[0043] The alarm prompt module is used to issue an alarm if it is determined that the vehicle has deviated from the lane.

[0044] In a third aspect, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0045] Beneficial effects of the present invention: The present invention first processes the acquired image through the lane line detection module to obtain the intersection of the left and right lane lines of the lane where the vehicle is located and the bottom edge of the image; then judges whether the vehicle deviates from the lane according to the IOU (interaction ratio) of the bottom edge intersection line segment of the left and right lane lines and the preset line segment; when the lane deviation decision module judges that the vehicle reaches the deviation threshold, an event alarm judgment is performed. In the entire IOU (interaction ratio) calculation, it is only necessary to calculate whether the ratio of the intersection and the union reaches the deviation threshold to determine whether the lane is deviated. The calculation occupies a small space and has a fast response. Moreover, as long as there is a car with a camera, the preset line segment can be configured once when it is used for the first time. This method can be used without installing other equipment, only using the camera, without a tedious calibration process, and does not rely on other sensors or reading information such as vehicle speed and yaw angle. Therefore, it can be easily and quickly installed on the vehicle, and the cost is low. It can be used offline, and no expenses will be incurred during use, and it is ensured that when the driver deviates from the driving, an alarm can be issued in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration and are not intended to limit the present invention.

[0047] In the accompanying drawings,

[0048] Figure 1 is a flow chart of the lane departure warning method of the present invention;

[0049] Figure 2 The video image frames obtained by the present invention;

[0050] Figure 3 This is the visualization result of the lane detection model of the present invention;

[0051] Figure 4 An example diagram of preset line segments is configured for the present invention;

[0052] Figure 5 A schematic diagram of a preset line segment is configured for the user of the present invention;

[0053] Figure 6 This is a schematic diagram of IOU calculation of the present invention;

[0054] Figure 7 This is the alarm logic flow chart of the present invention;

[0055] Figure 8 This is a schematic diagram of the actual deviation of the right deviation of the present invention;

[0056] Fig. 9 This is a schematic diagram of the actual deviation of the left deviation of the present invention;

[0057] Fig.10This is a schematic diagram of the present invention not deviating from the actual situation;

[0058] Fig.11 This is a block diagram of the lane departure warning device of the present invention;

[0059] Fig.12 This is a block diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0060] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0062] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0063] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0064] According to statistics, about 50% of automobile accidents are caused by vehicles deviating from their normal lanes. The main reasons are that the driver is distracted, inattentive or tired. When the vehicle deviates from the lane, an alarm is sounded or the steering wheel starts to vibrate to remind the driver of the current vehicle deviation, providing the driver with more reaction time. When the execution module is installed, it will even actively intervene in the steering wheel, turning the steering wheel to keep the vehicle body in the correct lane. Therefore, it is very necessary to install a lane departure warning device. The existing lane departure warning system is mainly composed of a camera, a controller and a sensor. When the lane departure system is turned on, the camera (generally placed on the side of the vehicle body or the rearview mirror position) will always collect the marking line of the driving lane, analyze the position of the car in the current lane through image processing and high-precision map. When the car is detected to deviate from the lane, the sensor will collect vehicle data and the driver's operating status in time, and then the controller will send an alarm signal; but in the actual lane deviation process, the reaction time is very short, and the calculation complexity of the sensor and the controller is high, resulting in untimely reaction at the critical moment, causing significant losses; therefore, the present invention designs a lane departure warning method for the reaction speed and equipment requirements. The method only needs to use a camera, does not need to perform a cumbersome calibration process, and does not rely on other sensors or read information such as vehicle speed yaw angle, so it can be easily and quickly installed on the vehicle, and the calculation logic is simple, and it can quickly determine whether it deviates from the lane, shortening the alarm time. At the same time, due to the reduction of some equipment, the overall cost of the device is reduced.

[0065] Example

[0066] See also Figure 1-10 , an embodiment of the present invention provides a lane departure warning method, comprising:

[0067] S101, acquiring a video image of the road condition ahead of the moving vehicle in real time;

[0068] Specifically, after the camera is installed at a suitable position on the front windshield of the vehicle, a video stream of the road condition in front of the moving vehicle is obtained, and the video is framed to obtain a video image.

[0069] The specific steps are as follows: First, park the vehicle on a flat road and wipe the glass inside and outside the camera installation area of ​​the vehicle to ensure that there are no impurities or dirt on the glass surface. This step is critical to ensure that the camera can clearly capture the road conditions and improve the accuracy of subsequent processing. Next, carefully install the forward-facing camera. In terms of the selection of the camera installation position, it is usually chosen to be installed at the front of the vehicle so as to capture the conditions of the road ahead. The installation position of the camera should be accurately measured and adjusted to ensure that it occupies an appropriate position in the field of view in front of the vehicle. Adjust the angle of the camera to ensure that the field of view remains horizontal and that the sky and the road can be displayed in a balanced manner so that they each occupy half of the picture. This can be achieved by carefully rotating and tilting the camera. After the camera is installed, a video stream of the road directly in front of the vehicle is captured by a camera connected to the vehicle system. In order to perform lane departure detection, the video needs to be framed to decompose the continuous video stream into separate image frames. In terms of lane departure detection, different camera installation positions may require different methods. The example of the present invention only takes the case where the camera is installed in front of the vehicle as an example. In this case, computer vision technology can be used to analyze each frame of the image, identify the road lines and monitor the position changes of the vehicle relative to these lane lines. This can help determine if the vehicle has strayed from its intended lane.

[0070] S102, sending the acquired image to a trained lane line detection model to obtain a coordinate set of left and right lane line points, and fitting to obtain a parameter equation;

[0071] Specifically, the image frame is preprocessed and then input into the trained lane line model, and the output detection point coordinates are fitted to obtain the lane line parameter equation; this embodiment adopts the UFLD lane line detection model. UFLD is based on a series of sparse coordinates of mixed anchors (row and column anchors) to represent lane lines. With the help of anchor-drive representation, an ordered classification method is used to learn lane line coordinates. With the help of anchor-driven representation, the lane line detection task is restated as an ordinal classification (there is a close ordinal relationship between adjacent classes), which can effectively solve the lane line occlusion situation, and achieve high performance in both speed and accuracy. The model inputs the preprocessed image and outputs a set of point coordinates of each lane line.

[0072] 1) Image preprocessing

[0073] First, pre-process the video image of the road condition in front of the moving vehicle. Specific steps:

[0074] (1) Convert the RGB color of the image to BGR format;

[0075] (2) Scale the result of step (1) to (width, height / crop_ratio), where width and height are the input width and height of the model, respectively, and crop_ratio is the image cropping ratio. In the specific implementation of the present invention, width is 800, height is 320, and crop_ratio is 0.6;

[0076] (3) normalizing the result of step (2);

[0077] (4) The result of step (3) is cropped in the height direction, and only the lower part with height = 320 is retained to achieve the purpose of cropping the sky part.

[0078] 2) Lane detection

[0079] The preprocessed image is sent to the lane line detection model. In the embodiment of the present invention, the trained UFLD lane line detection model is used. The model outputs the coordinate set of the left and right lane line points of the lane where the vehicle is located, and each coordinate includes the values ​​of the horizontal coordinate (x) and the vertical coordinate (y).

[0080] 3) Lane line parameterization unit

[0081] The parametric equations of the left and right lane lines are obtained by quadratic polynomial fitting. The specific steps are as follows:

[0082] (1) Through step 2), the coordinates of the data points on the left and right lane lines are obtained (x1, y1), (x2, y2), ..., (xn, yn), where y is the independent variable, representing the y-axis coordinate on the image, and x is the dependent variable, representing the x-axis coordinate on the image;

[0083] (2) Use the least squares method to estimate the coefficients a0, a1, and a2 of the quadratic polynomial to fit the data points. The fitting equation of the quadratic polynomial is: x = a0 + a1y + a2y 2 , where y is the independent variable, i.e., the y-axis coordinate on the image; x is the dependent variable, i.e., the x-axis coordinate on the image; a0, a1, a2 are the parameters to be fitted in the equation, which are obtained by the least squares estimation of the data points (x1, y1), (x2, y2), ...., (xn, yn);

[0084] (3) Fitting the left and right lane lines respectively, and obtaining two quadratic polynomial equations, which respectively represent the fitting results of the left lane line and the right lane line;

[0085] (4) The final obtained parameter equations of the left and right lane lines are:

[0086] Left lane line equation: x L =a 0L +a 1L y+a 2L y 2 ;

[0087] Right lane line equation: x R =a 0R +a 1R y+a 2R y 2 .

[0088] S103, calculating the intersection point between the left and right lane lines of the lane where the vehicle is located and the bottom edge of the image;

[0089] Specifically, the intersection point of the left and right lane lines of the lane where the vehicle is located and the bottom edge of the image is calculated. According to the lane line parameter equation obtained above, the value of x is calculated as follows when y=h, and the intersection point X of the left and right lane lines and the bottom edge of the image can be obtained. L , X R .

[0090]

[0091]

[0092] Among them, h is the image height, w is the image width, in particular, when X L <0,X L Take 0, X R >w, X R Take w.

[0093] S104, judging whether the vehicle has deviated from the lane according to the interaction ratio between the line segment of the bottom edge intersection of the left and right lane lines and the preset line segment;

[0094] Specifically, after the camera is installed for the first time, the preset line segments must be configured as required. When the vehicle is driving, the interaction ratio between the intersection line segment at the bottom edge of the left and right lane lines and the preset line segment is calculated to determine whether the vehicle has deviated from the lane. The specific implementation steps are:

[0095] 1) Configure preset line segments

[0096] The process of configuring the preset line segments is quite critical. The vehicle needs to be driven on a wide and level road, and the forward-facing camera is installed at the front of the vehicle to ensure that the vehicle is located in the middle of the lane to keep the distance from the left and right lane lines as equal as possible. Then, the user can connect the line segments at the intersection of the bottom edges of the left and right lane lines (PLPR) through the parameter configuration module to form the required preset line segments. It should be noted that the vehicle must be configured on a standard lane, and the width of a standard lane is usually 3.25 meters, 3.5 meters, 3.75 meters, etc., so it cannot be configured on narrow roads, such as roadside parking lanes, where the width is less than 2.8 meters. When implementing the present invention, the preset line segments only need to be configured once when the camera is first installed, but if the camera position moves, the preset line segments need to be reconfigured. This process ensures that the vehicle can accurately identify and remain on the standard lane, providing key information for road safety and navigation.

[0097] 2) Lane departure detection

[0098] When the vehicle is driving, as the vehicle moves relative to the left and right lane lines, the intersection point X L X R The line segment position will continue to move left and right. L X R The line segment does not deviate from the preset line segment P L P R The degree of overlap can be used to determine whether the vehicle has deviated. L X R Line segment and P L P R The degree of line segment overlap can be achieved by calculating the IOU interaction ratio of the two. IOU is a concept in object detection that is used to describe the degree of overlap between two boxes. The larger the overlapping area, the greater the IOU value.

[0099] When the vehicle is driving, the position of the intersection line segment of the bottom edge of the left and right lane lines will change as the vehicle moves relative to the left and right lane lines. In order to determine whether the vehicle has deviated, the bottom edge intersection line segment X can be calculated in real time. L X R With the preset line segment P L P R The degree of overlap between line segments is used to judge, and the degree of overlap between line segments can usually be calculated using the IOU interaction ratio.

[0100] The calculation formula is as follows:

[0101]

[0102] Where X L X R ∩P L PR Indicates the overlapping area (intersection area) of the bottom intersection line segment and the preset line segment, X L X R ∪P L P R Indicates the total area (union area) of the bottom intersection line segment and the preset line segment. In the above formula, when the overlap of the two line segments is high, the numerator becomes larger (the intersection area becomes larger) and the numerator becomes smaller (the union area becomes larger), then the IOU becomes larger; when the overlap of the two line segments is low, the numerator becomes smaller (the intersection area becomes smaller) and the numerator becomes larger (the union area becomes smaller), then the IOU becomes smaller. Therefore, the overlap degree of the two line segments can be expressed by calculating the IOU of the two.

[0103] Therefore, when the IOU is less than the threshold value T, it is judged that lane departure has occurred. In the embodiment of the present invention, T can be 0.3-0.6. It should be noted that the specific value of the threshold value T can be adjusted according to the actual usage. If T is set to a larger value, the warning sensitivity of lane departure will be improved, but the probability of false alarms may increase. Conversely, if T is set to a smaller value, the sensitivity of the warning will be reduced, but the risk of false alarms may be reduced. Therefore, the selection of the threshold value T should be weighed and adjusted according to the specific application scenarios and requirements to ensure that the system can provide accurate deviation warnings in various situations.

[0104] S105, when it is determined that the vehicle has reached the deviation threshold, an alarm service logic judgment is performed;

[0105] Specifically, when the lane departure decision module determines that the vehicle has reached the departure threshold, an event alarm is made. Optionally, the module can also add other business alarm logic, such as event alarm, interval alarm, and frequent detection filtering.

[0106] Combine the following Figure 7 Further explanation of alarm logic judgment:

[0107] 1) When the algorithm is initialized, set the variable event_mark = 0 to record new alarm events;

[0108] 2) After the lane departure decision module is executed, abnormal detection and judgment are performed. Specifically, when the intersection point X L X R When the line length is less than 500 pixels (X R -X L <500), event_mark=0 and jump to the image acquisition module again; when the intersection point X L = 0 and the intersection point of the bottom edge of the right lane line X R =0 (X L = = 0 and X R= =w), event_mark = 0 and jump to the image acquisition module again. If there is no abnormality in the detection, proceed to the next step. This step is used for filtering the lane line detection and fitting abnormal results to reduce false positives;

[0109] 3) When the abnormality judgment in step 2) is correct, determine whether the lane departure decision result is less than the threshold T. If it is less than T, proceed to the next step; otherwise, event_mark is accumulated by 1 and jumps back to the image acquisition module;

[0110] 4) When the condition of step 3) is met, determine whether event_mark is greater than 8. If so, the alarm prompt model will sound an alarm and event_mark=0; otherwise, event_mark=0 and jump to the image acquisition module again. This step is mainly used for new event alarm judgment, that is, after an alarm in a certain frame, no alarm will be sounded in the next frame even if the lane departure decision result is less than T. Only when the moving vehicle does not deviate and there is no abnormal driving for 8 frames can it be considered a new alarm event. For example, when the vehicle changes lanes and deviates, the deviation condition will always be met, but the algorithm will only sound an alarm once. The alarm will only be sounded when the car returns to the lane and deviates again, which is more reasonable.

[0111] S106: If it is determined that the vehicle has deviated from the lane, an alarm is issued.

[0112] Specifically, the alarm is a key measure taken when the vehicle determines that the lane has deviated. According to the specific implementation of the present invention, the alarm can be given in a variety of ways, such as voice or vibration reminders. This is to remind the driver to correct the behavior of deviating from the lane in time. Here, the present invention adopts a voice prompt, which is specifically manifested as a voice reminder of "Please keep the lane!"

[0113] This voice prompt can be played through the vehicle's built-in audio system or other related equipment. When the system detects that the vehicle has deviated from the preset lane and the IOU value is lower than the preset threshold T (for example, 0.5), the system will trigger a voice prompt to tell the driver to stay in the correct lane. This warning method is intuitive and clear to the driver, which can help them perceive the deviation more quickly and take corrective measures, thereby improving road safety.

[0114] See also Fig.11 The present invention provides a lane departure warning device, comprising:

[0115] An image acquisition module 101 is used to acquire a video image of the road condition ahead of the moving vehicle in real time;

[0116] The lane line detection module 102 is used to send the acquired image into the trained lane line detection model, obtain the coordinate set of the left and right lane line points, and fit the parameter equation, and calculate the intersection of the left and right lane lines of the lane where the vehicle is located with the bottom edge of the image; it is specifically used to detect the coordinates of the lane line points of the road ahead, and then obtain the lane line parameter equation for calculating the variables required by the lane departure decision module.

[0117] A parameter configuration module 103, used for configuring preset line segments;

[0118] The lane departure decision module 104 is used to determine whether the vehicle has deviated from the lane according to the interaction ratio between the bottom edge intersection line segment of the left and right lane lines and the preset line segment; specifically, it is used to determine whether the driving vehicle has deviated from the lane according to the detected lane line;

[0119] The alarm logic judgment module 105 is used to perform alarm business logic judgment when it is determined that the vehicle reaches the deviation threshold, such as event alarm, interval alarm, etc.;

[0120] The alarm prompt module 106 is used to issue an alarm prompt, such as a voice prompt, a vibration prompt, etc., if it is determined that the vehicle has deviated from the lane.

[0121] From the above, it can be seen that the lane departure warning device provided by the embodiment of the present invention obtains a video image of the road condition in front of the moving vehicle; detects the coordinates of the lane line points on the road ahead, and then obtains the lane line parameter equation for calculating the variables required by the lane departure decision module, and configures a preset line segment, and judges whether the moving vehicle has lane deviation according to the detected lane line; judges event alarms and interval alarms through alarm business logic, and then gives alarm prompts through language reminders, vibration reminders, etc. The present invention only needs to use a camera, does not require a cumbersome calibration process, and does not rely on other sensors or read information such as vehicle speed and yaw angle, so it can be easily and quickly installed on the vehicle, and the cost is low.

[0122] See also Fig.12 The present invention also provides an electronic device, comprising: a memory 100 and a processor 200, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0123] The embodiment of the present invention further provides a storage medium, and the embodiment of the present invention further provides a storage medium, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, some or all of the steps in each embodiment of the lane departure warning method provided by the present invention are implemented. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0124] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0125] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the lane departure warning device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0126] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A lane departure warning method, characterized in that: include: Real-time acquisition of video images of road conditions ahead of the moving vehicle; The acquired image is sent to the trained lane line detection model to obtain the coordinate set of the left and right lane line points, and the parametric equation is fitted; Calculate the intersection point between the left and right lane lines of the lane where the vehicle is located and the bottom edge of the image; Determine whether the vehicle has deviated from the lane based on the interaction ratio between the intersection line segment of the bottom edge of the left and right lane lines and the preset line segment; When it is determined that the vehicle has reached the deviation threshold, an alarm business logic judgment is performed; If it is determined that the vehicle has deviated from the lane, an alarm will be issued.

2. The lane departure warning method according to claim 1, characterized in that: Real-time acquisition of video images of the road conditions ahead of the moving vehicle, including: Acquire the video stream of the road in front of the vehicle in real time, and take frames of the video to obtain image frames; A preprocessing operation is performed on the image frame to obtain a preprocessed image.

3. The lane departure warning method according to claim 2, characterized in that: The preprocessed image is fed into the trained lane detection model to obtain the coordinate set of the left and right lane points, and the parametric equation is fitted, including: The preprocessed image is fed into the trained UFLD lane line detection model, and the model outputs the coordinate set of the left and right lane line points of the lane where the vehicle is located; For the coordinates of the left and right lane line points (x1, y1), (x2, y2), ...., (x n ,y n ), and perform quadratic polynomial fitting according to the following formula to obtain the parameter equations of the left and right lane lines; x=a0+a1y+a2y 2 Among them, y is the independent variable, that is, the y-axis coordinate on the image, x is the dependent variable, that is, the x-axis coordinate on the image, a0, a1, a2 are the parameters to be fitted in the equation, which are based on the data points (x1, y1), (x2, y2), ...., (x n ,y n ) is obtained by least squares estimation.

4. The lane departure warning method according to claim 1, characterized in that: Calculate the intersection point of the left and right lane lines of the lane where the vehicle is located with the bottom edge of the image, including: using the left and right lane line parameter equations, calculate the value of x when y=h according to the following formula, and calculate the intersection point X of the left and right lane lines with the bottom edge of the image L , X R ; Among them, h is the image height and w is the image width.

5. The lane departure warning method according to claim 1, characterized in that: The method determines whether the vehicle has deviated from the lane based on the interaction ratio between the intersection line segment of the bottom edge of the left and right lane lines and the preset line segment, including: After the vehicle reaches the middle of the lane, the user connects the intersection point P of the bottom edge of the left and right lane lines in the figure. L P R The line segment is the preset line segment; Calculate the intersection point X of the bottom edge of the left and right lane lines L X R Line segment and preset line segment P L P R The IOU is calculated as follows: Where X L X R ∩P L P R represents the intersection area of ​​two line segments, X L X R ∪P L P R represents the area of ​​the union of two line segments; When the IOU is less than the threshold T, it is judged that lane departure has occurred.

6. The lane departure warning method according to claim 5, characterized in that: The threshold T is 0.3-0.

6.

7. The lane departure warning method according to claim 1, characterized in that: The specific steps of alarm business logic judgment include: S1. When the algorithm is initialized, the variable event_mark is set to 0 to record new alarm events; S2. After the lane departure decision module is executed, abnormal detection and judgment are performed; specifically, when the intersection point X L X R When the line length is less than 500 pixels, X R -X L <500, event_mark=0 and jump to the image acquisition module again; when the intersection point X L = 0 and the intersection point X of the bottom edge of the right lane line R =0, that is, X L = = 0 and X R ==w, event_mark=0 and jump to the image acquisition module again. If no abnormality is detected, proceed to the next step, which is used for filtering lane line detection and fitting abnormal results to reduce false alarms; S3. When the abnormality in step S2 is judged to be normal, determine whether the lane departure decision result is less than the threshold value T. If it is less than T, proceed to the next step; otherwise, event_mark is accumulated by 1 and jumps to the image acquisition module again; When the condition of step S3 is met, it is determined whether event_mark is greater than 8. If so, the alarm prompt model will sound an alarm and event_mark=0; otherwise, event_mark=0 and the process jumps back to the image acquisition module.

8. A lane departure warning device, characterized in that: include: An image acquisition module is used to acquire a video image of the road condition ahead of the moving vehicle in real time; The lane detection module is used to send the acquired image to the trained lane detection model, obtain the coordinate set of the left and right lane points, fit the parameter equation, and calculate the intersection point of the left and right lane lines of the lane where the vehicle is located with the bottom edge of the image; Parameter configuration module, used for configuring preset line segments; The lane departure decision module is used to determine whether the vehicle has deviated from the lane based on the interaction ratio between the intersection line segment of the bottom edge of the left and right lane lines and the preset line segment; The alarm logic judgment module is used to make alarm business logic judgment when it is judged that the vehicle reaches the deviation threshold; The alarm prompt module is used to issue an alarm if it is determined that the vehicle has deviated from the lane.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

Images