Lane departure warning method, device, equipment, storage medium

Abstract

The application belongs to the technical field of vehicle early warning, and particularly relates to a lane departure early warning method, device, equipment and storage medium. The method obtains a feature image photographed by a vehicle camera and extracts a first lane line; according to the first lane line, a re-projection transformation equation is used to obtain a second lane line; according to the second left lane line and the second right lane line, a first included angle is determined, which is the included angle of the second left lane line and the second right lane line in an image internal memory buffer coordinate system; when the dynamically obtained first included angle decreases and the decrease time length exceeds a preset departure time length, it is determined that the vehicle departs from the lane, and lane departure early warning is performed; the method only needs to obtain simple parameters to realize lane departure early warning, avoids the defect of complex calculation caused by too many sensor parameters, meanwhile, the method can accurately and timely perform lane departure early warning, and reduces the loss of life and property of the driver.

Description

Technical Field

[0001] This application relates to the field of vehicle warning technology, and in particular to a lane departure warning method, device, equipment, and storage medium. Background Technology

[0002] With the development of automotive intelligence, more and more advanced driver assistance functions are being applied to cars. Lane departure warning is one of the more popular advanced driver assistance systems. The lane departure warning system uses environmental perception sensors to detect the distance between the vehicle and the lanes on either side while driving. When the environmental perception sensors detect that the vehicle has left the designated lane, it will issue warning signals such as vibrating the steering wheel or emitting a beeping sound to remind the driver to pay attention.

[0003] Most existing lane departure warning methods use lidar sensors to monitor in real time whether the vehicle is deviating from its lane, or use the relative distance between the vehicle and the guardrail to determine whether the vehicle is deviating from its lane, and prompt the driver to take appropriate measures when the vehicle deviates from its lane.

[0004] However, among the above methods, while the method of adding a lidar sensor can be well applied in harsh environments, it suffers from the drawback of excessively high modification costs; the method based on lateral distance judgment is prone to recognition delays at high vehicle speeds. Summary of the Invention

[0005] This application provides a lane departure warning method, device, equipment, and storage medium to address the shortcomings of existing warning methods, such as high modification costs and delayed identification.

[0006] On the one hand, this application provides a lane departure warning method, including:

[0007] The feature image captured by the vehicle's onboard camera is acquired, and the first lane line is extracted. The first lane line is extracted based on the vehicle's body coordinate system with the vehicle as the origin. The first lane line includes: a first left lane line and a first right lane line.

[0008] Based on the first lane line, a second lane line is obtained by using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line;

[0009] Based on the second left lane line and the second right lane line, a first included angle is determined, wherein the first included angle is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system;

[0010] When the dynamically acquired first included angle decreases and the decrease time exceeds the preset deviation time, it is determined that the vehicle has deviated from the lane and a lane departure warning is issued.

[0011] Optionally, acquiring the feature image captured by the vehicle's onboard camera and extracting the first lane line includes:

[0012] The feature image captured by the vehicle's onboard camera is obtained, and the image is subjected to inverse perspective transformation to obtain a top-view feature map;

[0013] The first lane line in the vehicle body coordinate system is extracted from the top-view feature map.

[0014] Optionally, obtaining the second lane line based on the first lane line using a reprojection transformation equation includes:

[0015] Based on the position of the first lane line and the lane information, determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system, wherein the lane information includes the lane width and the lane center.

[0016] Using the reprojection transformation equation, coordinate transformation is performed on the first left lane line equation and the first right lane line equation to obtain the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system.

[0017] The second lane line is obtained in the image memory buffer coordinate system based on the second left lane line equation and the second right lane line equation.

[0018] Optionally, determining the equations of the first left lane line and the first right lane line in the vehicle body coordinate system based on the position and lane information of the first lane line includes:

[0019] Based on the position of the first lane line, obtain the second angle between the first lane line and the horizontal axis of the vehicle body coordinate system;

[0020] Based on the lane width, the distance from the vehicle's center of gravity to the lane center, and the second included angle, the equations of the first left lane line and the first right lane line in the vehicle's body coordinate system are determined.

[0021] Optionally, determining the first included angle based on the second left lane line and the second right lane line includes:

[0022] Based on the second left lane line, the second right lane line, and the width of the lane in the image memory buffer coordinate system, the first angle between the second left lane line and the second right lane line is determined using the cosine formula.

[0023] Optionally, acquiring the feature image captured by the vehicle's onboard camera includes:

[0024] After receiving lane departure warning information from the roadside sensing device, the system acquires feature images captured by the vehicle's onboard camera. The lane departure warning information is sent by the roadside device when it determines that the vehicle has deviated from its lane based on the lane line angle.

[0025] Optionally, before acquiring the feature image captured by the vehicle's onboard camera, the method includes:

[0026] Obtain the vehicle status information of the vehicle, which includes: vehicle turn signal information and steering wheel information;

[0027] Determine whether the vehicle's turn signals are on;

[0028] When the vehicle's turn signal is not activated, determine whether the steering wheel torque of the vehicle is less than a preset torque;

[0029] The acquisition of feature images captured by the vehicle's onboard camera includes:

[0030] When the steering wheel torque of the vehicle is less than a preset torque, a feature image captured by the vehicle's onboard camera is obtained.

[0031] Optionally, the vehicle status information further includes: driving duration; after the steering wheel torque of the vehicle is less than a preset torque, the method further includes:

[0032] Determine whether the driving time of the vehicle is greater than the preset driving time;

[0033] If the driving time exceeds the preset driving time, then the preset deviation time is reduced.

[0034] Secondly, this application provides a lane departure warning device, comprising:

[0035] The acquisition module is used to acquire feature images captured by the vehicle's onboard camera and extract the first lane line. The first lane line is extracted based on the vehicle's body coordinate system with the vehicle as the origin. The first lane line includes: a first left lane line and a first right lane line.

[0036] The acquisition module is further configured to acquire a second lane line based on the first lane line using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line;

[0037] The determining module is used to determine a first angle based on the second left lane line and the second right lane line, wherein the first angle is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system;

[0038] The determining module is further configured to determine that the vehicle is deviating from the lane and issue a lane departure warning when the dynamically acquired first included angle decreases and the decrease time exceeds a preset deviation time.

[0039] Optionally, the acquisition module is specifically used to acquire feature images captured by the vehicle's onboard camera, perform inverse perspective transformation on the images to obtain a top-view feature map, and extract the first lane line in the vehicle body coordinate system from the top-view feature map.

[0040] Optionally, the determining module is specifically used to determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system based on the position of the first lane line and the lane information, wherein the lane information includes the lane width and the lane center.

[0041] The device further includes: a processing module;

[0042] The processing module is used to perform coordinate transformation on the first left lane line equation and the first right lane line equation using the reprojection transformation equation to obtain the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system.

[0043] The acquisition module is further configured to acquire the second lane line in the image memory buffer coordinate system based on the second left lane line equation and the second right lane line equation.

[0044] Optionally, the acquisition module is further configured to acquire, based on the position of the first lane line, the second included angle between the first lane line and the horizontal axis of the vehicle body coordinate system;

[0045] The determining module is specifically used to determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system based on the lane width, the distance from the vehicle's center of gravity to the lane center, and the second included angle.

[0046] Optionally, the determining module is specifically used to determine the first angle between the second left lane line and the second right lane line using a cosine formula, based on the width of the lane in the coordinate system of the second left lane line, the second right lane line, and the image memory buffer.

[0047] Optionally, the acquisition module is specifically used to acquire feature images captured by the vehicle's onboard camera after receiving lane departure warning information sent by the roadside sensing device, wherein the lane departure warning information is sent by the roadside device when it determines that the vehicle has deviated from the lane based on the lane line angle.

[0048] Optionally, the acquisition module is further configured to acquire vehicle status information of the vehicle, the vehicle status information including: vehicle turn signal information and steering wheel information;

[0049] The device further includes: a judgment module;

[0050] The judgment module is used to determine whether the vehicle's turn signals are turned on.

[0051] The judgment module is also used to determine whether the steering wheel torque of the vehicle is less than a preset torque when the vehicle turn signal is not turned on.

[0052] The acquisition module is specifically used to acquire feature images captured by the vehicle's onboard camera when the steering wheel torque of the vehicle is less than a preset torque.

[0053] Optionally, the vehicle status information further includes: driving duration; the judgment module is also used to determine whether the driving duration of the vehicle is greater than a preset driving duration;

[0054] The processing module is also used to reduce the preset deviation time when the driving time exceeds the preset driving time.

[0055] Thirdly, this application provides a lane departure warning device, comprising:

[0056] Memory;

[0057] processor;

[0058] The memory stores computer-executed instructions;

[0059] The processor executes computer execution instructions stored in the memory to implement the lane departure warning method as described in the first aspect and various possible implementations of the first aspect above.

[0060] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the lane departure warning method as described in the first aspect and various possible implementations of the first aspect above.

[0061] The lane departure warning method provided in this application acquires feature images captured by a vehicle's onboard camera and extracts a first lane line. The first lane line is extracted based on a vehicle body coordinate system with the vehicle as the origin and includes a first left lane line and a first right lane line. Based on the first lane line, a reprojection transformation equation is used to obtain a second lane line, which is a lane line in an image memory buffer coordinate system and includes a second left lane line and a second right lane line. Based on the second left lane line and the second right lane line, a first included angle is determined, which is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system. When the dynamically acquired first included angle decreases, and the decrease time exceeds a preset deviation time, the vehicle is determined to have deviated from the lane, and a lane departure warning is issued. This method only requires simple parameters to achieve lane departure warning, thus avoiding the computational complexity caused by too many sensor parameters. Furthermore, this method can accurately and promptly issue lane departure warnings, reducing the risk of loss of life and property for drivers. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0063] Figure 1 The process of the lane departure warning method provided in this application Figure 1 ;

[0064] Figure 2 The process of the lane departure warning method provided in this application Figure 2 ;

[0065] Figure 3 A schematic diagram of the top view feature provided in this application;

[0066] Figure 4 The process of the lane departure warning method provided in this application Figure 3 ;

[0067] Figure 5 A schematic diagram of the lane departure warning device provided in this application;

[0068] Figure 6 This is a structural schematic diagram of the lane departure warning device provided in this application.

[0069] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0072] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] With the development of automotive intelligence, more and more advanced driver assistance functions are being applied to cars. Lane departure warning is one of the more popular advanced driver assistance systems. The lane departure warning system uses environmental perception sensors to detect the distance between the vehicle and the lanes on either side while driving. When the environmental perception sensors detect that the vehicle has left the designated lane, it will issue warning signals such as vibrating the steering wheel or emitting a beeping sound to remind the driver to pay attention.

[0074] Existing lane departure warning methods typically use lidar sensors to monitor whether the vehicle is deviating from its lane in real time, or use the relative distance between the vehicle and the guardrail to determine whether the vehicle is deviating from its lane. When the vehicle deviates from its lane, the system prompts the driver to take appropriate measures, or uses a rear camera to recognize the driver's actions to issue a lane departure warning.

[0075] However, among the above methods, while the method of adding a lidar sensor can be well applied in harsh environments, it suffers from the drawback of excessively high modification costs; the methods based on lateral distance judgment and the methods of recognizing driver operation behavior are prone to recognition delays at high vehicle speeds.

[0076] To address the aforementioned issues, this application provides a lane departure warning method. This method converts a first lane line in a feature image captured by an onboard camera into a second lane line in the image memory buffer coordinate system. The included angle of the second lane line is then used as a condition for lane departure determination. When the dynamically acquired lane line angle decreases, and the decrease lasts for a period exceeding a preset departure time, the vehicle is determined to be deviating from its lane, and a lane departure warning is issued. This method not only avoids drawbacks such as high modification costs and recognition delays, but also, by using the lane line angle as the condition for lane departure determination, simple parameters are required to achieve lane departure warning, thus solving the problem of computational complexity caused by too many sensor parameters. Furthermore, this method can accurately and promptly issue lane departure warnings, reducing the risk of loss of life and property for drivers.

[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0078] Figure 1 This is the flow chart of the lane departure warning method provided in the embodiments of this application. Figure 1 The executing entity in this embodiment can be, for example, a vehicle that is currently in motion, such as... Figure 1 As shown in the figure, the lane departure warning method illustrated in this embodiment includes:

[0079] S101: Acquire the feature image captured by the vehicle's onboard camera and extract the first lane line, wherein the first lane line is extracted based on the vehicle's body coordinate system with the vehicle as the origin, and the first lane line includes: a first left lane line and a first right lane line.

[0080] Among them, the vehicle-mounted camera is a front-view vehicle-mounted camera. The feature image captured by the vehicle-mounted camera is an image in the direction of vehicle travel, and the feature image includes lane lines located on both sides of the vehicle.

[0081] The first lane line is the lane line in the vehicle's coordinate system with the vehicle as the origin, including the first left lane line and the first right lane line. After acquiring the feature image captured by the onboard camera, the first lane line needs to be extracted from the feature image.

[0082] In this step, for example, the feature image can be mapped to a vehicle body coordinate system with the vehicle as the origin, and then the first lane line can be extracted from the feature image; or a vehicle body coordinate system with the vehicle as the origin can be constructed first, and then an inverse perspective transformation can be performed on the feature image to obtain a feature image in the vehicle body coordinate system, and then the first lane line can be extracted from the feature image in the vehicle body coordinate system. This application does not impose any special restrictions on the specific implementation method of extracting the first lane line, as long as the first lane line in the vehicle body coordinate system can be extracted.

[0083] In this step, feature images captured by the vehicle's onboard camera can be acquired in real time, or feature images captured by the vehicle's onboard camera can be acquired according to a preset period. This application does not impose any special restrictions.

[0084] S102: Based on the first lane line, a second lane line is obtained by using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line.

[0085] Reprojection transformation refers to the process of transforming the coordinates of a point in one map projection to the coordinates of a point in another map projection. The reprojection transformation is explained in detail below:

[0086] Assuming any point pi in the spatial plane has homogeneous coordinates pi = (xi, yi, 1): (xi, yi) on the left camera image plane, then the coordinates pi in 3D space and the homogeneous coordinates p'i = (x'i, y'i, 1): (x'i, y'i) of pi on the right camera image plane can be transformed using kip'i = A·pi. Here, ki is the scaling operator, and A = h·R + t·n T h is the distance between the plane and the left camera, n is the normal direction of the ground plane relative to the left camera, t is the distance required for the left camera to translate to the right camera, and R represents the transformation required to rotate from the left camera coordinates to the right camera coordinates.

[0087] Under the influence of ki, Substituting kip′i=A·pi, we can obtain After transformation, we get

[0088]

[0089] If n points belong to the same plane, then their projected coordinates (xi, yi) and (x′i, y′i) (i = 1, 2, ..., n) on the image planes of the two cameras satisfy the following formula:

[0090]

[0091] Therefore, the transformation relationship between the image plane coordinates of the left and right cameras can be obtained as follows:

[0092]

[0093] In this step, the coordinates of the first lane line in the vehicle body coordinate system need to be transformed to obtain the second lane line in the image memory buffer coordinate system. The specific implementation process is as follows: the coordinates of the first lane line in the vehicle body coordinate system are considered as the image plane coordinates of the left camera, and the coordinates of the second lane line in the image memory buffer coordinate system are considered as the image plane coordinates of the right camera. Using the transformation relationship described in the reprojection transformation above, the coordinates of the first lane line are transformed to obtain the second lane line in the image memory buffer coordinate system. It can be understood that the coordinates of the first lane line are in the vehicle body coordinate system, and the coordinates of the second lane line are in the image memory buffer coordinate system.

[0094] Understandably, the origin of the image memory buffer coordinate system could be, for example, the vehicle's camera or the vehicle itself. In this image memory buffer coordinate system, the second left lane line and the second right lane line are not parallel; they form an angle with each other in the vehicle's direction of travel.

[0095] The purpose of this embodiment is to use the lane line angle as a condition for lane departure determination. In the vehicle's coordinate system, the left and right lane lines are parallel to each other and cannot form an angle. Therefore, it is necessary to convert the first lane line in the vehicle's coordinate system into the second lane line in the image memory buffer coordinate system to facilitate the subsequent acquisition of the angle of the second lane line.

[0096] S103: Determine a first included angle based on the second left lane line and the second right lane line, wherein the first included angle is the angle between the second left lane line and the second right lane line in the coordinate system of the image memory buffer.

[0097] Once the second lane line is obtained, the first included angle can be determined based on the second left lane line and the second right lane line.

[0098] Understandably, as a vehicle moves laterally, the coordinates and angles of the lane line image change accordingly, which in turn causes the first included angle to change as well.

[0099] S104: When the dynamically acquired first included angle decreases and the decrease time exceeds the preset deviation time, it is determined that the vehicle has deviated from the lane and a lane departure warning is issued.

[0100] The deviation duration refers to the duration during which the first included angle continues to decrease. It is understandable that as a vehicle continuously deviates to one side, its distance from the lane line on that side continuously decreases, leading to an increase in the image coordinates and angle of that lane line. Consequently, the first included angle obtained through the above steps will continue to decrease.

[0101] Therefore, when the dynamically acquired first included angle decreases and the duration of the decrease exceeds the preset deviation duration, it can be determined that the vehicle is currently deviating from the lane, and a lane departure warning can be issued at this time.

[0102] In this step, lane departure can be alerted by means such as vibrating the steering wheel or emitting a beeping sound. This application does not restrict the warning method.

[0103] The lane departure warning method provided in this embodiment acquires feature images captured by the vehicle's onboard camera and extracts a first lane line, which is extracted based on the vehicle's body coordinate system with the vehicle as the origin. Based on the first lane line, a reprojection transformation equation is used to obtain a second lane line, which is a lane line in the image memory buffer coordinate system. A first angle is determined based on the second left lane line and the second right lane line, which is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system. When the dynamically acquired first angle decreases, and the decrease time exceeds a preset deviation time, the vehicle is determined to have deviated from the lane, and a lane departure warning is issued. This method uses the lane line angle as the condition for lane departure determination, allowing for lane departure warning with simple parameters, thus avoiding the computational complexity caused by too many sensor parameters. Furthermore, this method can accurately and promptly issue lane departure warnings, reducing the risk of loss of life and property for drivers.

[0104] Figure 2 The flow chart of the lane departure warning method provided in the embodiments of this application Figure 2 .like Figure 2 As shown, this embodiment is... Figure 1 Based on the embodiments, the lane departure warning method is described in detail. The lane departure warning method shown in this embodiment includes:

[0105] S201: Acquire the feature image captured by the vehicle's onboard camera, and perform inverse perspective transformation on the image to obtain a top-view feature map.

[0106] In the feature images captured by the forward-looking vehicle camera, the perspective effect causes objects that are actually parallel to intersect in the image, resulting in a discrepancy between their coordinates in the image and their actual coordinates. Inverse Perspective Mapping (IPM) is designed to eliminate this perspective effect.

[0107] By performing inverse perspective transformation on the feature images captured by the vehicle-mounted camera, a top-view feature map in the world coordinate system can be obtained, that is, a top-view feature map in the vehicle body coordinate system with the vehicle as the origin. The specific implementation of the inverse perspective transformation can be found in existing technologies, and will not be elaborated upon here.

[0108] S202: Extract the first lane line in the vehicle body coordinate system from the top-view feature map.

[0109] Specifically, after obtaining the top-view feature map in the vehicle's body coordinate system, the first lane line is extracted from this overall top-view feature map. The first lane line is the lane line in the vehicle's body coordinate system, and it can reflect the true coordinates of the current lane line.

[0110] S203: Based on the position of the first lane line and the lane information, determine the equations of the first left lane line and the first right lane line in the vehicle body coordinate system, wherein the lane information includes the lane width and the lane center.

[0111] After the first lane line is extracted, the equations of the first left lane line and the first right lane line can be determined based on the position of the first lane line in the vehicle body coordinate system and the lane information.

[0112] Optionally, the second angle between the first lane line and the horizontal axis of the vehicle coordinate system can be obtained based on the position of the first lane line; the equations of the first left lane line and the first right lane line in the vehicle coordinate system can be determined based on the lane width, the distance from the vehicle's center of gravity to the lane center, and the second angle.

[0113] Figure 3 This is a schematic diagram of a top-view feature diagram provided in an embodiment of this application. For example... Figure 3 As shown, the first left lane line and the first right lane line are parallel to each other. The horizontal axis of the vehicle's coordinate system represents the vehicle's current direction of travel, and the vertical axis is the direction perpendicular to the horizontal axis. The lane width is D, and the distance from the vehicle's center of gravity to the center of the lane is d.

[0114] The second included angle δ can be determined based on the vehicle's current driving direction and the direction of the first lane line; then, the equations of the first left lane line and the first right lane line can be determined based on the lane width D, the distance d from the vehicle's center of gravity to the center of the lane, and the second included angle δ.

[0115] For example, the equation for the first left lane line could be:

[0116]

[0117] The equation for the first right lane line could be, for example:

[0118]

[0119] S204: Using the reprojection transformation equation, perform coordinate transformation on the first left lane line equation and the first right lane line equation to obtain the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system.

[0120] After obtaining the equations for the first left lane and the first right lane, the coordinate transformation equations can be used to transform the above equations, thereby obtaining the equations for the second left lane and the second right lane in the image memory buffer coordinate system.

[0121] For example: Let P(x,y,z) represent the vehicle's coordinate system, where the x-axis points in the vehicle's forward direction, the y-axis points to the right of the vehicle, and the z-axis is perpendicular to the vehicle and points downwards. Let P(c,r) represent the image memory buffer coordinate system, where the c-axis is parallel to the scan line direction and points in the direction of increasing scan pixels, and the r-axis is perpendicular to the camera scan line direction and points in the direction of increasing scan lines.

[0122] This is understandable, because the position and angle of lane lines in the captured feature images will vary depending on the height of the forward-looking vehicle camera, the distance of the camera from the center of the vehicle, and the pitch angle. Therefore, the height h, distance l from the center of the vehicle, and pitch angle θ0 of the forward-looking vehicle camera need to be fully considered during the calculation.

[0123] By replacing the coordinates in the image memory buffer coordinate system with coordinates from the vehicle body coordinate system, we can obtain the coordinates in the image memory buffer coordinate system represented by the coordinates from the vehicle body coordinate system. The formula for the transformation from the vehicle body coordinate system to the image memory buffer coordinate system, obtained from the reprojection transformation, is as follows:

[0124]

[0125] in, This is the value obtained by dividing the focal length by the length of one pixel along the scan line direction. N is the value obtained by dividing the focal length by the width of one pixel perpendicular to the scan line direction. c and N r These correspond to the number of pixel columns and rows in the image memory buffer per unit length in the image memory buffer coordinate system, respectively.

[0126] To obtain the equation of the second left lane line in the image memory buffer coordinate system, for example, the following method can be used:

[0127] Substituting the equation of the first left lane line into the formula for the transformation from the vehicle body coordinate system to the image memory buffer coordinate system, that is, substituting Formula 3 above into Formula 5, we get:

[0128]

[0129]

[0130] Simplifying Formula 6, we get:

[0131]

[0132] Substituting Equation 8 into Equation 7, we obtain the equation of the second left lane line in the image memory buffer coordinate system:

[0133]

[0134] Understandably, the coordinates of the second left lane line in the image memory buffer coordinate system can be obtained from the equation of the second left lane line in the image memory buffer coordinate system.

[0135] The specific process for obtaining the second right lane line equation in the image memory buffer coordinate system is similar to that for obtaining the second left lane line equation, except that the first right lane line equation is substituted into the transformation formula from the vehicle body coordinate system to the image memory buffer coordinate system, that is, substituted into Formula 5. Therefore, this embodiment will not be repeated here; please refer to the process for obtaining the second right lane line equation in the image memory buffer coordinate system described above.

[0136] The equation for the second left lane line in the image memory buffer coordinate system can be, for example, as follows:

[0137]

[0138] Understandably, the coordinates of the second right lane line in the image memory buffer coordinate system can be obtained from the equation of the second right lane line in the image memory buffer coordinate system.

[0139] S205: Obtain the second lane line in the image memory buffer coordinate system according to the second left lane line equation and the second right lane line equation.

[0140] After obtaining the equations for the second left lane line and the second right lane line, the coordinates of the second left lane line and the second right lane line can be obtained based on the above equations, thereby obtaining the second lane line.

[0141] S206: Based on the second left lane line, the second right lane line, and the width of the lane in the coordinate system of the image memory buffer, the first angle between the second left lane line and the second right lane line is determined using the cosine formula.

[0142] Since the second left lane line and the second right lane line are not parallel in the image memory buffer coordinate system, there is an angle between them.

[0143] The angle between the second left lane line and the second right lane line can be determined using the cosine formula based on the equations of the second left lane line, the second right lane line, and the width of the lane.

[0144] For example, the first included angle α can be expressed by the following formula:

[0145]

[0146] Where m1, m2, n1, and n2 are the parameters in the cosine formula.

[0147]

[0148] S207: When the dynamically acquired first included angle decreases and the decrease time exceeds the preset deviation time, it is determined that the vehicle has deviated from the lane and a lane departure warning is issued.

[0149] Step S207 is similar to step S104 above, and will not be repeated here.

[0150] The lane departure warning method provided in this embodiment extracts the first lane line from the top-view feature map in the vehicle body coordinate system, performs coordinate transformation on the coordinates of the first lane line using a reprojection transformation equation, and obtains the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system. Then, it determines the lane line angle of the second lane line in the image memory buffer coordinate system. When the lane line angle decreases and the decrease time exceeds a preset deviation time, it determines that the vehicle has deviated from the lane and issues a lane departure warning. This method not only avoids the defects of high modification costs and recognition delays, but also solves the problem of computational complexity caused by too many sensor parameters because lane departure warnings can be issued with simple parameters. At the same time, this method can promptly determine the current vehicle's lane departure and issue a lane departure warning, thereby improving the timeliness of the warning and reducing the loss of life and property of drivers.

[0151] Figure 4 The flow chart of the lane departure warning method provided in the embodiments of this application Figure 3 .like Figure 4 As shown, this embodiment is... Figure 1 and Figure 2 Based on the embodiments, the method for acquiring feature images captured by the vehicle's onboard camera is described in detail. The lane departure warning method shown in this embodiment includes:

[0152] S301: Receives lane departure warning information sent by roadside sensing devices.

[0153] The roadside sensing device will also determine whether a vehicle is deviating from its lane. When the roadside sensing device determines that a vehicle is deviating from its lane, it will send a lane departure warning message to the vehicle. The lane departure warning message is used to indicate that the vehicle is currently deviating from its lane.

[0154] It is understandable that the roadside sensing device can determine whether a lane has deviated from its lane using the method shown in the above embodiments, or it can use existing lane departure warning methods. This application does not impose any special restrictions on this.

[0155] Understandably, a vehicle will only determine lane departure after receiving lane departure warning information from the roadside sensing device. That is, in this embodiment, lane departure determination will be performed twice. The first determination is performed by the roadside sensing device. When the roadside sensing device determines that the vehicle has deviated from the lane, it will send lane departure warning information to the vehicle. After receiving the lane departure warning information, the vehicle will perform a second lane departure determination.

[0156] This step saves computational costs and time on the vehicle side by executing the lane departure warning method in the above embodiment only after receiving lane departure warning information sent by the roadside sensing device; at the same time, the warning result is more accurate because two judgments are made.

[0157] S302: Obtain the vehicle status information of the vehicle.

[0158] The vehicle status information includes at least one of the following: vehicle turn signal information, steering wheel information, and driving duration. Optionally, the vehicle status information may also include information such as lane width, camera position relative to the vehicle, and vehicle direction of travel.

[0159] The purpose of obtaining vehicle status information is to determine whether the current lane departure is a conscious action by the driver.

[0160] S303: Determine whether the vehicle's turn signals are turned on; if yes, proceed to step S304; if no, proceed to step S305.

[0161] In this scenario, when a vehicle turns left or right, the driver activates the turn signal to alert other vehicles to avoid it. Therefore, the purpose of determining whether the turn signal is activated in this step is to ascertain whether the current lane departure is a conscious action by the driver. If the turn signal is activated, it indicates that the vehicle is consciously deviating from its lane, meaning the driver is aware that the vehicle has left the lane, and in this case, the lane departure warning method described in the above embodiment is unnecessary.

[0162] If the vehicle's turn signals are not activated, it is necessary to further determine whether the steering wheel torque is less than the preset torque. This is because some drivers may forget to activate their turn signals when changing lanes, but they are still aware that the vehicle has deviated from its lane. Therefore, the absence of turn signals cannot be used to directly determine whether the vehicle is being operated unconsciously; further judgment based on steering wheel torque is required.

[0163] S304: Determined that the vehicle is being operated consciously.

[0164] S305: Determine whether the steering wheel torque of the vehicle is less than the preset torque; if not, proceed to step S304; if yes, proceed to step S306.

[0165] The preset torque can be, for example, the minimum torque obtained from historical data of vehicle lane changes. When the steering wheel torque of the vehicle is greater than the preset torque, it indicates that the vehicle is currently changing lanes and is in a conscious operation.

[0166] When the steering wheel torque is less than the preset torque, it indicates that the vehicle is not currently changing lanes, and the vehicle's deviation from the lane is an unconscious operation by the driver. The vehicle is currently in an unconscious operation state.

[0167] S306: It is determined that the vehicle is operating unconsciously.

[0168] Optionally, after determining that the vehicle is operating unconsciously, the method further includes: determining whether the driving time of the vehicle is greater than a preset driving time; if the driving time is greater than the preset driving time, then reducing the preset deviation time.

[0169] After determining that the vehicle is being operated unconsciously, it can also be determined whether the driving time exceeds the preset driving time. For example, when it is determined that the vehicle is being operated unconsciously, it can be determined whether the current driving time exceeds the preset three hours. If so, it indicates that the driver is currently driving while fatigued, and the preset deviation time needs to be further reduced in order to adjust the judgment time of the warning in a timely manner, making the lane departure warning more sensitive and timely, thereby reducing the loss of life and property of the driver.

[0170] S307: Acquire the feature image captured by the vehicle's onboard camera.

[0171] Step S307 is similar to step S101 above, and will not be described again here.

[0172] The vehicle departure warning method provided in this embodiment determines the driver's conscious operation based on vehicle status information, and only acquires feature images when it is determined that the vehicle is operating unconsciously, thereby reducing the number of false alarms. At the same time, lane departure detection of the vehicle segment is only performed after receiving lane departure warning information sent by the roadside terminal, thereby saving the computational cost and time of the vehicle terminal.

[0173] Figure 5 This is a structural schematic diagram of the lane departure warning device provided in this application. Figure 5 As shown, the lane departure warning device 300 provided in this embodiment includes:

[0174] The acquisition module 301 is used to acquire feature images captured by the vehicle's onboard camera and extract the first lane line, wherein the first lane line is extracted based on the vehicle body coordinate system with the vehicle as the origin, and the first lane line includes: a first left lane line and a first right lane line.

[0175] The acquisition module 301 is further configured to acquire a second lane line based on the first lane line using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line.

[0176] The determining module 302 is used to determine a first angle based on the second left lane line and the second right lane line, wherein the first angle is the angle between the second left lane line and the second right lane line in the coordinate system of the image memory buffer.

[0177] The determining module 302 is further configured to determine that the vehicle is deviating from the lane and issue a lane departure warning when the dynamically acquired first included angle decreases and the decrease time exceeds a preset deviation time.

[0178] Optionally, the acquisition module 301 is specifically used to acquire feature images captured by the vehicle's onboard camera, perform inverse perspective transformation on the images to obtain a top-view feature map, and extract the first lane line in the vehicle body coordinate system from the top-view feature map.

[0179] Optionally, the determining module 302 is specifically used to determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system based on the position of the first lane line and the lane information, wherein the lane information includes the lane width and the lane center.

[0180] The device further includes: a processing module 303;

[0181] The processing module 303 is used to perform coordinate transformation on the first left lane line equation and the first right lane line equation using the reprojection transformation equation to obtain the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system.

[0182] The acquisition module 301 is further configured to acquire the second lane line in the image memory buffer coordinate system based on the second left lane line equation and the second right lane line equation.

[0183] Optionally, the acquisition module 301 is further configured to acquire, based on the position of the first lane line, the second included angle between the first lane line and the horizontal axis of the coordinate system in the vehicle body coordinate system;

[0184] The determining module 302 is specifically used to determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system based on the lane width, the distance from the vehicle's center of gravity to the lane center, and the second included angle.

[0185] Optionally, the determining module 302 is specifically used to determine the first angle between the second left lane line and the second right lane line using a cosine formula based on the width of the lane in the coordinate system of the image memory buffer, the second left lane line, the second right lane line, and the width of the lane.

[0186] Optionally, the acquisition module 301 is specifically used to acquire feature images captured by the vehicle's onboard camera after receiving lane departure warning information sent by the roadside sensing device, wherein the lane departure warning information is sent by the roadside device when it determines that the vehicle has deviated from the lane based on the lane line angle.

[0187] Optionally, the acquisition module 301 is further configured to acquire vehicle status information of the vehicle, the vehicle status information including: vehicle turn signal information and steering wheel information;

[0188] The device further includes: a judgment module 304;

[0189] The judgment module 304 is used to determine whether the vehicle's turn signals are turned on.

[0190] The judgment module 304 is also used to determine whether the steering wheel torque of the vehicle is less than a preset torque when the vehicle body turn signal is not turned on.

[0191] The acquisition module 301 is specifically used to acquire feature images captured by the vehicle's onboard camera when the steering wheel torque of the vehicle is less than a preset torque.

[0192] Optionally, the vehicle status information further includes: driving duration; the judgment module 304 is also used to determine whether the driving duration of the vehicle is greater than a preset driving duration;

[0193] The processing module 303 is also used to reduce the preset deviation time when the driving time exceeds the preset driving time.

[0194] Figure 6 This is a structural schematic diagram of the lane departure warning device provided in this application. Figure 6 As shown, this application provides a lane departure warning device 400, which includes a receiver 401, a transmitter 402, a processor 403, and a memory 404.

[0195] Receiver 401 is used to receive instructions and data;

[0196] Transmitter 402 is used to send commands and data;

[0197] Memory 404 is used to store instructions executed by the computer;

[0198] The processor 403 is used to execute computer execution instructions stored in the memory 404 to implement the various steps of the lane departure warning method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing embodiments of the lane departure warning method.

[0199] Alternatively, the memory 404 can be either standalone or integrated with the processor 403.

[0200] When the memory 404 is set up independently, the electronic device also includes a bus for connecting the memory 404 and the processor 403.

[0201] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the lane departure warning method performed by the lane departure warning device described above.

[0202] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0203] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0204] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A lane departure warning method, characterized in that, The method includes: The feature image captured by the vehicle's onboard camera is acquired, and the first lane line is extracted. The first lane line is extracted based on the vehicle's body coordinate system with the vehicle as the origin. The first lane line includes: a first left lane line and a first right lane line. Based on the first lane line, a second lane line is obtained by using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line; Based on the second left lane line and the second right lane line, a first included angle is determined, wherein the first included angle is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system; When the dynamically acquired first included angle decreases and the decrease time exceeds the preset deviation time, it is determined that the vehicle has deviated from the lane and a lane departure warning is issued.

2. The method according to claim 1, characterized in that, The step of acquiring the feature image captured by the vehicle's onboard camera and extracting the first lane line includes: The feature image captured by the vehicle's onboard camera is obtained, and the image is subjected to inverse perspective transformation to obtain a top-view feature map; The first lane line in the vehicle body coordinate system is extracted from the top-view feature map.

3. The method according to claim 1, characterized in that, The step of obtaining the second lane line based on the first lane line using a reprojection transformation equation includes: Based on the position of the first lane line and the lane information, determine the first left lane line equation and the first right lane line equation in the vehicle body coordinate system, wherein the lane information includes the lane width and the lane center. Using the reprojection transformation equation, coordinate transformation is performed on the first left lane line equation and the first right lane line equation to obtain the second left lane line equation and the second right lane line equation in the image memory buffer coordinate system. The second lane line is obtained in the image memory buffer coordinate system based on the second left lane line equation and the second right lane line equation.

4. The method according to claim 3, characterized in that, The step of determining the equations of the first left lane line and the first right lane line in the vehicle body coordinate system based on the position and lane information of the first lane line includes: Based on the position of the first lane line, obtain the second angle between the first lane line and the horizontal axis of the vehicle body coordinate system; Based on the lane width, the distance from the vehicle's center of gravity to the lane center, and the second included angle, the equations of the first left lane line and the first right lane line in the vehicle's body coordinate system are determined.

5. The method according to claim 4, characterized in that, Determining the first included angle based on the second left lane line and the second right lane line includes: Based on the second left lane line, the second right lane line, and the width of the lane in the image memory buffer coordinate system, the first angle between the second left lane line and the second right lane line is determined using the cosine formula.

6. The method according to claim 1, characterized in that, The acquisition of feature images captured by the vehicle's onboard camera includes: After receiving lane departure warning information from the roadside sensing device, the system acquires feature images captured by the vehicle's onboard camera. The lane departure warning information is sent by the roadside device when it determines that the vehicle has deviated from its lane based on the lane line angle.

7. The method according to claim 6, characterized in that, Before acquiring the feature image captured by the vehicle's onboard camera, the method includes: Obtain the vehicle status information of the vehicle, which includes: vehicle turn signal information and steering wheel information; Determine whether the vehicle's turn signals are on; When the vehicle's turn signal is not activated, determine whether the steering wheel torque of the vehicle is less than a preset torque; The acquisition of feature images captured by the vehicle's onboard camera includes: When the steering wheel torque of the vehicle is less than a preset torque, a feature image captured by the vehicle's onboard camera is obtained.

8. The method according to claim 7, characterized in that, The vehicle status information also includes: driving duration; after the vehicle's steering wheel torque is less than a preset torque, the method further includes: Determine whether the driving time of the vehicle is greater than the preset driving time; If the driving time exceeds the preset driving time, then the preset deviation time is reduced.

9. A lane departure warning device, characterized in that, The device includes: The acquisition module is used to acquire feature images captured by the vehicle's onboard camera and extract the first lane line. The first lane line is extracted based on the vehicle's body coordinate system with the vehicle as the origin. The first lane line includes: a first left lane line and a first right lane line. The acquisition module is further configured to acquire a second lane line based on the first lane line using a reprojection transformation equation, wherein the second lane line is a lane line in the image memory buffer coordinate system, and the second lane line includes: a second left lane line and a second right lane line; The determining module is used to determine a first angle based on the second left lane line and the second right lane line, wherein the first angle is the angle between the second left lane line and the second right lane line in the image memory buffer coordinate system; The determining module is further configured to determine that the vehicle is deviating from the lane and issue a lane departure warning when the dynamically acquired first included angle decreases and the decrease time exceeds a preset deviation time.

10. A lane departure warning device, characterized in that, include: Memory; processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the lane departure warning method as described in any one of claims 1-8.

11. A computer storage medium, characterized in that, The computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the lane departure warning method as described in any one of claims 1-8.

Images