Lane departure identification method and device

Abstract

The invention provides a lane departure identification method and device, and the method comprises the steps: carrying out the lane line identification of a driving video collected by a corner camera, and obtaining an actual measurement lane line; obtaining the pixel distance from the intersection point of the actually measured lane line and the lane vertical line to the reference lane line, the pixel width of the actually measured lane line, the actual width of the actually measured lane line and the pixel width of the reference lane line; the current driving scene of the vehicle is determined based on the pixel distance, and a lane deviation value corresponding to the angle camera is obtained through calculation according to a deviation value calculation formula preset for the driving scene, the pixel distance, the pixel width of the actually measured lane line, the actual width of the actually measured lane line and the pixel width of the reference lane line; and calculating a lane departure angle based on the lane departure value and a preset departure angle formula. According to the scheme, the purpose of high-precision lane departure calculation and recognition is achieved in a complex environment by using an angle camera vehicle body self-reference technology.

Description

Technical Field

[0001] The present invention relates to the field of automotive electronic technology, and in particular to a lane departure recognition method and device. Background Art

[0002] Lane departure recognition technology is widely used in modern automobile intelligent driving assistance systems. Lane departure recognition refers to the technology that uses cameras and other equipment to monitor and identify whether the vehicle deviates from the lane during driving. This technology can help the driver detect whether the vehicle has deviated from the lane in time, and remind the driver to take timely measures through warning or intervention systems to avoid potential accidents.

[0003] In the existing lane departure recognition process, the lane recognition function is usually implemented by a forward-facing camera, such as a monocular camera or a binocular camera. The forward-facing camera is easily restricted by the environment, which affects the recognition accuracy and reliability, thus causing errors in lane departure recognition. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a lane departure recognition method and device to achieve the purpose of high-precision lane departure calculation and recognition.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of an embodiment of the present invention discloses a lane departure recognition method, which is applied to an ECU in a vehicle, wherein the vehicle is provided with a corner camera, and the method comprises:

[0007] Performing lane line recognition on the driving video collected by the corner camera to obtain the measured lane line;

[0008] Obtaining a pixel distance from an intersection of the measured lane line and the lane vertical line to a reference lane line, a pixel width of the measured lane line, an actual width of the measured lane line, and a pixel width of the reference lane line; the reference lane line is pre-calibrated based on a side profile of the vehicle; the lane vertical line is pre-calibrated based on a front profile or a rear profile of the vehicle;

[0009] The current driving scene of the vehicle is determined based on the pixel distance, and the lane deviation value corresponding to the corner camera is calculated using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line; the actual width of the measured lane line is obtained from map data;

[0010] A lane departure angle is calculated based on the lane departure value and a preset departure angle formula.

[0011] Preferably, the performing lane line recognition on the driving video collected by the corner camera to obtain the measured lane line includes:

[0012] If the driving video collected by the corner camera does not have a shadow coverage area, the measured lane line is identified from the driving video using video enhancement recognition technology; the shadow coverage area is the projection formed by sunlight irradiating the vehicle on the road;

[0013] If the driving video captured by the corner camera has the shadow coverage area, a shadow boundary line is obtained by identifying the driving video captured by the corner camera, and the image of the driving video is split based on the shadow boundary line to obtain a shadow area image and a non-shadow area image;

[0014] Using video enhancement recognition technology to perform lane recognition on the shadow area image and the non-shadow area image respectively to obtain a first lane line and a second lane line;

[0015] The first lane line and the second lane line are merged to obtain a measured lane line.

[0016] Preferably, the determining the current driving scene of the vehicle based on the pixel distance, and calculating the lane deviation value corresponding to the corner camera using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line, include:

[0017] When the pixel distance L3 meets the preset condition: When the vehicle is determined to be in a lane keeping scenario, a deviation value calculation formula pre-set for the lane keeping scenario is used: Calculating and obtaining a lane deviation value d3 corresponding to the corner camera;

[0018] Wherein, n represents an integer greater than or equal to 2, L1 represents the pixel width of the reference lane line, L2 represents the pixel width of the measured lane line, and d0 represents the actual width of the measured lane line;

[0019] When the pixel distance L3 satisfies the preset condition: L3≤L1+L2, it is determined that the vehicle is in a lane-pressing scene, and the deviation value calculation formula pre-set for the lane-pressing scene is used: Calculating a lane deviation value d3 corresponding to the corner camera;

[0020] When the pixel distance L3 meets the preset condition: L3>L1+L2 and When the vehicle is determined to be in a lane crossing scenario, the deviation value calculation formula pre-set for the lane crossing scenario is used: A lane deviation value d3 corresponding to the corner camera is calculated.

[0021] Preferably, the calculating the lane departure angle based on the lane departure value and a preset departure angle formula includes:

[0022] Based on the preset deviation angle formula: The lane departure angle θ is calculated;

[0023] Wherein, d3 is the lane deviation value, d'3 is the past lane deviation value, and the past lane deviation value is the lane deviation value calculated before a preset time; and S is the distance traveled by the vehicle within the preset time.

[0024] Preferably, the right front corner, left front corner, right rear corner and left rear corner of the vehicle are respectively provided with a first corner camera, a second corner camera, a third corner camera and a fourth corner camera;

[0025] The calculating the lane departure angle based on the lane departure value and a preset departure angle formula includes:

[0026] Based on the preset deviation angle formula: θ=arcsin(δd r / L V ), the lane departure angle θ is calculated;

[0027] Among them, δd r is the difference between the lane departure value corresponding to the first angle camera and the lane departure value corresponding to the third angle camera, or is the difference between the lane departure value corresponding to the second angle camera and the lane departure value corresponding to the fourth angle camera; L V is the length of the vehicle.

[0028] Preferably, the method further comprises:

[0029] When the lane deviation value is equal to 0, the intelligent driving system of the vehicle is triggered to issue a lane crossing alarm;

[0030] When the lane departure value is equal to -d t , triggering the intelligent driving system of the vehicle to issue a line crossing alarm; d t is the preset critical threshold.

[0031] Preferably, the method further comprises:

[0032] Calculate the distance from the pre-calibrated positioning point used to characterize the vehicle position to the measured lane line: X=d3+W*cosθ-L*sinθ; wherein d3 is the lane deviation value, θ is the lane deviation angle, W is the lateral distance between the positioning point and the corner point, L is the longitudinal distance between the positioning point and the corner point, and the corner point is the position where the corner camera is set corresponding to the lane deviation value;

[0033] Obtaining historical moments when the vehicle has passed through linear reference points in the past; the linear reference points are reference points in map data that are used to locate the longitudinal position of the vehicle on the road and have fixed intervals;

[0034] Calculate the travel distance of the vehicle along the measured lane line after the historical moment: Among them, T i is the historical moment, T1 is the current moment, θ is the lane departure angle, and S is the mileage measured by the wheel speedometer on the vehicle;

[0035] The distance from the positioning point to the measured lane line and the driving distance are converted into earth longitude and latitude coordinates to obtain a fused positioning position of the vehicle.

[0036] Preferably, the calibration process of the reference lane line includes:

[0037] Controlling the vehicle to travel along the lane line of the calibrated road section and gradually approach the lane line;

[0038] When it is detected that a protruding point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line close to the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as a first reference line;

[0039] When it is detected that a protruding point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line away from the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as a second reference line;

[0040] A reference lane line is calibrated based on the first reference line and the second reference line.

[0041] Preferably, the calibration process of the lane vertical line includes:

[0042] Controlling the vehicle to travel along the lane line of the calibrated road section, and using the corner camera to collect the calibration video; the calibrated road section is pre-set with a standard vertical line perpendicular to the lane line;

[0043] The calibration video is identified by using color difference map recognition technology or AI recognition technology. When it is identified that a protruding point of the front contour or the rear contour of the vehicle overlaps with the standard vertical line, a straight line passing through the protruding point of the front contour or the rear contour and parallel to the standard vertical line is calibrated as a lane vertical line.

[0044] A second aspect of an embodiment of the present invention discloses a lane departure recognition device, which is applied to an ECU in a vehicle, wherein the vehicle is provided with a corner camera, and the device comprises:

[0045] A recognition unit, used to perform lane line recognition on the driving video collected by the corner camera to obtain the measured lane line;

[0046] an acquisition unit, for acquiring a pixel distance from an intersection of the measured lane line and the lane vertical line to a reference lane line, a pixel width of the measured lane line, an actual width of the measured lane line, and a pixel width of the reference lane line; the reference lane line is pre-calibrated based on a side profile of the vehicle; the lane vertical line is pre-calibrated based on a front profile or a rear profile of the vehicle; and the actual width of the measured lane line is acquired from map data;

[0047] a deviation value calculation unit, configured to determine a current driving scene of the vehicle based on the pixel distance, and calculate a lane deviation value corresponding to the corner camera using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line;

[0048] The deviation angle calculation unit is used to calculate the lane deviation angle based on the lane deviation value and a preset deviation angle formula.

[0049] Based on the lane departure recognition method and device provided by the above-mentioned embodiment of the present invention, the driving video collected by the corner camera is subjected to lane line recognition to obtain the measured lane line; the pixel distance from the intersection of the measured lane line and the lane vertical line to the reference lane line, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line are obtained; the reference lane line is obtained in advance based on the side profile calibration of the vehicle; the lane vertical line is obtained in advance based on the front profile or rear profile calibration of the vehicle; the current driving scene of the vehicle is determined based on the pixel distance, and the lane deviation value corresponding to the corner camera is calculated using the deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line; the lane deviation angle is calculated based on the lane deviation value and the preset deviation angle formula. In this solution, the corner camera body self-reference technology is used to achieve the purpose of high-precision lane departure calculation and recognition in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 A flowchart of a lane departure recognition method disclosed in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a driving video image disclosed in an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a shadow formed by the sun casting on a vehicle according to an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of shadow covered lane line recognition disclosed in an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the relationship between the shadow length and the vehicle heading angle disclosed in an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of a lane keeping scenario disclosed in an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of a lane line pressing scenario and a lane line crossing scenario disclosed in an embodiment of the present invention;

[0058] Figure 8 A schematic diagram of lane departure angle calculation disclosed in an embodiment of the present invention;

[0059] Fig. 9 A schematic diagram of fusion positioning of a vehicle disclosed in an embodiment of the present invention;

[0060] Fig.10 A schematic diagram of the correspondence between the linear coordinates disclosed in the embodiment of the present invention and the longitude and latitude coordinates of the earth;

[0061] Fig.11 A structural diagram of a lane departure recognition device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0064] like Figure 1 , which is a flow chart of a lane departure recognition method disclosed in an embodiment of the present invention, wherein the lane departure recognition method is applied to an ECU in a vehicle.

[0065] It should be noted that the configuration included in the vehicle is shown in the following table:

[0066]

[0067] The lane departure recognition method includes the following steps:

[0068] Step S1: Perform lane line recognition on the driving video captured by the diagonal camera to obtain the measured lane line.

[0069] like Figure 2 , which is a schematic diagram of a driving video image disclosed in an embodiment of the present invention.

[0070] from Figure 2 It can be seen that mask area A and mask area B are pre-calibrated on the driving video image, where mask area A is the video area used to identify lane lines. That is to say, during the lane line recognition process, only mask area A needs to be analyzed, saving computing resources.

[0071] It should be noted that the corner cameras mentioned here are usually devices used in driving assistance functions such as remote driving or blind spot monitoring BSM (Blind-Spot-Monitoring). Corner cameras are generally set at the corners of the vehicle, such as the right front corner, right rear corner, left front corner and left rear corner of the vehicle. The application of the corner camera of the present invention will greatly enhance the fusion perception ability of these devices.

[0072] In the specific implementation process of step S1, if the driving video collected by the corner camera does not have a shadow coverage area, the video enhancement recognition technology is used to identify the measured lane line from the driving video.

[0073] Among them, video enhanced recognition technology is a reverse recognition model of electronic frog eye technology. By using a mobile camera, calculating the color difference changes in photos, and comparing the contours of recognizable static objects, it can discover and identify geographic elements including lane lines.

[0074] like Figure 3 , which is a schematic diagram of the sun casting a shadow on a vehicle according to an embodiment of the present invention.

[0075] It is understandable that the shadow coverage area is the projection of the vehicle on the road caused by sunlight. At noon or near noon, the strong light shadow on the ground has a great impact on lane line recognition. When calculating the lane deviation value, the unshadowed side is preferentially selected for recognition and calculation.

[0076] like Figure 4 , which is a schematic diagram of shadow covered lane line recognition disclosed in an embodiment of the present invention.

[0077] If there is a shadow-covered area in the driving video collected by the corner camera, a shadow boundary line is obtained by identifying the driving video collected by the corner camera, and then the image of the driving video is split based on the shadow boundary line to obtain a shadow area image and a non-shadow area image; lane recognition is performed on the shadow area image and the non-shadow area image respectively using video enhancement recognition technology to obtain a first lane line and a second lane line; the first lane line and the second lane line are merged to obtain a measured lane line.

[0078] Furthermore, the present invention discloses a method for calculating the relationship between the shadow length and the vehicle heading angle, so that the length of the shadow on the ground can be obtained according to the vehicle heading angle, and then the shadow boundary line can be determined according to the length of the shadow, as follows:

[0079] like Figure 5 , which is a schematic diagram of the relationship between the shadow length and the vehicle heading angle disclosed in an embodiment of the present invention.

[0080] 1. Calculate the shadow length.

[0081] The calculation formula for the solar altitude angle at noon is: α=90°-|β-γ|.

[0082] Among them, α is the solar altitude angle at noon, β is the latitude of the observer, and γ is the latitude of the point where the sun is directly above the sun.

[0083] The calculation formula for the deflection angle in the longitude direction is: δθ=15*(T-12)+θ1-θ0.

[0084] Wherein, T is time (in hours, calculated on a 24-hour basis, with negative values ​​in the morning), θ1 is the longitude of the location, and θ0 is the longitude of the time zone (θ0 = 105° in China).

[0085] Taking noon as an example, when δθ=0, the north-south length of the shadow is: L N =H*ctg|(α)*cos(δθ); the east-west length of the shadow is: L E =H*tg(δθ); therefore, the total length of the shadow is: L = SQRT(L N 2 +L E 2 ).

[0086] according to Figure 5 The geometric relationship shown, the north-south length of the shadow, and the east-west length of the shadow, the relationship between the shadow length and the vehicle heading angle is as follows:

[0087] Φ0=arctg(L N / L E )

[0088] Among them, Φ0 is the heading angle of the vehicle.

[0089] 2. Calculation of shadow coverage area.

[0090] Exemplarily, when the heading angle of the vehicle's travel direction is 90-Φ0 or -90-Φ0, the longest vertical height of the front and rear edges of the vehicle is H, and then the shadow ground length L is calculated based on H and the relationship between the shadow length and the vehicle's heading angle.

[0091] It is understandable that a light source with the same solar altitude angle irradiating objects at different altitudes will project shadows of different lengths on the ground. Therefore, the ground length of the shadow can be calculated based on the height of the object, that is, the height of the vehicle edge.

[0092] Exemplarily, when the vehicle is traveling in the direction of -Φ0 (or 180-Φ0), the side (left and right) shadow is the longest, the height of the side edge of the vehicle is taken as H, and then the shadow ground length L is calculated based on H and the relationship between the shadow length and the vehicle heading angle.

[0093] The vehicle body edge feature points in the shadow are identified from the driving video image, and the shadow boundary line is determined based on the vehicle body edge feature points and the shadow ground length L. Then, the shadow coverage area is determined based on the pre-calibrated vehicle contour and shadow boundary line.

[0094] Step S2: Obtain the pixel distance from the intersection of the measured lane line and the lane vertical line to the reference lane line, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line.

[0095] In step S2, the reference lane line is pre-calibrated based on the side profile of the vehicle; the lane vertical line is pre-calibrated based on the front profile or rear profile of the vehicle.

[0096] The vehicle profile is required in the process of calibrating the reference lane line and the lane vertical line. The vehicle profile is also pre-calibrated, including the left and right side profiles, the front profile, and the rear profile.

[0097] It should be noted that the vehicle outline, reference lane line and lane vertical line are equivalent to the calibration parameters of the corner cameras. Each corner camera has corresponding calibration parameters. When the position of the corner camera changes, the calibration parameters need to be recalibrated. When calculating the lane deviation value and lane deviation angle for the driving video collected by each corner camera, the calibration parameters corresponding to the corner camera need to be used for calculation.

[0098] Using the color difference map to identify and calibrate the vehicle outline:

[0099] The detection vehicle continues to drive straight for 30-40 meters on a straight road (the deviation angle is less than 0.1 degrees, and the recommended calibration test speed is set between 1-5 meters / second). The video data of the middle 10-20 meters is selected to generate a color difference map and then a contour.

[0100] If the color difference map is not clear and affects the generation of the outline, you can choose one of the following two methods to solve it:

[0101] The first method is to lay black and white grids on the ground of the test section (the edges of the 10 cm square black and white grids are perpendicular to the lane lines) and then conduct detection and collect data;

[0102] The second method is to spray a 10 cm or 20 cm wide vertical line of the test lane covered with white paint on the ground of the test section. When the vehicle outline passes through this vertical line, a clear color difference map boundary line is generated.

[0103] The calibration process of the baseline lane line is as follows:

[0104] The first step is to control the vehicle to travel along the lane line of the calibrated road section and gradually approach the lane line.

[0105] Specifically, the vehicle is tested on a straight road for 30-40 meters close to the lane line or driving straight on the line, and the vehicle's deviation angle is less than 0.1 degrees. It is recommended that the calibration test speed be set between 1-5 meters per second.

[0106] In the second step, when it is detected that the protruding point of the side contour of the vehicle close to the lane line overlaps with the edge line of the lane line close to the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as the first reference line; when it is detected that the protruding point of the side contour of the vehicle close to the lane line overlaps with the edge line of the lane line away from the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as the second reference line.

[0107] Specifically, a straight line passing through a protruding point of the side contour and parallel to the lane line can be converted into a reference line by interpolation, and black and white grids laid on the ground can be used to calibrate the reference line.

[0108] It should be noted that the middle 10-20 meters of driving video are selected for detection and processing. Generally, the side contour of a vehicle is not a smooth straight line, so whether it overlaps is determined based on the protruding points of the side contour. If the side contour of the vehicle is a straight line, whether it overlaps is determined based on the side contour.

[0109] Among them, the protruding points of the side profile do not include external devices such as rearview mirrors, corner radars, corner cameras, etc.

[0110] The third step is to calibrate the baseline lane line based on the first baseline and the second baseline.

[0111] It should be noted that the reference lane line is a reference line for judging whether a vehicle crosses the lane line, and is expressed by a set of broken lines or pixel point interpolation curves in a pixel coordinate system.

[0112] The calibration process of the lane vertical line is as follows:

[0113] The first step is to control the vehicle to drive along the lane lines of the calibrated road section and use the corner camera to collect the calibration video.

[0114] Among them, standard vertical lines perpendicular to the lane lines are pre-set on the calibrated section. The standard vertical lines can be: thermal expansion joints between the cement pavement and the lane lines, vertical lines of test lane lines covered with 10 cm or 20 cm wide white paint sprayed on the road surface in advance, and black and white grids paved on the ground of the calibrated section, with the 10 cm square black and white grid edge lines perpendicular to the direction of the lane lines.

[0115] The second step is to use color difference map recognition technology or AI recognition technology to identify and calibrate the video. When the protruding points of the front or rear contour of the vehicle are found to overlap with the standard vertical line, the straight line passing through the protruding points of the front or rear contour and parallel to the standard vertical line is calibrated as the lane vertical line.

[0116] Specifically, by collecting images of standard vertical lines and performing color difference map recognition or AI recognition, vertical lines that overlap or are close to overlapping with the rear contour line or front contour line of the vehicle (front and back depend on the position of the corner camera) are obtained and marked as lane vertical lines.

[0117] If the corresponding rear or front contour of the vehicle is not a straight line (cannot overlap with the obtained vertical line), select the vertical line passing through the outermost protruding point of the contour. The vertical line here is recorded and expressed by a polyline or quadratic spline curve of discrete points in the pixel coordinate system.

[0118] Step S3: Determine the current driving scene of the vehicle based on the pixel distance, and use the deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line to calculate the lane deviation value corresponding to the corner camera.

[0119] In step S3, the lane deviation value is the shortest distance from the position of the corner camera to the measured lane line.

[0120] Driving scenarios include: lane keeping scenario, lane crossing scenario and lane line crossing scenario.

[0121] When the pixel distance L3 meets the preset conditions: When the vehicle is in a lane keeping scenario, it is determined that the vehicle is in a lane keeping scenario.

[0122] like Figure 6 , which is a schematic diagram of a lane keeping scenario disclosed in an embodiment of the present invention.

[0123] The deviation value calculation formula pre-set for lane keeping scenarios is used:

[0124]

[0125] The lane deviation value d3 corresponding to the corner camera is calculated.

[0126] It can be understood that, since the corner cameras are arranged at different positions on the vehicle, the corner camera at each position can calculate the corresponding lane deviation value d3.

[0127] It should be noted that in the present application, n is set to an integer greater than or equal to 2, L1 represents the pixel width of the reference lane line, L2 represents the pixel width of the measured lane line, and d0 represents the actual width of the measured lane line. That is to say, the measured lane line is obtained by identifying the real lane line, and d0 represents the actual width of the real lane line at the physical level, and d0 can come from map data.

[0128] like Figure 7 , which is a schematic diagram of a lane line pressing scenario and a lane line crossing scenario disclosed in an embodiment of the present invention.

[0129] When the pixel distance L3 meets the preset condition: L3≤L1+L2, it is determined that the vehicle is in the lane line pressing scene, and the deviation value calculation formula pre-set for the lane line pressing scene is used:

[0130]

[0131] The lane deviation value d3 corresponding to the corner camera is calculated.

[0132] When the pixel distance L3 meets the preset conditions: L3>L1+L2 and When the vehicle is determined to be in a lane crossing scenario, the deviation value calculation formula pre-set for the lane crossing scenario is used:

[0133]

[0134] The lane deviation value d3 corresponding to the corner camera is calculated.

[0135] It should be noted that: Note: k is a cycle from 1 to n-1, that is, the segmented interpolation calculation of d3, where the segmentation is equivalent to the multiple segmentation of d0.

[0136] In one embodiment, when the lane deviation value is equal to 0, the intelligent driving system of the vehicle is triggered to issue a lane crossing alarm; when the lane deviation value is equal to -d t , triggering the vehicle's intelligent driving system to issue a line crossing alarm; d t is the preset critical threshold.

[0137] It should be noted that the minimum vertical distance between the prohibited or moving objects outside the lane line and the lane line is d t +d s , where d s is the minimum safe distance for scratching, d t is the critical threshold of scratching, when d3=-d t Trigger the line crossing alarm.

[0138] Step S4: Calculate the lane departure angle based on the lane departure value and a preset departure angle formula.

[0139] In step S4, the lane departure angle is the angle between the vehicle's driving direction and the measured lane line.

[0140] In the specific implementation process of step S4, based on the preset deviation angle formula: The lane departure angle θ is calculated.

[0141] It should be noted that θ mentioned in the above embodiment of the present invention is the deviation angle between sunlight and the vertical ground direction in the east-west direction, which is different from the definition here.

[0142] Wherein, d3 is the lane deviation value, d'3 is the past lane deviation value, and the past lane deviation value is the lane deviation value calculated before the preset time; S is the distance traveled by the vehicle within the preset time.

[0143] from Figure 7 It can be seen that the current lane vertical line 0 coincides with the edge of the vehicle body (i.e., the pre-calibrated vehicle outline). Since the vehicle maintained a driving state at the past moment, the lane vertical line 1 coincided with the edge of the vehicle body, and the past lane deviation value, that is, the lane vertical line 1 coincided with the edge of the vehicle body, the lane deviation value calculated and recorded.

[0144] like Figure 8 , which is a schematic diagram of lane departure angle calculation disclosed in an embodiment of the present invention.

[0145] If the right front corner, left front corner, right rear corner and left rear corner of the vehicle are respectively provided with a first corner camera, a second corner camera, a third corner camera and a fourth corner camera, then step S4 has a more preferred specific implementation method:

[0146] Based on the preset deviation angle formula: θ=arcsin(δd r / L V )≈δd r / L V , the lane departure angle θ is calculated.

[0147] Among them, δd r is the difference between the lane departure value corresponding to the first angle camera and the lane departure value corresponding to the third angle camera, or is the difference between the lane departure value corresponding to the second angle camera and the lane departure value corresponding to the fourth angle camera; L V is the length of the vehicle.

[0148] In one embodiment, the maximum deflection angle in the lane keeping state is: θ max ≈(w l -w v -d0) / L V .

[0149] Among them, w l is the lane width, wv is the vehicle width; θ max is the maximum deflection angle of the vehicle in the lane keeping state. That is, if the actual lane departure angle exceeds θ max The vehicle will continue to drive over the lane line or even go out of the lane. The actual lane departure angle can be compared with θ max The size relationship is used to determine the state of the vehicle.

[0150] like Fig. 9 FIG. 1 is a schematic diagram of the fusion positioning of a vehicle disclosed in an embodiment of the present invention. The fusion positioning realizes the application of the lane deviation value and the lane deviation angle. The specific fusion positioning process is as follows:

[0151] Calculate the distance from the pre-calibrated positioning point used to represent the vehicle position to the measured lane line:

[0152] X=d3+W*cosθ-L*sinθ

[0153] Wherein, d3 is the lane deviation value, θ is the lane deviation angle, W is the lateral distance between the positioning point and the corner point, L is the longitudinal distance between the positioning point and the corner point, and the corner point is the position where the corner camera is set corresponding to the lane deviation value.

[0154] Obtain the historical moments when the vehicle has passed through linear reference points (i.e., the angle of the vehicle coincides with the linear reference points); linear reference points are reference points with fixed intervals in the map data that are used to locate the longitudinal position of the vehicle on the road.

[0155] Linear reference points can be mileage posts on highways, horizontal markings on roads, and expansion joints of vertical lane lines.

[0156] Calculate the distance traveled by the vehicle along the measured lane line after the historical moment:

[0157]

[0158] Among them, T i is the historical moment, T1 is the current moment, θ is the lane departure angle, and S is the mileage measured by the wheel speedometer on the vehicle.

[0159] The distance from the positioning point to the measured lane line and the driving distance are converted into the longitude and latitude coordinates of the earth to obtain the fused positioning position of the vehicle.

[0160] It should be noted that when the vehicle reaches a specific linear reference point, the four corner cameras can calculate the longitudinal position of the linear reference point four times, which can be used to calibrate the wheel speed meter error or the wheel speed meter distance cumulative error. The calculation method is as follows:

[0161] Assume that T1 is the time when the front corner camera reaches the linear reference point, Ti is the time it takes for the rear corner camera to reach the linear reference point, and S is the time from T1 to T calculated by the wheel speed. i The driving distance in the time interval is E, and the wheel speed meter error is:

[0162] E=|SL V |

[0163] The accumulated error of the wheel speed meter can also be calculated by two known linear reference points. Assume that T1 is the time when the front corner camera reaches the first linear reference point, T i is the time when the front corner camera reaches the second linear reference point, and S is the time from T1 to T calculated by the wheel speed. i The driving distance in the time interval, E is the wheel speed meter error, L is the calibration distance between the two calibration lines, then:

[0164] E=|SL|

[0165] When fusion positioning is used to realize vehicle auxiliary positioning, the cumulative error estimation of the wheel speed meter is an important positioning calculation parameter.

[0166] like Fig.10 , which is a schematic diagram of the correspondence between the linear coordinates disclosed in an embodiment of the present invention and the longitude and latitude coordinates of the earth.

[0167] It can be understood that the distance from the positioning point to the measured lane line and the driving distance are equivalent to the coordinates of the linear reference system, which can be used Fig.10 The linear coordinates shown are converted into the corresponding relationship between the longitude and latitude coordinates of the earth to obtain the fused positioning position of the vehicle.

[0168] In one embodiment, the lane departure value and lane departure angle may also be applied to the calibration of other sensors on the vehicle. The specific examples are as follows:

[0169] 1. Corner camera self-calibration:

[0170] When the width of the front of the vehicle is the same as the width of the rear of the vehicle, the relationship between the calculation results of the four corner cameras set at the right front corner, left front corner, right rear corner and left rear corner of the vehicle is:

[0171] d rf +d lf =d rb +d lb =w l -w v cosθ-d0

[0172] If d rf +d lf or rb +d lbIf the above equation is not satisfied, it is determined that the calibration parameters of the corner camera are wrong and need to be recalibrated. The calibration parameters of the corner camera are the vehicle profile, reference lane line and lane vertical line mentioned above.

[0173] 2. Calibration of other sensors:

[0174] If a monocular camera is installed on the vehicle, and the lane departure amount and lane departure angle calculated by the monocular camera are inconsistent with the calculation results obtained in this application, you can consider using the lane departure amount and lane departure angle obtained in this application to correct the monocular camera position calibration value.

[0175] If the vehicle is equipped with a corner radar, and the recognition range of the corner radar overlaps with that of the corner camera, the lane departure amount and lane departure angle obtained in this application can be used to correct the position calibration value of the corner radar.

[0176] For example, after the corner radar identifies the lane line, it can be compared with the measured lane line identified by this application. If there is an obvious deviation, and the deviation value is relatively stable, it can be considered that the deviation is caused by the deflection of the radar installation angle caused by vehicle vibration. The lane deviation amount and lane deviation angle obtained by this application can be used to correct the corner radar calibration parameters.

[0177] Based on a lane departure recognition method disclosed in the above-mentioned embodiment of the present invention, lane line recognition is performed on the driving video collected by the corner camera to obtain the measured lane line; the pixel distance from the intersection of the measured lane line and the lane vertical line to the reference lane line, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line are obtained; the reference lane line is pre-calibrated based on the side profile of the vehicle; the lane vertical line is pre-calibrated based on the front profile or rear profile of the vehicle; the current driving scene of the vehicle is determined based on the pixel distance, and the lane deviation value corresponding to the corner camera is calculated using the deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line; the lane deviation angle is calculated based on the lane deviation value and the preset deviation angle formula. In this solution, the corner camera body self-reference technology is used to achieve the purpose of high-precision lane departure calculation and recognition in complex environments.

[0178] Corresponding to the lane departure recognition method disclosed in the above embodiment of the present invention, as Fig.11 As shown, it is a structural diagram of a lane departure recognition device disclosed in an embodiment of the present invention, which is applied to an ECU in a vehicle. A corner camera is provided on the vehicle. The device includes: an identification unit 1, an acquisition unit 2, a deviation value calculation unit 3 and a deviation angle calculation unit 4.

[0179] Recognition unit 1, used for performing lane line recognition on the driving video collected by the diagonal camera to obtain the measured lane line;

[0180] Preferably, the identification unit 1 is specifically used for:

[0181] If the driving video collected by the corner camera does not contain shadow coverage area, the video enhancement recognition technology is used to identify the measured lane line from the driving video; the shadow coverage area is the projection formed by the sunlight shining on the vehicle on the road;

[0182] If there is a shadow-covered area in the driving video collected by the corner camera, a shadow boundary line is obtained by identifying the driving video collected by the corner camera, and the image of the driving video is split based on the shadow boundary line to obtain an image of the shadow area and an image of the non-shadow area;

[0183] The video enhancement recognition technology is used to perform lane recognition on the shadow area image and the non-shadow area image respectively to obtain the first lane line and the second lane line;

[0184] The first lane line and the second lane line are merged to obtain the measured lane line.

[0185] An acquisition unit 2 is used to acquire a pixel distance from an intersection of a measured lane line and a lane vertical line to a reference lane line, a pixel width of the measured lane line, an actual width of the measured lane line, and a pixel width of the reference lane line; the reference lane line is pre-calibrated based on a side profile of the vehicle; the lane vertical line is pre-calibrated based on a front profile or a rear profile of the vehicle;

[0186] The deviation value calculation unit 3 is used to determine the current driving scene of the vehicle based on the pixel distance, and calculate the lane deviation value corresponding to the corner camera using the deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line;

[0187] Preferably, the acquisition unit is specifically used for:

[0188] When the pixel distance L3 meets the preset conditions: When the vehicle is determined to be in a lane keeping scenario, the deviation value calculation formula pre-set for the lane keeping scenario is used: Calculate and obtain the lane deviation value d3 corresponding to the corner camera;

[0189] Wherein, n represents an integer greater than or equal to 2, L1 represents the pixel width of the reference lane line, L2 represents the pixel width of the measured lane line, and d0 represents the actual width of the measured lane line;

[0190] When the pixel distance L3 meets the preset condition: L3≤L1+L2, it is determined that the vehicle is in the lane line pressing scene, and the deviation value calculation formula pre-set for the lane line pressing scene is used: Calculate the lane deviation value d3 corresponding to the corner camera;

[0191] When the pixel distance L3 meets the preset conditions: L3>L1+L2 and When the vehicle is determined to be in a lane crossing scenario, the deviation value calculation formula pre-set for the lane crossing scenario is used: The lane deviation value d3 corresponding to the corner camera is calculated.

[0192] The deviation angle calculation unit 4 is used to calculate the lane deviation angle based on the lane deviation value and a preset deviation angle formula.

[0193] Preferably, the deviation angle calculation unit 4 is specifically used for:

[0194] Based on the preset deviation angle formula: The lane departure angle θ is calculated;

[0195] Wherein, d3 is the lane deviation value, d'3 is the past lane deviation value, and the past lane deviation value is the lane deviation value calculated before the preset time; S is the distance traveled by the vehicle within the preset time.

[0196] Preferably, the right front corner, the left front corner, the right rear corner and the left rear corner of the vehicle are respectively provided with a first corner camera, a second corner camera, a third corner camera and a fourth corner camera, and the deviation angle calculation unit 4 is specifically used for:

[0197] Based on the preset deviation angle formula: θ=arcsin(δd r / L V ), the lane departure angle θ is calculated;

[0198] Among them, δd r is the difference between the lane departure value corresponding to the first angle camera and the lane departure value corresponding to the third angle camera, or is the difference between the lane departure value corresponding to the second angle camera and the lane departure value corresponding to the fourth angle camera; L V is the length of the vehicle.

[0199] Preferably, the lane departure recognition device further includes:

[0200] An alarm unit is used to trigger the vehicle's intelligent driving system to issue a lane crossing alarm when the lane deviation value is equal to 0;

[0201] When the lane departure value is equal to -d t , triggering the vehicle's intelligent driving system to issue a line crossing alarm; d t is the preset critical threshold.

[0202] Preferably, the lane departure recognition device further includes:

[0203] A first calibration unit, used to control the vehicle to travel along the lane line of the calibrated road section and gradually approach the lane line;

[0204] When it is detected that a convex point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line close to the vehicle, a straight line passing through the convex point of the side contour and parallel to the lane line is calibrated as a first reference line;

[0205] When it is detected that a convex point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line away from the vehicle, a straight line passing through the convex point of the side contour and parallel to the lane line is calibrated as a second reference line;

[0206] Based on the first baseline and the second baseline, a baseline lane line is calibrated.

[0207] Preferably, the lane departure recognition device further includes:

[0208] The second calibration unit is used to control the vehicle to travel along the lane line of the calibration section and collect the calibration video using the corner camera; the calibration section is pre-set with a standard vertical line perpendicular to the lane line;

[0209] The calibration video is identified using color difference map recognition technology or AI recognition technology. When it is identified that the protruding points of the front or rear contour of the vehicle overlap with the standard vertical line, the straight line passing through the protruding points of the front or rear contour and parallel to the standard vertical line is calibrated as the lane vertical line.

[0210] Based on the lane departure recognition device disclosed in the above-mentioned embodiment of the present invention, the driving video collected by the corner camera is used to identify the lane line to obtain the measured lane line; the pixel distance from the intersection of the measured lane line and the lane vertical line to the reference lane line, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line are obtained; the reference lane line is pre-calibrated based on the side profile of the vehicle; the lane vertical line is pre-calibrated based on the front profile or rear profile of the vehicle; the current driving scene of the vehicle is determined based on the pixel distance, and the lane deviation value corresponding to the corner camera is calculated using the deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line and the pixel width of the reference lane line; the lane deviation angle is calculated based on the lane deviation value and the preset deviation angle formula. In this solution, the corner camera body self-reference technology is used to achieve the purpose of high-precision lane departure calculation and recognition in complex environments.

[0211] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0212] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0213] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying lane departure, characterized in that: An ECU applied to a vehicle, wherein the vehicle is provided with a corner camera, the method comprising: Performing lane line recognition on the driving video collected by the corner camera to obtain the measured lane line; Obtaining a pixel distance from an intersection of the measured lane line and the lane vertical line to a reference lane line, a pixel width of the measured lane line, an actual width of the measured lane line, and a pixel width of the reference lane line; the reference lane line is pre-calibrated based on a side profile of the vehicle; the lane vertical line is pre-calibrated based on a front profile or a rear profile of the vehicle; The current driving scene of the vehicle is determined based on the pixel distance, and the lane deviation value corresponding to the corner camera is calculated using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line; the actual width of the measured lane line is obtained from map data; A lane departure angle is calculated based on the lane departure value and a preset departure angle formula.

2. The method according to claim 1, characterized in that The performing lane line recognition on the driving video collected by the corner camera to obtain the measured lane line includes: If the driving video collected by the corner camera does not have a shadow coverage area, the measured lane line is identified from the driving video using video enhancement recognition technology; the shadow coverage area is the projection formed by sunlight irradiating the vehicle on the road; If the driving video captured by the corner camera has the shadow coverage area, a shadow boundary line is obtained by identifying the driving video captured by the corner camera, and the image of the driving video is split based on the shadow boundary line to obtain a shadow area image and a non-shadow area image; Using video enhancement recognition technology to perform lane recognition on the shadow area image and the non-shadow area image respectively to obtain a first lane line and a second lane line; The first lane line and the second lane line are merged to obtain a measured lane line.

3. The method according to claim 1, characterized in that The determining the current driving scene of the vehicle based on the pixel distance, and calculating the lane deviation value corresponding to the corner camera using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line, includes: When the pixel distance L3 meets the preset condition: When the vehicle is determined to be in a lane keeping scenario, a deviation value calculation formula pre-set for the lane keeping scenario is used: Calculating and obtaining a lane deviation value d3 corresponding to the corner camera; Wherein, n represents an integer greater than or equal to 2, L1 represents the pixel width of the reference lane line, L2 represents the pixel width of the measured lane line, and d0 represents the actual width of the measured lane line; When the pixel distance L3 satisfies the preset condition: L3≤L1+L2, it is determined that the vehicle is in a lane-pressing scene, and the deviation value calculation formula pre-set for the lane-pressing scene is used: Calculating a lane deviation value d3 corresponding to the corner camera; When the pixel distance L3 meets the preset condition: L3>L1+L2 and When the vehicle is determined to be in a lane crossing scenario, the deviation value calculation formula pre-set for the lane crossing scenario is used: A lane deviation value d3 corresponding to the corner camera is calculated.

4. The method according to claim 1, characterized in that: The calculating the lane departure angle based on the lane departure value and a preset departure angle formula includes: Based on the preset deviation angle formula: The lane departure angle θ is calculated; Wherein, d3 is the lane deviation value, d'3 is the past lane deviation value, and the past lane deviation value is the lane deviation value calculated before a preset time; and S is the distance traveled by the vehicle within the preset time.

5. The method according to claim 1, characterized in that The right front corner, left front corner, right rear corner and left rear corner of the vehicle are respectively provided with a first corner camera, a second corner camera, a third corner camera and a fourth corner camera; The calculating the lane departure angle based on the lane departure value and a preset departure angle formula includes: Based on the preset deviation angle formula: θ=arcsin(δd r / L V ), the lane departure angle θ is calculated; Among them, δd r is the difference between the lane departure value corresponding to the first angle camera and the lane departure value corresponding to the third angle camera, or is the difference between the lane departure value corresponding to the second angle camera and the lane departure value corresponding to the fourth angle camera; L V is the length of the vehicle.

6. The method according to claim 1, characterized in that The method further comprises: When the lane deviation value is equal to 0, the intelligent driving system of the vehicle is triggered to issue a lane crossing alarm; When the lane departure value is equal to -d t , triggering the intelligent driving system of the vehicle to issue a line crossing alarm; d t is the preset critical threshold.

7. The method according to claim 1, characterized in that The method further comprises: Calculate the distance from the pre-calibrated positioning point used to characterize the vehicle position to the measured lane line: X=d3+W*cosθ-L*sinθ; wherein d3 is the lane deviation value, θ is the lane deviation angle, W is the lateral distance between the positioning point and the corner point, L is the longitudinal distance between the positioning point and the corner point, and the corner point is the position where the corner camera is set corresponding to the lane deviation value; Obtaining historical moments when the vehicle has passed through linear reference points in the past; the linear reference points are reference points in map data that are used to locate the longitudinal position of the vehicle on the road and have fixed intervals; Calculate the travel distance of the vehicle along the measured lane line after the historical moment: Among them, T i is the historical moment, T1 is the current moment, θ is the lane departure angle, and S is the mileage measured by the wheel speedometer on the vehicle; The distance from the positioning point to the measured lane line and the driving distance are converted into earth longitude and latitude coordinates to obtain a fused positioning position of the vehicle.

8. The method according to claim 1, characterized in that The calibration process of the reference lane line includes: Controlling the vehicle to travel along the lane line of the calibrated road section and gradually approach the lane line; When it is detected that a protruding point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line close to the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as a first reference line; When it is detected that a protruding point of the side contour of the vehicle close to the lane line overlaps with a side line of the lane line away from the vehicle, a straight line passing through the protruding point of the side contour and parallel to the lane line is calibrated as a second reference line; A reference lane line is calibrated based on the first reference line and the second reference line.

9. The method according to any one of claims 1 to 8, characterized in that: The calibration process of the lane vertical line includes: Controlling the vehicle to travel along the lane line of the calibrated road section, and using the corner camera to collect the calibration video; the calibrated road section is pre-set with a standard vertical line perpendicular to the lane line; The calibration video is identified by using color difference map recognition technology or AI recognition technology. When it is identified that a protruding point of the front contour or the rear contour of the vehicle overlaps with the standard vertical line, a straight line passing through the protruding point of the front contour or the rear contour and parallel to the standard vertical line is calibrated as a lane vertical line.

10. A lane departure recognition device, characterized in that: An ECU applied to a vehicle, wherein the vehicle is provided with a corner camera, the device comprising: A recognition unit, used to perform lane line recognition on the driving video collected by the corner camera to obtain the measured lane line; an acquisition unit, configured to acquire a pixel distance from an intersection of the measured lane line and the lane vertical line to a reference lane line, a pixel width of the measured lane line, an actual width of the measured lane line, and a pixel width of the reference lane line; the reference lane line is pre-calibrated based on a side profile of the vehicle; the lane vertical line is pre-calibrated based on a front profile or a rear profile of the vehicle; a deviation value calculation unit, configured to determine a current driving scene of the vehicle based on the pixel distance, and calculate a lane deviation value corresponding to the corner camera using a deviation value calculation formula pre-set for the driving scene, the pixel distance, the pixel width of the measured lane line, the actual width of the measured lane line, and the pixel width of the reference lane line; the actual width of the measured lane line is obtained from map data; The deviation angle calculation unit is used to calculate the lane deviation angle based on the lane deviation value and a preset deviation angle formula.

Images