Sensitive and accurate real-time warning system for lane deviation in complex road conditions

Abstract

This application is based on the practical application of developing intelligent assisted driving systems. It designs and implements a very fast lane deviation real-time warning system on the vehicle-mounted ARM platform. It proposes a very fast lane line recognition algorithm with W‑r extended dynamic evaluation of images, detects lane information in real time, and considers the lateral speed of the motor vehicle in the lane. It proposes to combine two warning strategies. The first one uses the convenient relative distance between the motor vehicle and the lane for warning judgment when the lateral speed is not large. When the lateral speed is relatively large, the deviation rate of the left and right lane lines in the image is used to judge lane departure. This application does not require perspective transformation of the image, does not need to measure the external and internal parameters of the monocular camera, and has low requirements for the monocular camera, which brings great convenience to users. At the same time, the algorithm has low time complexity, fast speed, strong practicality and applicability, and has great value and a broad market in real-time warning of lane deviation in complex road conditions.

Description

Technical Field

[0001] The present application relates to a real-time warning system for lane deviation under complex road conditions, and in particular to a sensitive and accurate real-time warning system for lane deviation under complex road conditions, belonging to the technical field of lane deviation warning for intelligent assisted driving. Background Art

[0002] The rapid development of the automobile industry has also caused a gradual increase in traffic accidents. Various complex terrains have led to complicated road conditions. At the same time, the quality of drivers is uneven, so traffic accidents are very likely to occur. Traffic safety is facing serious challenges.

[0003] In order to avoid traffic accidents and improve driving safety and comfort, intelligent assisted driving has become popular, and motor vehicle safety assisted driving system has become a research and development hotspot. Among them, the lane departure warning system is an important component. Its function is to detect whether the lane has deviated or is about to deviate and remind the driver to avoid possible traffic accidents. At present, the lane departure warning system is divided into two important warning functions: horizontal and vertical. The vertical function refers to preventing lane departure traffic accidents caused by excessive speed or loss of control of the driving direction; the horizontal system refers to a system that prevents the driver from unconsciously giving up steering control under conditions such as excessive fatigue or long-term driving, causing lane departure traffic accidents. The system can alert the driver when the lane has deviated or is about to deviate, expand the driver's lane identification range, and thus buy more reaction time for the driver, so that the motor vehicle can drive in a safe lane and avoid traffic accidents. The lane departure warning system based on vehicle-mounted image detection has the characteristics of low cost, small size and easy installation, and has great application value.

[0004] At present, the lane deviation warning system is gradually installed on some high-end brand cars. Considering the practicality and cost of the system, most of them are based on monocular vision. It mainly uses the obtained lane information to judge the lateral position of the current motor vehicle in the lane for warning. At present, it is mainly divided into forward-looking system and visual measurement system according to the installation method of the camera. With the rise of artificial intelligence technology in recent years, many companies and institutions have begun to gradually carry out ADAS research and development. To design a good lane deviation real-time warning system, it is fundamental to detect the position information of the lane line. A good lane line detection method is the core of the system.

[0005] The existing technology uses spatial coordinate transformation to detect the most likely straight lines in the vehicle-mounted image. The lane lines have obvious features in the image and are easy to pick out. The lane line detection and tracking algorithm based on the B-Snake spline curve in the existing technology does not depend on the parameters of the camera. It constructs the perspective effect of parallel lines with the help of universal lane boundaries or markers. It can describe a wider range of lane structures than other lane models such as straight line and parabola models. The existing technology checks lane lines based on SVM, combines the RANSAC algorithm with deep learning, and uses two steps to detect lanes in real-world driving videos. The first step includes blurring and edge detection for removing environmental noise as a preprocessing part, and the second step combines the RANSAC algorithm with the CNN process to extract accurate lane information. The existing technology proposes a lane line detection method based on an extended particle filter. By using a universal lane model to support two independent particle filters to detect the left and right lane boundaries respectively, multiple particles are combined to identify the key particles of an image line, and local linear regression is used to adjust the boundary detection points. The existing technology uses the statistical method of Hough transform for line segment detection to complete line segment extraction. By considering quantization error, image noise, pixel interference and peak expansion, the image center is selected as the image origin coordinate, and the random variable peak area is defined in each column. The score value is assigned a probability distribution, and the endpoint coordinates of the detected segment are determined by fitting a sine curve.

[0006] With the development of hardware technology, ARM microprocessors have been applied in various fields of society. ARM microprocessors have the advantages of small size, low power consumption, low cost and high performance, and are very suitable for installation on motor vehicles. Therefore, lane line detection through the ARM platform in real time and lane deviation warning have better economic benefits and strong applicability compared with other warning platforms. Although there are many lane detection algorithms proposed at present, many algorithms have good effects, but their calculation amount is relatively large, the detection speed is too slow, and it is rare to achieve real-time lane line detection and warning on vehicle-mounted equipment. Therefore, it is necessary to develop real-time lane detection and warning on the ARM platform.

[0007] In summary, the lane departure real-time warning system in the prior art still has problems. The difficulties and problems to be solved in this application are mainly concentrated in the following aspects:

[0008] First, by classifying and analyzing the existing lane line detection algorithms, it is found that they all have various deficiencies when directly applied to the on-board lane departure warning system; although the lane line detection algorithm based on the basic features of the image has low computational complexity, its robustness is poor, and it cannot detect or misdetects many complex road conditions; although the lane line detection based on geometric information has a better effect, it involves calculation modules such as perspective transformation, which has a large amount of calculation and requires many parameters to be adjusted, making it difficult to achieve real-time performance; the lane line detection model based on color information has a good implementation for simple road conditions and is fast, but it is difficult to use on various complex road conditions; lane line detection based on deep learning is a relatively new field. Although deep learning has demonstrated powerful capabilities in big data processing and face recognition, its platform requirements are high. There are still many problems with its real-time performance and effect when used in lane line detection. The existing technology lacks a lane departure real-time warning algorithm and system that is fast, effective, practical and applicable;

[0009] Second, the existing lane departure warning hardware platform is not economical and practical, the hardware cost is high, the stability and real-time performance are poor when used in vehicle systems, and there are often obvious lags or even failures, which cannot effectively reduce traffic accidents and casualties; at the same time, the update and use costs are high and low, the scope of application is narrow, the system size is large, the installation is inconvenient, and the power consumption is weak and the economy is not good when used in motor vehicles. Although the lane detection algorithms in the existing technology are diverse, the calculation amount is relatively large, and the detection speed is too slow when it is actually used in vehicle systems. Real-time lane line detection and warning cannot be achieved on vehicle-mounted equipment. Therefore, the development of a lane departure warning technology based on the ARM platform is of great significance and great practical value;

[0010] Third, the existing technology lacks a lane line recognition algorithm that is fast enough to meet real-time requirements, lacks a fast lane line feature optimization modeling method, and cannot detect complex road lane information stably and quickly in combination with road features. The time complexity of the perspective transformation calculation module used by the existing technology algorithm is relatively large, which is not only of little use, but also greatly increases the amount of calculation when used in the vehicle system, slowing down the detection speed. It is impossible to select lane lines with high credibility by combining the edge point intensity and the parallel relationship characteristics of the lane edge, and it is impossible to estimate the parameters of the angle, offset, average edge intensity and custom curvature. It is impossible to use lane color information to distinguish yellow and white lane lines, and lacks a method to use edge point spacing to distinguish virtual and real lane lines; the existing technology method can only detect lane straight lines, and cannot reflect the characteristics of the degree of lane curvature. It requires the conversion of various coordinate systems. When applied in engineering, it is necessary to obtain complicated steps such as internal and external parameters of the camera, installation location and motor vehicle model. The lane edge extraction degree is too low, and it is difficult to achieve real-time detection of lane lines on ARM platforms with weak computing power;

[0011] Fourth, after accurately determining the position information of the lane line, the relative position information of the motor vehicle needs to be obtained, and the various possibilities of the motor vehicle deviating from the lane need to be calculated through the deviation strategy; the existing warning method based on the lane model and the real road surface coordinate information has weak adaptability. The geometric imaging coordinate system of the motor vehicle system, camera, and road surface needs to know the model size of the motor vehicle, the road type, the camera and the optical lens installation angle and height, the internal parameters and distortion parameters of the camera. Every time a car, road model or camera is changed, the coordinate system must be re-established, which is very troublesome in engineering; at the same time, the vehicle-mounted image undergoes perspective transformation and other processing involving image pixel coordinate conversion, and the camera needs to be calibrated, which greatly increases the trouble of actual use, and the calculation speed and ease of operation are poor; realizing the deviation warning through the establishment of the road model violates the original intention of the algorithm design, lacks the method of considering the lateral speed of the motor vehicle in the lane and integrating the two warning strategies, lacks the convenient relative distance between the motor vehicle and the lane for warning judgment when the lateral speed is not large; lacks the use of the deviation rate of the left and right lane lines in the image to judge the lane departure when the lateral speed is large, and the lane deviation warning cannot achieve the ideal effect. Summary of the invention

[0012] This application is based on the practical application of developing intelligent assisted driving system. On the ARM platform with RK3288 processor as the core, a lane deviation real-time warning system is designed. It mainly includes two parts: lane line detection part and lane deviation warning part. This application proposes a fast lane information extraction algorithm of image Wr extended dynamic evaluation, which can detect lane information on the ARM platform at a very fast speed. Considering the size of the lateral speed of the motor vehicle in the lane, it proposes to combine two warning strategies. The first one uses the convenient relative distance between the motor vehicle and the lane for warning judgment. This processing strategy has better results when the lateral speed is not large; when the lateral speed is relatively large, that is, when the angle between the vehicle head direction and the road centerline is relatively large, the lane deviation judgment is made by using the deviation rate of the left and right lane lines in the image, which can achieve better results; the algorithm of this application does not need to perform perspective transformation on the image, so it does not need to measure the external and internal parameters of the monocular camera, and has low requirements for the monocular camera, which brings great convenience to users. At the same time, the time complexity of the algorithm is much smaller than others, with fast speed, good effect, strong practicality and applicability. It has great value and a broad market for real-time warning of lane deviation in complex road conditions.

[0013] In order to realize the above technical features, the technical solutions adopted in this application are as follows:

[0014] The sensitive and accurate real-time warning system for lane deviation under complex road conditions includes, firstly, the hardware including RK3288 image processing unit, image acquisition unit, alarm display unit and CAN signal part of control unit. The real-time road conditions of driving process are obtained through CCD camera above the cab. The acquired road condition images are processed by RK3288 image processing unit. According to the identified lane line information, it is determined whether it is necessary to send an alarm signal to the alarm unit to inform the driver to pay attention to lane deviation. Secondly, the software algorithm is divided into two parts, namely, the extreme lane line recognition algorithm of image Wr extended dynamic evaluation and the real-time warning method of lane deviation:

[0015] First, the ultra-fast lane line recognition algorithm of Wr extended dynamic evaluation uses line scanning to extract edge points, and classifies edge points into two categories for custom parameter scoring to obtain straight lines. Candidate lane lines are obtained according to the lane line model, and Wr extended dynamic evaluation is used to estimate and update road parameters. Inter-frame information is used to enhance system stability and robustness. Finally, the parameters output by Wr extended dynamic evaluation are used to extract the final edge points. The algorithm detects lane lines without inverse perspective transformation, and uses Wr extended dynamic evaluation to estimate parameters of the established lane parameter model, realizing real-time detection of lane lines on ARM platforms with weak computing power; lane color information is used to distinguish yellow and white lane lines, and edge point spacing is used to distinguish virtual and real lane lines; specifically including: lane line ultra-fast detection algorithm architecture, edge point extraction, establishment of evaluation and scoring rules, acquisition of lane line detection verification values, Wr extended dynamic evaluation parameter estimation and tracking, and lane edge sensitive extraction;

[0016] Second, the real-time lane departure warning method integrates the warning strategy based on the relative distance between the motor vehicle and the lane line and the warning strategy based on the slope of the lane line. When the lateral speed is relatively low, the warning strategy based on the relative distance between the motor vehicle and the lane line is adopted; when the lateral speed is relatively high, the strategy based on the lane deviation rate is adopted to avoid the use of various complex parameters required by the lane model, which greatly enhances the scope of application in engineering; specifically, it includes: warning strategy based on the relative distance between the motor vehicle and the lane, and sensitive warning strategy based on the lane deviation rate.

[0017] A sensitive and accurate real-time warning system for lane deviation in complex road conditions. Furthermore, a fast lane line feature optimization modeling method is proposed. Combined with road characteristics, image Wr extended dynamic evaluation is used for estimation and correction, and complex road lane information is quickly detected. The perspective transformation calculation module is removed, and a simplified geometric Vight detection line is used as the basis. The edge point intensity and lane edge parallel relationship features are combined to select lane lines with high credibility. Image Wr extended dynamic evaluation is used to estimate parameters such as angle, offset, average edge intensity and custom curvature.

[0018] A sensitive and accurate real-time warning system for lane deviation in complex road conditions. Furthermore, the lane line extreme speed detection algorithm architecture is mainly divided into seven steps:

[0019] Step 1: Obtaining vehicle images: Obtaining road images from the camera above the cab;

[0020] Step 2: Near-field image enhancement: enhance the degree of edge point mutation;

[0021] Step 3: Obtain edge points: Use line scanning to obtain edge points, and divide the edge points into four categories, which are the maximum and minimum values ​​on the left and right sides of the image;

[0022] Step 4: Establish the evaluation and scoring rules: Create a table for the two types of edge points based on the custom parameter distribution, and obtain candidate lane lines based on the scores;

[0023] Step 5: Acquisition of lane lines and edge points: The measured value of the lane line of the current frame is obtained by combining the relationship between frames, and the abnormal points on the edge of the lane line are removed by using the triple standard deviation method to enhance the robustness of the algorithm.

[0024] Step 6: Image Wr extended dynamic evaluation for estimation and correction: establish a lane parameter model, use image Wr extended dynamic evaluation for parameter estimation, and use the lane line position between frames to eliminate interference;

[0025] Step 7: According to the updated state value, obtain the final lane line edge points and connect them into lane lines;

[0026] In step six, it is necessary to compare the current frame detection verification value with the predicted value of the previous frame using the image Wr extended dynamic evaluation to see if they match. If they match, the set counter value count is increased by 1. If the value of count is greater than or equal to 3, the output optimal estimate is the current lane line position information; if they do not match, proceed to the next frame, and the output of the current frame value is judged separately.

[0027] Sensitive and accurate real-time warning system for lane deviation in complex road conditions. Further, edge point extraction: the colors of lanes are divided into two types: yellow and white, which form a sharp contrast with the gray road surface. The mutation characteristics of the associated pixel values ​​are used to extract edge points. It is not necessary to extract each edge point. The fast line scanning method is used to scan and extract edge points. At the same time, the lane line information is mainly in the lower half of the image. The region of interest is set by the vanishing point method. According to the characteristics of the lane in the image, the detection area is reset into two parts: near domain and far domain. The proportion of near domain lane lines in the image is large, and the whole line is scanned; the width between the left and right lane lines in the far domain is narrow and concentrated in the middle of the image. Only most of the middle area is scanned. The near domain lane line features are obvious, and interlaced scanning is used to retain more lane information; the far domain lane information is weak and there is a lot of interference information. Every three lines are scanned to avoid excessive interference;

[0028] The edge intensity of each pixel is calculated using formula 1:

[0029]

[0030] Where: E(n,i) represents the edge intensity of the pixel at the nth row and the ith column, L represents the filter length, which is 8 according to experience. When the left edge of the lane line is scanned, a maximum value will appear, and when the right edge of the lane line is scanned, a minimum value will appear in the edge intensity. According to the maximum and minimum values, the edge points are divided into two categories, and the custom parameters are used to detect the straight line respectively.

[0031] Based on the queue, the value information of the previous edge intensity is fully utilized to obtain the value of the next edge intensity. The edge intensity of two adjacent points only needs to access the distance from the endpoint of the filter length and the values ​​of the two points themselves, which is reduced from the original 16 operations to 4 operations, as shown in Formula 2:

[0032] E(n,j+1)=E(n,j)+I(n,jL)+I(n,j+L+1)-I(n,j)-I(n,j+1) Equation 2

[0033] Where: I(n,i) represents the pixel value of the pixel at the nth row and the ith column;

[0034] The quick noise inclusion filter method is used to enhance the near-domain image part; the noise inclusion filter model relies on partial differential equations and image discretization, so that a beating signal is obtained at the inflection point of the image. 0 (x,y) after filtering process u 0 (x,y,t),t>0, to achieve the best enhancement effect u(x,y,t 1 ),u ηη Represents the second-order directional derivative of the image gradient direction n, The image gradient is the image gradient. The Gaussian function is introduced to smooth and denoise the second-order directional derivative of the image gradient. At the same time, the forward diffusion process is introduced, that is, the term perpendicular to the image gradient direction is introduced into the equation to denoise the image sharpening enhancement process. The specific equation of the model is as follows:

[0035]

[0036] Where η, ξ represent the directions parallel and perpendicular to the image gradient, * represents convolution, G σ represents the Gaussian function, c ξ represents a constant whose value is positive, u ξξ represents the forward diffusion process, the first term of Equation 3 plays an enhancement role, and the following term denoises the enhancement process;

[0037] To simplify the calculation, based on the edge features of road markings, the second term in Equation 3 is removed:

[0038]

[0039] The model has a better enhancement effect on larger edge areas in the image, and the enhanced image is more conducive to edge point detection.

[0040] A sensitive and accurate real-time warning system for lane deviation in complex road conditions. Furthermore, a scoring rule is established: the amount of geometric Vight calculation is reduced, and the scoring rule is established directly using the edge points of the original image without the need for inverse perspective transformation;

[0041] The coordinate point (k, θ) corresponds to a straight line in the image coordinate system. Each line in the image coordinate system corresponds to another spatial coordinate point (k, θ). In this application, all edge points are first calculated and a scoring rule is obtained. Then, the parameter space is scored in two steps: each edge point is traversed within the set range of angle θ, the corresponding offset k is calculated, and the edge line is obtained by comparing the scores;

[0042] (1) Traverse all edge points of the left and right regions with tilt angles θ, calculate the corresponding offset k, and then increase the score of (k, θ) by 1. The value ranges of θ and k are set based on experience. The value range of angle θ is [20 degrees, 70 degrees], and the value range of k is [0, 840].

[0043] (2) Compare the scores corresponding to the parameter pairs (k, θ). Each (k, θ) represents a straight line. The higher the score of the (k, θ) pair, the more edge points there are on the straight line, and the more likely it is a lane line. The algorithm retains the edge lines that have scores that reach the threshold.

[0044] Each point is mapped to the coordinate space (k, θ) with 50 lines. Among the numerous lines, the local maximum score retention method is adopted to ensure that unnecessary lines are removed and lines that may be lane lines are retained;

[0045] First, in each offset, traverse all θ, find the line with the highest score and save it. When removing unnecessary lines, avoid removing the edge lines of the lane line. Then, judge whether the lines in the left and right areas intersect each other. The judgment method is as follows: 1 and k 2 Represents the offset between two straight lines, k 3 and k 4 represents the offset of the intersection point of the two straight lines with the top of the region of interest. If (k 1 -k 2 )·(k 4 -k 3 )<0, it means that the two straight lines intersect, and the line with the lower score is removed.

[0046] A sensitive and accurate real-time warning system for lane deviation under complex road conditions. Further, the lane line detection verification value is obtained: the angle difference between the two lines on the left and right edges of the structured road lane is not too large, and the offset width of the two lines is fixed. According to these two features, candidate lane lines are matched from more straight lines. In the process of edge point detection, the edge points are divided into two categories, maximum and minimum. In the process of detecting straight lines based on edge points, the straight lines are further divided into the maximum, i.e., the straight line corresponding to the left edge of the lane line, and the minimum, i.e., the straight line corresponding to the right edge of the lane line. These two types of lines are matched in pairs according to parallelism and offset to obtain candidate lane lines.

[0047] After obtaining the candidate lane lines, the lane lines tend to be in the middle of the image, so the edge points that can be detected are denser and have more scores. The scores of the left and right sides of the lane lines are summed to obtain the two lane lines with the highest scores. If the lane line offset with the highest score meets the threshold range, it is determined to be the desired lane line.

[0048] A sensitive and accurate real-time warning system for lane deviation in complex road conditions. Further, the image Wr extended dynamic evaluation parameter estimation tracking: the image Wr extended dynamic evaluation is used to estimate and update parameters. The output result is used as the priority detection range for the next frame. The model is associated with the lane line angle θ, offset k, average intensity E of the lane line, and custom lane line curvature representation S. The average edge intensity E of the lane line edge points and a variable S representing the curvature of the lane line are used. The search line is determined according to the angle and offset corrected by the image Wr extended dynamic evaluation. The variable S of the curvature of the lane line is used to determine the points within a certain range of the line, thereby searching for the edge points of the line. According to the edge intensity E of the line, the triple standard deviation method is used to remove abnormal points in the edge points to obtain an accurate lane line edge point.

[0049] The system model established in this application using the lane line angle θ, offset k, average lane line intensity E and custom lane line curvature representation S is as follows:

[0050]

[0051] They represent the lane line angle θ, the offset k, the average intensity E of the lane line, and the speed at which the custom lane line curvature representation S changes. The state vector of the image Wr extended dynamic evaluation is defined as:

[0052]

[0053] The state transfer matrix is:

[0054]

[0055] At the initial moment, the state vector is set to:

[0056] x(0)=[k(0),θ(0),E(0),S(0),0,0,0,0] T Formula 8

[0057] k(0), θ(0), E(0), S(0) represent the lane line information obtained from the first frame image. Assume that w 1 (i), w 2 (i), w 3 (i), w 4 (i) is an independent statistic, assuming that the system noise is:

[0058]

[0059] Detection verification vector:

[0060] z(i)=[k(i), θ(i), E(i), S(i)] T Formula 10

[0061] The measurement equation is:

[0062] z(k)=Hx(k)+V(k) Formula 11

[0063] According to the relationship between z(k) and x(k), the measurement matrix is:

[0064]

[0065] The measurement noise vector is:

[0066]

[0067] The impact of detection verification noise is greater than system noise. Under the same assumption v 1 (k), v 2 (k) Statistical independence:

[0068]

[0069] To ensure the accuracy of the estimation, a larger value is assigned to P(t) at the beginning:

[0070]

[0071] Estimate and update parameters. The parameters involved in road modeling include lane curvature, pitch angle, horizontal tilt angle, lane width and lateral deviation of motor vehicles. Parameter estimation includes two parts. The first part is to estimate the road based on the information of the road lane line, and the second part is the position parameters, focal length and road environment information of the camera.

[0072] In order to better utilize the inter-frame information to achieve stability, a control variable is set when the image Wr is extended and dynamically evaluated. When the value of the counter count is greater than or equal to 3, the output result is the final result of this frame, otherwise the result of this frame needs to be processed separately. The rule for counter accumulation is: if the detection value of the current frame matches the output result value of the image Wr extended dynamic evaluation of the previous frame, the counter is increased by 1, and the maximum value is 3; when it does not match, the counter is set to 0, and the dynamic evaluation output result is not used as the final detection result.

[0073] In addition, the algorithm sets up two other counters, which are accumulated according to the detection verification values. If the detection verification values ​​of the previous and next frames match, the counter is increased by 1, with a maximum upper limit of 8, otherwise it is decreased by 1; when the value of the counter reaches 8, it corresponds to a mark, and the mark is set to 1 at this time, and the default setting is 0; after the mark is 1, if there are three consecutive needles that are different, the mark is set to 0; if the left and right marks are both 1, the current left and right lane information is recorded. When the counter of one lane line is less than 8 and the counter of the other lane line is 8, it means that the lane line on one side is stable, and the lane line on the other side is blocked or scratched by the car; according to the invariance of lane width, when the position of one lane line remains unchanged, the position of the other lane line will not change. At this time, the originally matched lane information is taken out as the lane information of the current frame to enhance the algorithm's anti-interference ability.

[0074] A sensitive and accurate real-time warning system for lane deviation under complex road conditions. Further, lane edge sensitive extraction: after obtaining the corrected parameters, the detected lane line angle θ and offset k are used to determine the position range of the edge point search, and the custom lane line curvature representation S is used to determine the search range; the algorithm sets a trapezoidal area search range, and this trapezoid takes the straight line determined by the angle θ and offset k as the center line. In the bottom area of ​​the image lane, the angle and offset of the lane line reflect the lane edge, and the search range is small, which is five pixels around the line. At the farthest point of the area of ​​interest, the search range is set to S+5 pixels around the line; the search range gradually increases from near to far in the middle part; in the formed edge point concentration, other noise points are included, such as points formed by puddle on the road surface, lane wear and deformation, etc. This application uses the triple standard deviation method to remove abnormal points based on the average intensity E of the edge points estimated by the extended dynamic evaluation of the image Wr;

[0075] Assume that the number of edge points detected on the left edge of the lane line of a certain frame is n, and the intensity of the edge points is as follows: X 1 , X 2 , X 3 ...X n , then the standard deviation is calculated using the average edge intensity:

[0076]

[0077] Take the values ​​within the range of E±3σ as the retained values, and remove the values ​​outside this range; after removal, recalculate the standard deviation based on the remaining data, and then remove the outliers; repeat this cycle until there are no outliers; connect the lane edge points finally obtained into lines to form the final lane line;

[0078] Finally, according to the obtained lane line edge position, the pixel values ​​of the three channels of the lane line RGB are obtained. According to the characteristic that the projection components of white and yellow in the B channel are the most different, B=88 is taken as the critical value. If it is greater than 88, it is judged as white, otherwise it is judged as yellow; add a judgment condition to distinguish based on the ratio of the red component R to the blue component B of the RGB color channel. If R / B>0.65, it is judged as white, otherwise it is judged as yellow.

[0079] A sensitive and accurate real-time warning system for lane deviation in complex road conditions. Further, a warning strategy based on the relative distance between the vehicle and the lane is used: OABC represents the image after lane line recognition, HKI represents the set area of ​​interest, and point P 1 (x 1 ,y 1 ), P 2 (x 2 ,y 2 ), P 3 (x 3 ,y 3 ), P 4 (x 4 ,y 4 ) indicates the intersection of the left and right lane lines and the HKIJ area, point C 1 (x 5 ,0),C 2 (x 6 ,0) represents the boundary points of the motor vehicle at the bottom of the image. The lane width is set to 3.75m, and the width of the small passenger car is set to 1.8m. Let L be the pixel width of the motor vehicle in the image, and let L be the pixel width of the motor vehicle in the image. According to the ratio of lane width to motor vehicle width r = L C / L D =1.8 / 3.75,L C =x 4 -x 2 , L D =x 4 -x 2 , according to the straight test video sequence to obtain L D The pixel width is reversed to get C 1 (x 5 ,0),C 2 (x 6 ,0) to obtain the relative distances between the left and right lane lines and the left and right sides of the motor vehicle:

[0080] d l =x 5 -x 2 Formula 17

[0081] d r =x 4 -x 6Formula 18

[0082] Convert the relative distances between the lane lines on the left and right sides and the left and right sides of the motor vehicle into proportions:

[0083]

[0084]

[0085] If R r <0.33, it is judged as right-side deviation. l If the yaw angle is less than 0.33, it is judged as a left deviation. The deviation judgment does not require complex camera calibration operations. This warning strategy based on relative distance is more effective when the yaw angle of the motor vehicle is smaller. When the yaw angle is relatively large, this application adopts a strategy based on the deviation rate of the lane line for warning.

[0086] A sensitive and accurate real-time warning system for lane deviation under complex road conditions. Further, a sensitive warning strategy based on lane deviation rate is developed: the lane deviation rate warning is based on detecting the yaw angle of the moving vehicle. If the yaw angle is greater than the critical value, it means that the lane line is about to deviate. At this time, an early warning is required to derive the deviation rate:

[0087]

[0088] k represents the slope of the lane line, x=tanθ represents the tangent value corresponding to the deviation angle, and when the vehicle is traveling in a straight line along the road, x=0; by setting a threshold deviation angle, the threshold deviation rate when the vehicle deviates from the lane is obtained; in order to achieve real-time processing results, the response time of the vehicle is considered and set to between 0.2s and 0.9s; the time for the program to process each frame is less than or equal to 0.1s; so when the lane deviates, the total time required for detection and correction is t=0.9+0.1=ls, and when the lane does not deviate:

[0089]

[0090] d is 1.025m, v is 40km / s, and the obtained deviation angle threshold is 9 degrees; if the deviation angle is less than or equal to 9 degrees, it is not considered that the deviation occurs; otherwise, it is considered that the deviation occurs; the parameters obtained in this application include the inclination angle of the lane line, so the simplified formula is:

[0091]

[0092] Among them, θ l Represents the inclination angle of the currently detected left lane line, Represents the average value of the left and right lane angles when the lane is straight and not deviating, that is:

[0093]

[0094] if And θ l >θ r , then it is considered that right deviation occurs; if And θ r >θ l , then it is considered that left deviation occurs; if It is deemed that no deviation has occurred.

[0095] Compared with the prior art, the innovations and advantages of this application are:

[0096] First, this application is based on the practical application of developing intelligent assisted driving systems. Aiming at the characteristics of the ARM platform, this application designs a very fast lane deviation real-time warning system on the ARM platform with Rockchip RK3288 processor as the core. The system mainly includes two parts: lane line detection part and lane deviation warning part. This application proposes a fast lane information extraction algorithm of image Wr extended dynamic evaluation, which can detect lane information on the ARM platform at a very fast speed. On this basis, considering the lateral speed of the motor vehicle in the lane, it is proposed to combine two warning strategies. The first one uses the convenient relative distance between the motor vehicle and the lane for warning judgment. This processing strategy has better results when the lateral speed is not large. When the lateral speed is relatively large, that is, when the angle between the vehicle head direction and the center line of the road surface is relatively large, the lane deviation judgment is made by using the deviation rate of the left and right lane lines in the image, which can achieve better results. The algorithm of this application does not need to perform perspective transformation on the image, so it does not need to measure the external and internal parameters of the monocular camera, and has low requirements for the monocular camera, which brings great convenience to users. At the same time, the algorithm time complexity is much smaller than others. It can be detected in real time on the ARM platform of Rockchip RK3288 development board. It has fast speed, good effect, strong practicality and applicability. It has great value and a broad market for real-time warning of lane deviation in complex road conditions.

[0097] Second, the hardware platform of this application adopts the low-power ARM platform as the hardware basis, and adopts the powerful and economical RK3288 development board as the hardware basis of the algorithm. Through the ordinary camera to collect data, the RK3288 image processing unit, image acquisition unit, alarm display unit and control unit CAN signal part are designed to develop a set of lane line detection and timely deviation warning algorithms with good effect and real-time operation. The real-time road conditions of the driving process are obtained through the CCD camera above the cab. The road condition images obtained are processed by the RK3288 image processing unit. According to the identified lane line information, it is determined whether it is necessary to send an alarm signal to the alarm unit to inform the driver to pay attention to lane deviation. The hardware design is economical and practical, with low cost. It has good stability and real-time performance when used in the vehicle system, low hysteresis and failure rate, and effectively reduces traffic accidents and casualties. At the same time, the update and use cost is low, the application range is wide, the system size is small, and the installation is convenient. With the lane deviation real-time warning algorithm of this application, it has strong adaptability to complex road conditions and is truly used in the vehicle system to realize real-time detection and warning, which is of great significance and great practical value.

[0098] Third, the present application proposes an extremely fast lane line recognition algorithm based on Wr extended dynamic evaluation of images, which uses line scanning to extract edge points, and classifies edge points into two categories for custom parameter scoring to obtain straight lines, obtains candidate lane lines according to lane line models, uses Wr extended dynamic evaluation of images to estimate and update road parameters, uses inter-frame information to enhance system stability and robustness, and uses parameters output by Wr extended dynamic evaluation of images to extract final edge points, overcoming the characteristics that traditional methods can only detect lane straight lines but cannot reflect the degree of lane curvature. The algorithm abandons the conversion of various coordinate systems and eliminates the complicated steps of obtaining camera internal and external parameters, installation location, and motor vehicle model when applied in engineering. In addition to achieving real-time detection, the algorithm also has relatively high accuracy and robustness, and still has good results in various complex situations. The algorithm has low time complexity and is real-time under the ARM platform. It can accurately realize lane departure warning function in good or complex road conditions. At the same time, it can also accurately detect the virtual and real lanes and yellow and white lines, and realize real-time and accurate recognition of lane lines in complex road conditions.

[0099] Fourth, this application proposes a real-time lane deviation warning method. In order to speed up the calculation speed and simplify the operation, the obtained vehicle-mounted image is directly processed without perspective transformation or other processing involving image pixel coordinate conversion. Therefore, there is no need to calibrate the camera, which greatly reduces the trouble of actual use. If the deviation warning is realized by building a road model, it violates the original intention of the algorithm design of this application. Therefore, this application considers the lateral speed of the motor vehicle in the lane and integrates two warning strategies. The first one uses the convenient relative distance between the motor vehicle and the lane for warning judgment. This processing strategy has better results when the lateral speed is not large; when the lateral speed is large, that is, when the angle between the vehicle head direction and the center line of the road surface is large, the deviation rate of the left and right lane lines in the image is used to judge the lane deviation, which can achieve better results, avoid various complex parameters that need to be obtained by using the lane model, greatly enhance the scope of application in engineering, and have a fast detection speed, which is more suitable for application on vehicle-mounted platforms such as ARM. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 This is the architecture diagram of the lane departure real-time warning system development platform.

[0101] Figure 2 This is the lane line extreme speed detection algorithm lane line extreme speed detection algorithm block diagram.

[0102] Figure 3 It is the image Wr extended dynamic evaluation lane line recognition update flow chart.

[0103] Figure 4 This is a comparison chart before and after the near-domain image is enhanced by the quick noise inclusion filter.

[0104] Figure 5 It is a schematic diagram of the parameter space of the scoring rules and the cross-line judgment.

[0105] Figure 6 It is a schematic diagram of the lane imaging plane coordinate system for warning the relative distance between a motor vehicle and a lane.

[0106] Figure 7 This is a diagram of the recognition effects of each test lane based on the ARM platform.

[0107] Figure 8 This is a good recognition result diagram of each test road based on the ARM platform.

[0108] Fig. 9 This is a diagram of complex road condition recognition results based on various tests on the ARM platform.

[0109] Fig.10 This is a schematic diagram of lane deviation warning for each test based on the ARM platform.

[0110] Fig.11This is a performance comparison chart of this application with the RANSAC algorithm and the extended Kalman algorithm.

[0111] Fig.12 This is a comparison chart of straight line recognition between this application and the RANSAC algorithm and the extended Kalman algorithm.

[0112] Fig.13 This is a comparison chart of turn recognition between this application and the RANSAC algorithm and the extended Kalman algorithm. Specific implementation methods

[0113] The following, in conjunction with the accompanying drawings, further describes the technical solution of the sensitive and accurate real-time warning system for lane departure under complex road conditions provided by the present application, so that technicians in this field can better understand the present application and implement it.

[0114] The automobile industry is developing rapidly, and at the same time, road traffic accidents are frequent. Among them, the highest incidence of traffic accidents is caused by lane departure due to driver visual fatigue or inattention. Since the ARM platform is economical and practical, the lane departure real-time warning system based on the ARM platform can effectively reduce traffic accidents and casualties; it also has the characteristics of low cost and wide applicability. Therefore, the development of a lane departure warning technology based on the ARM platform is of great significance and great practical value.

[0115] The present application proposes an extremely fast lane line recognition algorithm based on Wr extended dynamic evaluation of images, which detects lane lines without inverse perspective transformation, and estimates parameters of the established lane parameter model using Wr extended dynamic evaluation of images, so as to realize real-time detection of lane lines on ARM platforms with relatively weak computing power; uses lane color information to distinguish yellow and white lane lines, and uses edge point spacing to distinguish virtual and real lane lines; performs lane departure judgment based on two strategies: the relative distance between the motor vehicle and the lane, and the offset rate of the lane line in the image; the system first captures the road image through an ordinary monocular camera, sets the area of ​​interest, and uses RGB color space information to remove useless image information such as roadside grass; then detects edge points by horizontally scanning pixel points; then extracts lane lines using a simple custom parameter space scoring; establishes a lane parameter model, uses Wr extended dynamic evaluation of images to estimate parameters, and uses edge point color and spacing information to distinguish virtual and real yellow and white lane lines.

[0116] In order to realize a lane departure real-time warning system with low cost, fast speed, good effect, practicality and strong applicability, this application adopts the ARM platform with low power consumption, strong functions and good economy as the hardware basis. Among the many ARM series products, after systematic evaluation, the RK3288 development board is used as the hardware basis of the algorithm. Through the data collection of ordinary cameras, a set of lane line detection and timely deviation warning algorithms with good effect and real-time operation are developed, realizing a lane departure real-time warning system with low cost, fast speed, good effect, practicality and strong applicability.

[0117] 1. Lane Departure Real-time Warning System Development Platform and Hardware Design

[0118] The system mainly includes RK3288 image processing unit, image acquisition unit, alarm display unit and other control unit CAN signal parts. Its architecture diagram is as follows Figure 1 As shown in the figure, when the lane departure warning system is working, the real-time road conditions of the driving process are obtained through the CCD camera above the cab. The acquired road condition images are processed by the RK3288 image processing unit, and it is determined whether an alarm signal needs to be sent to the alarm unit according to the identified lane line information to inform the driver to pay attention to the lane departure.

[0119] The system is installed on a motor vehicle and uses the small-sized and high-performance Firefly-RK3288 development board. The development board uses the Rockchip RK3288 processor, supports Android 4.4 and Ubuntu dual systems, and HDMI2.04K@60Hz output. The quad-core processor provides good computing power. Its support for the Ubuntu system is the system platform required for this application, and its support for the Android system provides a good platform for porting the algorithm to mobile devices.

[0120] 2. Extreme Lane Line Recognition Algorithm Based on Image Wr Extended Dynamic Evaluation

[0121] This application proposes a fast lane line feature optimization modeling method, which uses image Wr extended dynamic evaluation to estimate and correct in combination with road features, stably and quickly detects lane information in complex road conditions, removes the perspective transformation calculation module with high time complexity used in the prior art algorithm, uses a simplified geometric Vight detection straight line as the basis, combines edge point intensity and lane edge parallel relationship features to select lane lines with high credibility, and finally uses image Wr extended dynamic evaluation to estimate parameters such as angle, offset, average edge intensity and custom curvature.

[0122] 1. Lane Line Speed ​​Detection Algorithm Architecture

[0123] Lane line speed detection algorithm Lane line speed detection algorithm block diagram is as follows Figure 2 As shown, it is mainly divided into seven steps:

[0124] Step 1: Obtaining vehicle images: Obtaining road images from the camera above the cab;

[0125] Step 2: Near-field image enhancement: enhance the degree of edge point mutation;

[0126] Step 3: Obtain edge points: Use line scanning to obtain edge points, and divide the edge points into four categories, which are the maximum and minimum values ​​on the left and right sides of the image;

[0127] Step 4: Establish the evaluation and scoring rules: Create a table for the two types of edge points based on the custom parameter distribution, and obtain candidate lane lines based on the scores;

[0128] Step 5: Acquisition of lane lines and edge points: The measured value of the lane line of the current frame is obtained by combining the relationship between frames, and the abnormal points on the edge of the lane line are removed by using the triple standard deviation method to enhance the robustness of the algorithm.

[0129] Step 6: Image Wr extended dynamic evaluation for estimation and correction: establish a lane parameter model, use image Wr extended dynamic evaluation for parameter estimation, and use the lane line position between frames to eliminate interference;

[0130] Step 7: According to the updated state value, obtain the final lane line edge points and connect them into lane lines.

[0131] In step 6, it is necessary to compare the current frame detection verification value with the predicted value of the previous frame using the image Wr extended dynamic evaluation to see if they match. If they match, the set counter value count is increased by 1. If the value of count is greater than or equal to 3, the output of the optimal estimated value is the current lane line position information; if they do not match, proceed to the next frame, and the output of the current frame value needs to be judged separately. The specific process diagram is as follows Figure 3 shown.

[0132] (II) Edge point extraction

[0133] There are two colors of lanes: yellow and white, which form a sharp contrast with the gray road surface. Therefore, the mutation characteristics of the associated pixel values ​​are used to extract edge points. In order to reduce the amount of calculation, it is not necessary to extract each edge point. Fast line scanning is used to scan and extract edge points. At the same time, the upper half of the image is mainly sky and house information, and the lane line information is mainly in the lower half of the image. The region of interest is set by the vanishing point method. According to the characteristics of the lane in the image, the detection area is reset into two parts: near domain and far domain. The proportion of near domain lane lines in the image is large. In order to prevent information omission, the whole line scan is used; the width between the left and right lane lines in the far domain is narrow and concentrated in the middle of the image. Only most of the middle area is scanned. The near domain lane line features are obvious, and interlaced scanning is used to retain more lane information; the far domain lane information is weak and there is a lot of interference information. Every three lines are scanned to avoid excessive interference.

[0134] The edge intensity of each pixel is calculated using formula 1:

[0135]

[0136] Where: E(n,i) represents the edge intensity of the pixel point in the nth row and the ith column. L represents the filter length, which is 8 according to experience. When the left edge of the lane line is scanned, a maximum value will appear. When the right edge of the lane line is scanned, the edge intensity will have a minimum value. According to the maximum and minimum values, the edge points are divided into two categories, and custom parameters are used to detect straight lines.

[0137] In order to reduce the amount of calculation, the queue is used to make full use of the value information of the previous edge intensity to obtain the value of the next edge intensity. The edge intensity of two adjacent points only needs to access the distance between the endpoints of the filter length and the values ​​of the two points themselves, which is reduced from the original 16 operations to 4 operations, as shown in Formula 2:

[0138] E(n,j+1)=E(n,j)+I(n,jL,)+I(n,j+L+1)-I(n,j)-I(n,j+1) Equation 2

[0139] Where: I(n,i) represents the pixel value of the pixel at the nth row and the ith column;

[0140] In actual detection, the acquired images contain various noises, especially when the camera is used for a long time or the light is weak, the acquired images are blurry, such as Figure 4 At this time, the edge points will be fuzzy, the intensity will be weak, and it is easy to fail to detect. This application uses a quick noise inclusion filtering method to enhance the near-domain image part.

[0141] The noise inclusion filter model relies on partial differential equations and the discretization of the image, so that a beating signal is obtained at the inflection point of the image. 0 (x,y) after filtering process u 0 (x,y,t),t>0, to achieve the best enhancement effect u(x,y,t 1 ),u ηη Represents the second-order directional derivative of the image gradient direction n, The image gradient is the image gradient. The Gaussian function is introduced to smooth and denoise the second-order directional derivative of the image gradient. At the same time, the forward diffusion process is introduced, that is, the term perpendicular to the image gradient direction is introduced into the equation to denoise the image sharpening enhancement process. The specific equation of the model is as follows:

[0142]

[0143] Where η, ξ represent the directions parallel and perpendicular to the image gradient, * represents convolution, G σ represents the Gaussian function, c ξ represents a constant whose value is positive, u ξξ It represents the forward diffusion process. The first term of Equation 3 plays an enhancement role, and the following term denoises the enhancement process.

[0144] To simplify the calculation, based on the edge features of road markings, the second term in Equation 3 is removed:

[0145]

[0146] The model has a better enhancement effect on the larger edge areas in the image. The enhanced image is more conducive to edge point detection. The enhancement effect is as follows Figure 4 shown.

[0147] (III) Establishing Judging and Scoring Rules

[0148] In order to reduce the amount of calculation of the geometric Vight and adapt to the use of the subsequent image Wr expansion dynamic evaluation parameters, this application directly uses the edge points of the original image to establish the evaluation and scoring rules without the need for inverse perspective transformation.

[0149] The coordinate point (k,θ) corresponds to the straight line in the image coordinate system, such as Figure 5 As shown in (a), each line in the image coordinates corresponds to another spatial coordinate point (k, θ). In this application, a table is first built and calculated for all edge points to obtain a scoring rule, and then the parameter space is scored in two steps: each edge point is traversed within the set angle θ, the corresponding offset k is calculated, and the edge line is obtained by comparing the scores.

[0150] (1) Traverse all the edge points in the left and right side regions for the tilt angle θ, calculate the corresponding offset k, and then increment the score for (k, θ) by 1. The value ranges of θ and k are set according to experience. The value range of the angle θ is [20 degrees, 70 degrees], and the value range of k is [0, 840];

[0151] (2) Compare the scores corresponding to the parameter pairs (k, θ). Each (k, θ) represents a straight line. The higher the score of the (k, θ) pair, the more edge points there are on this straight line, and the greater the possibility that it is a lane line. In the algorithm, the edge lines with scores reaching the threshold are retained;

[0152] Each point is mapped to 50 lines in the coordinate space (k, θ). Among numerous lines, the local maximum score retention method is adopted to ensure removing unnecessary lines and retaining the lines that may be lane lines.

[0153] First, in each offset, traverse all θ, find the line with the highest score and save it. According to the road characteristics, many interference lines on the road surface are parallel or approximately parallel to the lane lines. Therefore, when removing unnecessary lines in this step, avoid removing the edge lines of the lane lines. After that, to further reduce interference lines, judge whether the lines in the left and right side regions intersect pairwise. The judgment method is as follows: As shown in Figure 5 (b), k 1 and k 2 represent the offsets of two straight lines, k 3 and k 4 represent the offsets of the intersection points of two straight lines with the top of the region of interest. If (k 1 - k 2 )·(k 4 - k 3 ) < 0, it means the two straight lines intersect, and remove the line with the lower score.

[0154] (4) Obtain the lane line detection verification value

[0155] According to the structured road characteristics, the angles of the two lines at the left and right edges of the lane do not differ much. At the same time, the offset widths of the two lines are fixed. Based on these two characteristics, candidate lane lines are matched among many straight lines. During the edge point detection process, the edge points are divided into two categories: maximum values and minimum values. During the process of detecting straight lines based on edge points, the straight lines are further divided into straight lines corresponding to maximum values (left edge of the lane line) and straight lines corresponding to minimum values (right edge of the lane line). These two types of lines are pairwise matched according to the parallelism and offset to obtain candidate lane lines.

[0156] After obtaining the candidate lane lines, the lane lines tend to be in the middle of the image, so the edge points that can be detected are denser and have more scores. The scores of the left and right sides of the lane lines are summed to obtain the two lane lines with the highest scores. If the lane line offset with the highest score meets the threshold range, it is determined to be the desired lane line.

[0157] (V) Image Wr extension dynamic evaluation parameter estimation tracking

[0158] The image Wr extended dynamic evaluation is used for parameter estimation and update, and the output result is used as the priority detection range of the next frame, which greatly improves the detection efficiency. The image Wr extended dynamic evaluation parameter estimation model proposed in this application is associated with the angle θ, offset k, average intensity E of the lane line, and the custom lane line curvature representation S. The average edge intensity E of the edge point of the lane line is used, and a variable S representing the curvature of the lane line is introduced. The search line is determined according to the angle and offset corrected by the image Wr extended dynamic evaluation. The variable S of the curvature of the lane line is used to determine the points within a certain range of the line, so as to search for the edge point of the line. There are various interferences on the road surface, and there will be some interference points in the process of obtaining the edge point of the line, such as small potholes and puddles on the road surface. Based on the edge intensity E of the line, this application uses the triple standard deviation method to eliminate abnormal points in the edge points to obtain an accurate lane line edge point.

[0159] The system model established in this application using the lane line angle θ, offset k, average lane line intensity E and custom lane line curvature representation S is as follows:

[0160]

[0161] They represent the lane line angle θ, the offset k, the average intensity E of the lane line, and the speed at which the custom lane line curvature representation S changes. The state vector of the image Wr extended dynamic evaluation is defined as:

[0162]

[0163] The state transfer matrix is:

[0164]

[0165] At the initial moment, the state vector is set to:

[0166] x(0)=[k(0), θ(0), E(0), S(0), 0, 0, 0, 0] T Formula 8

[0167] k(0), θ(0), E(0), S(0) represent the lane line information obtained from the first frame image. Assume that w1 (i), w 2 (i), w 3 (i), w 4 (i) is an independent statistic, assuming that the system noise is:

[0168]

[0169] Detection verification vector:

[0170] z(i)=[k(i), θ(i), E(i), S(i)] T Formula 10

[0171] The measurement equation is:

[0172] z(k)=Hx(k)+V(k) Formula 11

[0173] According to the relationship between z(k) and x(k), the measurement matrix is:

[0174]

[0175] The measurement noise vector is:

[0176]

[0177] The impact of detection verification noise is greater than system noise. Under the same assumption, v 1 (k), v 2 (k) Statistical independence, then:

[0178]

[0179] To ensure the accuracy of the estimation, a larger value is assigned to P(t) at the beginning:

[0180]

[0181] Estimate and update the parameters. The parameters involved in road modeling include lane curvature, pitch angle, horizontal inclination angle, lane width and lateral deviation of motor vehicle. Parameter estimation includes two parts. The first part is to estimate the road based on the information of the road lane line, and the second part is the camera's position parameters, focal length and road environment information.

[0182] In the actual detection process, in order to better utilize the inter-frame information to achieve stability, a control variable is set when the image Wr is extended to dynamically evaluate the output. When the value of the counter count is greater than or equal to 3, the output result is the final result of this frame, otherwise the result of this frame needs to be processed separately; the counter accumulation rule is: if the detection value of the current frame matches the output result value of the image Wr extended dynamic evaluation of the previous frame, the counter is increased by 1, and the maximum is 3; when it does not match, the counter is set to 0, and the dynamic evaluation output result is not used as the final detection result. The process flow chart is as follows Figure 3 shown.

[0183] In addition, the algorithm sets up two other counters, which are accumulated according to the detection verification values. If the detection verification values ​​of the previous and next frames match, the counter is increased by 1, with a maximum upper limit of 8, otherwise it is decreased by 1; when the value of the counter reaches 8, it corresponds to a mark, and the mark is set to 1 at this time, and the default setting is 0; after the mark is 1, if there are three consecutive needles that are different, the mark is set to 0; if the left and right marks are both 1, the current left and right lane information is recorded. When the counter of one lane line is less than 8 and the counter of the other lane line is 8, it means that the lane line on one side is stable, and the lane line on the other side is blocked or scratched by the car; according to the invariance of lane width, when the position of one lane line remains unchanged, the position of the other lane line will not change. At this time, the originally matched lane information is taken out as the lane information of the current frame to enhance the algorithm's anti-interference ability.

[0184] (VI) Lane edge sensitive extraction

[0185] To overcome the problem that traditional methods cannot well reflect the information of curved lane lines, after obtaining the corrected parameters, the algorithm uses the detected lane line angle θ and offset k to determine the position range of the edge points to search for, and uses a custom lane line curvature representation S to determine the search range; the algorithm sets a trapezoidal area search range, and this trapezoid takes the straight line determined by the angle θ and offset k as the center line. In the bottom area of ​​the image lane, the angle and offset of the lane line reflect the lane edge, and the search range is smaller, which is five pixels around the line. At the farthest point of the area of ​​interest, the search range is set to S+5 pixels around the line; in the middle part, the search range gradually increases from near to far; in the formed edge point concentration, other noise points are included, such as points formed by puddles on the road surface, lane wear and deformation, etc. This application uses the triple standard deviation method to remove abnormal points based on the average intensity E of the edge points estimated by the extended dynamic evaluation of the image Wr.

[0186] Assume that the number of edge points detected on the left edge of the lane line of a certain frame is n, and the intensity of the edge points is as follows: X 1 , X 2 , X3 ...X n , then the standard deviation is calculated using the average edge intensity:

[0187]

[0188] Take the values ​​within the range of E±3σ as the retained values, and remove the values ​​outside this range; after removal, recalculate the standard deviation based on the remaining data, and then remove the outliers; repeat this cycle until there are no outliers; connect the lane edge points finally obtained into lines to form the final lane line.

[0189] Finally, according to the obtained lane line edge position, the pixel values ​​of the lane line RGB channels are obtained. According to the characteristic that the projection components of white and yellow in the B channel are the most different, B=88 is taken as the critical value. If it is greater than 88, it is judged as white, otherwise it is judged as yellow. After testing, this method has a good effect in bright environments, but in dark environments such as tunnels, white will be mistakenly judged as yellow. Therefore, a judgment condition is added to distinguish based on the ratio of the red component R to the blue component B of the RGB color channel. If R / B>0.65, it is judged as white, otherwise it is judged as yellow. Experiments have proved that this discrimination method has a good effect.

[0190] 3. Lane Departure Real-time Warning Method

[0191] After accurately determining the position information of the lane line, the relative position information of the motor vehicle needs to be obtained, and the various possibilities of the motor vehicle deviating from the lane are calculated through the deviation strategy; in the actual warning process, the warning method based on the lane model and the real road surface coordinate information is relatively weak in adaptability. To establish the geometric imaging coordinate system of the motor vehicle system, camera, and road surface, it is necessary to know the model size of the motor vehicle, the road type, the rotation angle and height of the camera and the optical lens, the internal parameters and distortion parameters of the camera. Every time a car, road model or camera is changed, the coordinate system must be re-established, which is relatively troublesome in engineering. At the same time, in order to speed up the calculation speed and ease of operation, the algorithm of this application directly processes the obtained vehicle-mounted image without processing involving image pixel coordinate conversion such as perspective transformation, so there is no need to calibrate the camera, which greatly reduces the trouble of actual use. If the deviation warning is realized through the establishment of the road model, it violates the original intention of the design of the algorithm of this application. Therefore, the present application considers the lateral speed of the motor vehicle in the lane and integrates two warning strategies. The first one uses the convenient relative distance between the motor vehicle and the lane for warning judgment. This processing strategy has better results when the lateral speed is not large; when the lateral speed is large, that is, when the angle between the front direction of the vehicle and the center line of the road surface is large, the lane departure judgment is performed by using the offset rate of the left and right lane lines in the image, which can achieve better results.

[0192] 1. Warning strategy based on the relative distance between the vehicle and the lane

[0193] Figure 6 The coordinates of the lane lines in the image are displayed. OABC represents the image after lane line recognition, HKI represents the set area of ​​interest, and point P 1 (x 1 ,y 1 ), P 2 (x 2 ,y 2 ), P 3 (x 3 ,y 3 ), P 4 (x 4 ,y 4 ) indicates the intersection of the left and right lane lines and the HKIJ area, point C 1 (x 5 ,0),C 2 (x 6 ,0) represents the boundary points of the motor vehicle at the bottom of the image. According to the highway engineering technical standards, the lane width is set to 3.75m, and the width of a small passenger car is set to 1.8m. Let L be the pixel width of the motor vehicle in the image, and let L be the pixel width of the motor vehicle in the image. According to the ratio of lane width to motor vehicle width r = L C / L D =1.8 / 3.75,L C =x 4 -x 2 , L D =x 4 -x 2 , according to the straight test video sequence to obtain L D The pixel width is reversed to get C 1 (x 5 ,0),C 2 (x 6 ,0) to obtain the relative distances between the left and right lane lines and the left and right sides of the motor vehicle:

[0194] d l =x 5 -x 2 Formula 17

[0195] d r =x 4 -x 6 Formula 18

[0196] Convert the relative distances between the lane lines on the left and right sides and the left and right sides of the motor vehicle into proportions:

[0197]

[0198]

[0199] If R r <0.33, it is judged as right-side deviation. l If the yaw angle is less than 0.33, it is judged as left deviation. The deviation judgment does not require the complicated calibration of the camera. The warning strategy based on relative distance is more effective when the yaw angle of the motor vehicle is smaller. In the case of a relatively large yaw angle, this application adopts a strategy based on the lane line deviation rate for warning.

[0200] 2. Lane deviation rate-based sensitive warning strategy

[0201] The lane deviation rate warning is based on detecting the yaw angle of the moving vehicle. If the yaw angle is greater than the critical value, it means that the lane line is about to deviate. At this time, a warning prompt is required, and the deviation rate is derived:

[0202]

[0203] k represents the slope of the lane line, x=tanθ represents the tangent value corresponding to the deviation angle, and when the vehicle is driving in a straight line along the road, x=0; by setting a threshold deviation angle, the threshold deviation rate when the vehicle deviates from the lane is obtained; in order to achieve real-time processing results, the response time of the vehicle is considered and set between 0.2s and 0.9s; the program processes each frame in less than or equal to 0.1s; so when the lane deviates, the total time required for detection and correction is t=0.9+0.1=ls. When the lane does not deviate:

[0204]

[0205] d is 1.025m, v is 40km / s, and the obtained deviation angle threshold is 9 degrees; if the deviation angle is less than or equal to 9 degrees, it is not considered that the deviation occurs; otherwise, it is considered that the deviation occurs; the parameters obtained in this application include the inclination angle of the lane line, so the simplified formula is:

[0206]

[0207] Among them, θ l Represents the inclination angle of the currently detected left lane line, Represents the average value of the left and right lane angles when the lane is straight and not deviating, that is:

[0208]

[0209] if And θ l >θ r , then it is considered that right deviation occurs; if And θ r >θl , then it is considered that left deviation occurs; if It is deemed that no deviation has occurred.

[0210] 4. Experimental Results and Analysis

[0211] (I) Experimental environment

[0212] In order to verify the feasibility of the system, this experiment uses an ordinary vehicle-mounted driving recorder to obtain road images of a highway and underground tunnel near a university. In order to demonstrate the advantages of the algorithm in this application, it is compared with the RANSAC algorithm based on template matching.

[0213] (II) Test results based on ARM platform

[0214] The effects of various road conditions were tested, including complex conditions such as lighting changes, motor vehicle interference, tree shadows, straight lines or curves, etc. Three test sequences were selected to fully evaluate the accuracy, robustness and real-time performance of the algorithm. Figure 7 The lane line recognition results are statistically analyzed, including the total number of test frames, recognition rate, and average processing time per frame.

[0215] from Figure 7 It can be seen that the algorithm of this application achieves real-time detection while having relatively high accuracy and robustness. The video road conditions are relatively complex, but the accuracy is above 90%.

[0216] If the road conditions are good, Figure 8 As shown, Figure (a) and Figure (b) show that the algorithm can recognize solid and dashed lines, and Figure (c) and Figure (d) show that the algorithm has good effects in identifying turns and yellow and white lines.

[0217] In complex road conditions, such as Fig. 9 As shown, Figure (a) can accurately extract the lane line even when there is a motor vehicle ahead and only a small half of it is exposed. Figure (b) can accurately identify the dotted lines on both sides even when there is interference from fonts on the road. Figure (c) shows that even in a dark and blurry tunnel, the solid lines on both sides can be identified. Figure (d) shows that in a strong light environment, the virtual and real lanes on both sides can still be detected. It can be seen that the algorithm proposed in this application still has good results and high robustness in various complex situations.

[0218] Lane departure warning is shown in Figure 10. Figures (a) and (b) show that when the motor vehicle is close to the right lane or is about to cross the lane, the right lane is displayed in black, indicating that a right lane departure warning has been issued. Figures (c) and (d) show that when the motor vehicle is close to the left lane or is about to cross the left lane, the lane detected on the left is displayed in black, indicating that a left lane departure warning has been issued.

[0219] The above test results show that the algorithm in this application can be real-time on the ARM platform, and can accurately realize the lane departure warning function in good or complex road conditions. At the same time, it can also accurately detect the virtual and real lanes and the yellow and white lines.

[0220] (III) Comparison based on window effects

[0221] In order to verify the advantages of the algorithm of this application, the template-based RANSAC algorithm and the extended Kalman algorithm were selected for comparison.

[0222] Depend on Fig.11 It can be seen that the time complexity of the algorithm of the present application is much smaller than that of the template-based and extended Kalman algorithms, so that the algorithm can achieve real-time detection under the ARM platform system. At the same time, the recognition rate is higher than that of the template-based algorithm.

[0223] Depend on Fig.12 It can be seen that in the case of turning, the algorithm of this application and the extended Kalman algorithm can accurately extract the lane lines, while the template-based algorithm has poor template matching in the case of turning due to the double yellow line on the left, and the detection results have some deviations.

[0224] Depend on Fig.13 It can be seen that in the case of turning and landmark interference, the template-based lane line detection algorithm detects errors due to the interference of road signs and the fuzziness of the double yellow lane line on the left. However, the algorithm proposed in this application and the extended Kalman algorithm can accurately extract the lane lines, indicating that both have strong robustness.

[0225] The lane deviation real-time warning system designed in this application has a high recognition rate and robustness on structured urban roads, and can distinguish the yellow and white colors of lanes and the real and virtual lane lines. In more complex road conditions, it can also achieve good results. At the same time, the algorithm has a low time complexity and can achieve real-time detection. Compared with the lane line detection based on templates and extended Kalman algorithms, the recognition rate is greater than that based on templates. The detection speed of the algorithm in this application is faster than the other two algorithms and is more suitable for application on the ARM platform.

Claims

1. Sensitive and accurate real-time warning system for lane deviation in complex road conditions. It is characterized in that First, the hardware includes RK3288 image processing unit, image acquisition unit, alarm display unit and control unit CAN signal part. The real-time road conditions of the driving process are obtained through the CCD camera above the cab. The acquired road condition images are processed by the RK3288 image processing unit. According to the identified lane line information, it is determined whether it is necessary to send an alarm signal to the alarm unit to inform the driver to pay attention to lane deviation. Second, the software algorithm is divided into two parts, namely the extreme lane line recognition algorithm of image Wr extended dynamic evaluation and the real-time warning method of lane deviation: First, the ultra-fast lane line recognition algorithm of Wr extended dynamic evaluation of images uses line scanning to extract edge points, and classifies the edge points into two categories for custom parameter scoring to obtain straight lines. Candidate lane lines are obtained according to the lane line model, and Wr extended dynamic evaluation of images is used to estimate and update road parameters. Finally, the parameters output by Wr extended dynamic evaluation of images are used to extract the final edge points, and Wr extended dynamic evaluation of images is used to estimate the parameters of the established lane parameter model. Lane color information is used to distinguish yellow and white lane lines, and edge point spacing is used to distinguish virtual and real lane lines. The high-speed lane line recognition algorithm of image Wr extended dynamic evaluation specifically includes: lane line high-speed detection algorithm architecture, edge point extraction, establishment of evaluation and scoring rules, acquisition of lane line detection verification value, image Wr extended dynamic evaluation parameter estimation and tracking, and lane edge sensitive extraction. Second, the lane deviation real-time warning method integrates the warning strategy based on the relative distance between the motor vehicle and the lane line and the warning strategy based on the slope of the lane line. When the lateral speed is low, the warning strategy based on the relative distance between the motor vehicle and the lane line is adopted; when the lateral speed is high, the strategy based on the lane deviation rate is adopted. The lane deviation real-time warning method specifically includes: the warning strategy based on the relative distance between the motor vehicle and the lane, and the sensitive warning strategy based on the lane deviation rate. A fast lane line feature optimization modeling method is established. The image Wr extended dynamic evaluation is used to estimate and correct the lane information in complex road conditions. The perspective transformation calculation module is removed. The simplified geometric Vight detection line is used as the basis. The lane line with high credibility is selected by combining the edge point intensity and the lane edge parallel relationship characteristics. The image Wr extended dynamic evaluation is used to estimate the parameters of the angle, offset, average edge intensity and custom curvature. Edge point extraction: The lanes are divided into two colors: yellow and white. The edge points are extracted by using the mutation characteristics of the associated pixel values. The fast line scanning method is used to scan and extract the edge points. At the same time, the lane line information is mainly in the lower half of the image. The region of interest is set by the vanishing point method. According to the characteristics of the lanes in the image, the detection area is reset into two parts: near domain and far domain. The near domain lane lines have a large proportion in the image, and the whole line is scanned; the width between the left and right lane lines in the far domain is narrow and concentrated in the middle of the image. Only most of the middle area is scanned. The near domain lane line features are obvious, and interlaced scanning is used; the far domain lane information is weak, and every three lines are scanned; The edge intensity of each pixel is calculated using formula 1: Formula 1 Where: E(n,i) represents the edge intensity of the pixel at the nth row and the ith column, L represents the filter length, which is 8 according to experience. When the left edge of the lane line is scanned, a maximum value will appear, and when the right edge of the lane line is scanned, a minimum value will appear in the edge intensity. According to the maximum and minimum values, the edge points are divided into two categories, and the custom parameters are used to detect the straight line respectively. Based on the queue, the value information of the previous edge intensity is fully utilized to obtain the value of the next edge intensity. The edge intensity of two adjacent points only needs to access the distance from the endpoint of the filter length and the values ​​of the two points themselves, which is reduced from the original 16 operations to 4 operations, as shown in Formula 2: Formula 2 Where: I(n,i) represents the pixel value of the pixel at the nth row and the ith column; The fast noise inclusion filter method is used to enhance the near-domain image part; the noise inclusion filter model relies on partial differential equations and image discretization, so that a beating signal is obtained at the inflection point of the image. 0 (x,y) after filtering process u 0 (x,y,t),t>0, to achieve the best enhancement effect u(x,y,t 1 ), Represents the second-order directional derivative of the image gradient direction η, The image gradient is the image gradient. The Gaussian function is introduced to smooth and denoise the second-order directional derivative of the image gradient. At the same time, the forward diffusion process is introduced, that is, the term perpendicular to the image gradient direction is introduced into the equation to denoise the image sharpening enhancement process. The specific equation of the model is as follows: Formula 3 in Respectively represent parallel and perpendicular to the image gradient direction, * represents convolution, represents the Gaussian function, represents a constant whose value is positive. represents the forward diffusion process, the first term of Equation 3 plays an enhancement role, and the following term denoises the enhancement process; To simplify the calculation, based on the edge features of road markings, the second term in Equation 3 is removed: Formula 4.

2. According to the sensitive and accurate real-time warning system for lane deviation in complex road conditions as claimed in claim 1, It is characterized in that The lane line speed detection algorithm architecture is mainly divided into seven steps: Step 1: Obtaining vehicle images: Obtaining road images from the camera above the cab; Step 2: Near-field image enhancement: enhance the degree of edge point mutation; Step 3: Obtain edge points: Use line scanning to obtain edge points, and divide the edge points into four categories, which are the maximum and minimum values ​​on the left and right sides of the image; Step 4: Establish the evaluation and scoring rules: Create a table for the two types of edge points based on the custom parameter distribution, and obtain candidate lane lines based on the scores; Step 5: Acquisition of lane lines and edge points: The measured value of the lane line of the current frame is obtained by combining the relationship between frames, and the abnormal points on the edge of the lane line are removed by using the triple standard deviation method to enhance the robustness of the algorithm. Step 6: Image Wr extended dynamic evaluation for estimation and correction: establish a lane parameter model, use image Wr extended dynamic evaluation for parameter estimation, and use the lane line position between frames to eliminate interference; Step 7: According to the updated state value, obtain the final lane line edge points and connect them into lane lines; In step 6, it is necessary to compare the current frame detection verification value with the predicted value of the previous frame using the image Wr extended dynamic evaluation to see if they match. If they match, the set counter value count is increased by 1. If the value of count is greater than or equal to 3, the output optimal estimated value is the current lane line position information; If they do not match, proceed to the next frame and the output of the current frame value is judged separately.

3. According to the sensitive and accurate real-time warning system for lane deviation in complex road conditions as claimed in claim 1, It is characterized in that Establishing evaluation and scoring rules: Reduce the amount of geometric Vight calculations and directly use the edge points of the original image to establish evaluation and scoring rules; The coordinate point (k, θ) corresponds to a straight line in the image coordinate system. Each line in the image coordinate system corresponds to another spatial coordinate point (k, θ). In this application, all edge points are first calculated and a scoring rule is obtained. Then, the parameter space is scored in two steps: each edge point is traversed within the set range of angle θ, the corresponding offset k is calculated, and the edge line is obtained by comparing the scores; (1) Traverse all edge points in the left and right regions with tilt angles θ, calculate the corresponding offset k, and then increase the score of (k, θ) by 1. The value ranges of θ and k are set based on experience. The value range of angle θ is [20 degrees, 70 degrees], and the value range of k is [0, 840]. (2) Compare the scores corresponding to the parameter pairs (k, θ). Each (k, θ) represents a straight line. The higher the score of the (k, θ) pair, the more edge points there are on the straight line, and the more likely it is a lane line. The algorithm retains the edge lines that have scores that reach the threshold. Each point is mapped to the coordinate space (k, θ) with 50 lines. Among the numerous lines, the local maximum score retention method is adopted to ensure that unnecessary lines are removed and lines that may be lane lines are retained; First, in each offset, traverse all θ, find the line with the highest score and save it. When removing unnecessary lines, avoid removing the edge lines of the lane lines. Then, judge whether the lines in the left and right areas intersect with each other. The judgment method is as follows: 1 and k 2 Represents the offset between two straight lines, k 3 and k 4 represents the offset of the intersection point of the two straight lines with the top of the region of interest. If (k 1 -k 2 )·(k 4 -k 3 )<0, it means that the two straight lines intersect, and the line with the lower score is removed.

4. According to claim 1, the sensitive and accurate real-time warning system for lane deviation in complex road conditions, It is characterized in that Obtain lane line detection verification value: The angle difference between the two lines on the left and right edges of the structured road lane is small, and the offset width of the two lines is fixed. Based on these two features, candidate lane lines are matched from multiple straight lines. In the process of edge point detection, edge points are divided into two categories, maximum and minimum. In the process of detecting straight lines based on edge points, straight lines are further divided into the maximum, i.e., the straight line corresponding to the left edge of the lane line, and the minimum, i.e., the straight line corresponding to the right edge of the lane line. These two types of lines are matched in pairs according to parallelism and offset to obtain candidate lane lines. After obtaining the candidate lane lines, the lane lines tend to be in the middle of the image, so the edge points that can be detected are dense and have high scores. The two lane lines with the highest scores are screened based on the scores of the left and right sides of the lane lines. If the offset of the lane line with the highest score meets the threshold range, it is determined to be the desired lane line.

5. According to claim 1, the sensitive and accurate real-time warning system for lane deviation in complex road conditions, It is characterized in that Image Wr extended dynamic evaluation parameter estimation tracking: Image Wr extended dynamic evaluation is used for parameter estimation and update. The output result is used as the priority detection range for the next frame. The angle θ between the model and the lane line, the offset k, and the average edge intensity E of the lane line are calculated. p The lane line curvature degree is associated with the custom lane line curvature degree representation S, and the lane line edge average intensity E is used. p A variable S representing the curvature of the lane line is introduced. The search line is determined by the angle and offset corrected by the dynamic evaluation of the image Wr. The variable S representing the curvature of the lane line is used to determine the points within a certain range of the line, thereby searching for the edge points of the line. According to the edge intensity E of the line, the triple standard deviation method is used to remove abnormal points in the edge points to obtain an accurate lane line edge point. This application uses the lane line angle θ, offset k, and the lane line edge average intensity E p The system model established by the custom lane line curvature representation S is as follows: Formula 5 They represent the lane line angle θ, the offset k, and the average edge intensity E of the lane line. p And the speed of change of the custom lane line curvature representation S, the image Wr extended dynamic evaluation state vector is defined as: Formula 6 The state transfer matrix is: Formula 7 At the initial moment, the state vector is set to: Formula 8 k(0),θ(0),E(0),S(0) respectively represent the lane line information obtained from the first frame image. Assume that w 1 (i), w 2 (i), w 3 (i), w 4 (i) is an independent statistic, assuming that the system noise is: Formula 9 Detection verification vector: Formula 10 The measurement equation is: z(k)=Hx(k)+V(k) Formula 11 According to the relationship between z(k) and x(k), the measurement matrix is: Formula 12 The measurement noise vector is: Formula 13 The impact of detection verification noise is greater than system noise. Under the same assumption, v 1 (i), v 2 (i) Statistical independence: Formula 14 To ensure the accuracy of the estimate, a large value is assigned to P(t) at the beginning: Formula 15 Estimate and update parameters. The parameters involved in road modeling include lane curvature, pitch angle, horizontal tilt angle, lane width and lateral deviation of motor vehicles. Parameter estimation includes two parts. The first part is to estimate the road based on the information of the road lane line, and the second part is the position parameters, focal length and road environment information of the camera. In order to better utilize the inter-frame information to achieve stability, a control variable is set when the image Wr is extended and dynamically evaluated. When the value of the counter count is greater than or equal to 3, the output result is the final result of this frame, otherwise the result of this frame needs to be processed separately. The rule for counter accumulation is: if the detection value of the current frame matches the output result value of the image Wr extended dynamic evaluation of the previous frame, the counter is increased by 1, and the maximum value is 3; when it does not match, the counter is set to 0, and the dynamic evaluation output result is not used as the final detection result. In addition, the algorithm sets up two other counters, which are accumulated according to the detection verification values. If the detection verification values ​​of the previous and next frames match, the counter is increased by 1, with a maximum upper limit of 8, otherwise it is decreased by 1; when the value of the counter reaches 8, it corresponds to a mark, and the mark is set to 1 at this time, and the default setting is 0; after the mark is 1, if three consecutive needles are different, the mark is set to 0; if the left and right marks are both 1, the current left and right lane information is recorded. When the counter of one lane line is less than 8 and the counter of the other lane line is 8, it means that the lane line on one side is stable, and the lane line on the other side is blocked or scratched by the car; according to the invariance of lane width, when the position of one lane line remains unchanged, the position of the other lane line will not change. At this time, the originally matched lane information is taken out as the lane information of the current frame.

6. According to claim 1, the sensitive and accurate real-time warning system for lane deviation in complex road conditions, It is characterized in that Sensitive lane edge extraction: After obtaining the corrected parameters, the detected lane line angle θ and offset k are used to determine the position range of the edge point search, and the custom lane line curvature representation S is used to determine the search range; the algorithm sets a trapezoidal area search range, and this trapezoid takes the straight line determined by the angle θ and offset k as the center line. In the bottom area of ​​the image lane, the angle and offset of the lane line reflect the lane edge, and the search range is five pixels around the line. At the farthest point of the area of ​​interest, the search range is set to β+5 pixels around the line; the search range gradually increases from near to far in the middle part; in the formed edge point concentration, other noise points, road puddles, and points formed by lane wear and deformation are included. This application extends the edge average intensity E estimated by dynamic evaluation based on the image Wr p The triple standard deviation method was used to eliminate outliers; Assume that the number of edge points detected on the left edge of the lane line of a certain frame is δ, and the intensity of the edge points is as follows: E 1 、E 2 、E 3 ……E δ , then the standard deviation is calculated using the edge average intensity: Formula 16 Take the values ​​within the range of E±3σ as the retained values, and remove the values ​​outside this range; after removal, recalculate the standard deviation based on the remaining data, and then remove the abnormal points; repeat this cycle until there are no abnormal points; The lane edge points finally obtained are connected into lines to form the final lane line; Finally, according to the obtained lane line edge position, the pixel values ​​of the three channels of the lane line RGB are obtained. According to the characteristic that the projection components of white and yellow in the B channel are the most different, B=88 is taken as the critical value. If it is greater than 88, it is judged as white, otherwise it is judged as yellow; add a judgment condition to distinguish based on the ratio of the red component R to the blue component B of the RGB color channel. If R / B>0.65, it is judged as white, otherwise it is judged as yellow.

7. According to claim 1, the sensitive and accurate real-time warning system for lane deviation in complex road conditions, It is characterized in that Warning strategy based on the relative distance between the vehicle and the lane: OABC represents the image after lane line recognition, HKIJ represents the set area of ​​interest, and point P 1 (x 1 ,y 1 ), P 2 (x 2 ,y 2 ), P 3 (x 3 ,y 3 ), P 4 (x 4 ,y 4 ) indicates the intersection of the left and right lane lines and the HKIJ area, point C 1 (x 5 ,0),C 2 (x 6 ,0) represents the boundary points of the motor vehicle at the bottom of the image. The lane width is set to 3.75m, and the width of the small passenger car is set to 1.8m. Let L C is the pixel width of the lane in the image, let L D is the pixel width of the motor vehicle in the image, then according to the ratio of lane width to motor vehicle width r=L C / L D =3.75 / 1.8, L C =x 4 -x 2 , L D =x 6 -x 5 , according to the straight test video sequence to obtain L D The pixel width is reversed to get C 1 (x 5 ,0),C 2 (x 6 ,0) to obtain the relative distances between the left and right lane lines and the left and right sides of the motor vehicle: d l =x 5 -x 2 Formula 17 d r =x 4 -x 6 Formula 18 Convert the relative distances between the lane lines on the left and right sides and the left and right sides of the motor vehicle into proportions: Formula 19 Formula 20 If R r <0.33, it is judged as right-side deviation. l If the yaw angle is less than 0.33, it is judged as a left deviation. The deviation judgment does not require complex camera calibration operations. This warning strategy based on relative distance is more effective when the yaw angle of the motor vehicle is smaller. When the yaw angle is large, this application adopts a strategy based on the deviation rate of the lane line for warning.

8. According to claim 1, the sensitive and accurate real-time warning system for lane deviation in complex road conditions, It is characterized in that Lane departure rate sensitive warning strategy: Lane departure warning based on lane departure rate detects the yaw angle of a moving vehicle. If the yaw angle is greater than the critical value, it indicates that a lane line deviation is about to occur, and a warning needs to be given at this time. The deviation rate is derived as follows: Formula 21 α represents the slope of the lane line, x=tanθ represents the tangent value corresponding to the deviation angle, and when the vehicle is traveling in a straight line along the road, x=0; by setting a threshold deviation angle, the threshold deviation rate when the vehicle deviates from the lane is obtained; in order to achieve real-time processing results, the response time of the vehicle is considered and set between 0.2s and 0.9s; the time for the program to process each frame is less than or equal to 0.1s; so when the lane deviates, the total time required for detection and correction is t 2 =0.9+0.1=ls, when the lane does not deviate: Formula 22 When d is 1.025m and v is taken as 40km / s, the obtained threshold of the deflection angle is 9 degrees; if the deflection angle is less than or equal to 9 degrees, it is not considered that a deviation has occurred; otherwise, it is considered that a deviation has occurred; among the parameters obtained in this application, the inclination angle of the lane line is included, so the simplification is based on the formula: Formula 23 Among them, θ l Represents the inclination angle of the currently detected left lane line, Represents the average value of the left and right lane angles when the lane is straight and not deviating, that is: Formula 24 if and , then it is considered that right deviation occurs; if and , then it is considered that left deviation occurs; if , then it is deemed that no deviation occurs.

Images