A vehicle active collision avoidance method based on road pothole detection

Abstract

The present invention discloses a vehicle active collision avoidance method based on road pothole detection. The vehicle active safety obstacle avoidance method based on road pothole detection proposed by the present invention detects the potholes on the road ahead by using multi-sensor fusion, and performs preprocessing by using grayscale and multi-scale Retinex algorithms to improve image processing efficiency; uses Hough transform to remove road marking interference in the image; accurately calculates the pothole area based on threshold segmentation theory and chain code method; judges passability by comprehensively considering slope obstacles and depth step obstacles; and can more accurately calculate the maximum speed of vehicles passing through potholes by combining the above-mentioned various information, thereby reducing the occurrence of traffic accidents caused by vehicles passing through potholes.

Description

Technical Field

[0001] The present invention belongs to the technical field of driving safety, and in particular relates to an active collision avoidance method based on road pothole detection. Background Art

[0002] According to statistics from the Ministry of Transport, by the end of 2019, the total mileage of highways in China was 5.0125 million kilometers, an increase of 548,600 kilometers from 2014. The highway density is 52.21 kilometers per 100 square kilometers, an increase of 5.71 kilometers per 100 square kilometers from 2014, showing a steady growth trend. And with the increase in the age of roads and the increasing traffic volume, the pressure of passenger volume on roads is increasing, causing damage to the road structure and generating a large number of road diseases, among which potholes are the most common and most likely to affect driving safety. Obtaining the specific size information of potholes in the road in a timely and accurate manner and avoiding dangerous conditions in advance are of great practical significance to driving safety, but the existing active collision avoidance algorithms applied to vehicles do not take into account the impact of potholes on vehicle driving safety. Summary of the invention

[0003] The present invention discloses an active collision avoidance method based on road pothole detection, which can accurately predict the current vehicle's maximum speed limit for crossing a pothole, and avoid the impact of road potholes on driving safety during vehicle driving. The specific scheme is as follows:

[0004] An active collision avoidance method based on road pothole detection comprises the following steps:

[0005] Step 1: Get pothole information;

[0006] Step 2: Grayscale and enhance the pothole image, and remove the road marking area;

[0007] Step 3: Extract the contour features of road potholes;

[0008] Step 4: Track the pothole contour obtained in step 3 and calculate the pothole area;

[0009] Step 5: Combine the information obtained in step 1 with the slope of the pothole boundary to determine the passability;

[0010] Step 6: Combine the information obtained in steps 1, 4, and 5 to determine the maximum speed for crossing a pit.

[0011] Furthermore, in step 1, the depth of the pothole is obtained by using a radar, and the image of the pothole on the road surface is obtained by using a camera;

[0012] Furthermore, the weighted average method for graying the image in step 2 is specifically:

[0013] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j)

[0014] Where (i, j) represents the coordinates of the pixel point, and Gray(i, j) represents the pixel value of the grayscale image.

[0015] Furthermore, in step 2, the image is enhanced using the MSR multi-scale algorithm as follows:

[0016] s(i,j)=l(i,j)r(i,j)

[0017] Where (i, j) represents the pixel coordinates, l(i, j) represents the incident illumination image, r(i, j) represents the reflected illumination image, and s(i, j) represents the product of the incident illumination and reflected illumination images.

[0018] The Gaussian convolution function F(i,j) is used to estimate the brightness component from the known image s(i,j). The specific formula is as follows:

[0019] l(i,j)=F(i,j)*s(i,j)

[0020]

[0021] Among them, * is the convolution operation, λ is the normalization coefficient, and σ is the scale parameter of the Gaussian function.

[0022] The MSR multi-scale algorithm is used for linear weighted summation. The specific formula is:

[0023]

[0024] Where N is the number of Gaussian kernels, usually 3, and

[0025] Furthermore, the specific method for extracting the road marking area in step 2 is:

[0026] Based on the Canny edge detection operator, the road surface image containing the road markings is subjected to Hough transform to extract the straight edges of the road markings:

[0027] ρ=xcosθ+ysinθ

[0028] Among them, ρ is the distance from the straight line to the origin of the coordinate system, ranging from ρ∈(0,r), r is the length of the image diagonal, and θ is the angle between the straight line and the x-axis, ranging from θ∈(0,180°);

[0029] The grayscale mean of each area formed by the extracted road markings is calculated, and the road marking area is segmented by setting a threshold and the extracted road marking area is removed from the original image to obtain a new road detection area.

[0030] Furthermore, the specific method for extracting the road pothole contour in step 3 is:

[0031] The Canny operator is used to extract contours. First, the calcHist function and min-MaxLoc function are used to calculate the optimal image segmentation threshold T. k ; Then the image is binarized; Finally, the median filter algorithm is used to obtain the edge contour of the pothole;

[0032] Furthermore, the specific method of tracking the pothole contour and calculating the pothole area in step 4 is as follows:

[0033] The 8-directional chain code method is used to track the similarity of pothole boundaries. First, the total number of pixels A on the pothole boundary of the road is counted. 1 ; Then set the pothole boundary points as vectors; and then count the total number of pixels A within the pothole boundary 2 ;Finally calculate the pothole area s;

[0034] Furthermore, the traversability in step 5 is comprehensively judged by the slope obstacle and the depth step obstacle function;

[0035] Furthermore, the pit-crossing speed limit v in step 6 l The pothole area s, depth d, and passability f ΔT , current vehicle speed v, maximum absolute value of vertical acceleration |a z-max | and weighted acceleration RMS value Waiting for decision;

[0036] Beneficial effects of the present invention:

[0037] The active safety obstacle avoidance method for automobiles based on road pothole detection proposed in the present invention uses multi-sensor fusion to detect potholes on the road ahead, and uses grayscale and multi-scale Retinex algorithms for preprocessing to improve image processing efficiency; uses Hough transform to remove road marking interference in the image; accurately calculates the area of ​​the pothole based on threshold segmentation theory and chain code method; comprehensively considers slope obstacles and depth step obstacles to determine passability; and comprehensively considers the above-mentioned various information to more accurately calculate the maximum speed for passing potholes, thereby reducing the occurrence of traffic accidents caused by vehicles passing potholes. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 It is a geometric model diagram for pothole depth measurement;

[0040] Figure 3 This is a schematic diagram of a slope obstacle. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings.

[0042] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0043] like Figure 1 As shown, the present invention provides a method for active vehicle collision avoidance based on road pothole detection, comprising the following steps:

[0044] Step 1: Get pothole information;

[0045] The pothole image can be obtained by a CCD industrial camera installed on the vehicle; the single-point ranging laser radar is installed on the front bumper of the vehicle with a pitch angle of β, which is used to calculate the pothole depth d, such as Figure 2 As shown, the specific calculation formula is as follows:

[0046]

[0047] in, Indicates the distance when the laser point reaches the deepest part of the pit. It indicates the distance from the laser point of the single-point ranging laser radar to the road surface, which is a constant value.

[0048] Step 2: Grayscale and enhance the image, extract and remove the road marking area;

[0049] 2.1) Use the weighted average method to grayscale the image. The specific formula is:

[0050] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j)

[0051] Among them, (i, j) represents the coordinates of the pixel point, and Gray(i, j) represents the pixel value of the grayscale image.

[0052] 2.2) Based on the Retinex theory, the MSR multi-scale algorithm is used to perform weighted summation of different scales. The specific formula is:

[0053] s(i,j)=l(i,j)r(i,j)

[0054] Among them, (i, j) represents the pixel coordinates, l(i, j) represents the incident illumination image, r(i, j) represents the reflected illumination image, and s(i, j) represents the product of the incident illumination and reflected illumination images.

[0055] The Gaussian convolution function F(i,j) is used to estimate the brightness component from the known image s(i,j). The specific formula is as follows:

[0056] l(i,j)=F(i,j)*s(i,j)

[0057]

[0058] Among them, * is the convolution operation, λ is the normalization coefficient, and σ is the scale parameter of the Gaussian function.

[0059] The multi-scale Retinex algorithm is used to perform weighted summation of different scales. The specific formula is:

[0060] Among them, r MSR (i, j) represents the reflected illumination image calculated by the MSR algorithm, F n (i,j) represents the Gaussian convolution function, N is the number of Gaussian kernels, usually 3, and

[0061] 2.3) Perform Hough transform line detection on the image based on the Canny edge detection operator, extract the road marking area and remove it from the original image. The specific method is as follows:

[0062] ρ=xcosθ+ysinθ

[0063] Among them, ρ is the distance from the straight line to the origin of the coordinate system, ranging from ρ∈(0,r), r is the length of the image diagonal, and θ is the angle between the straight line and the x-axis, ranging from θ∈(0,180°).

[0064] Calculate the grayscale mean of each area formed by the extracted road markings. Set a threshold T. If the grayscale mean difference between areas is less than the threshold T, the different areas are defined as the same area. Based on the characteristic that the grayscale value of the road marking area is higher than that of the ordinary road surface, the road marking area can be accurately segmented. Then subtract the extracted road marking area from the original image to obtain a new road detection area.

[0065] Step 3: Extract the contour features of road potholes;

[0066] The Canny operator is used to extract the contour of potholes on the road. The specific steps are as follows:

[0067] 3.1) Use the calcHist function to obtain the grayscale histogram corresponding to the image and determine the grayscale range of the two peaks. The specific formula is:

[0068]

[0069] Among them, l represents the pixel mean of the entire image, and m and n are the length and width of the pixels of the image respectively.

[0070] 3.2) Use the min-MaxLoc function to calculate the maximum pixel value and grayscale value t within the left and right peak ranges respectively l and t r The code is as follows:

[0071] void minMaxLoc(InputArray src, CV_OUT double*minVal, CV_OUT double*maxVal=0, CV_OUT Point*minLoc=0, CV_OUT Point*maxLoc=0, InputArray mask=noArray());

[0072] 3.3) First calculate the initial image segmentation threshold T 0 , segment the image into two parts, the target and the background, and then calculate the grayscale mean t of the two parts l-m and t r-m , then the new threshold T k+1 It can be obtained by the grayscale average of the two parts. The specific formula is as follows:

[0073]

[0074]

[0075]

[0076]

[0077] Among them, Q(i,j) is the grayscale of any point in the image, N(i,j) is the number of pixels at the pixel point (i,j), and T k is the threshold value.

[0078] The optimal segmentation threshold T of the final image k It can be obtained by satisfying the equation T k =T k+1 get.

[0079] 3.4) The image is binarized to obtain a grayscale value of 255 (white) for all pixels in the pothole area and a grayscale value of 0 (black) for all pixels outside the pothole area.

[0080] 3.5) Use the median filter algorithm to get the edge contour of the pothole:

[0081] y i =med{f i-n ,...,f i-1,f i ,f i+1 ,...,f i+n}

[0082] Among them, f i It is the median value sorted from small to large and replaces the grayscale value of the specified point.

[0083] Step 4: Track the pothole contour obtained in step 3 and calculate the pothole area;

[0084] The 8-direction chain code method is used to track the similarity of the pothole boundary, and the chain code of the curve is: 57670013234.

[0085] 4.1) Divide the binarized pothole image into multiple columns, and start from the upper left corner and track the pothole edge in reverse order from small to large. The largest number at the end is the total number of boundary pixels A 1 .

[0086] 4.2) According to the boundary point sequence number in step 4.1, define the vectorization settings of the previous boundary point pothole boundary point: define the vector link from the previous boundary point (N-1) to the current boundary point (N) as the front vector; the vector chain code from the current boundary point (N) to the next boundary point (N+1) is the back vector, the vector direction and vector value refer to the 8-direction chain code mark, and the center pixel of the chain code is the current boundary point.

[0087] 4.3) Perform vector analysis on the pixels within the boundary in ascending order, and record the previous and next vectors of each boundary point. If the pixel to the right of the current boundary point is a point within the boundary, calculate the number of pixels between the current pixel and the next adjacent boundary point, expressed as N i+1 -N i -1, the sum of the number of pixels is the total number of pixels within the boundary A 2 Among them, N i is the column value of the current boundary point, N i+1 is the column value of the next boundary point. If the judgment condition shown in the formula is met, the point is defined as a point inside the boundary:

[0088] bv≠8, and fv=5, or fv<3 and |fv-bv|>4, or fv>5 and |fv-bv|<4

[0089] Among them, fv is the front vector and bv is the back vector.

[0090] 4.4) The area of ​​the pothole s is given by A 1 , A 2 , u gets:

[0091]

[0092] Among them, A 1 is the total number of pixels at the boundary of the pothole, A 2 is the total number of pixels within the boundary of the pothole, and u is the calibrated pixel equivalent.

[0093] Step 5: Combine the information obtained in step 1 with the slope of the pothole boundary to determine the passability;

[0094] The slope of the pothole boundary β exceeds the threshold β max (In the embodiment of the present invention, max Take 0.236) as a slope obstacle, and vehicles cannot pass through. Figure 3 As shown, the boundary slope β of the pothole on the road surface is obtained:

[0095]

[0096] The barrier function is as follows:

[0097]

[0098] Among them, h z is the vehicle ground clearance and b is the vehicle wheelbase.

[0099] The depth of the pothole d exceeds the ground clearance h of the car z It is regarded as a deep step obstacle, and the vehicle cannot pass through. The obstacle function is as follows:

[0100]

[0101] The vehicle passability can be determined by combining slope boundary obstacles and depth step obstacles:

[0102] f Δ =max(f Δ (β),f Δ (d))

[0103] If the passability factor f Δ If it is greater than 1, it means that the boundary obstacle of the pothole on the road exceeds the limit condition, and the pothole road surface is defined as an impassable area, otherwise it can be passed.

[0104]

[0105] Among them, f ΔT It is the passability factor.

[0106] Step 6: Combining the information obtained in steps 1, 4, and 5, the maximum speed for driving over a pit is obtained;

[0107] Using the fifth-order polynomial to fit the maximum absolute value of the vehicle vertical acceleration |a z-max | and weighted acceleration RMS value

[0108] |a z-max |=0.0064s 5 -0.0568s 4 +1560.5d 4 +0.1895s 3 +851.58d 3 -0.0001v 3 -0.2918s 2 -243.30d 2 +0.0033v 2 +0.2036s+19.572d-0.0441v

[0109]

[0110] Maximum speed for driving over a pit l The pothole area s, depth d, and passability f ΔT , current vehicle speed v, maximum absolute value of vertical acceleration |a z-max | and weighted acceleration RMS value Waiting for decision:

[0111] First, use the passability factor f ΔT Determine whether the road surface with potholes can be passed. If it can be passed, calculate the maximum speed v for passing the potholes using the known pothole area s and pothole depth d. l The specific method is as follows:

[0112] Among them, a z-limit is the maximum absolute vertical acceleration of the vehicle, is the maximum weighted acceleration.

[0113] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent methods or changes that do not deviate from the technical creation of the present invention should be included in the scope of protection of the present invention.

Claims

1. A vehicle active collision avoidance method based on road pothole detection, It is characterized in that The steps include: Step 1: Get pothole information; Step 2: Grayscale and enhance the pothole image, extract and remove the road marking area; Step 3: Extract the contour features of road potholes; Step 4: Track the pothole contour obtained in step 3 and calculate the pothole area; Step 5: Determine feasibility; Step 6: Determine the maximum speed for crossing a pit according to the information obtained in steps 1, 4, and 5; In step 2, the image is grayed using a weighted average method, specifically: Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j) In the formula, (i, j) represents the coordinates of the pixel point, and Gray(i, j) represents the pixel value of the grayscale image; In step 2, the image is enhanced using the MSR multi-scale algorithm, specifically: Let s(i,j)=l(i,j)r(i,j) Where (i, j) represents the pixel coordinates, l(i, j) represents the incident illumination image, r(i, j) represents the reflected illumination image, and s(i, j) represents the product of the incident illumination and reflected illumination images; The Gaussian convolution function F(i,j) is used to estimate the brightness component from the known image s(i,j). The specific formula is as follows: l(i,j)=F(i,j)*s(i,j) Among them, * is the convolution operation, λ is the normalization coefficient, and σ is the scale parameter of the Gaussian function; The MSR multi-scale algorithm is used for linear weighted summation. The specific formula is: Among them, r MSR (i, j) represents the reflected illumination image calculated by the MSR algorithm, F n (i,j) represents the Gaussian convolution function, N is the number of Gaussian kernels, which is 3, and The specific method for extracting and removing the road marking area in step 2 is: Based on the Canny edge detection operator, the road surface image containing the road markings is subjected to Hough transform to extract the straight edges of the road markings: ρ=xcosθ+ysinθ Among them, ρ is the distance from the straight line to the origin of the coordinate system, ranging from ρ∈(0,r), r is the length of the image diagonal, and θ is the angle between the straight line and the x-axis, ranging from θ∈(0,180°); Calculate the grayscale mean of each area formed by the extracted road markings, set a threshold T, and if the grayscale mean difference between areas is less than the threshold T, define different areas as the same area. Based on the characteristic that the grayscale value of the road marking area is higher than that of the ordinary road surface, the road marking area can be accurately segmented; then subtract the extracted road marking area from the original image to obtain a new road detection area; In step 3, the Canny operator is used to extract the contour of the pothole on the road. The specific method is as follows: 3.1, use the calcHist function to obtain the grayscale histogram corresponding to the image and determine the grayscale range of the two peaks, specifically: Among them, l represents the pixel mean of the entire image, m and n are the length and width of the image pixels respectively; 3.2, use the min-MaxLoc function to calculate the maximum pixel value and grayscale value t within the left and right peak ranges respectively l and t r ; 3.3, Calculate the initial image segmentation threshold T 0 , segment the image into two parts, the target and the background, and then calculate the grayscale mean t of the two parts l-m and t r-m , then the new threshold T k+1 It can be obtained by the grayscale average of the two parts. The specific formula is as follows: Among them, Q(i,j) is the grayscale of any point in the image, N(i,j) is the number of pixels at the pixel point (i,j), and T k is the threshold value; The optimal segmentation threshold T of the final image k It can be obtained by satisfying the equation T k =T k+1 get; 3.4, binarize the image to obtain that the grayscale value of all pixels in the pothole area is 255, and the grayscale value of the external pixels is 0; 3.5, use the median filter algorithm to get the edge contour of the pothole: y i =med{f i-n ,...,f i-1 ,f i ,f i+1 ,...,f i+n } Among them, f i It is the median value sorted from small to large and replaces the gray value of the specified point; The specific method of tracking the pothole contour and calculating the pothole area in step 4 is as follows: The 8-directional chain code method is used to track the similarity of pothole boundaries. Specifically: 4.1, divide the binarized pothole image into multiple columns, start from the upper left corner and track the pothole edge in reverse order from small to large, and the largest number at the end is the total number of boundary pixels A 1 ; 4.2, according to the boundary point sequence number in step 4.1, define the vector setting of the previous boundary point pothole boundary point: define the vector link from the previous boundary point (N-1) to the current boundary point (N) as the front vector; the vector chain code from the current boundary point (N) to the next boundary point (N+1) is the back vector, the vector direction and vector value refer to the 8-direction chain code mark, and the center pixel of the chain code is the current boundary point; 4.3, perform vector analysis on the pixels within the boundary in ascending order, and record the previous vector and the next vector of each boundary point; if the right pixel of the current boundary point is a point within the boundary, calculate the number of pixels between the current pixel and the next adjacent boundary point, and the expression is N i+1 -N i -1, the sum of the number of pixels is the total number of pixels within the boundary A 2 ; Among them, N i is the column value of the current boundary point, N i+1 is the column value of the next boundary point; if the judgment condition shown in the formula is met, the point is defined as an inner point of the boundary: bv≠8, and fv=5, or fv<3 and |fv-bv|>4, or fv>5 and |fv-bv|<4 Among them, fv is the front position vector, bv is the back position vector; 4.4, the area of ​​the pothole s is given by A 1 , A 2 , u gets: Among them, A 1 is the total number of pixels at the boundary of the pothole, A 2 is the total number of pixels within the boundary of the pothole, and u is the calibration pixel equivalent; The passability judgment in step 5 is obtained based on the information obtained in step 1 and the slope judgment of the pothole boundary. Specifically: When the pitch angle β exceeds the threshold β of the pothole boundary slope max When , it is regarded as a slope obstacle and the vehicle cannot pass through. The threshold β of the boundary slope of the pothole is obtained. max As shown below: The barrier function is designed as follows: Among them, h z is the vehicle ground clearance, b is the vehicle wheelbase; The depth of the pothole d exceeds the ground clearance h of the car z It is regarded as a deep step obstacle, and vehicles cannot pass through it. The obstacle function is designed as follows: The comprehensive slope and depth of step obstacles can determine vehicle passability: f △ =max(f △ (β),f △ (d)) If the passability factor f △ If it is greater than 1, it means that the boundary obstacle of the pothole on the road exceeds the limit condition, and the pothole road is defined as an impassable area, otherwise it can be passed; Among them, f △T It is the passability judgment factor; The implementation of step 6 includes: First, the fifth-order polynomial is used to fit the maximum absolute value of the vehicle vertical acceleration |a z-max | and weighted RMS acceleration a z-max |=0.0064s 5 -0.0568s 4 +1560.5d 4 +0.1895s 3 +851.58d 3 -0.0001v 3 -0.2918s 2 -243.30d 2 +0.0033v 2 +0.2036s+19.572d-0.0441v Maximum speed for driving over a pit l The pothole area s, depth d, and passability f △T , current vehicle speed v, maximum absolute value of vertical acceleration |a z-max | and weighted acceleration RMS value Waiting for decision: First, use the passability factor f △T Determine whether the road surface with potholes can be passed. If it can be passed, calculate the maximum speed v for passing the potholes using the known pothole area s and pothole depth d. l The specific method is as follows: Among them, a z-limit is the maximum absolute vertical acceleration of the vehicle, is the maximum weighted acceleration.

2. The vehicle active collision avoidance method based on road pothole detection according to claim 1, It is characterized in that The implementation of step 1 includes: the pothole image is obtained by a CCD industrial camera installed on the vehicle; the pothole depth is obtained by a single-point ranging laser radar installed on the front bumper of the vehicle. Assuming the pitch angle is β, the pothole depth d is calculated according to the following formula: in, Indicates the distance when the laser point reaches the deepest part of the pit. It indicates the distance from the laser point of the single-point ranging laser radar to the road surface, which is a constant value.

Images