An image recognition-based road surface anti-skid performance prediction method

Abstract

The application relates to a road surface anti-skid performance prediction method based on image recognition, which comprises the following steps: collecting current road information and current road surface topography information through a vehicle-mounted forward-looking camera and a monocular camera respectively; inputting the obtained current road information into a trained neural network to identify a current road surface state and identify current road surface configuration parameters based on the obtained current road surface topography information; determining a road surface friction coefficient according to a friction coefficient model according to the current road surface state and the current road surface configuration parameters, so as to predict the current road surface anti-skid performance. Compared with the prior art, the application avoids the problem that real-time estimation cannot be realized due to complex modeling in dynamic analysis, quantifies specific asphalt pavement configuration parameters, improves prediction accuracy and real-time performance, and is suitable for various road conditions and driving environments.

Description

Technical Field

[0001] This invention relates to the field of road surface skid resistance prediction technology, and in particular to a road surface skid resistance prediction method based on image recognition. Background Technology

[0002] The increasing number of vehicles on the road each year has led to increasingly serious traffic problems. The skid resistance of asphalt pavement is a major factor affecting driving safety; insufficient skid resistance will reduce vehicle braking efficiency. Therefore, assessing the skid resistance of asphalt pavement is crucial during the road construction and operation phases. The coefficient of friction between tires and the road surface is the main evaluation indicator of skid resistance, and the coefficient of friction of the road surface is closely related to its current condition and texture.

[0003] From the moment asphalt pavement is constructed, its surface is continuously subjected to vehicle loads, wind, sand, and rainwater, causing changes in its surface condition and texture. The pavement's skid resistance deteriorates over time, and even for the same road section, weather factors such as rainfall further affect its skid resistance. Previous methods for measuring skid resistance under fixed road surface conditions did not consider weather and pavement texture, making them unsuitable for driving in environments with real-time changes. Therefore, dynamic and real-time prediction of the pavement friction coefficient is necessary.

[0004] Currently, there are two main methods for real-time estimation of road surface friction coefficient. The first method involves collecting data through sensors and establishing a mathematical model of the sensor data and friction coefficient for estimation. However, this requires additional sensor installation, is complex to debug, and increases the cost. The second method uses a dynamic approach to establish a mathematical model for estimation. While this method is more reliable for estimating road surface skid resistance, the model is complex, requires collecting a large number of parameters, and struggles to cope with varying road conditions or guarantee real-time performance. Existing vision-based methods for estimating road surface skid resistance, such as an online method and system for estimating road surface adhesion coefficient based on vehicle cameras, and an automatic emergency braking control method based on vision-based road surface adhesion coefficient estimation, all utilize neural networks to classify the road surface ahead and predict the current road surface skid resistance based on experience gained from previous studies of samples. They do not consider the texture and roughness characteristics of the current road surface and can only map the adhesion coefficient to a numerical value or range based on experience, making it difficult to guarantee accuracy.

[0005] A search revealed Chinese invention patent CN201710589940.2, which discloses a method for detecting the distribution of road surface texture and anti-skid performance. The method involves moving a road surface image acquisition system to the road surface to be tested, adjusting the angle of the image acquisition device using a telescopic connector, turning on the power switch and processor switch, and capturing images of the road surface. The captured images are transmitted to a control box via a data transmission line. Simultaneously, the captured images are transmitted sequentially to an image processor via a wireless signal transmitter and a wireless signal transceiver for image processing, yielding a road surface texture image, a uniformity analysis result of the road surface texture distribution, and a road surface anti-skid performance analysis result. This existing patent suffers from problems such as using complex equipment and incomplete, non-real-time test data after fixing the position, resulting in insufficient real-time performance and accuracy in prediction.

[0006] Therefore, how to establish a reasonable road surface skid resistance prediction model based on visual technology, while taking into account weather conditions and current road surface texture roughness characteristics, and to make dynamic, real-time, and accurate predictions, is a problem that needs to be solved. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for predicting road surface skid resistance performance based on image recognition.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] According to one aspect of the present invention, a method for predicting the anti-skid performance of road surfaces based on image recognition is provided, the method comprising the following steps:

[0010] Step S1: Collect current road information and current road surface topography information using the vehicle-mounted forward-facing camera and monocular camera respectively;

[0011] Step S2: Input the current road information obtained in step S1 into the trained neural network to identify the current road surface condition;

[0012] Step S3: Identify the current road surface structure parameters based on the current road surface morphology information obtained in step S1;

[0013] Step S4: Based on the road surface condition in step S2 and the road surface structure parameters in step S3, the road surface friction coefficient is determined using a friction coefficient model, thereby predicting the current road surface anti-skid performance.

[0014] Preferably, if the road surface condition is a dry road surface, the friction coefficient model is:

[0015]

[0016] Where f is the real-time road surface friction coefficient, N is the number of load applications, and H is the average road surface texture depth.

[0017] Preferably, if the road surface condition is a wet road surface, the friction coefficient model is:

[0018]

[0019] Where f is the real-time road surface friction coefficient, h is the water film thickness, and H is the average road surface texture depth.

[0020] More preferably, if the road surface condition is a waterlogged road surface, the minimum value of the empirical friction coefficient range of the wet road surface is taken as the friction coefficient of the waterlogged road surface.

[0021] Preferably, the images acquired in step S1 include at least road images of dry road surfaces, wet road surfaces, and waterlogged road surfaces, wherein the road images are road images of various sizes, scales, weather conditions, and time conditions.

[0022] Preferably, the neural network is a ResNet network, and the pre-trained model is trained using transfer learning to obtain a road surface state classifier.

[0023] More preferably, the road surface condition includes dry road surface, wet road surface, and waterlogged road surface.

[0024] Preferably, the process of identifying the current road surface structure parameters includes:

[0025] 1) Median filtering is applied to the road surface topography information;

[0026] 2) Convert the RGB image to a single-channel grayscale image, and then convert the single-channel grayscale image into a grayscale matrix A;

[0027] 3) Filter single-channel grayscale images. If the coefficient of variation of the grayscale image matrix A exceeds the threshold, then remove the single-channel grayscale image.

[0028] 4) Solve the image construction depth of each single-channel grayscale image through matrix operations; correct the image construction depth of each single-channel grayscale image with correction coefficients to obtain the average road surface construction depth represented by each image;

[0029] 5) Calculate the average of the road surface texture depths represented by all images to obtain the current average road surface texture depth.

[0030] More preferably, the formula for calculating the coefficient of variation of the grayscale image matrix A is:

[0031]

[0032] Where A(x,y) is the gray value of the pixel at coordinates (x,y); m and n are the number of rows and columns of matrix A, respectively. C is the mean of all elements in matrix A. v is the coefficient of variation of the grayscale image matrix.

[0033] More preferably, the formula for calculating the image construction depth is:

[0034]

[0035] Where A(x,y) is the grayscale value of the pixel at coordinates (x,y); A max is the largest gray value in the gray matrix; D is the integration region, which is the product of the number of rows and columns of the matrix; H P Construct depth for road surface images;

[0036] The current average road surface texture depth is specifically calculated as follows: the image texture depth of each single-channel grayscale image is corrected to obtain the average road surface texture depth H, and the calculation formula is as follows:

[0037]

[0038] Among them, A max Let A be the maximum value of all elements in matrix A; A min Let H be the minimum value of all elements in matrix A; β is the correction coefficient obtained experimentally; H P H represents the texture depth of the asphalt pavement image, and H represents the average texture depth of the pavement.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention inputs the current road information collected in real time into the trained neural network to identify the current road surface condition in real time; it quantifies the current road surface structure parameters through the road surface morphology information collected in real time, and finally determines the road surface friction coefficient based on the friction coefficient model. Compared with the current method of directly predicting the friction coefficient based on the road surface condition ahead, the present invention considers the differences of different asphalt pavements, quantifies the specific asphalt pavement structure parameters, improves the prediction accuracy, and quickly predicts the anti-skid performance of the current road surface in real time, avoiding the problem of not being able to estimate in real time due to complex modeling in dynamic analysis.

[0041] (2) The test method of the present invention is simple and easy to use, and can predict in real time. It does not require the installation of complicated equipment. The predicted road friction coefficient is applicable to various road conditions and environments, ensuring real-time performance while improving accuracy. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the prediction method of the present invention;

[0043] Figure 1 (a) is Figure 1 Detailed process diagram;

[0044] Figure 2 This is a schematic diagram of the camera position arrangement according to the present invention;

[0045] Figure 2 (a) is Figure 2 A quantitative diagram illustrating the camera placement;

[0046] Figure 3 This is a schematic diagram of the road surface condition classifier of the present invention;

[0047] Figure 4 This is a schematic diagram illustrating the classification of dry road surfaces in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of a dry road surface in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of the road surface structure in an embodiment of the present invention;

[0050] In the attached diagram, 1 is a forward-facing camera and 2 is a monocular camera. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] This embodiment relates to a method for predicting road surface skid resistance performance based on image recognition, such as... Figure 1 and Figure 1 (a) includes the following steps:

[0053] (1) Obtain real-world environmental information through a camera;

[0054] Based on the vehicle-mounted forward-facing camera 1 and monocular camera 2, current road surface information is collected to provide real-time data for predicting anti-skid performance. Specifically: the forward-facing camera collects information about the road ahead, and through a trained neural network, identifies the road surface as one of three conditions: dry, wet, or flooded. This road surface condition is used as one of the bases for predicting anti-skid performance. The monocular camera collects road surface morphology information, and based on this information, obtains current road surface structure parameters to quantitatively describe the roughness of the current road surface, which is also used as one of the bases for predicting anti-skid performance. In this embodiment, the camera positions and arrangement methods are as follows: Figure 2 and Figure 2 As shown in (a).

[0055] (2) Identify the current road surface condition based on neural networks;

[0056] Images of at least several different types of road surfaces, including dry, wet, and flooded surfaces, were collected. These images included varying sizes, scales, weather conditions, times of day, and lighting conditions, and also featured pedestrians and various types of vehicles. Road surface conditions were labeled for each type of image. This dataset was then used to train a neural network. A ResNet101 network was trained on the dataset using transfer learning to obtain a complete road surface condition classifier.

[0057] In this embodiment, to ensure the trained classification model has good generalization performance in various scenarios, images of different sizes and proportions are collected. Preferably, based on the resolution of mainstream cameras currently on the market, the pixels in the width and height of the collected images are mostly distributed in the range of 500 to 2000, with a maximum of 4096 pixels. Images are collected under different lighting conditions, such as cloudy and sunny days, and at different times of day. At the same time, attention is paid to including pedestrians and various vehicles to simulate real road images during driving, and road surface conditions are labeled for different types of road surface images.

[0058] The collected dataset images and the neural network training process are as follows: Figure 3 As shown. Preferably, at least 600 images of each road surface are collected as a deep learning dataset. The training and validation sets are divided in a 4:1 ratio to obtain the neural network training dataset. Transfer learning is used to train the model on the dataset using ResNet101. In this example, a dry road surface is selected, such as... Figure 4 As shown, the road surface condition classifier identifies the current road surface type in less than 0.1 seconds, and can be used in real time for subsequent prediction of the current road surface's anti-skid performance.

[0059] (3) Identify the current road surface structure parameters;

[0060] During driving, road surface topography information is collected by a monocular camera 2 to identify the current road surface structure parameters. An image of the current road surface is captured at regular intervals as the vehicle travels along the current road segment. To improve image quality, median filtering is applied to all images to make them smooth and flat. The asphalt pavement image data structure acquired by the digital device is RGB.

[0061] Since only the grayscale information of the asphalt pavement is needed, the RGB image must first be converted into a single-channel grayscale image. This is typically done using a floating-point algorithm. Then, the single-channel grayscale image is converted into a grayscale matrix A. Because the image is sensitive to lighting conditions—for example, if some parts of the image are shaded by trees—the grayscale values ​​will fluctuate significantly, with the overall grayscale values ​​in brighter areas being significantly higher than in shaded areas. This can cause substantial errors in calculating the pavement texture depth; therefore, such uneven light distribution should be avoided as much as possible.

[0062] If the coefficient of variation of the grayscale image matrix exceeds the threshold, the image is considered to have uneven light distribution, and the single-channel grayscale image is discarded.

[0063] Matrix operations are performed on the retained single-channel grayscale images to solve the image construction depth of each single-channel grayscale image. The image construction depth of each single-channel grayscale image is corrected with a correction coefficient to obtain the average road surface construction depth represented by each image. The average road surface construction depth represented by all images is averaged to obtain the current average road surface construction depth.

[0064] Convert an RGB image to a single-channel grayscale image using the following formula:

[0065] G = 0.299R + 0.587G + 0.114B (1)

[0066] Where G is the gray value of the grayscale image, and R, G, and B are the corresponding components of the color image;

[0067] The converted single-channel grayscale image can be mathematically represented as an m×n two-dimensional matrix A, i.e., the grayscale image matrix.

[0068] The formula for calculating the coefficient of variation of the grayscale image matrix A is:

[0069]

[0070] Where A(x,y) is the gray value of the pixel at coordinates (x,y); m and n are the number of rows and columns of matrix A, respectively. It is the mean of all elements in matrix A.

[0071] Based on the data from the two-dimensional digital matrix, the formula for calculating the depth of the image structure is:

[0072]

[0073] Where A(x,y) is the grayscale value of the pixel at coordinates (x,y); A max Let H be the maximum gray value in the gray matrix, D be the integration region, and H be the product of the number of rows and columns of the matrix. PDepth is constructed for asphalt pavement images.

[0074] Find H P Then, this value is corrected to obtain the average pavement texture depth H, calculated using the following formula:

[0075]

[0076] Among them, A max Let A be the maximum value of all elements in matrix A; A min Let H be the minimum value of all elements in matrix A; β is the correction coefficient obtained experimentally; H P Depth is constructed for asphalt pavement images.

[0077] In this embodiment, preferably, an image of the current road surface is collected every 10 meters within a 100m range along the vehicle's direction of travel on the current road segment. The coefficient of variation threshold is set to 5.8%. Unqualified images are discarded, and one of the qualified images is as follows: Figure 5 As shown, the surface texture diagram of this area is as follows. Figure 6 As shown, the image texture depth obtained through surface texture is 90.19. The dry road surface β is set to 1.92. After performing the above operation on the remaining qualified surface images, the average value is calculated, resulting in an average road surface texture depth of 1.21 mm in this embodiment.

[0078] (4) Predict the current anti-skid performance of the road surface based on the road surface condition and structural parameters;

[0079] When the road surface is dry, the road surface friction coefficient is preferably determined according to the following dry road surface friction coefficient model:

[0080]

[0081] Where f is the real-time road surface friction coefficient, N is the number of load applications, and H is the average road surface texture depth;

[0082] When the road surface is wet, the preferred method is to determine the road surface friction coefficient based on the following wet road surface friction coefficient model:

[0083]

[0084] Where f is the real-time road surface friction coefficient, h is the water film thickness, and H is the average road surface texture depth;

[0085] When the road surface is in the state of water accumulation, the minimum value of the empirical friction coefficient range of wet road surface, 0.45, is taken as the friction coefficient of water accumulation road surface.

[0086] In this embodiment, the friction coefficient is calculated to be 0.67 using the above dry road surface friction coefficient model.

[0087] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting road surface skid resistance performance based on image recognition, characterized in that, The method includes the following steps: Step S1: Collect current road information and current road surface topography information using the vehicle-mounted forward-facing camera and monocular camera respectively; Step S2: Input the current road information obtained in step S1 into the trained neural network to identify the current road surface condition; Step S3: Identify the current road surface structure parameters based on the current road surface morphology information obtained in step S1; Step S4: Based on the road surface condition in step S2 and the road surface structure parameters in step S3, the road surface friction coefficient is determined using a friction coefficient model, thereby predicting the current road surface skid resistance performance. The process of identifying the current road surface structure parameters includes: 1) Median filtering is applied to the road surface topography information; 2) Convert the RGB image to a single-channel grayscale image, and then convert the single-channel grayscale image into a grayscale matrix. ; 3) Filter single-channel grayscale images. If the coefficient of variation of the grayscale image matrix A exceeds the threshold, then remove the single-channel grayscale image. 4) Solve the image construction depth of each single-channel grayscale image through matrix operations; correct the image construction depth of each single-channel grayscale image with correction coefficients to obtain the average road surface construction depth represented by each image; 5) Calculate the average surface texture depth of all images to obtain the current average surface texture depth; The formula for calculating the coefficient of variation of the grayscale image matrix A is as follows: in, coordinates The grayscale value of the pixel; , They are matrices The number of rows and columns, It is a matrix The mean of all elements in the set. The coefficient of variation of the grayscale image matrix; The formula for calculating the image construction depth is: in, coordinates The grayscale value of the pixel; This is the largest gray value in the grayscale matrix; The region of integration is denoted as , and is the product of the number of rows and columns of the matrix. Construct depth for road surface images; The current average road surface texture depth is specifically obtained by correcting the image texture depth of each single-channel grayscale image. H The calculation formula is as follows: in, For matrix The maximum value of all elements in the array; For matrix The minimum value of all elements in the set; The correction factor is obtained through experiments. To construct depth for asphalt pavement images. H The average surface texture depth.

2. The method for predicting road surface skid resistance performance based on image recognition according to claim 1, characterized in that, If the road surface condition is dry, the friction coefficient model is: in, This represents the real-time road surface friction coefficient. The number of load applications. The average surface texture depth.

3. The method for predicting road surface skid resistance performance based on image recognition according to claim 1, characterized in that, If the road surface condition is a wet road surface, the friction coefficient model is: in, This represents the real-time road surface friction coefficient. For water film thickness, The average surface texture depth.

4. The method for predicting road surface skid resistance performance based on image recognition according to claim 3, characterized in that, If the road surface condition is a waterlogged road surface, the minimum value of the empirical friction coefficient range of the wet road surface shall be taken as the friction coefficient of the waterlogged road surface.

5. The method for predicting road surface skid resistance performance based on image recognition according to claim 1, characterized in that, The images acquired in step S1 include at least road images of dry road surfaces, wet road surfaces, and waterlogged road surfaces, wherein the road images are road images of various sizes, scales, weather conditions, and time conditions.

6. The method for predicting road surface skid resistance performance based on image recognition according to claim 1, characterized in that, The neural network is a ResNet network, and the pre-trained model is trained using transfer learning to obtain a road surface state classifier.

7. The method for predicting road surface skid resistance performance based on image recognition according to claim 6, characterized in that, The road surface conditions include dry road surface, wet road surface, and waterlogged road surface.

Images