Road surface skid resistance prediction method based on image recognition

Through image recognition technology, combined with neural network and friction coefficient model, dynamic and real-time prediction of road surface anti-slip performance is solved, and the problem of insufficient real-time and accuracy of prediction in the existing technology is achieved, and efficient use of various road conditions and environments is achieved predict.

CN118918069AActive Publication Date: 2024-11-08TONGJI UNIV

Patent Information

Application Number
CN202410952749.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

The prior art is difficult to predict the anti-slip performance of the pavement in a real-time changing environment, especially the failure to effectively consider the weather conditions and rough pavement texture characteristics, resulting in insufficient real-time and accuracy of the prediction.

Method used

Using an image recognition method, road information and surface morphology information are collected through vehicle-mounted front-view cameras and monocular cameras, trained neural networks to identify road surface state and structural parameters, and use friction coefficient model to predict road surface friction coefficient to predict resistance. Sliding performance. The method includes three state models of dry, wet and water-stabilized pavement. It uses the ResNet network for transfer learning, combines median filtering and grayscale image processing to quantify the pavement construction depth to improve prediction accuracy.

Benefits of technology

It realizes rapid, real-time and accurate prediction of the anti-slip performance of the pavement under a variety of road conditions and environments, avoids the complexity of traditional dynamic analysis and the complexity of equipment installation, and improves the accuracy and real-time prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918069A_ABST
    Figure CN118918069A_ABST
Patent Text Reader

Abstract

The invention relates to a road surface anti-skid performance prediction method based on image recognition. The method comprises the steps that current road information and current road surface appearance information are collected through a vehicle-mounted front-view camera and a monocular camera respectively; inputting the obtained current road information into the trained neural network, identifying a current road surface state, and identifying current road surface construction parameters based on the obtained current road surface topography information; and according to the current pavement state and the current pavement construction parameters, determining a pavement friction coefficient according to a friction coefficient model so as to predict the anti-skid performance of the current pavement. Compared with the prior art, the method has the advantages that the problem that real-time estimation cannot be achieved due to complex modeling in kinetic analysis is solved, specific asphalt pavement construction parameters are quantified, prediction precision and real-time performance are improved, and the method is suitable for various road conditions and driving environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of road surface anti-skid performance prediction, and in particular to a road surface anti-skid performance prediction method based on image recognition. Background Art

[0002] The number of vehicles on the road increases year by year, causing increasingly serious traffic problems. The anti-skid performance of asphalt pavement is the main factor affecting driving safety. Insufficient anti-skid ability will reduce the braking efficiency of vehicles. Therefore, evaluating the anti-skid performance of asphalt pavement is crucial in the road construction and operation stage. The friction coefficient between tires and pavement is the main evaluation index of anti-skid performance, and the friction coefficient of the pavement is closely related to the current state of the pavement and the roughness of the texture.

[0003] Since the construction of asphalt pavement is completed, the road surface will continue to be affected by automobile loads, wind and sand, and rain. Its surface state and texture roughness characteristics will change, and the pavement's anti-skid performance will decay over time. In addition, for the same road section, weather factors such as rainfall will also affect the pavement's anti-skid performance. The previous method of experimentally measuring anti-skid performance under fixed conditions on a fixed road surface did not take into account weather and pavement texture, and could not be applied to driving in a real-time changing environment. Therefore, it is necessary to dynamically and real-time predict the pavement friction coefficient.

[0004] At present, there are two main methods for real-time estimation of road friction coefficient. The first method is to collect data through sensors and establish a mathematical model of sensor data and friction coefficient for estimation. However, it requires additional installation of sensors, which is more complicated to debug and increases the cost of use. The second method is to establish a mathematical model based on the dynamic method for estimation. Although this method is more reliable for estimating the anti-skid performance of the road, the model is complex and requires the collection of a large number of parameters. It is difficult to cope with changing road conditions and it is difficult to ensure real-time performance. The existing vision-based anti-skid performance estimation methods of the road, such as an online estimation method and system for the road adhesion coefficient based on a vehicle-mounted camera, and an automatic emergency braking control method for the road adhesion coefficient estimation based on vision, all use neural networks to classify the road ahead and predict the anti-skid performance of the current road based on the experience gained from previous research on samples. The texture roughness characteristics of the current road are not considered, and the adhesion coefficient can only be mapped to a value or range based on experience, which makes it difficult to ensure accuracy.

[0005] After searching, Chinese invention patent CN201710589940.2 discloses a method for detecting the surface structure distribution and anti-skid performance of a road surface, including moving the road surface image acquisition system to the road surface to be tested, adjusting the angle of the image acquisition instrument through a telescopic connector, turning on the power switch button and the processor switch button, and taking images of the road surface to be tested. The captured images are transmitted to the control box through a data transmission line, and the captured images are transmitted to the image processor in turn through a wireless signal transmitter and a wireless signal transceiver for image processing, so as to obtain a road surface structure image, a road surface structure distribution uniformity analysis result, and a road surface anti-skid performance analysis result. This existing patent has the problem of using a complex device, and the test data is not comprehensive and real-time after the fixed position, so the real-time and accuracy of the prediction are insufficient.

[0006] Therefore, how to establish a reasonable road skid resistance prediction model based on visual technology, taking into account weather conditions and the roughness of the current road surface texture, and conduct dynamic, real-time and accurate predictions is a problem that needs to be solved. Summary of the invention

[0007] The purpose of the present invention is to provide a method for predicting the anti-skid performance of a road surface based on image recognition in order to overcome the defects of the above-mentioned prior art.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, a method for predicting the anti-skid performance of a road surface based on image recognition is provided, the method comprising the following steps:

[0010] Step S1, collecting current road information and current road surface morphology information respectively through the vehicle-mounted front-view camera and the monocular camera;

[0011] Step S2, inputting the current road information obtained in step S1 into the trained neural network to identify the current road surface state;

[0012] Step S3, identifying current pavement construction parameters based on the current road surface topography information obtained in step S1;

[0013] Step S4, based on the road surface state of step S2 and the road surface structural parameters of step S3, and using the friction coefficient model to determine the road surface friction coefficient, thereby predicting the current road surface anti-skid performance.

[0014] Preferably, if the road surface state is a dry road surface, the friction coefficient model is:

[0015]

[0016] Among them, f is the real-time road friction coefficient, N is the number of load actions, and H is the average structural depth of the road surface.

[0017] Preferably, if the road surface state is a wet road surface, the friction coefficient model is:

[0018]

[0019] Among them, f is the real-time road friction coefficient, h is the water film thickness, and H is the average road surface construction depth.

[0020] More preferably, if the road surface condition is a flooded road surface, the minimum value of the empirical friction coefficient range of the wet road surface is taken as the friction coefficient of the flooded road surface.

[0021] Preferably, the images collected in step S1 at least include road images of dry road surface, wet road surface and flooded road surface types, wherein the road images are road images under various sizes, ratios, weather and time conditions.

[0022] Preferably, the neural network is a ResNet network, and the pre-trained model is trained using a transfer learning method to obtain a road surface state classifier.

[0023] More preferably, the road surface conditions include dry road surface, wet road surface and flooded road surface.

[0024] Preferably, the process of identifying the current pavement structural parameters includes:

[0025] 1) Perform median filtering on the road surface morphology information;

[0026] 2) Convert the RGB image into a single-channel grayscale image, and then convert the single-channel grayscale image into a grayscale image matrix A;

[0027] 3) Screening the single-channel grayscale image, if the coefficient of variation of the grayscale image matrix A exceeds the threshold, the single-channel grayscale image is eliminated;

[0028] 4) Solving the image construction depth of each single-channel grayscale image through matrix operation; correcting the image construction depth of each single-channel grayscale image with a correction coefficient to obtain the average construction depth of the road surface represented by each image;

[0029] 5) Calculate the average structural depth of the road surface represented by all images to obtain the average structural depth of the current road surface.

[0030] More preferably, the calculation formula of the coefficient of variation of the grayscale image matrix A is:

[0031]

[0032] Among them, A(x,y) is the grayscale value of the pixel at coordinate (x,y); m and n are the number of rows and columns of matrix A respectively. is the mean of all elements in matrix A, C v is the coefficient of variation of the grayscale image matrix.

[0033] More preferably, the calculation formula of the image structure depth is:

[0034]

[0035] Among them, A(x,y) is the gray value of the pixel at coordinate (x,y); A max is the maximum gray value in the gray matrix; D is the integral area, which is the product of the number of matrix rows and columns; H P Construct depth for road surface images;

[0036] The current average road surface structural depth is specifically: the image structural depth of each single-channel grayscale image is corrected to obtain the average road surface structural depth H, and the calculation formula is as follows:

[0037]

[0038] Among them, A max is the maximum value of all elements in matrix A; A min is the minimum value of all elements in matrix A; β is the correction coefficient obtained through experiments, H P is the construction depth of the asphalt pavement image, and H is the average construction depth of the pavement.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention inputs the current road information collected in real time into a trained neural network to identify the current road surface condition in real time; quantifies the current road surface structural parameters through the road surface morphology information collected in real time, and finally determines the road surface friction coefficient based on the friction coefficient model. Compared with the current method of directly predicting the friction coefficient based on the road surface condition ahead, the present invention takes into account the differences between different asphalt pavements, quantifies the specific asphalt pavement structural parameters, improves the prediction accuracy, and quickly predicts the current road surface anti-skid performance in real time, avoiding the problem of being unable to make real-time estimates due to complex modeling in dynamic analysis.

[0041] (2) The testing method of the present invention is simple and easy to use, and can provide real-time prediction on the move without the need to install complex equipment. The predicted road friction coefficient is applicable to a variety of road conditions and environments, ensuring real-time performance while improving accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the flow chart of the prediction method of the present invention;

[0043] Figure 1 (a) Figure 1 Detailed process diagram of

[0044] Figure 2 A schematic diagram of the camera position arrangement of the present invention;

[0045] Figure 2 (a) Figure 2 Quantitative schematic diagram of camera position layout;

[0046] Figure 3 is a schematic diagram of a road surface state classifier of the present invention;

[0047] Figure 4 This is a schematic diagram of dry road surface classification in an embodiment of the present invention;

[0048] Figure 5 Schematic diagram of a dry road surface in an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the surface structure of a road surface in an embodiment of the present invention;

[0050] In the accompanying drawings, 1 is a front-view camera and 2 is a monocular camera. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] This embodiment relates to a method for predicting the anti-skid performance of a road surface based on image recognition, such as Figure 1 and Figure 1 (a), comprising the following steps:

[0053] (1) Obtain real environment information through the camera;

[0054] According to the on-board front-view camera 1 and monocular camera 2, the current road surface information is collected to provide real-time data for predicting the anti-skid performance. Among them: the front-view camera collects the road ahead information, and the trained neural network identifies the road ahead as one of the three road conditions: dry road surface, wet road surface and flooded road surface, and uses the road condition as one of the bases for predicting the anti-skid performance; the monocular camera collects the road surface morphology information, obtains the current road surface structural parameters based on the road surface morphology information, and quantitatively describes the roughness of the current road surface as one of the bases for predicting the anti-skid performance. In this embodiment, the camera position and arrangement method are as follows: Figure 2 and Figure 2 (a) shown.

[0055] (2) Identify the current road surface status based on a neural network;

[0056] Collect at least several different types of road images, including dry road, wet road and flooded road, including images of different sizes, scales, different weather and time and other lighting conditions, including pedestrians and various types of vehicles, and annotate the road conditions of different types of road images. Use them as the data set for training neural networks, use the transfer learning method to train the data set using the ResNet101 network, and obtain a complete road condition classifier.

[0057] In this embodiment, in order to make the trained classification model have good generalization performance in various scenes, images of different sizes and proportions are collected. Preferably, based on the resolution of the mainstream cameras in the current market, most of the pixels in the width and height directions of the collected images are distributed in the range of 500 to 2000, and the maximum can reach 4096 pixels. Images are collected under different weather and time conditions such as cloudy and sunny days, and at the same time, attention is paid to including pedestrians and various vehicles to simulate the real front road image during driving, and the road state of different types of road images is annotated.

[0058] Some of the collected data set images and the neural network training process are shown below: Figure 3 Preferably, at least 600 images of each road surface are collected as a deep learning data set, and the training set and the validation set are divided into a 4:1 ratio to obtain a neural network training data set. The transfer learning method is used to train the model using ResNet101 on the data set. In the embodiment, a dry road surface is selected, such as Figure 4 The road surface state classifier takes no more than 0.1 seconds to identify the current road surface type, and can be used in real time for subsequent current road surface anti-skid performance prediction.

[0059] (3) Identify current pavement structural parameters;

[0060] During driving, the current road surface structural parameters are identified through the road surface morphology information collected by the monocular camera 2. A current road surface image is collected at a certain distance during the vehicle driving on the current road section. To improve the image quality, all images are processed by median filtering to make the image smooth and flat. The asphalt road surface image data structure collected by the digital device is RGB.

[0061] Since only the grayscale information of the asphalt pavement is needed, the RGB image must be converted into a single-channel grayscale image first. Generally, the RGB image is converted into a single-channel grayscale image through a floating-point algorithm. Then the single-channel grayscale image is converted into a grayscale matrix A. Because the image is sensitive to lighting conditions, for example, if part of the image is shaded by trees, the grayscale value of the image will fluctuate greatly. The grayscale value of the part with strong light is generally larger than that of the shadow, which will cause a large error in calculating the road surface structure depth. Such uneven light distribution should be avoided as much as possible.

[0062] Select a suitable threshold. If the coefficient of variation of the grayscale image matrix exceeds the threshold, it is considered that the light distribution of the image is uneven, and the single-channel grayscale image is discarded.

[0063] Perform matrix operations on the retained single-channel grayscale images to solve the image construction depth of each single-channel grayscale image, use the correction coefficient to correct the image construction depth of each single-channel grayscale image, and obtain the average construction depth of the road surface represented by each image. The average construction depth of the road surface represented by all images is averaged to obtain the average construction depth of the current road surface.

[0064] Convert the RGB image into a single-channel grayscale image according to the following formula:

[0065] G=0.299R+0.587G+0.114B (1)

[0066] Among them, G is the grayscale value of the grayscale image, and R, G, and B are the corresponding components of the color image;

[0067] The converted single-channel grayscale image can be mathematically represented as an m×n two-dimensional matrix A, namely the grayscale image matrix.

[0068] The calculation formula of the coefficient of variation of the grayscale image matrix A is:

[0069]

[0070] Among them, A(x,y) is the grayscale value of the pixel at coordinate (x,y); m and n are the number of rows and columns of matrix A respectively. is the mean of all elements in the matrix A.

[0071] According to the data of the two-dimensional digital matrix, the calculation formula of the image construction depth is:

[0072]

[0073] Among them, A(x,y) is the gray value of the pixel at coordinate (x,y); A max is the maximum gray value in the gray matrix, D is the integral area, which is the product of the number of matrix rows and columns; H PConstruct depth for asphalt road images.

[0074] Find H P After that, the value is corrected to obtain the average road surface construction depth H, which is calculated as follows:

[0075]

[0076] Among them, A max is the maximum value of all elements in matrix A; A min is the minimum value of all elements in matrix A; β is the correction coefficient obtained through experiments, H P Construct depth for asphalt road images.

[0077] In this embodiment, preferably, a current road surface image is collected every 10m within 100m of the current road section along the vehicle travel direction, and the coefficient of variation threshold is selected as 5.8%, and unqualified images are eliminated. One of the qualified images is as follows Figure 5 The surface structure of the road in this area is shown in Figure 6 As shown, the image structure depth obtained by surface structure is 90.19. The dry road surface β is 1.92, and the remaining qualified surface images are subjected to the above operation and the average value is obtained, and the average structure depth of the road surface in this embodiment is 1.21 mm.

[0078] (4) Predict the current pavement skid resistance based on pavement conditions and structural parameters;

[0079] When the road surface is dry, preferably, the road surface friction coefficient is determined according to the following dry road surface friction coefficient model:

[0080]

[0081] Among them, f is the real-time road friction coefficient, N is the number of load actions, and H is the average structural depth of the road surface;

[0082] When the road surface state is a wet road surface, preferably, the road surface friction coefficient is determined according to the following wet road surface friction coefficient model:

[0083]

[0084] Among them, f is the real-time road friction coefficient, h is the water film thickness, and H is the average road surface structural depth;

[0085] When the road surface is flooded, the minimum value of 0.45 in the empirical friction coefficient range of wet road surfaces is taken as the friction coefficient of the flooded road surface.

[0086] In this embodiment, the friction coefficient is calculated to be 0.67 using the above dry road friction coefficient model.

[0087] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for predicting road surface anti-skid performance based on image recognition, characterized in that: The method comprises the following steps: Step S1, collecting current road information and current road surface morphology information respectively through the vehicle-mounted front-view camera and the monocular camera; Step S2, inputting the current road information obtained in step S1 into the trained neural network to identify the current road surface state; Step S3, identifying current pavement construction parameters based on the current road surface topography information obtained in step S1; Step S4, based on the road surface state of step S2 and the road surface structural parameters of step S3, and using the friction coefficient model to determine the road surface friction coefficient, thereby predicting the current road surface anti-skid performance.

2. The method for predicting road surface anti-skid performance based on image recognition according to claim 1, characterized in that: If the road surface is dry, the friction coefficient model is: Among them, f is the real-time road friction coefficient, N is the number of load actions, and H is the average structural depth of the road surface.

3. The method for predicting the anti-skid performance of a road surface based on image recognition according to claim 1, characterized in that: If the road surface is a wet road, the friction coefficient model is: Among them, f is the real-time road friction coefficient, h is the water film thickness, and H is the average road surface construction depth.

4. The method for predicting the anti-skid performance of a road surface based on image recognition according to claim 3, characterized in that: If the road surface condition is a flooded road surface, the minimum value of the empirical friction coefficient range of the wet road surface is taken as the friction coefficient of the flooded road surface.

5. The method for predicting the anti-skid performance of a road surface based on image recognition according to claim 1, characterized in that: The images collected in step S1 at least include road images of dry road surface, wet road surface and flooded road surface, wherein the road images are road images under various sizes, ratios, weather and time conditions.

6. The method for predicting the anti-skid performance of a road surface based on image recognition according to claim 1, characterized in that: The neural network is a ResNet network, and the pre-trained model is trained using a transfer learning method to obtain a road surface state classifier.

7. The method for predicting road surface anti-skid performance based on image recognition according to claim 6, characterized in that: The road surface conditions include dry road surface, wet road surface and flooded road surface.

8. The method for predicting road surface anti-skid performance based on image recognition according to claim 1, characterized in that: The process of identifying the current pavement structural parameters includes: 1) Perform median filtering on the road surface morphology information; 2) Convert the RGB image into a single-channel grayscale image, and then convert the single-channel grayscale image into a grayscale image matrix A; 3) Screening the single-channel grayscale image, if the coefficient of variation of the grayscale image matrix A exceeds the threshold, the single-channel grayscale image is eliminated; 4) Solving the image construction depth of each single-channel grayscale image through matrix operation; correcting the image construction depth of each single-channel grayscale image with a correction coefficient to obtain the average construction depth of the road surface represented by each image; 5) Calculate the average structural depth of the road surface represented by all images to obtain the average structural depth of the current road surface.

9. The method for predicting the anti-skid performance of a road surface based on image recognition according to claim 8, characterized in that: The calculation formula of the coefficient of variation of the grayscale image matrix A is: Among them, A(x,y) is the grayscale value of the pixel at coordinate (x,y); m and n are the number of rows and columns of matrix A respectively. is the mean of all elements in matrix A, C v is the coefficient of variation of the grayscale image matrix.

10. The method for predicting road surface anti-skid performance based on image recognition according to claim 8, characterized in that: The calculation formula of the image structure depth is: Among them, A(x,y) is the gray value of the pixel at coordinate (x,y); A max is the maximum gray value in the gray matrix; D is the integral area, which is the product of the number of matrix rows and columns; H P Construct depth for road surface images; The current average road surface structural depth is specifically: the image structural depth of each single-channel grayscale image is corrected to obtain the average road surface structural depth H, and the calculation formula is as follows: Among them, A max is the maximum value of all elements in matrix A; A min is the minimum value of all elements in matrix A; β is the correction coefficient obtained through experiments, H P is the construction depth of the asphalt pavement image, and H is the average construction depth of the pavement.

Citation Information

Patent Citations

  • Machine vision based rapid detection method for pavement skid resistance

    CN108760740A

  • Full-automatic processing method for safe driving under water accumulation condition

    CN111563478A

  • Construction method of asphalt pavement anti-slide performance prediction system and constructed system thereof

    CN115128074A

  • Comfort-based self-driving planning method

    US20200406925A1

Cited By

  • Pavement surface skid resistance prediction method based on image recognition

    CN120013899A