A method and device for identifying ice and snow road surface, a system and a storage medium

Abstract

The present invention discloses a method and device, system and storage medium for identifying an icy and snowy road surface, including: obtaining road surface reflection infrared spectrum data and vehicle driving road surface image data; judging whether the road surface on which the vehicle is driving has ice, snow accumulation or snow-covered ice according to the road surface reflection infrared spectrum data and vehicle driving road surface image data. The technical solution of the present invention is adopted to realize the identification of the ice and snow coverage on the road, and improve the safety factor of vehicles driving on roads covered with ice and snow in high-cold areas.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method and device, a system, and a storage medium for identifying an icy and snowy road surface. Background Art

[0002] Highways are the transportation infrastructure with the widest coverage and the largest population. On highways in high-altitude and cold regions, the road surface is often covered with ice and snow. Traffic accidents caused by drivers' inaccurate judgment of road conditions during driving lead to traffic jams, affecting road traffic and causing great safety hazards to driving and road safety. At present, it is still difficult for vehicles to identify the conditions of icy and snowy roads when driving in high-altitude and cold regions. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device, system, and storage medium for identifying ice and snow road surfaces, so as to realize the identification of ice and snow coverage conditions on roads; during driving, the image information and spectral information of ice and snow coverage on the road surface in front of the vehicle are obtained in real time, the road surface conditions are identified through a deep learning algorithm, and the safe vehicle speed under the current road surface conditions is calculated, thereby improving the safety factor of vehicles driving on ice and snow covered roads in high-altitude and cold areas.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A method for identifying an icy and snowy road surface, comprising:

[0006] Step S1, obtaining road surface reflection infrared spectrum data and vehicle driving road surface image data;

[0007] Step S2, preprocessing the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction;

[0008] Step S3, dividing the road surface in front of the vehicle into multiple sub-areas according to the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and finding the spectrum data corresponding to the position of each sub-area, and fusing the spectrum data with the image data;

[0009] Step S4, obtaining a road ice and snow cover recognition model for identifying road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions based on the fused spectral data and image data;

[0010] Step S5: input the real-time road surface reflection infrared spectrum data and the vehicle driving road surface image data into the road ice and snow cover recognition model to perform ice and snow road surface recognition.

[0011] Preferably, step S4 comprises:

[0012] According to the fused spectral data and image data, the number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road surface covering, and the texture characteristics are extracted;

[0013] The number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road cover, and the texture characteristics are used as input data, and the road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions are used as output data to train the BP neural network to obtain a road ice and snow cover recognition model.

[0014] Preferably, the method further comprises: step S6, obtaining the safe driving speed under different road conditions according to the recognition result of the road ice and snow cover recognition model, and the calculation formula is:

[0015]

[0016] Among them, V is the safe driving speed of the vehicle; M is the weight of the vehicle; t is the driver's braking reaction time; T is the friction coefficient between the vehicle and the road surface; g is the gravitational acceleration exerted on the vehicle; Z is other resistances such as air resistance; θ is the angle between the horizontal plane and the road surface, when θ>0, it is uphill, and when θ<0, it is downhill; S is the safe braking distance of the vehicle; the road surface is icy, snowy, black ice, snow-covered ice, or dry.

[0017] The present invention also provides an ice and snow road surface recognition device, comprising:

[0018] An acquisition module is used to acquire road surface reflection infrared spectrum data and vehicle driving road surface image data;

[0019] A preprocessing module is used to preprocess the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction;

[0020] A fusion module is used to divide the road surface in front of the vehicle into multiple sub-areas based on the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and find the spectrum data corresponding to the position of each sub-area, and fuse the spectrum data with the image data;

[0021] A training module is used to obtain a road ice and snow cover recognition model for identifying road ice, snow accumulation, black ice, snow-covered ice, and dry coverage based on the fused spectral data and image data;

[0022] The recognition module is used to input the real-time road surface reflection infrared spectrum data and the vehicle driving road surface image data into the road ice and snow cover recognition model to perform ice and snow road surface recognition.

[0023] Preferably, the training module includes:

[0024] An extraction unit, used to extract the number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road covering, and the texture characteristics according to the fused spectral data and image data;

[0025] The training unit is used to train the BP neural network using the number of wave peaks, the number of wave troughs, the signal strength at the corresponding wavelength, the color of the road cover, and the texture characteristics as input data and using the road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions as output data to obtain a road ice and snow cover recognition model.

[0026] Preferably, a calculation module is also included for obtaining the safe driving speed under different road conditions according to the recognition result of the road ice and snow cover recognition model.

[0027] The present invention also provides an icy and snowy road surface recognition system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an icy and snowy road surface recognition method when executed by the processor.

[0028] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the ice and snow road surface recognition method when running.

[0029] The present invention can accurately identify the ice and snow coverage of the road; reduce the probability of traffic accidents caused by ice and snow covered roads in high-altitude cold areas, and ensure driving safety; obtain the friction coefficient between the vehicle and the current road surface through the current road surface condition, and then calculate the safe speed of the vehicle under different ice and snow coverage conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 A flowchart of a method for identifying an icy and snowy road surface;

[0032] Figure 2 Provide a schematic diagram for the layout of data acquisition equipment;

[0033] Figure 3This is a schematic diagram of diffuse reflection monitoring;

[0034] Figure 4 It is a schematic diagram of the method for fusing spectral data and image data;

[0035] Figure 5 Schematic diagram of road surface ice and snow coverage status identification. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Embodiment 1:

[0039] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying icy and snowy roads. By real-time collection and analysis of vehicle road status data, it is determined whether there is ice, snow accumulation, or snow-covered ice on the road surface on which the vehicle is traveling. At the same time, the safe driving speed of the vehicle on the road surface is analyzed based on the recognition result, thereby fully ensuring the safety of vehicles traveling on roads in high-altitude and cold regions.

[0040] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying an icy and snowy road surface, comprising:

[0041] Step S1, obtaining road surface reflection infrared spectrum data and vehicle driving road surface image data;

[0042] Step S2, preprocessing the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction;

[0043] Step S3, dividing the road surface in front of the vehicle into multiple sub-areas according to the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and finding the spectrum data corresponding to the position of each sub-area, and fusing the spectrum data with the image data;

[0044] Step S4, obtaining a road ice and snow cover recognition model for identifying road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions based on the fused spectral data and image data;

[0045] Step S5: Obtain safe driving speeds under different road conditions according to the recognition results of the road ice and snow cover recognition model.

[0046] As an implementation method of the present invention, in step S1, Figure 2 As shown, considering that the road spectrum and image data need to be collected in real time during movement during the data collection process, the micro-spectrometer and high-definition camera are fixedly installed on the top of the vehicle to prevent the data collection equipment from tilting due to road bumps and ensure the consistency of data collection. The collected spectral data and image information are transmitted to the edge computer installed on the vehicle by wire to ensure stable data transmission, and finally subsequent analysis and processing are carried out at the edge computer.

[0047] like Figure 3 As shown, due to the uneven road surface in high-altitude cold areas, diffuse reflection monitoring is considered to be more stable and reliable than near-infrared transmission detection and near-infrared reflection detection, suitable for long-term operation and harsh environments, and can achieve high-precision measurement in spectral signal acquisition. Therefore, diffuse reflection monitoring is used to collect spectral signals of road coverings, where the formula for calculating reflected light intensity is shown below.

[0048]

[0049] Among them, I γ is the reflected light intensity; I0 is the incident light intensity; S(r) represents the correlation function of the material's light absorption coefficient r; P(d) represents the diffuse reflection coefficient function of the detection sample thickness d; R represents the detection distance from the detection sample surface to the receiving device.

[0050] By fusing spectral data and image data for identification, both data need to be preprocessed before data fusion to improve data quality and availability, provide better input for subsequent model training, and thus improve the accuracy and generalization of the model. In terms of spectral data preprocessing, after the spectral data reflected by the road is collected and transmitted to the edge calculator, the spectral data is first preliminarily processed by the reflectivity normalization method to reduce the error in the spectral data collection process and enhance the data sample difference. The so-called normalization is to find the maximum value of the reflectivity curve and calculate the ratio of all reflectivities to this maximum value. The reflectivity normalization method can eliminate the measurement error during spectral data collection, accelerate the subsequent model convergence speed and improve the model accuracy, and prevent the instability of the neural network caused by the model gradient explosion during the model training. The calculation method is shown in the formula.

[0051]

[0052] Among them, R γ is the normalized reflectivity at wavelength γ; I γ is the reflected light intensity; D(I γ ) is the background light intensity; Ref(I γ ) is the reference background light intensity; max(R) is the maximum reflectivity at different wavelengths in the spectral curve.

[0053] Because the vehicle is in motion during the process of collecting spectral data, there may be factors such as unstable light source or instrument measurement offset, or the spectral data may have too much noise and severe signal intensity jitter due to factors such as instrument noise, measurement error and environmental noise, resulting in the spectral curve not being smooth enough and many burrs appearing. In order to eliminate the influence of these factors on the feature extraction of spectral data, the model effect is not good. In order to retain the shape and characteristics of the original data, effectively remove unnecessary noise, and retain the edges or transitions in the data to facilitate feature extraction of subsequent models, a five-point cubic smoothing filter algorithm is used to eliminate the noise in the spectral curve. The algorithm is shown in the formula.

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] Where n (n≥5) is the number of spectral data nodes, i = 3, 4, 5...n-2; where Y~ n and Y~ i It is the value of the nth or ith spectral data node after smoothing.

[0060] In terms of road image preprocessing, first, after the road image in front of the vehicle is collected by a high-definition camera, the vehicle is in motion during the collection process, and when driving in high-altitude cold areas, it often faces a variety of complex weather environments such as strong winds, snow and fog, which leads to the road image being blurred, low resolution and too dark, which is difficult to be recognized by the model. In order to reduce the influence of environmental factors, the image super-resolution reconstruction based on the super-resolution generative adversarial network (Super Resolution Generative Adversarial Networks) is used for super-definition processing. SRGAN can convert low-resolution images into high-resolution images, greatly improving the details and clarity of the image. SRGAN introduces a generator network and a discriminator network for adversarial training. The generator network is responsible for converting low-resolution road images into high-resolution images, while the discriminator network is used to compare the real road high-resolution images with the high-resolution images generated by the generator network. Finally, the generator network is gradually trained to generate more realistic and detailed high-resolution road images. In the process of training the generator network, SRGAN uses a generative adversarial approach to perform super-resolution reconstruction of images. At the same time, it proposes a loss function consisting of content loss and adversarial loss, and weights them with certain weights. The weighted formula is shown below.

[0061]

[0062] Among them, l SR To reconstruct the image; Content Loss; It is Adversarial Loss.

[0063] There are two Content Loss schemes. One is the traditional MSE loss function, which can obtain a very high signal-to-noise ratio, but the images generated in this way have the problem of missing high-frequency details. The other is the VGG loss function. VGG loss uses the feature vector of a pre-trained 19-layer VGG network to calculate the Euclidean distance between the feature representation of the generated image and the reference image, and proposes a feature map of a certain layer on the trained VGG network, and compares the generated image with the feature map of the output layer of the real image.

[0064] The formulas of the two loss functions are shown below.

[0065]

[0066]

[0067] in, is the MSE loss; W is the image width; H is the image height; xy is the pixel position; l HR For high-resolution images; is the generator network; l LR For low-resolution images; is the VGG loss; define φ i,j W represents the feature map output after the jth convolution layer of the VGG network and before the i-th maximum pooling layer; i,j and H i,j Respectively identify the width and height of the feature map output after the j-th convolution layer of the VGG network and before the i-th maximum pooling layer.

[0068] Among them, Adversarial Loss is a commonly used form in GAN. Its purpose is to generate data distribution that the discriminant network cannot discriminate. Its formula is as follows.

[0069]

[0070] in, For the judgement network.

[0071] Then, the image is intelligently segmented through the Segment Anything Model (SAM), and the SAM is adjusted to segment the road and the surrounding environment in the image in a targeted manner. The adjustment of the image segmentation model requires the training model of the ice and snow road image dataset, and the image segmentation model parameters are optimized according to the MSE loss function to achieve continuous optimization of the image segmentation model. The specific steps for fine-tuning SAM are as follows.

[0072] The image segmentation model fine-tuning has the following steps;

[0073] ① Create a dataset of icy and snowy road images

[0074] In order to enable the image segmentation model to efficiently and accurately segment the images collected by the vehicle, it is necessary to select or create a dataset of icy and snowy road images and use this dataset to train the image segmentation model to enhance the image segmentation model's segmentation effect on icy and snowy roads.

[0075] ②Data preprocessing

[0076] First, the image needs to be resized to a standard size, and then the scan results are processed from NumPy arrays to PyTorch tensors, and finally the SAM preprocessing method is used to complete the preprocessing. The SAM preprocessing method first needs to scale the longest side of the image to a fixed value, and fix the aspect ratio of the image while scaling to prevent the image from being out of proportion; then, the color of the image is normalized by subtracting the mean from each pixel and dividing by the standard deviation. The purpose of color normalization is to unify the color range of different images and reduce the impact of different lighting conditions and device differences on model training, thereby improving the generalization ability of the image segmentation model and speeding up the training; finally, padding is applied to the image to ensure that all images have the same dimensions.

[0077] ③Training Loop

[0078] In the main training loop, a faster GPU is used to traverse each data item, then generate masks and compare them with the true masks in order to optimize the image segmentation model parameters according to the loss function MSE, and finally obtain a fine-tuned image segmentation model.

[0079]

[0080] Among them, y i is the true value; is the predicted value; n is the number of samples

[0081] By fine-tuning the image segmentation model through the ice and snow road image dataset, the image segmentation model can be continuously optimized and the segmentation efficiency and accuracy of the SAM segmentation model can be improved.

[0082] After the road image segmentation is completed, due to the complex situation faced during the photo acquisition process, the grayscale value conversion of the image is often different from the actual situation, which will affect the subsequent image processing. Therefore, the grayscale correction method is used to process the image to enhance the grayscale range and enrich the grayscale level to achieve enhanced image contrast and resolution.

[0083] As an implementation of the present invention, in step S3, considering that in actual situations, the road surface is covered with ice and snow in a complex manner, and often multiple coverage conditions such as ice, snow accumulation, and snow-covered ice exist simultaneously in the same road section. Therefore, in order to accurately identify the road surface coverage and determine the ice and snow coverage of the current driving road surface, the road surface in front of the vehicle is divided into multiple sub-areas, and the spectral data and image data of each sub-area are fused.

[0084] like Figure 4As shown in the figure, in the data fusion process, the pixel coordinate system of the road image is first converted into the world coordinate system, and the world coordinate position of each area of ​​the road surface in the current road image is obtained through the world coordinate system. The conversion relationship between the pixel coordinate system and the world coordinate system is shown in the following formula.

[0085]

[0086] Among them, Z C is the position variable; u and v are the coordinate positions in the pixel coordinate system; f is the focal length of the camera; is the shooting posture angle of the high-definition camera, is the transformation matrix between the fuselage coordinates and the ground coordinates; (x E ,y E ,z E )Coordinates in the world coordinate system.

[0087] From the above formula, we can see that there is only one position variable (scale variable) Z when converting from a point in the world coordinate system to a point in the pixel coordinate system. C , when the camera height h and attitude angle are known And the projection of a point in the world coordinate system in the physical coordinate system (x I ,y I ), the world coordinate system coordinates (x E ,y E ,z E ).

[0088] Then, the road surface is divided into multiple sub-areas according to actual needs, and the world coordinate system coordinate value of each sub-area is obtained to fully ensure the accuracy of road surface recognition. At the same time, the spectral data of each sub-area is extracted, corresponding to the coordinate position of each sub-area, and the corresponding spectral data is bound to the coordinate position of the road surface sub-area; finally, each sub-area of ​​the road surface in front of the vehicle has corresponding image data and spectral data.

[0089] As an implementation of the embodiment of the present invention, in step S4, Figure 5As shown in the figure, the BP neural network is selected for training to obtain a road ice and snow cover recognition model; in the training process, firstly, the processed data sample set is selected and the input features (number of peaks, number of troughs, signal strength at the corresponding wavelength, road surface cover color, texture features) and output labels (road icing, snow accumulation, black ice, snow-covered ice, dry coverage) of the samples are marked; then, a hidden layer is constructed, the input sample is forward propagated, the error of the output layer is calculated, and the error is back-propagated to update the weights and bias of the neural network, and the forward propagation and back-propagation process is repeated continuously; secondly, by testing the performance of the trained BP neural network on the test set, the BP neural network parameters are adjusted, and the BP neural network is continuously optimized to finally obtain a road ice and snow cover recognition model.

[0090] In the process of building the BP neural network model, the number of layers of the input layer and the output layer is fixed to a single layer, and the number of layers of the hidden layer is set according to the sample characteristics and the required requirements. The number of nodes in the input layer and the output layer is determined according to the actual needs of ice and snow road recognition, and the number of nodes in the hidden layer is determined by an empirical formula, which is shown below.

[0091]

[0092] Among them, l is the number of hidden layer neuron nodes; m is the number of input layer neuron nodes; n is the number of output layer neuron nodes; a is the adjustment constant.

[0093] The loss function in the BP neural network defines the difference measure between the predicted result and the actual result. The loss function is used to quantify the difference between the predicted result and the true value, and is a calculation method for the gap between the predicted value and the true value. In order to improve the training speed and effect of the road ice and snow cover recognition model, find the optimal parameters, and improve the generalization ability of the road ice and snow cover recognition model, considering that the cross entropy error loss function is more suitable for some classification scenarios, the cross entropy error loss function is selected, and its formula is shown as follows.

[0094]

[0095] Where N is the number of sample data; t nk Represents the value of the kth element of the nth data; y nk Output of the neural network.

[0096] In order to find the optimal solution for the parameters that minimize the loss function, the gradient descent method is used to gradually search the parameter space, and the parameters are continuously updated iteratively to calculate the gradient of the loss function relative to the parameters, and the parameters are updated according to the direction of the gradient and the learning rate, and finally the optimal solution that minimizes the loss function is found. In order to reduce the number of iterations required for convergence and make the converged result closer to the effect of gradient descent, the mini-batch gradient descent method is used to train the neural network model, and its formula is shown below.

[0097]

[0098] Where η represents the learning rate, which is usually a small value and determines how large the step size will be to update the parameters; J(θ; x i:i+n ;y i:i+n ) indicates that the loss function calculates the first-order derivative of the parameter θ on each data.

[0099] As an implementation method of the present invention, in step S6, the road condition is identified according to step S5, and the safe driving speed of the vehicle on the current road is obtained by combining the friction coefficient between each road condition and the vehicle. The calculation formula is:

[0100]

[0101] Among them, V is the safe driving speed of the vehicle; M is the weight of the vehicle; t is the driver's braking reaction time; T is the friction coefficient between the vehicle and the road; g is the gravitational acceleration exerted on the vehicle; Z is other resistances such as air resistance; θ is the angle between the horizontal plane and the road surface, when θ>0, it is uphill, and when θ<0, it is downhill; S is the safe braking distance of the vehicle.

[0102] Finally, the driver is informed of the road conditions, safe driving speed and safe driving skills of the current road through the display system or voice system of the vehicle terminal, and the vehicle assisted driving system is used for assisted driving to ensure the vehicle's driving safety under various complex road conditions such as icing, snow accumulation, and snow-covered ice in high-altitude areas, fully protecting the driver's life and property.

[0103] The embodiment of the present invention collects infrared spectral data through a micro-spectrometer, and considers the unevenness of the road surface, and uses a diffuse reflection monitoring method in the process of collecting spectral data; in image data collection, a high-definition camera is used to obtain image data of the road surface on which the vehicle is traveling in real time. In data preprocessing, the spectral data and image data need to be preprocessed respectively. First, the road spectral data is processed using a reflectivity normalization method to avoid the influence of errors generated during the acquisition process, and then a five-point cubic smoothing filter algorithm is used for smoothing and noise reduction processing to make the spectral curve smoother, which is convenient for subsequent training models and improving the accuracy of model recognition. Secondly, the road image data is ultra-cleared to enhance the clarity of the road image and reduce the influence of image blur caused by road bumps or vehicle movement. Thirdly, the image segmentation model (Segment Anything Model) is used to segment and extract the road image to remove the influence of the surrounding environment in the road image; finally, the grayscale correction method is used to process the image to enhance the grayscale variation range and enrich the grayscale level, so as to enhance the contrast and resolution of the image. In terms of spectral and image data fusion, considering that ice, snow, snow-covered ice, etc. may appear simultaneously on the same road section in high-cold areas, the road image is divided into multiple sub-areas for identification, and the spectral data corresponding to the position of each sub-area is found based on the world coordinate system, and finally the spectral data is fused with the image data; in terms of model training, based on the fused spectral data and image data, a road ice and snow cover recognition model for identifying road ice, snow accumulation, black ice, snow-covered ice, and dry coverage is obtained, and the real-time road surface reflection infrared spectral data and vehicle driving road surface image data are input into the road ice and snow cover recognition model for ice and snow road surface identification to determine the road surface coverage with ice and snow; in terms of decision support, the safe vehicle speed under the current conditions is calculated to provide decision support to the driver.

[0104] Embodiment 2:

[0105] The embodiment of the present invention further provides an icy and snowy road surface recognition device, comprising:

[0106] An acquisition module is used to acquire road surface reflection infrared spectrum data and vehicle driving road surface image data;

[0107] A preprocessing module is used to preprocess the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction;

[0108] A fusion module is used to divide the road surface in front of the vehicle into multiple sub-areas based on the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and find the spectrum data corresponding to the position of each sub-area, and fuse the spectrum data with the image data;

[0109] A training module is used to obtain a road ice and snow cover recognition model for identifying road ice, snow accumulation, black ice, snow-covered ice, and dry coverage based on the fused spectral data and image data;

[0110] The recognition module is used to input the real-time road surface reflection infrared spectrum data and the vehicle driving road surface image data into the road ice and snow cover recognition model to perform ice and snow road surface recognition.

[0111] As an implementation of an embodiment of the present invention, the training module includes:

[0112] An extraction unit, used to extract the number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road covering, and the texture characteristics according to the fused spectral data and image data;

[0113] The training unit is used to train the BP neural network using the number of wave peaks, the number of wave troughs, the signal strength at the corresponding wavelength, the color of the road cover, and the texture characteristics as input data and using the road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions as output data to obtain a road ice and snow cover recognition model.

[0114] As an implementation manner of an embodiment of the present invention, it also includes a calculation module for obtaining a safe driving speed under different road conditions according to the recognition result of the road ice and snow cover recognition model.

[0115] Embodiment 3:

[0116] An embodiment of the present invention further provides an icy and snowy road surface recognition system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an icy and snowy road surface recognition method when executed by the processor.

[0117] Embodiment 4:

[0118] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the ice and snow road recognition method when running.

[0119] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for identifying icy and snowy roads, characterized in that: include: Step S1, obtaining road surface reflection infrared spectrum data and vehicle driving road surface image data; Step S2, preprocessing the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction; Step S3, dividing the road surface in front of the vehicle into multiple sub-areas according to the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and finding the spectrum data corresponding to the position of each sub-area, and fusing the spectrum data with the image data; Step S4, obtaining a road ice and snow cover recognition model for identifying road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions based on the fused spectral data and image data; Step S5, inputting the data obtained by integrating the real-time road surface reflection infrared spectrum data and the vehicle driving road surface image data into the road ice and snow cover recognition model to perform ice and snow road surface recognition; Step S4 includes: According to the fused spectral data and image data, the number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road surface covering, and the texture characteristics are extracted; The number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road cover, and the texture characteristics are used as input data, and the road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions are used as output data to train the BP neural network to obtain a road ice and snow cover recognition model.

2. The method for identifying an icy and snowy road surface according to claim 1, characterized in that: The method also includes: step S6, obtaining the safe driving speed under different road conditions according to the recognition result of the road ice and snow cover recognition model, and the calculation formula is: Among them, V is the safe driving speed of the vehicle; M is the weight of the vehicle; t is the driver's braking reaction time; T is the friction coefficient between the vehicle and the road surface; g is the gravitational acceleration exerted on the vehicle; Z is other resistances such as air resistance; θ is the angle between the horizontal plane and the road surface, when θ>0, it is uphill, and when θ<0, it is downhill; S is the safe braking distance of the vehicle; the road surface is icy, snowy, black ice, snow-covered ice, or dry.

3. A device for identifying ice and snow road surface, characterized in that: include: An acquisition module is used to acquire road surface reflection infrared spectrum data and vehicle driving road surface image data; A preprocessing module is used to preprocess the road surface reflection infrared spectrum data and the vehicle driving road surface image data, wherein the road surface reflection infrared spectrum data is subjected to reflection normalization processing and smoothing filtering processing, and the vehicle driving road surface image data is subjected to image ultra-definition processing, image segmentation and grayscale correction; A fusion module is used to divide the road surface in front of the vehicle into multiple sub-areas based on the pre-processed road surface reflection infrared spectrum data and the vehicle driving road surface image data, and find the spectrum data corresponding to the position of each sub-area, and fuse the spectrum data with the image data; A training module is used to obtain a road ice and snow cover recognition model for identifying road ice, snow accumulation, black ice, snow-covered ice, and dry coverage based on the fused spectral data and image data; The recognition module is used to input the real-time road surface reflection infrared spectrum data and the vehicle driving road surface image data into the road ice and snow cover recognition model to perform ice and snow road surface recognition; The training modules include: An extraction unit, used to extract the number of peaks, the number of troughs, the signal strength at the corresponding wavelength, the color of the road covering, and the texture characteristics according to the fused spectral data and image data; The training unit is used to train the BP neural network using the number of wave peaks, the number of wave troughs, the signal strength at the corresponding wavelength, the color of the road cover, and the texture characteristics as input data and using the road icing, snow accumulation, black ice, snow-covered ice, and dry coverage conditions as output data to obtain a road ice and snow cover recognition model.

4. The icy and snowy road surface recognition device according to claim 3, characterized in that: It also includes a calculation module for obtaining the safe driving speed under different road conditions according to the recognition result of the road ice and snow cover recognition model. The calculation formula is: Among them, V is the safe driving speed of the vehicle; M is the weight of the vehicle; t is the driver's braking reaction time; T is the friction coefficient between the vehicle and the road surface; g is the gravitational acceleration exerted on the vehicle; Z is other resistances such as air resistance; θ is the angle between the horizontal plane and the road surface, when θ>0, it is uphill, and when θ<0, it is downhill; S is the safe braking distance of the vehicle; the road surface is icy, snowy, black ice, snow-covered ice, or dry.

5. A snow and ice road recognition system, comprising: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for identifying an icy and snowy road surface as described in any one of claims 1-2 is executed.

6. A storage medium having a computer program stored thereon, wherein the computer program, when run, executes the method for identifying an icy and snowy road surface as described in any one of claims 1 to 2.

Images