Road icing visual detection method based on deep learning

Abstract

The present invention provides a road icing visual detection method based on deep learning, the method comprising: step S1: acquiring road images in different states to form a road image data set; step S2: constructing and training an improved Resnet-18 model; constructing a feature extraction module, the feature extraction module comprising an average pooling layer, a convolution layer, four parallel hole convolution layers, a first feature fusion module, an average pooling layer and a convolution layer connected in sequence; using the feature extraction module to enhance the shallow features extracted by the pre-trained Resnet-18 model, and then integrating them into the deep features extracted by the pre-trained Resnet-18 model, thereby obtaining an improved Resnet-18 model; step S3: using the improved Resnet-18 model to process road image data to obtain the probabilities of different road categories. The present invention has the advantages of high detection accuracy and intelligence.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a road icing visual detection method based on deep learning. Background Art

[0002] Road icing caused by temperature and precipitation reduces the friction of the road, posing a serious threat to traffic safety. Therefore, timely identification of road icing is crucial to ensuring traffic safety and improving transportation efficiency. At present, research and related patents on road icing condition detection are mainly divided into two categories: sensor detection and visual detection.

[0003] The sensor detection method refers to the use of various types of sensors to detect physical quantities such as the temperature and humidity of the road surface, and then determine whether the road surface is frozen. For example, the Chinese patent application "A road ice condition detection and automatic ice melting system" with application number CN201510070262.X, the Chinese patent application "Highway road icing monitoring device" with application number CN202020741447.5, and the Chinese patent application "A highway tunnel entrance road icing detection and early warning device" with application number CN202022293038.6 all use ice sensors to detect road conditions in real time, and when the detected value exceeds the threshold, it is determined that the road surface is frozen. However, compared with the visual detection method, the sensor detection method is more expensive and has certain limitations when deployed on a large scale.

[0004] The visual detection method uses a camera to capture road images and uses image processing technology to identify the icing state. For example, the Chinese patent application "A road icing monitoring method and system based on road cameras" with application number CN201811507302.2 uses traditional image feature extraction and classification methods to detect icing images. However, traditional methods rely on artificially designed image features and are difficult to fully capture the complex features of road icing, resulting in low detection accuracy.

[0005] In summary, traditional road icing detection methods rely on manually designed features, which often cannot fully capture the complex characteristics of road icing, resulting in low detection accuracy. Especially under different weather conditions and lighting conditions, the performance of traditional methods may be greatly reduced. Existing road icing detection methods, whether based on sensors or traditional vision, have the problems of low intelligence and low detection efficiency. Summary of the invention

[0006] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a road icing visual detection method based on deep learning, which has a simple principle, a high degree of intelligence, a small number of parameters, and can improve detection efficiency and accuracy.

[0007] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0008] A road icing visual detection method based on deep learning includes the following steps:

[0009] Step S1: Acquire road surface images in different states to form a road surface image data set;

[0010] Step S2: Get the pre-trained Resnet-18 model;

[0011] Constructing a feature extraction module, wherein the feature extraction module includes an average pooling layer, a convolution layer, four parallel hole convolution layers, a first feature fusion module, an average pooling layer and a convolution layer connected in sequence;

[0012] The four parallel atrous convolutional layers include: a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 1, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 3, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 6, and a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 9;

[0013] The first feature fusion module performs weighted fusion on the outputs of four parallel dilated convolutional layers through a channel attention mechanism;

[0014] After enhancing the shallow features extracted by the pre-trained Resnet-18 model using the feature extraction module, multi-scale fusion is performed with the deep features extracted by the pre-trained Resnet-18 model through skip connections, thereby obtaining an improved Resnet-18 model;

[0015] Using the road image dataset to train the improved Resnet-18 model, and saving the trained improved Resnet-18 model;

[0016] Step S3: Use the improved Resnet-18 model to process the road surface image data to obtain the probabilities of different road surface categories, and determine the final road surface state according to the maximum probability.

[0017] As a further improvement of the method of the present invention, it also includes constructing a dynamic threshold model based on the relationship between temperature and road icing, and the model corrects the output probability of the improved Resnet-18 model through the following formula:

[0018] ,

[0019] in, P i ′ represents the corrected icing probability, P irepresents the icing probability output by the improved Resnet-18 model, T Indicates the road surface temperature, L Indicates the preset critical temperature threshold, k Represents the temperature sensitivity coefficient.

[0020] As a further improvement of the method of the present invention, it also includes constructing a dynamic threshold model based on the relationship between temperature and road icing, and the dynamic threshold model corrects the dry road surface and wet road surface probabilities output by the improved Resnet-18 model through the following formula:

[0021] ,

[0022] ,

[0023] Wherein, L represents the preset critical temperature threshold, T Indicates the road surface temperature, k represents the temperature sensitivity coefficient, represents the corrected dry road probability, P d Represents the dry road probability output by the improved Resnet-18 model; represents the corrected wet road probability, P w Represents the wet road probability output by the improved Resnet-18 model.

[0024] As a further improvement of the method of the present invention: after using the improved Resnet-18 model to process the road surface image data, the probability of each category of road surface conditions is obtained, wherein the road surface condition corresponding to the maximum probability is used as the final output road surface condition, and the road surface condition includes one or more of dry road surface, wet road surface, and icy road surface.

[0025] As a further improvement of the method of the present invention: in the step S2, the pre-trained Resnet-18 model includes a convolutional layer, a maximum pooling layer, a multi-layer convolutional layer and a second feature fusion module connected in sequence; the shallow feature map output by the maximum pooling layer in the pre-trained Resnet-18 model is input into the feature extraction module for feature enhancement, and then the enhanced features are input into the second feature fusion module of the pre-trained Resnet-18 model to perform feature fusion with the deep feature map output by the last convolutional layer of the pre-trained Resnet-18 model, and the deep features of the fused feature map are extracted.

[0026] As a further improvement of the method of the present invention: in the step S1, a three-fold cross-validation method is used to randomly divide the acquired road image data set under different conditions into multiple parts to obtain a training set and a test set, and the operation is repeated multiple times so that each image is used as test data.

[0027] As a further improvement of the method of the present invention: it also includes calculating the recall rate and precision rate when the road surface state in the image detected by the improved Resnet-18 model belongs to an icy road surface, the recall rate refers to the proportion of samples that are correctly predicted by the model as icy roads among all the samples that are actually icy roads, and the precision rate refers to the proportion of samples that are actually icy roads among all the samples predicted by the model as icy roads.

[0028] As a further improvement of the method of the present invention: the function expressions of the recall rate Recall and the precision rate Precision are:

[0029] ,

[0030] ,

[0031] Among them, TP represents the number of roads that are predicted to be icy but are actually icy, FP represents the number of roads that are predicted to be icy but are actually not icy, and FN represents the number of roads that are predicted to be not icy but are actually icy.

[0032] Compared with the prior art, the advantages of the present invention are:

[0033] 1. The present invention improves the pre-trained Resnet-18 model by constructing a feature extraction module, wherein the feature extraction module uses four parallel hole convolution layers, which can effectively capture ice surface features of different scales, and train and optimize the improved model, and finally uses the trained model to process the road surface image data to obtain the road surface state. The present invention solves the problems of high model complexity, high ice surface false detection rate, and insufficient real-time performance in the prior art by introducing a combination of multi-scale hole convolutions, a channel attention mechanism, a jump connection fusion, and a dynamic probability judgment logic, and significantly improves the detection accuracy, efficiency, and scene adaptability.

[0034] 2. By improving the Resnet-18 model, the present invention can effectively reduce the number of parameters, reduce the computing resource requirements while maintaining high performance, and is easier to deploy and run, thereby improving the feasibility in practical applications. Especially for scenarios with limited resources, efficient road condition monitoring and management can be achieved without sacrificing performance.

[0035] 3. The present invention further constructs a mathematical model based on the relationship between road surface temperature and road surface icing, and incorporates temperature information into the model output probability system, which can more accurately describe the impact of temperature changes on the probability of road surface icing. This enables the system to predict the possibility of icing based on real-time temperature data, rather than relying solely on fixed threshold judgments. This prediction is more dynamic and flexible and can adapt to different environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for visually detecting road icing based on deep learning in an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of road surface status under different conditions in an embodiment of the present invention.

[0038] Figure 3 It is a schematic diagram of the principle of visual detection of road icing based on deep learning in a specific application example of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, this embodiment provides a road icing visual detection method based on deep learning, comprising the following steps:

[0041] Step S1: Acquire road surface images in different states to form a road surface image dataset.

[0042] Specifically, Figure 2 (a) shows a dry road surface. Figure 2 (b) shows a wet road surface. Figure 2 (c) shows an icy road surface. A dataset of road images in different states is obtained. The dataset consists of two parts: one is 150 images collected, and the other is 240 images obtained from the UA-DETRAC dataset, totaling 390 images. Among these 390 images, there are 150 dry road images, 150 wet road images, and 90 icy road images. Road images in different states will provide data support for subsequent model training and testing.

[0043] In this embodiment, a three-fold cross-validation method is used to randomly divide the acquired road surface image data set in different states into multiple parts to obtain a training set and a test set, and the operation is repeated multiple times so that each image is used as test data.

[0044] Specifically, in order to ensure the effectiveness of model training and testing, the acquired road image data set is divided by a three-fold cross-validation method. The specific operation is as follows: the images of dry road surface, wet road surface and icy road surface are randomly divided into three parts respectively, wherein 2 dry road surface images, 2 wet road surface images and 2 icy road surface images are selected each time and combined together to form a training set, and the remaining 1 dry road surface image, 1 wet road surface image and 1 icy road surface image are combined into a test set. This operation will be repeated 3 times, and finally 3 sets of training set-test set combinations are obtained. It can be understood that the three-fold cross-validation here is only an option in this embodiment, and the specific number of random divisions can be flexibly determined according to actual needs, with the purpose of giving each image the opportunity to be used as test data to improve the generalization ability of the model and the accuracy of the evaluation.

[0045] Step S2: Get the pre-trained Resnet-18 model;

[0046] The pre-trained Resnet-18 model refers to a Resnet-18 model pre-trained on the ImageNet dataset, and the pre-trained Resnet-18 model is trained on the dataset constructed by the present invention. When training on the dataset constructed by the present invention, the model training method is consistent with the improved Resnet-18 model.

[0047] Constructing a feature extraction module, the feature extraction module includes an average pooling layer, a convolution layer, four parallel hole convolution layers, a first feature fusion module, an average pooling layer and a convolution layer connected in sequence;

[0048] The shallow features extracted by the pre-trained Resnet-18 model are enhanced using the feature extraction module and then integrated into the deep features extracted by the pre-trained Resnet-18 model to obtain an improved Resnet-18 model.

[0049] Use the road image dataset to train the improved Resnet-18 model and save the trained improved Resnet-18 model.

[0050] In this embodiment, the pre-trained Resnet-18 model includes a convolutional layer, a maximum pooling layer, a multi-layer convolutional layer and a second feature fusion module connected in sequence; the shallow feature map output by the maximum pooling layer in the pre-trained Resnet-18 model is input into the feature extraction module for feature enhancement, and then the enhanced feature is input into the second feature fusion module of the pre-trained Resnet-18 model for feature fusion with the deep feature map output by the last convolutional layer of the pre-trained Resnet-18 model, and deep features are extracted from the fused feature map.

[0051] In this embodiment, the four parallel dilated convolutional layers include a 3×3 convolutional layer with a convolution kernel of 128, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 3, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 6, and a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 9, and the first feature fusion module performs weighted fusion on the outputs of the four parallel dilated convolutional layers through a channel attention mechanism.

[0052] In a specific application embodiment, the pre-trained Resnet-18 model is improved using a feature extraction module. The specific improvement method is as follows: Figure 3 As shown in the figure, the shallow feature map output by the 3×3 maximum pooling layer in the Resnet-18 model is input into the feature extraction module, which contains multiple 3×3 convolutional layers and dilated convolutional layers, and each convolutional layer has a different expansion rate (for example, 1, 3, 6, 9). By setting different expansion rates and convolutional layers, the model can capture feature information of different scales, thereby enhancing the shallow feature map output by the 3×3 maximum pooling layer in the Resnet-18 model. After that, the enhanced feature map output by the feature extraction module is added to the deep feature map output by the last 3×3 convolutional layer of the Resnet-18 model, further enhancing and improving the extraction of deep features of the image by the Resnet-18 model, which helps the model to make more accurate outputs.

[0053] In this embodiment, the improved Resnet-18 model has a lower number of parameters. The calculation formula of the number of convolution parameters is:

[0054] , (1)

[0055] Among them, F CH Indicates the height of the convolution kernel, F CW Represents the width of the convolution kernel, C I Indicates the number of channels of the input feature map, C O Indicates the number of channels of the output feature map. If convolution adds BN or bias, its parameters need to be combined with C O Add. The parameters of all convolutions in the network model are added to the parameters of the fully connected layer to get the parameters of the network model.

[0056] In this embodiment, by reducing the number of parameters, the improved Resnet-18 model has lower computing resource requirements while maintaining high performance. It can not only reduce the dependence on computing resources while ensuring performance, but also be more conducive to deployment and efficient operation on various hardware platforms, bringing more convenience to practical applications, especially suitable for resource-constrained environments or scenarios with high requirements for computing performance.

[0057] Step S3: Use the improved Resnet-18 model to process the road surface image data to obtain the probabilities of different road surface categories, and determine the final road surface state based on the maximum probability.

[0058] In this embodiment, after using the improved Resnet-18 model to process the road image data, the probability of each category of road conditions is obtained, wherein the road condition corresponding to the maximum probability is used as the final output road condition, and the road condition includes one or more of dry road surface, wet road surface, and icy road surface.

[0059] In this embodiment, multi-category classification of road conditions facilitates comprehensive road condition monitoring, which has wider applicability and stronger practicality than existing technologies that can only detect a single state (such as whether it is icy). In actual traffic management and road maintenance, it can not only accurately identify the risks brought by icy roads, but also distinguish between dry and wet roads, providing richer information for traffic decisions under different road conditions, thereby helping to formulate more refined traffic control strategies.

[0060] In this embodiment, the probability of each category of road surface state is obtained by improving the Resnet-18 model, and the road surface state corresponding to the maximum probability is used as the final output, taking into account the relative probability between different categories, and avoiding the risk of misjudgment caused by simple binary judgment (such as only judging whether it is icy). In actual situations, the road surface state may be in a critical situation. The probability representation can better reflect the uncertainty of the model about the current road surface state and provide more information for decision-making. For example, assuming that when a certain image is classified, the probability of an icy road surface is 45%, the probability of a wet road surface is 50%, and the probability of a dry road surface is 5%. According to the maximum probability, it can be known that the road surface is in a wet road state. This result can not only clarify the road surface state, but also indicate the possible icing risk (because the probability of an icy road surface is also high), so that the traffic management department can adjust the corresponding response measures according to different probability values, rather than simply operating according to a single category judgment result.

[0061] In this embodiment, a dynamic threshold model is constructed according to the relationship between temperature and road icing. The dynamic threshold model corrects the icing probability output by the improved Resnet-18 model through the following formula:

[0062] , (2)

[0063] in, P i ′ represents the icing probability output by the modified model, P i represents the icing probability output by the improved Resnet-18 model, T Indicates the road surface temperature, L Indicates the preset critical temperature threshold, k Represents the temperature sensitivity coefficient.

[0064] In this embodiment, a dynamic threshold model is constructed according to the relationship between temperature and road icing. The dynamic threshold model corrects the dry road and wet road probabilities output by the improved Resnet-18 model through the following formula:

[0065] , (3)

[0066] , (4)

[0067] Wherein, L represents the preset critical temperature threshold, T Indicates the road surface temperature, k represents the temperature sensitivity coefficient, represents the corrected dry road probability, P d Represents the dry road probability output by the improved Resnet-18 model; represents the corrected wet road probability, P w Represents the wet road probability output by the improved Resnet-18 model.

[0068] In this embodiment, for a dry road surface, when the road surface temperature T > L hour, P d 'near P d , indicating that the probability of dry road surface is higher at high temperature; when the road surface temperature T < L hour, P d ′ is significantly reduced, indicating that the probability of dry pavement at low temperature is small. For wet pavement, when the pavement temperature T > L hour, P w 'near P w , indicating that the probability of wet road surface is higher under high temperature; when the road surface temperature T < L hour,P w ′ is significantly reduced, indicating that the probability of wet road surface at low temperature is small (it may turn into icy road surface). For icy road surface, when the road surface temperature T < L The corrected icing probability is P i ' will increase as the temperature decreases. T > L hour, P i ′ is significantly reduced, indicating that the probability of the road surface being considered icy is low in this case.

[0069] In this embodiment, by introducing the relationship between temperature and road icing, the impact of environmental factors on road conditions is fully considered. In real scenarios, temperature is one of the key factors for road icing. Relying solely on image recognition models may be disturbed by factors such as light and shadows and cause misjudgments. Combined with temperature information, the probability of the output of the improved Resnet-18 model can be corrected, so that the final detection result is more in line with the actual road conditions, greatly reducing the possibility of misjudgment, thereby improving the accuracy of road icing detection. Due to the large differences in temperature conditions in different regions and seasons, this solution can adapt to such changes. In cold areas or winter, lower temperatures make the road more prone to icing, and the mathematical model can reasonably adjust the model output according to the local temperature characteristics; in areas or seasons with higher temperatures, even if the image may have some visual features similar to icing, the model can also eliminate misjudgments in combination with temperature, enhancing the adaptability of the entire detection system in different environments and ensuring that it can work effectively under various climatic conditions.

[0070] In this embodiment, it also includes calculating the recall rate and precision rate when the road surface condition in the detected image by the improved Resnet-18 model is icy road surface. The recall rate refers to the proportion of samples that are correctly predicted by the model as icy roads among all samples that are actually icy roads. The precision rate refers to the proportion of samples that are actually icy roads among all samples that are predicted by the model as icy roads.

[0071] In this embodiment, when the output state is an icy road, the function expressions of the recall rate Recall and the precision rate Precision are:

[0072] , (5)

[0073] , (6)

[0074] Among them, true positive TP indicates the number of roads predicted to be icy but actually are icy, false positive FP indicates the number of roads predicted to be icy but actually are not icy, and false negative FN indicates the number of roads predicted to be not icy but actually are icy.

[0075] Recall and precision are important indicators for evaluating the performance of classification models, which help to gain a deeper understanding of the performance of the model in different situations. Specifically, the recall rate reflects the model's ability to find all actual icy roads, and the precision rate reflects the accuracy of the samples predicted by the model as icy roads that are truly icy roads. In this embodiment, the recall rate and precision rate of the road surface state in the image detected by the improved Resnet-18 model are calculated to provide quantitative indicators for evaluating the performance of the model in icy road detection. For traffic monitoring, a high recall rate means that potential icy roads can be detected to the greatest extent, reducing traffic accidents caused by undetected icy roads and ensuring road safety; while a high precision rate ensures reliability when predicting icy roads, avoiding unnecessary traffic control and waste of resources due to misjudgment.

[0076] For example, in an actual road monitoring system, if the recall rate is low, some actual icy sections may be missed, thereby increasing the risk of traffic accidents; if the precision rate is low, icy conditions may be frequently misreported, leading to traffic jams and increased management costs. By calculating the recall rate and precision rate, the model can be optimized and adjusted in a targeted manner to improve the reliability and stability of the system.

[0077] In a specific application embodiment, the recall rates of icy roads are detected using a pre-trained Resnet-18 model, an improved Resnet-18 model, and a pre-trained Resnet-50 model on three different test sets, respectively. The recall rate values ​​of different models are shown in Table 1:

[0078] Table 1 Recall rate table of icy roads

[0079]

[0080] The accuracy of detecting icy roads using the pre-trained Resnet-18 model, the improved Resnet-18 model, and the pre-trained Resnet-50 model on three test sets is shown in Table 2.

[0081] Table 2 Icy road surface detection accuracy

[0082]

[0083] The pre-trained Resnet-50 model refers to a model obtained by obtaining a Resnet-50 model pre-trained on the ImageNet dataset, and then training the Resnet-50 model on the dataset constructed by the present invention. When training on the dataset constructed by the present invention, the model training method is consistent with the improved Resnet-18 model.

[0084] Comparing the recall performance of the pre-trained Resnet-18 model, the improved Resnet-18 model, and the pre-trained Resnet-50 model on three different test sets, it is not difficult to find that the improved Resnet-18 model performs well in terms of recall. In the three test sets, the recall rates of the improved Resnet-18 model in the first and second groups reached 93.3%, and the recall rate of the third group was as high as 100.0%, with an average of 95.5%. In contrast, the average value of the pre-trained Resnet-18 model was 91.1%, 80.0% in the first test set, and although it reached 96.7% in the second and third groups, the overall average was still lower than that of the improved Resnet-18 model. The average recall rate of the pre-trained Resnet-50 model was 87.8%, and the three test sets were 90.0%, 86.7%, and 86.7%, respectively, all lower than the improved Resnet-18 model. This shows that the improved Resnet-18 model can more effectively identify the actual icy road conditions when detecting icy roads, and the number of missed detections is relatively small.

[0085] Similarly, when focusing on the accuracy performance of the three models on three different test sets, the improved Resnet-18 model still performs well. The accuracy of the improved Resnet-18 model is relatively stable in different test sets, while the pre-trained Resnet-18 model fluctuates greatly in different test sets, ranging from 92.3% to 76.3%. This shows that the improved Resnet-18 model has a higher accuracy in judging icy roads, and there are relatively few misjudgments.

[0086] And the number of parameters of the improved Resnet-18 model is 14.1M, while the number of parameters of the pre-trained Resnet-50 model is 23.5M. Fewer parameters means that the model structure is relatively simple. When computing resources are limited, the improved Resnet-18 model has greater advantages. For example, in environments with relatively low computing power and storage capacity such as edge computing devices and mobile devices, the improved Resnet-18 model is easier to deploy and run, which can reduce memory usage and computing time, and improve the real-time response capability of the model. Due to the small number of parameters, during the model training process, the improved Resnet-18 model needs to update and optimize relatively fewer parameters, which makes the training speed faster and the convergence speed may also be faster. This advantage will be more obvious when facing large-scale data training, which can save a lot of training time and computing resources, and improve the training efficiency and iteration speed of the model.

[0087] Generally speaking, models with too many parameters are more likely to overfit, that is, the model performs well on the training set, but performs poorly on the test set or new data. The pre-trained Resnet-50 model has many parameters, and when the amount of data is limited, it may be easier to overfit the noise and details in the training data, resulting in a decrease in generalization ability. The improved Resnet-18 model has relatively fewer parameters and has a better chance of learning the general characteristics of the data, thus having better generalization ability in different data sets and actual application scenarios, being more adaptable to new data, and being able to output accurate results more stably.

[0088] In summary, although the average detection accuracy of the pre-trained Resnet-50 model is slightly higher than that of the improved Resnet-18 model (93.1% vs. 86.8%), the average recall of the improved Resnet-18 model is higher than that of the pre-trained Resnet-50 model (95.5% vs. 87.8%). Considering that the improved Resnet-18 model uses significantly fewer parameters but can achieve an advantage in recall, and the accuracy is not much different from that of the pre-trained Resnet-50, it shows that the improved Resnet-18 model has achieved a good balance between performance and parameter quantity, and has a higher cost-effectiveness.

[0089] This embodiment also provides a road icing visual detection device based on deep learning, including a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to perform a road icing visual detection method based on deep learning.

[0090] It is understandable that the above method of this embodiment can be executed by a single device, such as a computer or server, etc., and can also be applied to a distributed scenario and completed by multiple devices in cooperation with each other. In the case of a distributed scenario, one of the multiple devices can only execute one or more steps in the above method of this embodiment, and multiple devices interact to complete the above method. The processor can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, etc., for executing related programs to implement the above method of this embodiment. The memory can be implemented in the form of a read-only memory ROM, a random access memory RAM, a static storage device, and a dynamic storage device. The memory can store an operating system and other applications. When the above method of this embodiment is implemented by software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0091] This embodiment also provides a computer-readable storage medium storing a road icing visual detection program based on deep learning, which implements a road icing visual detection method based on deep learning when the road icing visual detection program based on deep learning is executed.

[0092] Those skilled in the art should understand that the above-mentioned embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The above are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

[0093] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A road icing visual detection method based on deep learning, characterized in that: The following steps are involved: Step S1: Acquire road surface images in different states to form a road surface image data set; Step S2: Get the pre-trained Resnet-18 model; Constructing a feature extraction module, wherein the feature extraction module includes an average pooling layer, a convolution layer, four parallel hole convolution layers, a first feature fusion module, an average pooling layer and a convolution layer connected in sequence; The four parallel atrous convolutional layers include: a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 1, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 3, a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 6, and a 3×3 convolutional layer with a convolution kernel of 128 and a dilation rate of 9; The first feature fusion module performs weighted fusion on the outputs of four parallel dilated convolutional layers through a channel attention mechanism; After enhancing the shallow features extracted by the pre-trained Resnet-18 model using the feature extraction module, multi-scale fusion is performed with the deep features extracted by the pre-trained Resnet-18 model through skip connections, thereby obtaining an improved Resnet-18 model; Using the road image dataset to train the improved Resnet-18 model, and saving the trained improved Resnet-18 model; Step S3: using the improved Resnet-18 model to process the road surface image data to obtain the probabilities of different road surface categories, and determining the final road surface state according to the maximum probability; In step S2, the pre-trained Resnet-18 model includes a convolutional layer, a maximum pooling layer, a multi-layer convolutional layer and a second feature fusion module connected in sequence; the shallow feature map output by the maximum pooling layer in the pre-trained Resnet-18 model is input into the feature extraction module for feature enhancement, and then the enhanced features are input into the second feature fusion module of the pre-trained Resnet-18 model for feature fusion with the deep feature map output by the last convolutional layer of the pre-trained Resnet-18 model, and the deep features of the fused feature map are extracted.

2. The method for visually detecting road icing based on deep learning according to claim 1, characterized in that: It also includes building a dynamic threshold model based on the relationship between temperature and road icing, and the dynamic threshold model corrects the icing probability output by the improved Resnet-18 model through the following formula: ， in, P i ′ represents the corrected icing probability, P i represents the icing probability output by the improved Resnet-18 model, T Indicates the road surface temperature, L Indicates the preset critical temperature threshold, k Represents the temperature sensitivity coefficient.

3. The road icing visual detection method based on deep learning according to claim 1 is characterized in that: It also includes constructing a dynamic threshold model based on the relationship between temperature and road icing, and the dynamic threshold model corrects the dry road and wet road probabilities output by the improved Resnet-18 model through the following formula: ， ， Wherein, L represents the preset critical temperature threshold, T Indicates the road surface temperature, k represents the temperature sensitivity coefficient, represents the corrected dry road probability, P d Represents the dry road probability output by the improved Resnet-18 model; represents the corrected wet road probability, P w Represents the wet road probability output by the improved Resnet-18 model.

4. The method for visually detecting road icing based on deep learning according to any one of claims 1 to 3, characterized in that: After using the improved Resnet-18 model to process the road image data, the probability of each category of road conditions is obtained, wherein the road condition corresponding to the maximum probability is used as the final output road condition, and the road condition includes one or more of dry road surface, wet road surface, and icy road surface.

5. The road icing visual detection method based on deep learning according to claim 1 is characterized in that: In step S1, a three-fold cross validation method is used to randomly divide the acquired road surface image data set under different conditions into multiple parts to obtain a training set and a test set, and the operation is repeated multiple times so that each image is used as test data.

6. The road icing visual detection method based on deep learning according to claim 1, characterized in that: It also includes calculating the recall rate and precision rate when the road surface state in the detected image by the improved Resnet-18 model belongs to an icy road surface. The recall rate refers to the proportion of samples that are correctly predicted as icy roads by the model among all the samples that are actually icy roads. The precision rate refers to the proportion of samples that are actually icy roads among all the samples that are predicted as icy roads by the model.

7. The method for visually detecting road icing based on deep learning according to claim 6, characterized in that: The function expressions of recall rate and precision rate are as follows: ， ， Among them, Recall represents the recall rate, Precision represents the accuracy rate, TP represents the number of roads predicted to be icy but actually are icy, FP represents the number of roads predicted to be icy but actually are not icy, and FN represents the number of roads predicted to be not icy but actually are icy.

Images