A road surface image recognition method and system for intelligent transportation

Abstract

The application provides a pavement image recognition method and system for intelligent traffic, and relates to the technical field of intelligent traffic systems.The method comprises the following steps: acquiring pavement images with different pavement states; preprocessing a pavement image dataset; constructing a pavement image recognition model based on an EfficientNet efficient convolutional neural network architecture, wherein the pavement image recognition model comprises a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence; inputting the pavement image dataset to the pavement image recognition model as a training set until the loss function value of the pavement image recognition model is less than a preset loss function value; acquiring a to-be-recognized pavement image; inputting the to-be-recognized pavement image to the trained pavement image recognition model, and outputting a predicted pavement image category.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation systems, in particular to a road surface image recognition method and system for intelligent transportation. BACKGROUND

[0002] Intelligent transportation road surface refers to the road surface in the intelligent transportation system, which is monitored and analyzed by sensors, cameras or other detection devices. These road surface information can include the friction of the road surface, the material type (such as asphalt or concrete), the flatness (such as flat or undulating), and the environmental conditions (such as wet, water accumulation, snow accumulation, etc.). Intelligent transportation road surface recognition technology helps the intelligent transportation system to accurately classify road surface images by automatically collecting and analyzing these road surface information, so as to improve driving safety and riding comfort.

[0003] With the continuous increase of vehicle ownership, more and more people choose to drive as the preferred travel solution. However, when the vehicle is driving in special road conditions, the controllability and stability will be significantly reduced. Wet, snow and uneven road surfaces are the main reasons for affecting the driving experience and causing frequent traffic accidents. In addition, uneven road surfaces also exacerbate vehicle body vibration and transmit vibration and noise to drivers and passengers through vehicles, directly affecting the steering stability and riding comfort. Therefore, as an important part of the human-vehicle-road information coordination system, the road surface type recognition capability of the vehicle is the key to improving driving safety and riding comfort, and the accurate recognition of the road surface type of the vehicle is of great significance to driving safety and riding comfort.

[0004] Traditional road surface type recognition methods mainly include the adhesion coefficient observation method based on dynamic modeling and the unevenness estimation method based on vibration response. Although the dynamic modeling method is accurate and reliable, it relies on the model, which is usually based on assumptions and simplifications under specific conditions and is difficult to completely cover complex scenarios in reality. When facing different road conditions, vehicles or environments, the adjustment of model parameters is difficult to adapt to a wide range of situations, resulting in poor adaptability in new scenarios and limited generalization ability. With the advancement of computer vision and artificial intelligence technology, environmental perception technology in driving systems has been widely applied. The technical path of this task mainly includes traditional machine learning methods and deep learning methods. Although machine learning methods are effective, they rely on manual design and feature extraction, which is difficult to capture complex patterns or high-level features in images. In recent years, benefiting from the advantages of convolutional neural networks (CNN) in automatic feature extraction and improving recognition accuracy, deep learning has been widely applied in road surface type recognition. However, most CNN-based deep neural networks have the problem of losing local highly correlated information when processing high-dimensional data, resulting in decreased model performance and low road surface recognition accuracy. SUMMARY

[0005] In order to solve the problem that the CNN-based deep neural network in the prior art loses local high correlation information when processing high-dimensional data, resulting in low model performance and low road surface recognition accuracy, the application provides a road surface image recognition method and system for intelligent transportation.

[0006] The technical scheme provided by the embodiments of the application is as follows:

[0007] The first aspect

[0008] The road surface image recognition method for intelligent transportation provided by the embodiments of the application comprises:

[0009] S1: acquiring road surface images with different road surface states, wherein each road surface image and the corresponding road surface state form a road surface image data set;

[0010] S2: preprocessing the road surface image data set;

[0011] S3: constructing a road surface image recognition model based on the EfficientNet high-efficiency convolutional neural network architecture, wherein the road surface image recognition model comprises a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence, wherein the second EMBConv layer comprises an ECA attention module, wherein the ECA module comprises an input unit, a global average pooling unit, an adaptive one-dimensional convolution kernel unit, a Sigmoid activation function unit and an output unit connected in sequence, wherein the input information of the input unit and the input information of the output unit are connected through a skip connection mode, wherein the MLP classifier comprises a fully connected layer and a KAN layer;

[0012] S4: inputting the road surface image data set as a training set to the road surface image recognition model to train the road surface image recognition model until the loss function value of the road surface image recognition model is less than a preset loss function value;

[0013] S5: acquiring a road surface image to be recognized;

[0014] S6: inputting the road surface image to be recognized into the trained road surface image recognition model to output a predicted road surface image category.

[0015] The second aspect

[0016] The road surface image recognition system for intelligent transportation provided by the embodiments of the application comprises:

[0017] A processor;

[0018] The memory stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the intelligent traffic-oriented road surface image recognition method of the first aspect.

[0019] The third aspect

[0020] The computer readable storage medium provided by the embodiment of the present application stores a computer program, and the program is executed by the processor to implement the intelligent traffic-oriented road surface image recognition method of the first aspect.

[0021] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0022] In the present application, based on the obtained road surface images of different road surface states, a road surface image recognition model is constructed by connecting a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier in sequence based on the basic framework of the EfficientNet model. In the road surface image recognition model, the second EMBConv layer including the ECA module is introduced into the basic architecture EfficientNet model. The ECA efficiently calculates the channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependency between channels. Then, the feature map is mapped to a fixed dimension through the pooling layer. Subsequently, the Transformer layer is introduced, and the model can establish long-distance dependencies between parts of the feature vector through the multi-head self-attention mechanism. Then, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and the KAN layer process the feature vector respectively, combining linear and nonlinear feature extraction, thereby improving the explainability and classification accuracy of the model. The introduced simplified Transformer layer enhances the global feature extraction capability of the model. At the same time, the KAN module with stronger explainability is added to improve the generalization ability of the model, enhance the local feature extraction of road surface recognition, improve the explainability, increase the classification accuracy of the model, and at the same time realize the optimization of the use of computing resources, which is conducive to realizing the accurate identification of real-time road surface, reducing traffic accidents, improving driving safety, and having a positive impact on social traffic safety, providing economic benefits for the construction and maintenance of the intelligent traffic system. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 A flowchart of a road surface image recognition method for intelligent transportation provided by an embodiment of the present application is shown in FIG. 1.

[0025] Figure 2 A structural diagram of a road surface image recognition model provided by an embodiment of the present application is shown in FIG. 2.

[0026] Figure 3 A structural diagram of a second EMBConv layer provided by an embodiment of the present application is shown in FIG. 3.

[0027] Figure 4 A structural diagram of a simplified Transformer layer provided by an embodiment of the present application is shown in FIG. 4.

[0028] Figure 5 A structural diagram of an ECA module provided by an embodiment of the present application is shown in FIG. 5.

[0029] Figure 6 A structural diagram of a road surface image recognition system for intelligent transportation provided by an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0030] The technical solutions in the present application will be described below with reference to the drawings.

[0031] In the embodiments of the present application, the words such as “example”, “for example” are used to represent an example, illustration or description. Any embodiment or design scheme described as “example” in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word “example” is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by “and / or” can be both, or can be either one of the two.

[0032] To make the technical problems, technical solutions and advantages of the present application clearer, the following will be described in detail with reference to the drawings and specific embodiments.

[0033] Reference is made to the drawings accompanying the specification Figure 1 , a flowchart of a road surface image recognition method for intelligent transportation provided by an embodiment of the present application is shown in FIG. 1.

[0034] The present application provides a road surface image recognition method for intelligent transportation, which can be implemented by a road surface image recognition device for intelligent transportation. The road surface image recognition device for intelligent transportation can be a terminal or a server. The processing flow of the road surface image recognition method for intelligent transportation can include the following steps:

[0035] S1: Obtain road surface images with different road surface states.

[0036] Each road surface image and the corresponding road surface state form a road surface image dataset.

[0037] The road surface image refers to a photo of a road surface taken under different conditions, which can include different weather, different road materials, and different road flatness, etc. The road surface image dataset is a picture collection composed of different road surface images and their corresponding road surface states, serving as the basic data for model training.

[0038] It should be noted that by collecting images with diverse road surface states, a rich dataset can be constructed, which can reflect the complex road conditions in reality. The diversity of such a dataset provides a wider range of samples for subsequent model training, improves the model's recognition ability for various road conditions, and ensures the robustness and accuracy of the recognition system in different traffic environments.

[0039] In one possible implementation, each road surface state includes a road friction level attribute, a road material attribute, and a road unevenness attribute. The road friction level attribute includes a dry state road friction level attribute, a wet state road friction level attribute, a water state road friction level attribute, a new snow state road friction level attribute, a thawed snow state road friction level attribute, and an ice state road friction level attribute. The road material attribute includes an asphalt road material attribute, a concrete road material attribute, a mud road material attribute, and a gravel road material attribute. The road unevenness attribute includes a gentle undulation state road unevenness attribute, a slight undulation state road unevenness attribute, and a severe undulation state road unevenness attribute.

[0040] The road surface state includes a first road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a gentle undulation state road unevenness attribute; a second road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a slight undulation state road unevenness attribute; a third road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a severe undulation state road unevenness attribute; a fourth road surface sub-state with an ice state road friction level attribute; a fifth road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a gentle undulation state road unevenness attribute; a sixth road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a slight undulation state road unevenness attribute; and a seventh road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a severe undulation state road unevenness attribute.

[0041] S2: Preprocess the road surface image dataset.

[0042] The preprocessing includes dividing the road surface image dataset according to a preset ratio, cropping, random horizontal flipping, tensor format conversion and normalization.

[0043] The dividing of the road surface image dataset according to a preset ratio specifically includes dividing the preprocessed road surface image dataset into a training set, a verification set and a test set according to a preset ratio, and specifically, the preset ratio is 8:1:1.

[0044] It should be noted that through the preprocessing steps of the road surface image dataset, such as cropping, random horizontal flipping, tensor format conversion and normalization, the quality and diversity of the data are significantly improved, the cropping and flipping can increase the variability of the training data, avoid overfitting of the model, the tensor format conversion makes the image more suitable for deep learning framework processing, and the normalization speeds up the convergence speed of the model.

[0045] Referring to the accompanying drawings Figure 2 , a structural schematic diagram of a road surface image recognition model provided by an embodiment of the application is shown.

[0046] Figure 2 In the formula, xn represents the first EMBconv layer, specifically n layers, and n is specifically 3; and xm represents the second EMBConv layer, specifically m layers, and m is specifically 4.

[0047] For example, if the input image size is 224x224x3, the first convolutional layer, the three first EMBConv layers, the four second EMBConv layers, the second convolutional layer, the pooling layer, the simplified Transformer layer with a feedforward neural network and the MLP classifier are repeatedly executed 1 time, 1 time, 2 times, 3 times, 3 times, 4 times, 1 time, 1 time, 1 time, 1 time and 1 time respectively, then the image sizes obtained after passing through the first convolutional layer, the three first EMBConv layers, the four second EMBConv layers, the second convolutional layer, the pooling layer, the simplified Transformer layer with a feedforward neural network and the MLP classifier in turn are 112x112x32, 112x112x16, 56x56x24, 28x28x40, 14x14x80, 14x14x112, 7x7x192, 7x7x320, 7x7x1280, 1x1x1280, 1x1x1280 and 1x1x7 respectively, wherein the image size can be uniformly represented as axbxc, a and b represent the spatial resolution of the image, i.e. the width and height, and c represents the number of image channels, also known as image depth.

[0048] S3: Construct a road surface image recognition model based on the EfficientNet efficient convolutional neural network architecture, wherein the road surface image recognition model comprises a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence, wherein the second EMBConv layer comprises an ECA attention module, wherein the ECA module comprises an input unit, a global average pooling unit, an adaptive one-dimensional convolution kernel unit, a Sigmoid activation function unit, and an output unit connected in sequence, wherein the input information of the input unit and the input information of the output unit are connected through a skip connection mode, wherein the MLP classifier comprises a fully connected layer and a KAN layer.

[0049] EfficientNet (Efficient Network) is an efficient convolutional neural network architecture that maintains high performance at different computational costs by scaling model parameters, and the road image recognition model is a deep learning-based model specifically designed to recognize images of different road conditions. The first and second convolutional layers are basic layers in the deep learning network responsible for extracting features from images through convolution operations. The first and second EMBConv layers are improved convolutional layers that combine convolution operations and embedded convolution kernel optimization for more efficient feature extraction. The pooling layer is used for dimensionality reduction and reducing computational complexity while preserving important feature information. The feedforward neural network is a non-cyclic neural network structure with a one-way data flow from input to output. The Transformer layer is a neural network layer used to process sequential data, which is a simplified version here to capture long-range dependencies in images. The MLP (Multilayer Perceptron) classifier is a multilayer perceptron used to classify the extracted features and output the final prediction class. The ECA (Efficient Channel Attention) module is used to enhance the channel attention mechanism in the network by dynamically adjusting the importance of different channels to enhance feature representation. The input unit is the first module of the neural network model that receives raw data and passes it to subsequent network layers. The global average pooling unit is a pooling operation that generates a feature map by taking the average of all pixel values in each channel. The adaptive one-dimensional convolution kernel unit is used to dynamically adjust the size of the convolution kernel by adaptively computing the input features to select the most appropriate convolution kernel size to better capture the correlation between channels. The Sigmoid activation function is a common activation function that compresses input values between 0 and 1, used in binary classification problems or attention mechanisms. The output unit is the last layer of the network that receives the processing results from the previous layer and generates the final output. The KAN (Kernel Attention Network) layer is a network module used to improve the performance of the classifier by focusing on different dimensional features to optimize classification decisions.

[0050] Specifically, both the first sum-normalization unit and the second sum-normalization unit first sum the input features and then normalize them.

[0051] It needs to be explained that by combining EfficientNet and simplified Transformer layer, a high-efficiency and powerful pavement image recognition model is formed, which utilizes the powerful feature extraction capability of convolutional neural network and the channel attention mechanism of ECA module to enhance the recognition performance of complex images, and at the same time, the long-distance dependence relationship in the image is captured through the Transformer layer, the whole architecture design effectively improves the precision and robustness of the model, and at the same time, the efficient use of computing resources is maintained, which adapts to the real-time recognition demand in intelligent transportation.

[0052] Referring to the drawings accompanying the specification Figure 3 , a structural schematic diagram of a second EMBConv layer provided by the embodiment of the application is shown.

[0053] Figure 3 In the figure, from left to right, Conv1x1 respectively represents a first convolution unit and a second convolution unit, from left to right, two BN Swish respectively represent a first batch normalization unit with a Swish activation function and a second batch normalization unit with a Swish activation function, DWConvkxk represents a depth convolution unit, ECA represents an ECA module, BN represents a third batch normalization unit, and Droupout represents a random inactivation unit.

[0054] Referring to the drawings accompanying the specification Figure 4 , a structural schematic diagram of a simplified Transformer layer provided by the embodiment of the application is shown.

[0055] Figure 4 In the figure, from input to output, the multi-head attention mechanism, summation & normalization, feedforward neural network, summation & normalization respectively represent the input module, the multi-head attention mechanism module, the first summation & normalization module, the feedforward neural network module, the second summation & normalization module and the output module.

[0056] In a possible implementation, the first EMBConv layer is 3 layers, and the second EMBConv layer is 4 layers, wherein the second EMBConv layer comprises an input unit, a first convolution unit, a first batch normalization unit with a Swish activation function, a depth convolution unit, a second batch normalization unit with a Swish activation function, a second convolution unit, a third batch normalization unit, a random inactivation unit and an output unit.

[0057] The input unit is connected with the output unit through a convolution unit, a first batch normalization unit with a Swish activation function, a depth convolution unit, a second batch normalization unit with a Swish activation function, an ECA module, a third batch normalization unit and a random inactivation unit in sequence, wherein the input information of the input unit and the input information of the output unit are connected through a skip connection mode.

[0058] The simplified Transformer layer further comprises an input module, a multi-head attention mechanism module, a first sum normalization module, a second sum normalization module, and an output module.

[0059] The input module is connected in sequence with the multi-head attention mechanism module, the first sum normalization module, a feedforward neural network, the second sum normalization module, and the output module.

[0060] Referring to the accompanying drawings Figure 5 , a structural schematic diagram of an ECA module is shown.

[0061] Figure 5 In the formula, x represents an input feature map, C represents the number of channels of the feature map, H and W represent the height and width of the feature map, GAP represents a global average pooling unit, 1x1xC represents the global feature size of each channel after compression by the global average pooling unit, and sigma represents an activation function. represents an output feature map after channel attention mechanism weighting.

[0062] The input feature map has a size of HxWxC. First, the feature map is subjected to a global average pooling unit to compress the height and width to 1, thereby generating statistical information in the channel dimension. Then, the statistical information is subjected to two 1x1 convolution layers and processed by a nonlinear activation function, thereby generating a channel weight vector. The weight vector is used to weight each channel of the original input feature map, emphasize important channel information, and suppress unimportant channels. Finally, the feature map after channel weighting is used for subsequent network layer calculation, thereby helping the model to better capture the dependency between channels and enhance the feature expression capability.

[0063] In a possible implementation, the ECA module is used to enhance the channel attention mechanism in a convolutional neural network, and the enhancement process specifically comprises:

[0064] An input feature map x is received, wherein the feature map has a width of W, a height of H, and a channel number of C.

[0065] A global average pooling operation is performed on the feature map to obtain global information of each channel.

[0066] According to the global information, weights of the channels are calculated by applying an adaptive one-dimensional convolution kernel.

[0067] The weights of the channels obtained after the calculation are normalized by a Sigmoid activation function, thereby obtaining attention weight values of the channels.

[0068] The attention weight values of the channels are multiplied by channel features corresponding to the feature map, thereby obtaining a weighted feature map.

[0069] It should be noted that the channel attention mechanism enhances the feature expression ability of the model, the global average pooling extracts important global information, the adaptive convolution kernel calculates the weight of each channel, and the Sigmoid function is normalized to highlight key features and suppress irrelevant features, which not only improves the recognition accuracy of the model, but also optimizes the computing resources, makes the model more efficient, and has stronger generalization ability in diversified input scenarios.

[0070] In a possible implementation, the calculation method of the adaptive one-dimensional convolution kernel size is specifically as follows:

[0071]

[0072] wherein k represents the one-dimensional convolution kernel size, γ represents the scaling parameter, b represents the constant offset, log2(C) represents the logarithm operation with base 2 on the channel C, represents the take-and-function the nearest odd number.

[0073] By inputting the feature map into the multi-head attention mechanism, the multi-head attention mechanism then extracts important features from the input data, and outputs the extracted data, performs summation and normalization operations, and further extracts and processes the normalized features through the feedforward neural network to enhance the nonlinear expression ability of the network. Then, summation and normalization operations are performed again on the output of the feedforward network, so that the features are more robust and unified. Finally, the processed result is output as the prediction result or classification result of the model.

[0074] In a possible implementation, the MLP classifier is used to classify the output data of the simplified Transformer layer, and output the predicted road surface image category.

[0075] The process in which the MLP classifier classifies the output data of the simplified Transformer layer to determine the predicted road surface image category is specifically as follows:

[0076] The first predicted road surface image category probability of the output data of the simplified Transformer layer is extracted through the fully connected layer.

[0077] The second predicted road surface image category probability of the output data of the simplified Transformer layer is extracted through the KAN layer.

[0078] The sum value probability of the first predicted road surface image category probability and the second predicted road surface image category probability is calculated.

[0079] The predicted road surface image category corresponding to the maximum sum value probability is output.

[0080] It should be noted that by combining the full connection layer and the KAN layer to calculate two kinds of prediction probabilities respectively, the classification accuracy of the model for the road surface image categories can be enhanced, by calculating the sum of the two kinds of prediction probabilities, the model can more comprehensively consider different features, improve the stability and precision of classification, finally select the maximum sum value probability, ensure the output of the most possible road surface image category, this multi-level probability calculation mode improves the generalization ability and robustness of the model.

[0081] In a possible implementation, the first prediction road surface image category probability of the output data of the simplified Transformer layer is extracted by the full connection layer, specifically:

[0082]

[0083] wherein, MLP(x) represents the first prediction road surface image category probability, x represents an input vector representing the output data of the simplified Transformer layer, W i-1 represents the transformation matrix of the i-1 layer MLP, i = 1, 2, …, N, N represents the total number of MLP layers, σ represents an activation function, represents a function composition operator.

[0084] In a possible implementation, the second prediction road surface image category probability of the output data of the simplified Transformer layer is extracted by the KAN layer, specifically:

[0085]

[0086]

[0087] wherein, f(x) represents an n-dimensional continuous function, KAN represents the KAN layer, KAN(x) represents the second prediction road surface image category probability, x p represents an input variable, i.e. the output data of the simplified Transformer layer, n represents the total dimension of the input variable, the outer layer Φ q and the inner layer φ q,p both represent one-dimensional continuous functions that can be learned, q represents the serial number used to distinguish the outer layer continuous function, p represents the serial number used to distinguish the inner layer continuous function, T0 represents the initial layer conversion function of the KAN layer, T i represents the conversion function of the i layer KAN layer, i = 1, 2, …, M, M represents the total number of KAN layers.

[0088] S4: input the road surface image data set as a training set to the road surface image recognition model, to train the road surface image recognition model, until the loss function value of the road surface image recognition model is less than the preset loss function value.

[0089] Wherein, the training set is a part of data in the data set for training the model, the model adjusts the parameters by learning the samples of the training set, and the preset loss function value is a threshold, when the loss function value of the model is lower than the threshold, the training process ends, which means that the model is accurate enough.

[0090] Specifically, the loss function value is a cross-entropy loss function value, and the cross-entropy loss function is a loss function for classification problems, which measures the performance of the model by calculating the difference between the predicted distribution and the actual distribution.

[0091] It should be noted that by using the cross-entropy loss function to train the road surface image recognition model, the gap between the model prediction result and the actual situation can be accurately measured, and the model will continue to optimize until the loss function value is less than the preset threshold, ensuring that the model has good performance. In addition, using the training set to input the model for training enables the model to effectively distinguish different road surface states in actual application and has strong recognition ability, and the whole process guarantees the stable convergence and accuracy of the model.

[0092] S5: Obtain a to-be-identified road surface image.

[0093] Wherein, the to-be-identified road surface image is a road surface image that needs to be identified in an actual application scenario, which is input data for prediction of the model after training, and contains images with unknown labels.

[0094] It should be noted that by directly inputting new road surface images for classification, it is ensured that the model can effectively operate in the actual traffic environment, quickly and accurately identify the road surface state, and support the decision and management of intelligent transportation.

[0095] S6: Input the to-be-identified road surface image into the trained road surface image recognition model, and output a predicted road surface image category.

[0096] Wherein, the to-be-identified road surface image refers to a road surface image that needs to be predicted and classified by the model in an actual scenario, the trained road surface image recognition model is a model optimized through the training stage, and has the ability to identify different road surface states. The predicted road surface image category is the result output by the model after classifying the input to-be-identified road surface image, indicating the road surface state category to which the image belongs.

[0097] It should be noted that by applying the trained model to the actual road surface image classification task, by inputting the to-be-identified image, the model can quickly output the predicted road surface category, and this process significantly improves the recognition efficiency, enabling the model to have the ability of instant application.

[0098] The technical scheme provided by the embodiment of the application has at least the following beneficial effects:

[0099] In the present application, based on the obtained road surface images of different road surface states, a road surface image recognition model is constructed by connecting a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network and an MLP classifier in turn based on the basic framework of the EfficientNet model. In the road surface image recognition model, a plurality of second EMBConv layers including an ECA module are introduced into the basic architecture EfficientNet model. The ECA efficiently calculates the channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependency between channels. Then, the feature map is mapped to a fixed dimension through the pooling layer. Subsequently, the Transformer layer is introduced, which can establish long-distance dependencies between different parts of the feature vector through the multi-head self-attention mechanism. Then, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and the KAN layer process the feature vector respectively, combining linear and nonlinear feature extraction, thereby improving the explainability and classification accuracy of the model. The introduced simplified Transformer layer enhances the global feature extraction capability of the model. At the same time, the KAN module with stronger explainability is added to improve the generalization ability of the model, enhance the local feature extraction of road surface recognition, improve the explainability, increase the classification accuracy of the model, and at the same time realize the optimization of the use of computing resources, which is conducive to realizing the accurate identification of real-time road surface, reducing traffic accidents, improving driving safety, and having a positive impact on social traffic safety, providing economic benefits for the construction and maintenance of intelligent transportation systems.

[0100] Reference is made to the accompanying drawings Figure 6 Fig. 1 shows a structural schematic diagram of a road surface image recognition system for intelligent transportation provided by the present application.

[0101] The present application also provides a road surface image recognition system for intelligent transportation 20 applied to the road surface image recognition method for intelligent transportation described above, comprising:

[0102] A processor 201.

[0103] A memory 202, the memory 202 stores computer readable instructions, and the computer readable instructions are executed by the processor 201 to realize the road surface image recognition method for intelligent transportation as in the method embodiment.

[0104] The road surface image recognition system for intelligent transportation 20 provided by the present application can execute the road surface image recognition method for intelligent transportation described above, and realize the same or similar technical effects. To avoid repetition, the present application will not be described again.

[0105] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0106] In the present application, based on the obtained road surface images of different road surface states, a road surface image recognition model is constructed by connecting a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network and an MLP classifier in turn based on the basic framework of the EfficientNet model. In the road surface image recognition model, the multi-layer second EMBConv layer including the ECA module is introduced in the basic architecture EfficientNet model. The ECA efficiently calculates the channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependency between channels. Then, the feature map is mapped to a fixed dimension through the pooling layer. Subsequently, the Transformer layer is introduced, and through the multi-head self-attention mechanism, the model can establish long-distance dependencies between different parts of the feature vector. Then, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and the KAN layer process the feature vector respectively, combining linear and nonlinear feature extraction, thereby improving the explainability and classification accuracy of the model. The introduced simplified Transformer layer enhances the global feature extraction capability of the model. At the same time, the KAN module with stronger explainability is added to improve the generalization ability of the model, enhance the local feature extraction of road surface recognition, improve the explainability, increase the classification accuracy of the model, and at the same time realize the optimization of the use of computing resources, which is conducive to realizing the accurate identification of real-time road surface, reducing traffic accidents, improving driving safety, having a positive impact on social traffic safety, and providing economic benefits for the construction and maintenance of intelligent transportation systems.

[0107] It should be understood that the processor in the embodiment of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0108] It should also be understood that the memory in the embodiments of the present application can be volatile or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. The volatile memory can be random access memory (RAM) used as external cache. By way of example, and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0109] The above-described embodiments can be implemented in part or in whole through software, hardware (e.g., circuitry), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When loaded and executed by a computer, the computer instructions or computer programs can generate the flow or function according to the embodiments of the present application in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, such as from one website site, computer, server, or data center to another website site, computer, server, or data center through a wired (e.g., infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0110] It should be understood that the term "and / or" used herein is merely an association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship, which can be understood in the context before and after it.

[0111] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or the like means any combination of the items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0112] It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0113] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0115] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0117] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0118] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0119] The embodiment of the present application provides a computer readable storage medium, which stores a computer program. The program is executed by a processor to realize the intelligent traffic-oriented road surface image recognition method of the method embodiment.

[0120] The computer readable storage medium provided by the present application can realize the steps and effects of the intelligent traffic-oriented road surface image recognition method of the method embodiment. To avoid repetition, the present application will not be described again.

[0121] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0122] In the present application, based on the obtained road surface images of different road surface states, a road surface image recognition model is constructed by connecting a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network and an MLP classifier in turn based on the basic framework of the EfficientNet model. In the road surface image recognition model, the multi-layer second EMBConv layer including the ECA module is introduced into the basic architecture EfficientNet model. The ECA efficiently calculates the channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependence between channels. Then, the feature map is mapped to a fixed dimension through the pooling layer. Subsequently, the Transformer layer is introduced, and through the multi-head self-attention mechanism, the model can establish long-distance dependence between parts of the feature vector. Then, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and the KAN layer process the feature vector respectively, combining linear and nonlinear feature extraction, thereby improving the explainability and classification accuracy of the model. The introduced simplified Transformer layer enhances the global feature extraction capability of the model. At the same time, the KAN module with stronger explainability is added to improve the generalization ability of the model, enhance the local feature extraction of road surface recognition, improve the explainability, increase the classification accuracy of the model, and at the same time realize the optimization of the use of computing resources, which is conducive to realizing the accurate identification of real-time road surface, reducing traffic accidents, improving driving safety, and having a positive impact on social traffic safety. It provides economic benefits for the construction and maintenance of intelligent transportation systems.

[0123] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0124] The following points need to be explained:

[0125] (1) The drawings of the embodiments of the present application only involve the structures involved in the embodiments of the present application, and other structures can refer to the usual design.

[0126] (2) For the sake of clarity, the thickness of the layer or region is magnified or reduced in the drawings used to describe the embodiments of the present application, that is, these drawings are not drawn according to the actual proportion. It can be understood that when an element such as a layer, a film, a region or a substrate is referred to as being located "on" or "under" another element, the element can be "directly" located on or under another element or there can be an intermediate element.

[0127] (3) In the case of no conflict, the embodiments and features in the embodiments can be combined to obtain new embodiments.

[0128] The above merely illustrates the embodiments of the present application, but the protection scope of the present application is not limited thereto, and the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A road surface image recognition method for intelligent transportation, characterized in that, include: S1: Obtain road surface images with different road surface conditions, wherein each road surface image and its corresponding road surface condition constitute a road surface image dataset; S2: Preprocess the road surface image dataset; S3: Construct a road image recognition model based on the EfficientNet efficient convolutional neural network architecture. The road image recognition model includes a first convolutional layer, multiple first EMBConv layers, multiple second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence. The second EMBConv layer includes an ECA attention module. The ECA module includes an input unit, a global average pooling unit, an adaptive one-dimensional convolutional kernel unit, a Sigmoid activation function unit, and an output unit connected in sequence. The input information of the input unit and the input information of the output unit are connected through skip connections. The MLP classifier includes a fully connected layer and a KAN layer. S4: Input the road image dataset as a training set into the road image recognition model to train the road image recognition model until the loss function value of the road image recognition model is less than the preset loss function value; S5: Acquire the image of the road surface to be identified; S6: Input the road surface image to be identified into the trained road surface image recognition model and output the predicted road surface image category.

2. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that, Each road surface condition includes road friction level attributes, road material attributes, and road unevenness attributes. The road friction level attributes include dry road friction level attributes, wet road friction level attributes, water-covered road friction level attributes, fresh snow road friction level attributes, snow-melted road friction level attributes, and ice-covered road friction level attributes. The road material attributes include asphalt road material attributes, concrete road material attributes, mud road material attributes, and gravel road material attributes. The road unevenness attributes include gently undulating road unevenness attributes, slightly undulating road unevenness attributes, and severely undulating road unevenness attributes. The road surface conditions include: a first road surface sub-state composed of the dry road friction level attribute, the asphalt road material attribute, and the unevenness attribute of the gently undulating road surface; a second road surface sub-state composed of the dry road friction level attribute, the asphalt road material attribute, and the unevenness attribute of the slightly undulating road surface; a third road surface sub-state composed of the dry road friction level attribute, the asphalt road material attribute, and the unevenness attribute of the severely undulating road surface; a fourth road surface sub-state having the icy road friction level attribute; a fifth road surface sub-state composed of the wet road friction level attribute, the asphalt road material attribute, and the unevenness attribute of the gently undulating road surface; a sixth road surface sub-state composed of the wet road friction level attribute, the asphalt road material attribute, and the unevenness attribute of the slightly undulating road surface; and a seventh road surface sub-state composed of the wet road friction level attribute, the asphalt road material attribute, and the severely undulating road surface attribute.

3. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that, The first EMBConv layer has 3 layers, and the second EMBConv layer has 4 layers. The second EMBConv layer includes an input unit, a first convolutional unit, a first batch normalization unit with a Swish activation function, a depthwise convolutional unit, a second batch normalization unit with a Swish activation function, a second convolutional unit, a third batch normalization unit, a random deactivation unit, and an output unit. The input unit is sequentially connected to the output unit through the convolution unit, the first batch normalization unit with the Swish activation function, the depthwise convolution unit, the second batch normalization unit with the Swish activation function, the ECA module, the third batch normalization unit, and the random deactivation unit. The input information of the input unit and the input information of the output unit are connected by a skip connection method. The simplified Transformer layer also includes an input module, a multi-head attention mechanism module, a first summation and normalization module, a second summation and normalization module, and an output module; The input module is connected to the output module in sequence through the multi-head attention mechanism module, the first summation and normalization module, the feedforward neural network, and the second summation and normalization module.

4. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that, The ECA module is used to enhance the channel attention mechanism in convolutional neural networks. The enhancement process is as follows: Receive an input feature map x, wherein the feature map has a width of W, a height of H, and a number of channels of C; Perform global average pooling on the feature map to obtain global information for each channel; Based on the global information, the weights of each channel are calculated by applying an adaptive one-dimensional convolution kernel; The attention weight values ​​for each channel are obtained by normalizing the calculated weights of each channel using the Sigmoid activation function. The attention weight value of each channel is multiplied by the channel feature corresponding to the feature map to obtain the weighted feature map.

5. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that, The specific method for calculating the adaptive one-dimensional convolution kernel size is as follows: Where k represents the size of the one-dimensional convolution kernel, γ represents the scaling parameter, b represents the constant offset, and log2(C) represents the base-2 logarithm operation on channel C. Represents the AND function The most recent odd number.

6. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that, The MLP classifier is used to classify the output data of the simplified Transformer layer and output the predicted road surface image category; the process of the MLP classifier classifying the output data of the simplified Transformer layer and determining the predicted road surface image category is as follows: The first predicted road surface image category probability is extracted from the output data of the simplified Transformer layer through the fully connected layer; The second predicted road surface image category probability is extracted from the output data of the simplified Transformer layer through the KAN layer; Calculate the sum of the probabilities of the first predicted road surface image category and the second predicted road surface image category; Output the predicted road surface image category corresponding to the highest sum probability.

7. The road surface image recognition method for intelligent transportation according to claim 6, characterized in that, The first predicted road surface image category probability, obtained by extracting the output data of the simplified Transformer layer through the fully connected layer, is specifically as follows: Where MLP(x) represents the probability of the first predicted road surface image category, x represents the input vector representing the simplified output data of the Transformer layer, and W i-1 Let represent the transformation matrix of the (i-1)th layer of the MLP, where i = 1, 2, ..., N, N represents the total number of layers in the MLP, and σ represents the activation function. This represents the compound operators for functions.

8. The road surface image recognition method for intelligent transportation according to claim 7, characterized in that, The second predicted road surface image category probability, which is extracted from the output data of the simplified Transformer layer through the KAN layer, is specifically as follows: Where f(x) represents an n-dimensional continuous function, KAN represents a KAN layer, KAN(x) represents the probability of the second predicted road surface image category, and x p This represents the input variables, which is a simplified version of the Transformer layer's output data. 'n' represents the total dimension of the input variables, and 'Φ' represents the outer layer. q and inner layer φ q,p All represent learnable one-dimensional continuous functions, q represents the index used to distinguish outer continuous functions, p represents the index used to distinguish inner continuous functions, T0 represents the initial layer transition function of the KAN layer, and T i Let represent the transformation function of the i-th KAN layer, where i = 1, 2, ..., M, and M represents the total number of KAN layers.

9. A road surface image recognition system for intelligent transportation, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the road surface image recognition method for intelligent transportation as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the road surface image recognition method for intelligent transportation as described in any one of claims 1 to 8.

Images