Road surface image recognition method and system for intelligent traffic

Abstract

The invention provides a road surface image recognition method and system for intelligent traffic, and relates to the technical field of intelligent traffic systems, and the method comprises the steps: obtaining road surface images with different road surface states; preprocessing the road surface image data set; constructing a pavement image recognition model based on an EfficentNet efficient convolutional neural network architecture, wherein the pavement image recognition model comprises a first convolutional layer, a plurality of first EMBConv layers, a plurality of second EMBConv layers, a second convolutional layer, a pooling layer, a simplified Transform layer with a feedforward neural network and an MLP classifier which are connected in sequence; inputting the pavement image data set as a training set into a pavement image recognition model until a loss function value of the pavement image recognition model is smaller than a preset loss function value; obtaining a to-be-recognized pavement image; and inputting a to-be-recognized pavement image into the trained pavement image recognition model, and outputting a predicted pavement image category.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation systems, and in particular to a road surface image recognition method and system for intelligent transportation. Background Art

[0002] Intelligent traffic pavement refers to the road surface that is monitored and analyzed by sensors, cameras or other detection equipment in the intelligent traffic system. The road surface information may include the friction, material type (such as asphalt or concrete), flatness (such as flat or undulating), and environmental conditions (such as wet, water, snow, etc.). Intelligent traffic pavement recognition technology helps the intelligent traffic system to accurately classify road images by automatically collecting and analyzing the road surface information, so as to improve driving safety and ride comfort.

[0003] As the number of vehicles continues to rise, more and more people choose cars as their preferred travel option. However, when a vehicle is driving under special road conditions, its handling and stability will be significantly reduced. Wet, snowy and uneven roads are the main reasons that affect the driving experience and cause frequent traffic accidents. In addition, uneven roads will also aggravate the vibration of the vehicle body, and transmit vibration and noise to the driver and passengers through the vehicle, directly affecting the handling stability and ride comfort. Therefore, as an important part of the human-vehicle-road information collaborative system, the vehicle's road type recognition ability is the key to improving driving safety and ride comfort. Accurately identifying the vehicle's road type is of great significance to driving safety and ride comfort.

[0004] Traditional methods for road type recognition mainly include the adhesion coefficient observation method based on dynamic modeling and the roughness estimation method based on vibration response. Although the dynamic modeling method is accurate and reliable, it relies on models, which are usually based on assumptions and simplifications under specific conditions and are difficult to fully cover complex scenarios in reality. When facing different road conditions, vehicles or environments, the adjustment of model parameters is difficult to adapt to a wide range of situations, resulting in poor adaptability in new scenarios and limited generalization ability. With the advancement of computer vision and artificial intelligence technology, environmental perception technology in driving systems has been widely used. The technical paths for such tasks are mainly divided into traditional machine learning methods and deep learning methods. Although machine learning methods are effective, they rely on manual design and feature extraction and are difficult to capture complex patterns or high-level features in images. In recent years, thanks to the advantages of convolutional neural networks (CNN) in automatic feature extraction and improving recognition accuracy, deep learning has been widely used in the field of road type recognition. However, most CNN-based deep neural networks have the problem of losing local highly relevant information when processing high-dimensional data, resulting in reduced model performance and low road recognition accuracy. Summary of the invention

[0005] In order to solve the problem that the CNN-based deep neural network in the prior art loses local height-related information when processing high-dimensional data, resulting in reduced model performance and low road recognition accuracy, the present invention provides a road image recognition method and system for intelligent transportation.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides a road surface image recognition method for intelligent transportation, comprising:

[0009] S1: Acquire road images with different road conditions, wherein each road image and the corresponding road condition constitute a road image data set;

[0010] S2: preprocess the road image dataset;

[0011] S3: Construct a road image recognition model based on the EfficientNet efficient convolutional neural network architecture, wherein the road image recognition model includes a first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, a second convolution layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence, wherein the second EMBConv layer includes an ECA attention module, wherein the ECA module includes an input unit, a global average pooling unit, an adaptive one-dimensional convolution kernel unit, a Sigmoid activation function unit, and an output unit connected in sequence, wherein the input information of the input unit is connected to the input information of the output unit by a jump connection, wherein the MLP classifier includes a fully connected layer and a KAN layer;

[0012] S4: inputting the road surface image data set as a training set into the road surface image recognition model to train the road surface image recognition model until the loss function value of the road surface image recognition model is less than a preset loss function value;

[0013] S5: Acquire the road surface image to be identified;

[0014] S6: Input the road image to be identified into the trained road image recognition model, and output the predicted road image category.

[0015] Second aspect

[0016] An embodiment of the present invention provides a road surface image recognition system for intelligent transportation, comprising:

[0017] processor;

[0018] A memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the road surface image recognition method for intelligent transportation as described in the first aspect is implemented.

[0019] The third aspect

[0020] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the road surface image recognition method for intelligent transportation as described in the first aspect is implemented.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] In the present invention, based on the acquired road images of different road conditions, a road image recognition model is constructed by sequentially connecting the first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, second convolution layers, pooling layers, simplified Transformer layers with feedforward neural networks, and MLP classifiers with the EfficientNet model as the basic framework. The road image recognition model introduces a multi-layer second EMBConv layer including an ECA module into the basic architecture EfficientNet model. ECA efficiently calculates channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependencies between channels. After that, the feature map passes through the pooling layer to map to a fixed dimension. Subsequently, the Transformer layer is introduced, and the model is able to establish long-distance dependencies between the parts of the feature vector through the multi-head self-attention mechanism. Next, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and KAN layer process feature vectors respectively, combining linear and nonlinear feature extraction, thereby improving the interpretability and classification accuracy of the model. The simplified Transformer layer introduced enhances the global feature extraction capability of the model. At the same time, the addition of a KAN module with stronger interpretability improves the generalization ability of the model, enhances the local feature extraction of road surface recognition, improves interpretability, and increases the classification accuracy of the model. At the same time, it optimizes the use of computing resources, which is conducive to the accurate recognition of real-time road surfaces, reduces traffic accidents, improves driving safety, has a positive impact on social traffic safety, and provides economic benefits for the construction and maintenance of intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a process flow of a road surface image recognition method for intelligent transportation provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of the structure of a road surface image recognition model provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of the structure of a second EMBConv layer provided in an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a simplified Transformer layer structure provided by an embodiment of the present invention;

[0028] Figure 5 A schematic diagram of the structure of an ECA module provided in an embodiment of the present invention;

[0029] Figure 6 A schematic structural diagram of a road surface image recognition system for intelligent transportation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0031] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0032] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0033] Reference Manual Attached Figure 1 , showing a flow chart of a road surface image recognition method for intelligent transportation provided by an embodiment of the present invention.

[0034] The embodiment of the present invention provides a road surface image recognition method for intelligent transportation, which can be implemented by a road surface image recognition device for intelligent transportation, and the road surface image recognition device for intelligent transportation can be a terminal or a server. The processing flow of the road surface image recognition method for intelligent transportation can include the following steps:

[0035] S1: Acquire road surface images with different road surface conditions.

[0036] Among them, each road surface image and the corresponding road surface state constitute a road surface image dataset.

[0037] Among them, road surface images refer to photos of road surfaces taken under different conditions. These conditions may include different weather, different road materials, and different road flatness. The road surface image dataset is a collection of pictures consisting of different road surface images and their corresponding road surface conditions, which serves as the basic data for model training.

[0038] It should be noted that by collecting images with diverse road conditions and constructing a rich dataset that can reflect the complex road conditions in reality, the diversity of this dataset provides a wider range of samples for subsequent model training, improves the model's ability to recognize various road conditions, and ensures the robustness and accuracy of the recognition system in different traffic environments.

[0039] In a possible implementation, each road surface state includes a road friction level attribute, a road material attribute, and a road roughness attribute, wherein the road friction level attribute includes a dry state road friction level attribute, a wet state road friction level attribute, a water state road friction level attribute, a new snow state road friction level attribute, a melting snow state road friction level attribute, and an ice state road friction level attribute; the road material attributes include an asphalt road material attribute, a concrete road material attribute, a mud road material attribute, and a gravel road material attribute; and the road roughness attribute includes a gently undulating state road roughness attribute, a slightly undulating state road roughness attribute, and a severely undulating state road roughness attribute.

[0040] The road surface state includes a first road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a gently undulating state road unevenness attribute; a second road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a slightly undulating state road unevenness attribute; a third road surface sub-state composed of a dry state road friction level attribute, an asphalt road material attribute, and a severely undulating state road unevenness attribute; a fourth road surface sub-state composed of an icy state road friction level attribute; a fifth road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a gently undulating state road unevenness attribute; a sixth road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a slightly undulating state road unevenness attribute; and a seventh road surface sub-state composed of a water state road friction level attribute, an asphalt road material attribute, and a severely undulating state road unevenness attribute.

[0041] S2: Preprocess the road image dataset.

[0042] Among them, the preprocessing includes dividing the road image dataset into preset proportions, cropping, random horizontal flipping, tensor format conversion and normalization.

[0043] Dividing the road surface image dataset according to a preset ratio specifically involves dividing the preprocessed road surface image dataset into a training set, a validation set, and a test set according to a preset ratio. Specifically, the preset ratio is 8:1:1.

[0044] It should be noted that the quality and diversity of the data are significantly improved through preprocessing steps of the road image dataset, such as cropping, random horizontal flipping, tensor format conversion and normalization. Cropping and flipping can increase the variability of training data and avoid overfitting of the model. Tensor format conversion makes the image more suitable for deep learning framework processing, while normalization speeds up the convergence of the model.

[0045] Reference Manual Attached Figure 2 , shows a schematic structural diagram of a road surface image recognition model provided by an embodiment of the present invention.

[0046] Figure 2 In the figure, xn indicates that the first EMBconv layer is n layers, n is 3, and xm indicates that the second EMBConv layer is m layers, m is 4.

[0047] For example, if the input image size is 224×224×3, the first convolution layer, 3 first EMBConv layers, 4 second EMBConv layers, the second convolution layer, the pooling layer, the simplified Transformer layer with a feed-forward neural network, and the MLP classifier are repeated 1, 1, 2, 3, 3, 4, 1, 1, 1, 1, and 1 times respectively, then the first convolution layer, 3 first EMBConv layers, 4 second EMBConv layers, the second convolution layer, the pooling layer, the simplified Transformer layer with a feed-forward neural network, and the MLP classifier are repeated 1, 1, 2, 3, 3, 4, 1, 1, 1, 1, and 1 times respectively. The image sizes obtained after optimizing the Transformer layer and MLP classifier are 112×112×32, 112×112×16, 56×56×24, 28×28×40, 14×14×80, 14×14×112, 7×7×192, 7×7×320, 7×7×1280, 1×1×1280, 1×1×1280 and 1×1×7, respectively. The image size can be uniformly expressed as a×b×c, where a and b represent the spatial resolution of the image, i.e., width and height, and c represents the number of image channels, also known as image depth.

[0048] S3: Construct a road image recognition model based on the EfficientNet efficient convolutional neural network architecture, wherein the road image recognition model includes a first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, a second convolution layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence, wherein the second EMBConv layer includes an ECA attention module, wherein the ECA module includes an input unit, a global average pooling unit, an adaptive one-dimensional convolution kernel unit, a Sigmoid activation function unit, and an output unit connected in sequence, wherein the input information of the input unit is connected to the input information of the output unit by a jump connection, wherein the MLP classifier includes a fully connected layer and a KAN layer.

[0049] Among them, EfficientNet (Efficient Network) is an efficient convolutional neural network architecture that maintains efficient performance at different computing costs by compound scaling model parameters. The road image recognition model is a deep learning-based model specifically used to identify images of different road conditions. The first convolution layer and the second convolution layer are basic layers in the deep learning network, responsible for extracting features from the image and realizing feature mapping through convolution operations. The first EMBConv layer and the second EMBConv layer are improved convolution layers that combine convolution operations and embedded convolution kernel optimization for more efficient feature extraction. The pooling layer is used to reduce dimensionality and reduce the amount of computation while retaining important feature information. The feedforward neural network is a non-cyclic neural network structure with a unidirectional data flow from input to output. The Transformer layer is a neural network layer used to process sequence data. Here is a simplified version used to capture long-distance dependencies in images. The MLP (Multilayer Perceptron) classifier is a multilayer perceptron used to classify the extracted features and output the final predicted category. The ECA (Efficient Channel The Attention (Efficient Channel Attention) module is used to improve the channel attention mechanism in the network, and enhances the feature representation by dynamically adjusting the importance of different channels. The input unit is the first module of the neural network model. It receives the original data and passes it to the subsequent network layers. The global average pooling unit is a pooling operation that generates a feature map by averaging all pixel values ​​of each channel. The adaptive one-dimensional convolution kernel unit is used to dynamically adjust the size of the convolution kernel. It selects the most appropriate convolution kernel size by adaptively calculating the input features to better capture the correlation between channels. The Sigmoid activation function is a common activation function that compresses the input value to between 0 and 1 and is used in binary classification problems or attention mechanisms. The output unit is the last layer of the network. It receives the processing results of the previous layer and generates the final output. The KAN (Kernel Attention Network) layer is a network module used to improve the performance of the classifier, which optimizes classification decisions by focusing on features of different dimensions.

[0050] Specifically, both the first summation and normalization unit and the second summation and normalization unit first sum the input features and then normalize them.

[0051] It should be noted that by combining EfficientNet and the simplified Transformer layer, an efficient and powerful road image recognition model is formed. It uses the powerful feature extraction capability of the convolutional neural network and the channel attention mechanism of the ECA module to enhance the recognition performance of complex images. At the same time, the long-distance dependencies in the image are captured through the Transformer layer. The entire architecture design effectively improves the accuracy and robustness of the model while maintaining efficient use of computing resources to meet the real-time recognition needs in intelligent transportation.

[0052] Refer to the instruction manual Figure 3 , shows a schematic structural diagram of a second EMBConv layer provided in an embodiment of the present invention.

[0053] Figure 3 In the figure, Conv1×1 from left to right represents the first convolution unit and the second convolution unit, respectively. The two BN Swish from left to right represent the first batch normalization unit with Swish activation function and the second batch normalization unit with Swish activation function, respectively. DWConvk×k represents a deep convolution unit, ECA represents an ECA module, BN represents the third batch normalization unit, and Droupout represents a random dropout unit.

[0054] Reference Manual Attached Figure 4 , showing a schematic diagram of the structure of a simplified Transformer layer provided in an embodiment of the present invention.

[0055] Figure 4 The multi-head attention mechanism, summation & normalization, feedforward neural network, and summation & normalization from input to output respectively represent the input module, the multi-head attention mechanism module, the first summation and normalization module, the feedforward neural network module, the second summation and normalization module, and the output module.

[0056] In one possible implementation, the first EMBConv layer is 3 layers, and the second EMBConv layer is 4 layers, wherein the second EMBConv layer includes an input unit, a first convolution unit, a first batch normalization unit with a Swish activation function, a depthwise convolution unit, a second batch normalization unit with a Swish activation function, a second convolution unit, a third batch normalization unit, a random dropout unit, and an output unit.

[0057] The input unit is connected to the output unit in sequence through a convolution unit, a first batch normalization unit with a Swish activation function, a depth convolution unit, a second batch normalization unit with a Swish activation function, an ECA module, a third batch normalization unit, and a random inactivation unit, wherein the input information of the input unit is connected to the input information of the output unit through a jump connection.

[0058] The simplified Transformer layer also includes an input module, a multi-head attention mechanism module, a first sum normalization module, a second sum normalization module and an output module.

[0059] The input module is connected to the output module in sequence through the multi-head attention mechanism module, the first sum normalization module, the feedforward neural network, and the second sum normalization module.

[0060] Reference Manual Attached Figure 5 , showing a structural schematic diagram of an ECA module provided by an embodiment of the present invention.

[0061] Figure 5 In , x represents the input feature map, C represents the number of channels of the feature map, H and W represent the height and width of the feature map, GAP represents the global average pooling unit, 1×1×C represents the global feature size of each channel after compression by the global average pooling unit, σ represents the activation function, Represents the output feature map after weighting by the channel attention mechanism.

[0062] The size of the input feature map is H×W×C. First, the feature map passes through the global average pooling unit to compress the height and width to 1 and generate statistical information of the channel dimension. Then, the statistical information passes through two 1×1 convolutional layers and is processed by a nonlinear activation function to generate a channel weight vector. This weight vector is used to weight each channel of the original input feature map, emphasizing important channel information and suppressing unimportant channels. Finally, the feature map after channel weighting is used for calculation of subsequent network layers to help the model better capture the dependencies between channels and enhance feature expression capabilities.

[0063] In a possible implementation, the ECA module is used to enhance the channel attention mechanism in the convolutional neural network, and the enhancement process is specifically as follows:

[0064] Receive the input feature map x, where the width of the feature map is W, the height is H, and the number of channels is C.

[0065] Perform a global average pooling operation on the feature map to obtain the global information of each channel.

[0066] According to the global information, the weights of each channel are calculated by applying an adaptive one-dimensional convolution kernel.

[0067] The weights of each channel obtained after calculation are normalized through the Sigmoid activation function to obtain the attention weight value of each channel.

[0068] Multiply the attention weight value of each channel with the channel feature corresponding to the feature map to obtain a weighted feature map.

[0069] It should be noted that the feature expression ability of the model is enhanced through the channel attention mechanism, global average pooling is used to extract important global information, the adaptive convolution kernel calculates the weight of each channel, and normalizes it through the Sigmoid function to highlight key features and suppress irrelevant features. This not only improves the recognition accuracy of the model, but also optimizes computing resources, making the model more efficient and showing stronger generalization capabilities in diverse input scenarios.

[0070] In a possible implementation, the adaptive one-dimensional convolution kernel size is calculated as follows:

[0071]

[0072] Among them, k represents the one-dimensional convolution kernel size, γ represents the scaling parameter, b represents the constant offset, log 2 (C) indicates the logarithm operation with base 2 on channel C. Represents the AND function Nearest odd number.

[0073] The feature map is input into the multi-head attention mechanism, which then extracts important features from the input data. The multi-head attention mechanism outputs the extracted data, performs summation and normalization operations, and further extracts and processes the normalized features through a feedforward neural network to enhance the network's nonlinear expression capabilities. The output of the feedforward network is then summed and normalized again to make the features more robust and unified. The final result is processed and output as the prediction result or classification result of the model.

[0074] In a possible implementation, the MLP classifier is used to classify the output data of the simplified Transformer layer and output a predicted road surface image category.

[0075] The MLP classifier classifies the output data of the simplified Transformer layer and determines the predicted road image category as follows:

[0076] The first predicted road image category probability of the output data of the simplified Transformer layer is extracted through the fully connected layer.

[0077] The second predicted road image category probability of the output data of the simplified Transformer layer is extracted by the KAN layer.

[0078] The sum probability of the first predicted road surface image category probability and the second predicted road surface image category probability is calculated.

[0079] The predicted road image category corresponding to the maximum sum probability is output.

[0080] It should be noted that by combining the fully connected layer and the KAN layer to calculate the two prediction probabilities respectively, the classification accuracy of the model for road image categories can be enhanced. By calculating the sum of the two prediction probabilities, the model can consider different features more comprehensively, improve the stability and accuracy of classification, and finally select the largest sum probability to ensure the output of the most likely road image category. This multi-level probability calculation method improves the generalization ability and robustness of the model.

[0081] In a possible implementation, the first predicted road image category probability of the output data of the simplified Transformer layer is extracted by the fully connected layer, specifically:

[0082]

[0083] Where MLP(x) represents the first predicted road image category probability, x represents the input vector representing the output data of the simplified Transformer layer, and W i-1 represents the transformation matrix of the i-1th layer MLP, i=1,2,…,N, N represents the total number of MLP layers, σ represents the activation function, Represents a function composition operator.

[0084] In a possible implementation, the second predicted road image category probability of the output data of the simplified Transformer layer is extracted by the KAN layer, specifically:

[0085]

[0086]

[0087] Where f(x) represents an n-dimensional continuous function, KAN represents a KAN layer, KAN(x) represents the second predicted road image category probability, and x p represents the input variable, that is, the output data of the simplified Transformer layer, n represents the total dimension of the input variable, and the outer layer Φ q and the inner layer φ q,p Both represent learnable one-dimensional continuous functions, q represents the serial number used to distinguish the outer continuous function, p represents the serial number used to distinguish the inner continuous function, T 0 represents the initial layer conversion function of the KAN layer, T i represents the conversion function of the i-th KAN layer, i = 1, 2, ..., M, and M represents the total number of KAN layers.

[0088] S4: Inputting the road surface image data set as a training set into the road surface image recognition model to train the road surface image recognition model until the loss function value of the road surface image recognition model is less than a preset loss function value.

[0089] Among them, the training set is a part of the data in the dataset used to train the model. The model adjusts parameters by learning samples from the training set. The preset loss function value is a threshold. When the loss function value of the model is lower than this value, the training process ends, which means that the model is accurate enough.

[0090] Specifically, the loss function value is the cross entropy loss function value. The cross entropy loss function is a loss function used for classification problems. It measures the performance of the model by calculating the difference between the predicted distribution and the actual distribution.

[0091] It should be noted that by using the cross entropy loss function to train the road image recognition model, the gap between the model prediction results and the actual situation can be accurately measured. The model will continue to optimize until the loss function value is less than the preset threshold to ensure that the model has good performance. In addition, using the training set input model for training enables the model to effectively distinguish different road conditions in practical applications and has strong recognition capabilities. The whole process ensures the stable convergence and accuracy of the model.

[0092] S5: Acquire a road surface image to be identified.

[0093] Among them, the road image to be identified is the road image that needs to be identified in the actual application scenario. It is the input data used for prediction by the model after training and contains images with unknown labels.

[0094] It should be noted that by directly inputting new road surface images for classification, it is ensured that the model can operate effectively in the actual traffic environment, quickly and accurately identify the road surface status, and support the decision-making and management of intelligent transportation.

[0095] S6: Input the road image to be identified into the trained road image recognition model, and output the predicted road image category.

[0096] Among them, the road surface image to be identified refers to the road surface image that needs to be predicted and classified by the model in the actual scene. The trained road surface image recognition model is a model that has been optimized after the training stage and has the ability to identify different road surface conditions. The predicted road surface image category is the output result after the model classifies the input road surface image to be identified, which indicates the road surface condition category to which the image belongs.

[0097] It should be noted that by applying the trained model to the actual road image classification task and inputting the image to be identified, the model can quickly output the predicted road category. This process significantly improves the recognition efficiency and enables the model to be applied immediately.

[0098] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0099] In the present invention, based on the acquired road images of different road conditions, a road image recognition model is constructed by sequentially connecting the first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, second convolution layers, pooling layers, simplified Transformer layers with feedforward neural networks, and MLP classifiers with the EfficientNet model as the basic framework. The road image recognition model introduces a multi-layer second EMBConv layer including an ECA module into the basic architecture EfficientNet model. ECA efficiently calculates channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependencies between channels. After that, the feature map passes through the pooling layer to map to a fixed dimension. Subsequently, the Transformer layer is introduced, and the model is able to establish long-distance dependencies between the parts of the feature vector through the multi-head self-attention mechanism. Next, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and KAN layer process feature vectors respectively, combining linear and nonlinear feature extraction, thereby improving the interpretability and classification accuracy of the model. The simplified Transformer layer introduced enhances the global feature extraction capability of the model. At the same time, the addition of a KAN module with stronger interpretability improves the generalization ability of the model, enhances the local feature extraction of road surface recognition, improves interpretability, and increases the classification accuracy of the model. At the same time, it optimizes the use of computing resources, which is conducive to the accurate recognition of real-time road surfaces, reduces traffic accidents, improves driving safety, has a positive impact on social traffic safety, and provides economic benefits for the construction and maintenance of intelligent transportation systems.

[0100] Reference Manual Attached Figure 6 , showing a schematic structural diagram of a road surface image recognition system for intelligent transportation provided by the present invention.

[0101] The present invention further provides a road surface image recognition system 20 for intelligent transportation, which is applied to the above-mentioned road surface image recognition method for intelligent transportation, and comprises:

[0102] Processor 201.

[0103] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the road surface image recognition method for intelligent transportation according to the method embodiment is implemented.

[0104] The road surface image recognition system 20 for intelligent transportation provided by the present invention can execute the above-mentioned road surface image recognition method for intelligent transportation and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0105] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0106] In the present invention, based on the acquired road images of different road conditions, a road image recognition model is constructed by sequentially connecting the first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, second convolution layers, pooling layers, simplified Transformer layers with feedforward neural networks, and MLP classifiers with the EfficientNet model as the basic framework. The road image recognition model introduces a multi-layer second EMBConv layer including an ECA module into the basic architecture EfficientNet model. ECA efficiently calculates channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependencies between channels. After that, the feature map passes through the pooling layer to map to a fixed dimension. Subsequently, the Transformer layer is introduced, and the model is able to establish long-distance dependencies between the parts of the feature vector through the multi-head self-attention mechanism. Next, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and KAN layer process feature vectors respectively, combining linear and nonlinear feature extraction, thereby improving the interpretability and classification accuracy of the model. The simplified Transformer layer introduced enhances the global feature extraction capability of the model. At the same time, the addition of a KAN module with stronger interpretability improves the generalization ability of the model, enhances the local feature extraction of road surface recognition, improves interpretability, and increases the classification accuracy of the model. At the same time, it optimizes the use of computing resources, which is conducive to the accurate recognition of real-time road surfaces, reduces traffic accidents, improves driving safety, has a positive impact on social traffic safety, and provides economic benefits for the construction and maintenance of intelligent transportation systems.

[0107] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0108] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0109] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0110] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0111] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0112] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0113] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0119] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a road surface image recognition method for intelligent transportation is implemented as in the method embodiment.

[0120] A computer-readable storage medium provided by the present invention can implement the steps and effects of the road surface image recognition method for intelligent transportation in the above method embodiment. To avoid repetition, the present invention will not go into details.

[0121] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0122] In the present invention, based on the acquired road images of different road conditions, a road image recognition model is constructed by sequentially connecting the first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, second convolution layers, pooling layers, simplified Transformer layers with feedforward neural networks, and MLP classifiers with the EfficientNet model as the basic framework. The road image recognition model introduces a multi-layer second EMBConv layer including an ECA module into the basic architecture EfficientNet model. ECA efficiently calculates channel attention by adaptively selecting the convolution kernel size, so that the model can better capture the dependencies between channels. After that, the feature map passes through the pooling layer to map to a fixed dimension. Subsequently, the Transformer layer is introduced, and the model is able to establish long-distance dependencies between the parts of the feature vector through the multi-head self-attention mechanism. Next, the feature vector passes through the fully connected layer and the KAN layer, and then the output results of the two are added to generate the final classification prediction. The fully connected layer and KAN layer process feature vectors respectively, combining linear and nonlinear feature extraction, thereby improving the interpretability and classification accuracy of the model. The simplified Transformer layer introduced enhances the global feature extraction capability of the model. At the same time, the addition of a KAN module with stronger interpretability improves the generalization ability of the model, enhances the local feature extraction of road surface recognition, improves interpretability, and increases the classification accuracy of the model. At the same time, it optimizes the use of computing resources, which is conducive to the accurate recognition of real-time road surfaces, reduces traffic accidents, improves driving safety, has a positive impact on social traffic safety, and provides economic benefits for the construction and maintenance of intelligent transportation systems.

[0123] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0124] There are a few points to note:

[0125] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0126] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0127] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0128] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A road surface image recognition method for intelligent transportation, characterized in that: include: S1: Acquire road images with different road conditions, wherein each road image and the corresponding road condition constitute a road image data set; S2: preprocessing the road surface image dataset; S3: Construct a road image recognition model based on the EfficientNet efficient convolutional neural network architecture, wherein the road image recognition model includes a first convolution layer, multiple first EMBConv layers, multiple second EMBConv layers, a second convolution layer, a pooling layer, a simplified Transformer layer with a feedforward neural network, and an MLP classifier connected in sequence, wherein the second EMBConv layer includes an ECA attention module, wherein the ECA module includes an input unit, a global average pooling unit, an adaptive one-dimensional convolution kernel unit, a Sigmoid activation function unit, and an output unit connected in sequence, wherein the input information of the input unit is connected to the input information of the output unit by a jump connection, wherein the MLP classifier includes a fully connected layer and a KAN layer; S4: inputting the road surface image data set as a training set into the road surface image recognition model to train the road surface image recognition model until the loss function value of the road surface image recognition model is less than a preset loss function value; S5: Acquire the road surface image to be identified; S6: Input the road image to be recognized into the trained road image recognition model, and output the predicted road image category.

2. The road surface image recognition method for intelligent transportation according to claim 1 is characterized in that: Each road surface state includes a road friction level attribute, a road material attribute, and a road unevenness attribute, wherein the road friction level attribute includes a dry state road friction level attribute, a wet state road friction level attribute, a water state road friction level attribute, a new snow state road friction level attribute, a snowmelt state road friction level attribute, and an ice state road friction level attribute; the road material attribute includes an asphalt road material attribute, a concrete road material attribute, a mud road material attribute, and a gravel road material attribute; the road unevenness attribute includes a gently undulating state road unevenness attribute, a slightly undulating state road unevenness attribute, and a severely undulating state road unevenness attribute; The road surface state includes a first road surface sub-state composed of the dry state road friction level attribute, the asphalt road material attribute and the gently undulating state road unevenness attribute; a second road surface sub-state composed of the dry state road friction level attribute, the asphalt road material attribute and the slightly undulating state road unevenness attribute; a third road surface sub-state composed of the dry state road friction level attribute, the asphalt road material attribute and the severely undulating state road unevenness attribute; a fourth road surface sub-state composed of the ice state road friction level attribute; a fifth road surface sub-state composed of the water state road friction level attribute, the asphalt road material attribute and the gently undulating state road unevenness attribute; a sixth road surface sub-state composed of the water state road friction level attribute, the asphalt road material attribute and the slightly undulating state road unevenness attribute; and a seventh road surface sub-state composed of the water state road friction level attribute, the asphalt road material attribute and the severely undulating state road unevenness attribute.

3. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that: The first EMBConv layer has 3 layers, and the second EMBConv layer has 4 layers, wherein the second EMBConv layer includes an input unit, a first convolution unit, a first batch normalization unit with a Swish activation function, a depth convolution unit, a second batch normalization unit with a Swish activation function, a second convolution unit, a third batch normalization unit, a random dropout unit, and an output unit; The input unit is connected to the output unit in sequence through the convolution unit, the first batch normalization unit with the Swish activation function, the depth convolution unit, the second batch normalization unit with the Swish activation function, the ECA module, the third batch normalization unit, and the random inactivation unit, wherein the input information of the input unit is connected to the input information of the output unit by a jump connection; The simplified Transformer layer also includes an input module, a multi-head attention mechanism module, a first sum normalization module, a second sum normalization module and an output module; The input module is connected to the output module in sequence through the multi-head attention mechanism module, the first sum normalization module, the feedforward neural network, and the second sum normalization module.

4. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that: The ECA module is used to enhance the channel attention mechanism in the convolutional neural network. The enhancement process is as follows: Receive an input feature map x, where the width of the feature map is W, the height is H, and the number of channels is C; Performing a global average pooling operation on the feature map to obtain global information of each channel; According to the global information, the weight of each channel is calculated by applying an adaptive one-dimensional convolution kernel; The weights of each channel obtained after calculation are normalized by the Sigmoid activation function to obtain the attention weight value of each channel; The attention weight value of each channel is multiplied by the channel feature corresponding to the feature map to obtain a weighted feature map.

5. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that: The calculation method of the adaptive one-dimensional convolution kernel size is specifically as follows: Among them, k represents the one-dimensional convolution kernel size, γ represents the scaling parameter, b represents the constant offset, and log2(C) represents the logarithm operation with base 2 on channel C. Represents the AND function Nearest odd number.

6. The road surface image recognition method for intelligent transportation according to claim 1, characterized in that: The MLP classifier is used to classify the output data of the simplified Transformer layer and output the predicted road surface image category; the MLP classifier classifies the output data of the simplified Transformer layer and determines the predicted road surface image category in detail: Extracting a first predicted road surface image category probability of output data of the simplified Transformer layer through the fully connected layer; Extracting a second predicted road surface image category probability of the output data of the simplified Transformer layer through the KAN layer; Calculating a sum probability of the first predicted road surface image category probability and the second predicted road surface image category probability; The predicted road image category corresponding to the maximum sum probability is output.

7. The method for road surface image recognition for intelligent transportation according to claim 6, characterized in that: The first predicted road surface image category probability of the output data of the simplified Transformer layer extracted by the fully connected layer is specifically: Where MLP(x) represents the first predicted road image category probability, x represents the input vector representing the output data of the simplified Transformer layer, and W i-1 represents the transformation matrix of the i-1th layer MLP, i=1,2,…,N, N represents the total number of MLP layers, σ represents the activation function, Represents a function composition operator.

8. The road surface image recognition method for intelligent transportation according to claim 7, characterized in that: The second predicted road surface image category probability of the output data of the simplified Transformer layer extracted by the KAN layer is specifically: Where f(x) represents an n-dimensional continuous function, KAN represents a KAN layer, KAN(x) represents the second predicted road image category probability, and x p represents the input variable, that is, the output data of the simplified Transformer layer, n represents the total dimension of the input variable, and the outer layer Φ q and the inner layer φ q,p Both represent learnable one-dimensional continuous functions, q represents the serial number used to distinguish the outer continuous function, p represents the serial number used to distinguish the inner continuous function, T0 represents the initial layer conversion function of the KAN layer, T i represents the conversion function of the i-th KAN layer, i = 1, 2, ..., M, and M represents the total number of KAN layers.

9. A road image recognition system for intelligent transportation, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the road surface image recognition method for intelligent transportation as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the road surface image recognition method for intelligent transportation as described in any one of claims 1 to 8 is implemented.

Images