Road surface recognition method and device based on improved ShuffleNetV2, medium and vehicle-mounted intelligent system

Abstract

The invention discloses a road surface recognition method and device based on improved ShuffleNetV2, a medium and a vehicle-mounted intelligent system, and the method comprises the steps: obtaining to-be-recognized road surface image data; road surface image data to be recognized are input into a pre-trained improved ShuffleNetV2 model, a prediction result of a road surface type is obtained, a SENet module is introduced into a main branch of each basic unit of the improved ShuffleNetV2 model, and a ReLU activation function in the SENet module is replaced with an H-Sash activation function; and feeding back the prediction result of the road surface type to the intelligent driving system, so that the intelligent driving system realizes an automatic driving decision function and a vehicle driving mode regulation and control function according to the prediction result of the road surface type. Therefore, the SENet module is introduced into the main branch of the basic unit of the improved ShuffleNetV2 model to improve the pavement feature extraction capability of the model, and the ReLU activation function in the SE module adopts an H-Sash activation function to accelerate the convergence rate and improve the nonlinear expression capability of the model, so that the efficient recognition of the complex pavement type is realized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a road surface recognition method based on an improved ShuffleNetV2, a computer-readable storage medium, a road surface recognition device based on an improved ShuffleNetV2, and an in-vehicle intelligent system. Background Art

[0002] In modern transportation systems, accurate identification of road conditions is of great significance for ensuring driving safety and optimizing road maintenance strategies. With the development of computer vision technology, image-based road recognition methods have gradually emerged. However, the problem with related technologies is that when existing deep learning models are applied to road recognition, the number of parameters is too large and the computational complexity is high, which places stringent requirements on hardware equipment resources. At the same time, when faced with the complex and changeable actual road environment, the robustness is poor and the recognition accuracy is unstable. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first object of the present invention is to propose a road surface recognition method based on an improved ShuffleNetV2, which can achieve efficient recognition of complex road surface types while reducing model complexity.

[0004] A second object of the present invention is to provide a computer-readable storage medium.

[0005] The third object of the present invention is to propose a road surface recognition device based on improved ShuffleNetV2.

[0006] A fourth objective of the present invention is to provide an in-vehicle intelligent system.

[0007] To achieve the above-mentioned purpose, the first embodiment of the present invention proposes a road surface recognition method based on the improved ShuffleNetV2, including: obtaining road surface image data to be identified; inputting the road surface image data to be identified into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type, wherein a SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by an H-Swish activation function; the prediction result of the road surface type is fed back to the intelligent driving system, so that the intelligent driving system realizes the automatic driving decision-making function and the vehicle driving mode control function according to the prediction result of the road surface type.

[0008] According to an embodiment of the present invention, a road surface recognition method based on an improved ShuffleNetV2 is used to obtain road surface image data to be recognized. The road surface image data to be recognized is then input into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type. A SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced with an H-Swish activation function. Furthermore, the predicted road surface type result is fed back to the intelligent driving system, enabling the intelligent driving system to implement an autonomous driving decision-making function and a vehicle driving mode control function based on the predicted road surface type result. Thus, by introducing the SENet module into the main branch of the basic unit of the improved ShuffleNetV2 model to improve the model's road surface feature extraction capability, and replacing the ReLU activation function in the SE module with an H-Swish activation function to accelerate convergence and improve the model's nonlinear expression capability, efficient recognition of complex road surface types is achieved while reducing model complexity.

[0009] In addition, the road surface recognition method based on the improved ShuffleNetV2 according to the above embodiment of the present invention may also have the following additional technical features:

[0010] According to one embodiment of the present invention, the prediction result of the road surface type includes each road surface type and a prediction probability value corresponding to each road surface type.

[0011] According to one embodiment of the present invention, the SENet module performs a global average pooling operation on the road surface image data to be identified to obtain global information of each channel, calculates the channel attention weight through two fully connected layers, and multiplies the channel attention weight with the road surface image data to be identified to readjust the weight of the feature pixel value, wherein the weight of the feature pixel value is used to generate a prediction result of the road surface type.

[0012] According to one embodiment of the present invention, the autonomous driving decision-making function includes: adjusting the vehicle driving decision according to the prediction result of the road type through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving decision.

[0013] According to one embodiment of the present invention, the vehicle driving mode control function includes: adjusting the vehicle driving mode according to the prediction result of the road type and the vehicle status information through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving mode.

[0014] According to one embodiment of the present invention, training the improved ShuffleNetV2 model includes: collecting road image data and preprocessing the road image data; constructing a training data set based on the preprocessed road image data; dividing the training data set into a training set and a test set according to a preset ratio; training the improved ShuffleNetV2 model using the training set and the test set according to preset training parameters; updating the model parameters using an optimization algorithm, monitoring the model performance using a validation set during training, and adjusting the learning rate based on the validation results.

[0015] According to one embodiment of the present invention, the method further includes: obtaining evaluation index parameters of the improved ShuffleNetV2 model, the evaluation index parameters including accuracy, precision, recall rate, F1 score and specificity; and verifying the model performance of the improved ShuffleNetV2 model based on the evaluation index parameters.

[0016] In order to achieve the above-mentioned purpose, the computer-readable storage medium proposed in the second embodiment of the present invention stores a road surface recognition program based on the improved ShuffleNetV2. When the road surface recognition program based on the improved ShuffleNetV2 is executed by the processor, the road surface recognition method based on the improved ShuffleNetV2 of the above-mentioned embodiment of the present invention is implemented.

[0017] According to the computer-readable storage medium of an embodiment of the present invention, by executing the road surface recognition program based on the improved ShuffleNetV2 stored thereon, it is possible to achieve efficient recognition of complex road surface types while reducing model complexity.

[0018] To achieve the above-mentioned purpose, the road surface recognition device based on the improved ShuffleNetV2 proposed in the third aspect of the embodiment of the present invention includes: an acquisition module for acquiring road surface image data to be identified; an identification module for inputting the road surface image data to be identified into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type, wherein a SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by an H-Swish activation function; a control module for feeding back the prediction result of the road surface type to the intelligent driving system, so that the intelligent driving system realizes the automatic driving decision-making function and the vehicle driving mode control function according to the prediction result of the road surface type.

[0019] According to an embodiment of the present invention, a road surface recognition device based on an improved ShuffleNetV2 acquires road surface image data to be recognized through an acquisition module. Furthermore, the road surface image data to be recognized is input into a pre-trained improved ShuffleNetV2 model through a recognition module to obtain a road surface type prediction result. In this case, an SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced with the H-Swish activation function. Furthermore, the road surface type prediction result is fed back to the intelligent driving system through a control module, enabling the intelligent driving system to implement autonomous driving decision-making functions and vehicle driving mode control functions based on the road surface type prediction result. Thus, by introducing the SENet module into the main branch of the basic unit of the improved ShuffleNetV2 model to improve the model's road surface feature extraction capability, and replacing the ReLU activation function in the SE module with the H-Swish activation function to accelerate convergence and improve the model's nonlinear expression capability, efficient recognition of complex road surface types is achieved while reducing model complexity.

[0020] To achieve the above-mentioned purpose, the in-vehicle intelligent system proposed in the fourth embodiment of the present invention includes the road surface recognition device based on the improved ShuffleNetV2 of the above-mentioned embodiment of the present invention.

[0021] According to the in-vehicle intelligent system of an embodiment of the present invention, by adopting the aforementioned road surface recognition device based on the improved ShuffleNetV2, it is possible to achieve efficient recognition of complex road types while reducing model complexity.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 2 is a flow chart of a road surface recognition method based on an improved ShuffleNetV2 according to an embodiment of the present invention;

[0024] Figure 2 2 is a schematic diagram of the network structure of an improved ShuffleNetV2 model according to an embodiment of the present invention;

[0025] Figure 3 2 is a schematic diagram of a network structure of a SENet module according to an embodiment of the present invention;

[0026] Figure 4 1 is a flow chart of a road surface recognition method based on an improved ShuffleNetV2 according to an embodiment of the present invention;

[0027] Figure 5 is a flowchart of a road surface recognition method based on an improved ShuffleNetV2 according to another embodiment of the present invention;

[0028] Figure 6 is a block diagram of a road surface recognition device based on an improved ShuffleNetV2 according to an embodiment of the present invention;

[0029] Figure 7 FIG. 4 is a block diagram of an in-vehicle intelligent system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0031] The following describes, with reference to the accompanying drawings, a road surface recognition method based on an improved ShuffleNetV2, a computer-readable storage medium, a road surface recognition device based on an improved ShuffleNetV2, and an in-vehicle intelligent system according to embodiments of the present invention.

[0032] Figure 1 3 is a flow chart of a road surface recognition method based on the improved ShuffleNetV2 according to an embodiment of the present invention.

[0033] Specifically, in some embodiments of the present invention, Figure 1 As shown in the figure, the road surface recognition method based on the improved ShuffleNetV2 includes:

[0034] S101, obtaining road surface image data to be identified.

[0035] It can be understood that in this embodiment of the present invention, the road surface image in front of the vehicle is captured by the vehicle-mounted camera as the road surface image data to be identified, and then the road surface type recognition is implemented based on the road surface image data to be identified.

[0036] S102: Input the road image data to be identified into the pre-trained improved ShuffleNetV2 model to obtain the prediction result of the road type, wherein the SENet module is introduced into each basic unit main branch of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by the H-Swish activation function.

[0037] It is understandable that in this embodiment of the present invention, the lightweight ShuffleNetV2 architecture shown in Table 1 is improved to obtain Figure 2 The improved ShuffleNetV2 model shown in Figure 1, where the feature map size represents the resolution of the feature map output by each module. Specifically, Figure 2 As shown in Figures 2b and 2c, the SENet (Squeeze-and-Excitation) module is introduced after the last 1×1 convolution layer of the main branches of basic units 1 and 2. Specifically, by introducing the SENet module into each basic unit main branch of the improved ShuffleNetV2 model, the improved ShuffleNetV2 model can automatically learn the importance of different channel features, enhance the improved ShuffleNetV2 model's ability to extract key road features, and replace the ReLU activation function in the SENet module with the H-Swish activation function, which alleviates the gradient vanishing problem to a certain extent, accelerates the convergence speed of the improved ShuffleNetV2 model, and improves the nonlinear expression ability of the improved ShuffleNetV2 model, which helps to better fit complex road features. As a result, the improved ShuffleNetV2 model has stronger road image feature extraction and classification performance, enabling it to efficiently identify complex road types in a lightweight architecture, and then obtain the corresponding road type prediction results based on the aforementioned road image data to be identified.

[0038]

[0039] Specifically, based on the improved ShuffleNetV2 model of the embodiment of the present invention, while ensuring the recognition accuracy, the model size (only 5.64MB) and parameter amount (1.43M) are compressed, so that the model can run efficiently on resource-constrained mobile devices or edge computing devices, and realize real-time road surface recognition. In addition, through the effective extraction of key road surface features by the SENet module and the improvement of the model expression ability by the H-Swish activation function, the model can accurately distinguish different types of road surface features, effectively reducing misjudgments and missed judgments.

[0040] S103, feeding back the prediction result of the road surface type to the intelligent driving system, so that the intelligent driving system can realize the automatic driving decision function and the vehicle driving mode control function according to the prediction result of the road surface type.

[0041] It can be understood that in this embodiment of the present invention, the prediction results of the road type can be applied to the intelligent driving system (for example, providing the vehicle with road condition information in advance), so that the intelligent driving system can realize the automatic driving decision-making function (for example, vehicle speed decision, path planning decision, etc.) and vehicle driving mode control function (for example, power (comfort, standard, sports, off-road) control, braking control, steering control, suspension height (low / medium / high) control, suspension damping (soft / moderate / hard) control, etc.) according to the prediction results of the road type.

[0042] Furthermore, in some embodiments of the present invention, the prediction result of the road surface type includes each road surface type and a prediction probability value corresponding to each road surface type.

[0043] It can be understood that in this embodiment of the present invention, the road surface image data to be identified is first processed by the preprocessing module, cropped (removing irrelevant parts of the image edge and retaining only the key areas containing road surface information), uniformly resized to 224×224 pixels, and normalized (mapping pixel values ​​to a specific range, such as the [0,1] interval to optimize model calculation efficiency). Then, the preprocessed road surface image data to be identified is input into the improved ShuffleNetV2 model, which performs calculations through the forward propagation algorithm. Specifically, each layer of the network structure of the improved ShuffleNetV2 model extracts and analyzes the image features of the road surface image data to be identified in turn, for example, from the basic features of the shallow network (such as edges, textures, etc.) to the semantic features of the deep network. Finally, the improved ShuffleNetV2 model outputs the prediction result of the road surface type, where the prediction result of the road surface type is presented in the form of a probability distribution, including each road surface type (such as cement road, asphalt road, damaged road, ice and snow road, gravel road, dirt road, etc.) and the corresponding prediction probability value of each road surface type.

[0044] Furthermore, in some embodiments of the present invention, the SENet module performs a global average pooling operation on the road surface image data to be identified to obtain global information of each channel, calculates the channel attention weight through two fully connected layers, and multiplies the channel attention weight with the road surface image data to be identified to readjust the weight of the feature pixel value, wherein the weight of the feature pixel value is used to generate a prediction result of the road surface type.

[0045] It can be understood that in this embodiment of the present invention, the SENet module obtains the global information of each channel by performing a global average pooling operation on the feature map (i.e., the road surface image data to be identified), and then calculates the channel attention weight through two fully connected layers. Finally, the weight is multiplied by the original feature map to readjust the weight of the feature pixel value, so that the network can automatically learn the importance of different channel features. Among them, the weight of the feature pixel value is used to generate the prediction result of the road surface type, which can effectively enhance the ability to extract key features of the road surface.

[0046] Specifically, in the above embodiment of the present invention, the SENet module obtains the weight of the road surface features in each channel dimension through Squeeze and Excitation operations, and then assigns the feature weights to the feature map through Scale operation, so that the improved ShuffleNetV2 model focuses more on the extraction of key features of the road surface.

[0047] For example, if Figure 3 As shown in the figure, assuming the input to the SENet module is image data with a resolution of H×W×C, the SENet module then performs global average pooling through a Squeeze operation, compressing the feature map to generate a 1×1×C feature vector. Furthermore, through an Excitation operation, two fully connected layers with H-Swish and H-Sigmoid nonlinear activation functions are used to generate corresponding weights for each channel, establishing associations between the channels in the feature map. Finally, a Scale operation is used to assign weights to the feature map. Thus, by emphasizing the main features of the road surface and suppressing unimportant features such as background, the SENet module enables the improved ShuffleNetV2 model to focus on important features of the road surface.

[0048] In addition, in the above embodiment of the present invention, the ReLU activation function in the SENet module is replaced by the H-Swish activation function, wherein the H-Swish activation function is improved from the Swish function, which is a function that can closely approximate the Swish function and exhibit higher computational efficiency on mobile devices, as shown in the following equations (1)(2)(3):

[0049] ReLU6(x)=min(max(0,x),6)…………(1)

[0050]

[0051]

[0052]

[0053] Therefore, in the training process of the improved ShuffleNetV2 model, the application of the H-Swish activation function can alleviate the gradient vanishing problem to a certain extent, accelerate the model convergence speed, and improve the nonlinear expression ability of the model, which helps to better learn the complex feature relationships in road images.

[0054] Furthermore, in some embodiments of the present invention, the autonomous driving decision-making function includes: adjusting the vehicle driving decision according to the prediction results of the road type through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving decision.

[0055] It can be understood that in this embodiment of the present invention, based on the prediction results of the road type output by the improved ShuffleNetV2 model, the intelligent driving system can obtain the road condition information in front of the vehicle in advance, realize road preview, and then adjust the vehicle driving decision, and control the vehicle to perform intelligent driving according to the adjusted vehicle driving decision.

[0056] For example, when the road type prediction results output by the improved ShuffleNetV2 model determine that the road ahead is an icy or snowy road, the intelligent driving system can immediately initiate a series of vehicle driving decisions to respond. For example, the intelligent driving system will reduce the vehicle speed in advance, and build a vehicle dynamics model based on the relationship between the adhesion coefficient and slip rate of the icy or snowy road to calculate the safe driving speed, and send instructions to the vehicle's power control system to adjust the engine output power or motor torque to ensure that the vehicle travels on special roads with optimal power and avoid the vehicle skidding out of control. At the same time, it optimizes path planning to avoid the vehicle making sharp turns or frequent lane changes on the ice, and chooses a safer and more stable driving route.

[0057] Furthermore, in some embodiments of the present invention, the vehicle driving mode control function includes: adjusting the vehicle driving mode according to the prediction results of the road type and the vehicle status information through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving mode.

[0058] It can be understood that in this embodiment of the present invention, based on the prediction results of the road type output by the improved ShuffleNetV2 model, the intelligent driving system can obtain the road condition information in front of the vehicle in advance, realize road preview, and then adjust the vehicle driving mode in combination with the vehicle status information perceived by the chassis domain sensors (such as accelerometers, gyroscopes, wheel speed sensors, suspension height sensors, etc.), and control the vehicle to perform intelligent driving according to the adjusted vehicle driving mode.

[0059] For example, when the road type prediction results output by the improved ShuffleNetV2 model determine that the road ahead is a gravel road, and the chassis domain sensor detects changes in wheel speed, suspension compression, etc. that are consistent with gravel road driving characteristics, the driving mode control system in the intelligent driving system will decisively switch the driving mode to off-road mode in advance. At this time, the power system will adjust the power output to a high-torque mode adapted to off-road conditions to ensure that the vehicle has sufficient power to climb and drive on gravel roads; the braking system adjusts the brake pressure distribution and response speed to enhance the braking effect on loose roads; the steering system increases steering damping to provide a more stable and precise steering feel, avoiding steering loss of control due to bumpy roads; the suspension system adjusts the height to high to improve the vehicle's passability and prevent the chassis from scraping gravel, while adjusting the suspension damping to hard to reduce the shaking of the vehicle body on bumpy roads, reduce the risk of vehicle rollover, and improve driving comfort and handling stability.

[0060] Furthermore, in some embodiments of the present invention, Figure 4 As shown, the training improved ShuffleNetV2 model includes:

[0061] S201: collecting road surface image data and preprocessing the road surface image data.

[0062] It can be understood that in this embodiment of the present invention, collecting road image data includes: 1) camera selection and installation: selecting a camera with high resolution (such as 1280×720 pixels or higher), high frame rate (30fps or above) and good low-light performance, and installing it in a suitable position on the vehicle (such as above the front windshield, below the rearview mirror, etc.) according to the application scenario to ensure that the image of the road ahead can be clearly captured. At the same time, it is equipped with corresponding sensors (such as light sensors, GPS locators, etc.) to record environmental information and vehicle position information when collecting images; 2) image acquisition: performing image acquisition under different road types, weather conditions and time periods. For example, in different scenarios such as urban roads, highways, rural roads, etc., acquisition is performed under weather conditions such as sunny days, cloudy days, rainy days, snowy days, and different time periods (such as morning, noon, and evening). During the acquisition process, the vehicle maintains a normal driving speed to obtain real road image data.

[0063] In addition, in this embodiment of the present invention, preprocessing of road surface image data includes: 1) image preprocessing: removing areas in the image that are not related to road surface recognition, such as the sky, vehicle body, roadside scenery, etc., and retaining only the road surface part, for example, by setting a fixed cropping area or automatically identifying the road surface area based on image segmentation technology for cropping, and uniformly adjusting the cropped image to the size required for model input; 2) normalization processing: normalizing the image pixel values ​​to a specific range, such as [0,1] or [-1,1], for example, by calculating the mean and standard deviation of the image pixel values, and then performing a normalization operation on each pixel value; 3) random transformation enhancement: performing random transformations on the normalized image, including random rotation, random horizontal and vertical flipping, random brightness adjustment, random contrast adjustment, etc. These random transformation operations are performed on the training set images to increase the diversity of the data set and improve the robustness of the model.

[0064] S202: Construct a training data set based on the preprocessed road surface image data.

[0065] It can be understood that in this embodiment of the present invention, a training data set for training the improved ShuffleNetV2 model is constructed based on the preprocessed road image data.

[0066] S203: Divide the training data set into a training set and a test set according to a preset ratio.

[0067] It can be understood that in this embodiment of the present invention, the training data set is divided into five categories according to the road surface type, namely concrete road, asphalt road, ice and snow road, dirt road and gravel road, and is divided into training set and test set according to a preset ratio (for example, 8:2).

[0068] S204: Train the improved ShuffleNetV2 model using the training set and the test set according to preset training parameters.

[0069] It can be understood that in this embodiment of the present invention, the improved ShuffleNetV2 model is trained using the training set and the test set according to the preset training parameters shown in Table 2 below, such as the initial learning rate (0.001), momentum (0.9), weight decay coefficient (0.0005), number of training rounds (Epoch = 100) and batch size (determined according to the device video memory and data set size, such as 32).

[0070] Table 2

[0071]

[0072] S205, using an optimization algorithm to update the model parameters, and using a validation set to monitor the model performance during the training process, and adjusting the learning rate based on the validation results.

[0073] It can be understood that in this embodiment of the present invention, based on the above Table 2, an optimization algorithm (such as stochastic gradient descent method SGD) is used to update the model parameters, and a validation set is used to monitor the model performance during the training process, and the learning rate is adjusted according to the validation results (such as using a cosine annealing strategy) to prevent overfitting (such as L2 regularization).

[0074] Among them, the learning rate adjustment strategy adopts the cosine annealing strategy to adjust the learning rate. In the early stage of training, the learning rate is large so that the model converges quickly, and as the number of training rounds increases, the learning rate gradually decreases to avoid unstable model parameter updates caused by excessive learning rate in later training. For example, after the number of training rounds reaches a certain proportion of the total number of rounds (such as 70%), the learning rate begins to gradually decrease according to the cosine annealing curve. At the same time, the model performance is monitored and optimized during the training process, and the model is evaluated after each certain number of training rounds (such as 10 rounds).

[0075] Furthermore, in some embodiments of the present invention, Figure 6 As shown, the method further includes:

[0076] S301, obtaining evaluation index parameters of the improved ShuffleNetV2 model, the evaluation index parameters including accuracy, precision, recall, F1 score and specificity.

[0077] It should be understood that since road type recognition is a multi-classification problem, the confusion matrix contains more information than a single evaluation metric for multi-classification tasks. For example, higher values ​​on the diagonal indicate higher prediction accuracy. Therefore, in this embodiment of the present invention, a confusion matrix is ​​used to intuitively display the prediction results of the classification model corresponding to each category. The following indicators are combined as evaluation parameters for testing model performance: Accuracy (A), Precision (P), Recall (R), F1 score, and Specificity, to comprehensively evaluate the improved model.

[0078] Accuracy refers to the proportion of correctly classified samples in the total number of samples, and its calculation formula is:

[0079]

[0080] The F1 score is an evaluation indicator that comprehensively considers precision and recall. Its calculation formula is:

[0081]

[0082] Specificity represents the false detection rate of the model on the road surface recognition dataset. A higher specificity value indicates that the model can correctly identify more road samples and reduce the number of false positives. Its calculation formula is:

[0083]

[0084] Precision and recall measure the model's ability to classify road surface types.

[0085]

[0086] Among them, TP means the model predicts a positive example and the true label is also a positive example (correct prediction), FP means the model predicts a positive example but the true label is a negative example (false positive), TN means the model predicts a negative example and the true label is also a negative example (correct prediction), and FN means the model predicts a negative example but the true label is a positive example (false negative).

[0087] S302: Verify the model performance of the improved ShuffleNetV2 model based on the evaluation index parameters.

[0088] It is understood that in this embodiment of the present invention, by calculating the ShuffleNetV2 model

[0089] Accuracy, precision, recall and other indicators on the test set are used to observe the changing trend of the loss function value to verify the model performance of the improved ShuffleNetV2 model according to the evaluation index parameters. If it is found that the performance of the model on the test set no longer improves or shows a downward trend (i.e., overfitting), corresponding optimization measures can be taken, such as terminating training early, increasing the regularization strength (such as increasing the L2 regularization coefficient), adjusting the model structure (such as reducing the number of network layers or channels), etc. At the same time, through the Tensorboard visualization method, the loss function curve and accuracy curve are drawn, which can intuitively monitor the model training process, identify problems in time and make adjustments.

[0090] Optionally, in some embodiments of the present invention, in terms of model improvement, other attention mechanisms (such as CBAM-ConvolutionalBlockAttentionModule) can be tried to replace the SENet module, or different network structure combinations can be explored, such as combining ShuffleNetV2 with the residual structure in ResNet to further improve the model's ability to extract road features. In addition, for activation functions, in addition to the H-Swish activation function, other new activation functions (such as Mish, etc.) can be invented and used to evaluate their performance in road recognition tasks, and the most suitable activation function can be selected to optimize the model. In addition, in terms of data acquisition, it is possible to consider adding multi-camera data fusion to obtain road image information from different angles to improve the model's ability to understand complex road conditions. At the same time, the diversity of the data set can be further enriched, including collecting road images in different seasons and different geographical regions to enhance the generalization ability of the model.

[0091] In summary, according to the road surface recognition method based on the improved ShuffleNetV2 in an embodiment of the present invention, road surface image data to be recognized is obtained, and then the road surface image data to be recognized is input into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type, wherein the SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, the ReLU activation function in the SENet module is replaced with the H-Swish activation function, and the prediction result of the road surface type is fed back to the intelligent driving system, so that the intelligent driving system implements the automatic driving decision function and the vehicle driving mode control function based on the prediction result of the road surface type. Therefore, by introducing the SENet module into the main branch of the basic unit of the improved ShuffleNetV2 model to improve the road surface feature extraction capability of the model, and adopting the H-Swish activation function as the ReLU activation function in the SE module to accelerate the convergence speed and improve the nonlinear expression capability of the model, efficient recognition of complex road surface types is achieved while reducing the complexity of the model.

[0092] Based on the aforementioned road surface recognition method based on the improved ShuffleNetV2 of the embodiment of the present invention, the embodiment of the present invention also proposes a computer-readable storage medium, on which a road surface recognition program based on the improved ShuffleNetV2 is stored. When the road surface recognition program based on the improved ShuffleNetV2 is executed by the processor, the road surface recognition method based on the improved ShuffleNetV2 of the embodiment of the present invention is implemented.

[0093] It should be understood that the specific implementation of the computer-readable storage medium of the embodiment of the present invention can refer to the specific implementation of the road surface recognition method based on the improved ShuffleNetV2 in the aforementioned embodiment of the present invention. To reduce redundancy, it will not be repeated here.

[0094] In summary, according to the computer-readable storage medium of an embodiment of the present invention, by executing the road surface recognition program based on the improved ShuffleNetV2 stored thereon, it is possible to achieve efficient recognition of complex road surface types while reducing model complexity.

[0095] Figure 6 2 is a block diagram of a road surface recognition device based on an improved ShuffleNetV2 according to an embodiment of the present invention.

[0096] Specifically, in some embodiments of the present invention, Figure 6 As shown, the road surface recognition device 100 based on the improved ShuffleNetV2 includes: an acquisition module 10, a recognition module 20 and a control module 30.

[0097] Among them, the acquisition module 10 is used to obtain the road surface image data to be identified; the recognition module 20 is used to input the road surface image data to be identified into the pre-trained improved ShuffleNetV2 model to obtain the prediction result of the road surface type, wherein the SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by the H-Swish activation function; the control module 30 is used to feed back the prediction result of the road surface type to the intelligent driving system, so that the intelligent driving system can realize the automatic driving decision-making function and the vehicle driving mode control function according to the prediction result of the road surface type.

[0098] Furthermore, in some embodiments of the present invention, the prediction result of the road surface type includes each road surface type and a prediction probability value corresponding to each road surface type.

[0099] Furthermore, in some embodiments of the present invention, the SENet module performs a global average pooling operation on the road surface image data to be identified to obtain global information of each channel, calculates the channel attention weight through two fully connected layers, and multiplies the channel attention weight with the road surface image data to be identified to readjust the weight of the feature pixel value, wherein the weight of the feature pixel value is used to generate a prediction result of the road surface type.

[0100] Furthermore, in some embodiments of the present invention, the autonomous driving decision-making function includes: adjusting the vehicle driving decision according to the prediction results of the road type through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving decision.

[0101] Furthermore, in some embodiments of the present invention, the vehicle driving mode control function includes: adjusting the vehicle driving mode according to the prediction results of the road type and the vehicle status information through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving mode.

[0102] Furthermore, in some embodiments of the present invention, training the improved ShuffleNetV2 model includes: collecting road image data and preprocessing the road image data; constructing a training data set based on the preprocessed road image data; dividing the training data set into a training set and a test set according to a preset ratio; training the improved ShuffleNetV2 model using the training set and the test set according to preset training parameters; updating the model parameters using an optimization algorithm, and monitoring the model performance using a validation set during the training process, and adjusting the learning rate based on the validation results.

[0103] Furthermore, in some embodiments of the present invention, the acquisition module 10 is also used to obtain evaluation index parameters of the improved ShuffleNetV2 model, the evaluation index parameters including accuracy, precision, recall rate, F1 score and specificity; and verify the model performance of the improved ShuffleNetV2 model according to the evaluation index parameters.

[0104] It should be understood that the specific implementation of the road surface recognition device 100 based on the improved ShuffleNetV2 in the embodiment of the present invention corresponds one-to-one to the specific implementation of the road surface recognition method based on the improved ShuffleNetV2 in the aforementioned embodiment of the present invention. In order to reduce redundancy, they will not be repeated here.

[0105] In summary, according to an embodiment of the present invention, a road surface recognition device based on an improved ShuffleNetV2 acquires road surface image data to be recognized through an acquisition module. Furthermore, the road surface image data to be recognized is input into a pre-trained improved ShuffleNetV2 model through a recognition module to obtain a prediction result of the road surface type. In this regard, an SENet module is introduced into the main branch of each basic unit of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced with an H-Swish activation function. Furthermore, the predicted results of the road surface type are fed back to the intelligent driving system through a control module, so that the intelligent driving system implements an autonomous driving decision-making function and a vehicle driving mode control function based on the predicted results of the road surface type. Thus, by introducing the SENet module into the main branch of the basic unit of the improved ShuffleNetV2 model to improve the model's road feature extraction capability, and replacing the ReLU activation function in the SE module with an H-Swish activation function to accelerate convergence and improve the model's nonlinear expression capability, efficient recognition of complex road surface types is achieved while reducing model complexity.

[0106] Figure 7 FIG. 4 is a block diagram of an in-vehicle intelligent system according to an embodiment of the present invention.

[0107] Specifically, in some embodiments of the present invention, Figure 7 As shown, the in-vehicle intelligent system 1000 includes the road surface recognition device 100 based on the improved ShuffleNetV2 according to the above-mentioned embodiment of the present invention.

[0108] It should be understood that the specific implementation of the in-vehicle intelligent system 1000 of the embodiment of the present invention can refer to the specific implementation of the road surface recognition method based on the improved ShuffleNetV2 in the aforementioned embodiment of the present invention. To reduce redundancy, it will not be repeated here.

[0109] In summary, the in-vehicle intelligent system according to an embodiment of the present invention, by adopting the aforementioned road surface recognition device based on the improved ShuffleNetV2, can achieve efficient recognition of complex road types while reducing model complexity.

[0110] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0111] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0112] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0113] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0114] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0115] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0116] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0117] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A road surface recognition method based on improved ShuffleNetV2, characterized in that: The method comprises: Acquiring road surface image data to be identified; Inputting the road surface image data to be identified into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type, wherein a SENet module is introduced into each basic unit main branch of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by the H-Swish activation function; The prediction result of the road surface type is fed back to the intelligent driving system, so that the intelligent driving system can realize the automatic driving decision function and the vehicle driving mode control function according to the prediction result of the road surface type.

2. The road surface recognition method based on the improved ShuffleNetV2 according to claim 1 is characterized in that: The prediction result of the road surface type includes each road surface type and a prediction probability value corresponding to each road surface type.

3. The road surface recognition method based on the improved ShuffleNetV2 according to claim 2 is characterized in that: The SENet module performs a global average pooling operation on the road surface image data to be identified to obtain global information of each channel, calculates channel attention weights through two fully connected layers, and multiplies the channel attention weights with the road surface image data to be identified to readjust the weights of feature pixel values, wherein the weights of the feature pixel values ​​are used to generate the prediction results of the road surface type.

4. The road surface recognition method based on the improved ShuffleNetV2 according to claim 3 is characterized in that: The automatic driving decision-making function includes: adjusting the vehicle driving decision according to the prediction result of the road type through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving decision.

5. The road surface recognition method based on the improved ShuffleNetV2 according to claim 3 is characterized in that: The vehicle driving mode control function includes: adjusting the vehicle driving mode according to the prediction result of the road type and the vehicle status information through the intelligent driving system, and controlling the vehicle to perform intelligent driving according to the adjusted vehicle driving mode.

6. The road surface recognition method based on the improved ShuffleNetV2 according to any one of claims 1 to 5, characterized in that: Training the improved ShuffleNetV2 model includes: Collecting road surface image data and preprocessing the road surface image data; Construct a training dataset based on the preprocessed road image data; Dividing the training data set into a training set and a test set according to a preset ratio; According to preset training parameters, the improved ShuffleNetV2 model is trained using the training set and the test set; The optimization algorithm is used to update the model parameters, and the validation set is used to monitor the model performance during the training process, and the learning rate is adjusted according to the validation results.

7. The road surface recognition method based on improved ShuffleNetV2 according to claim 6, characterized in that: The method further comprises: Obtain evaluation index parameters of the improved ShuffleNetV2 model, wherein the evaluation index parameters include accuracy, precision, recall, F1 score and specificity; The model performance of the improved ShuffleNetV2 model is verified according to the evaluation index parameters.

8. A computer-readable storage medium, characterized in that A road surface recognition program based on the improved ShuffleNetV2 is stored thereon, and when the road surface recognition program based on the improved ShuffleNetV2 is executed by the processor, the road surface recognition method based on the improved ShuffleNetV2 according to any one of claims 1 to 7 is implemented.

9. A road surface recognition device based on improved ShuffleNetV2, characterized in that: The device comprises: An acquisition module, used for acquiring road surface image data to be identified; A recognition module is configured to input the road surface image data to be recognized into a pre-trained improved ShuffleNetV2 model to obtain a prediction result of the road surface type, wherein a SENet module is introduced into each basic unit main branch of the improved ShuffleNetV2 model, and the ReLU activation function in the SENet module is replaced by an H-Swish activation function; The control module is used to feed back the prediction result of the road surface type to the intelligent driving system, so that the intelligent driving system can realize the automatic driving decision function and the vehicle driving mode control function according to the prediction result of the road surface type.

10. An in-vehicle intelligent system, characterized in that: The in-vehicle intelligent system includes the road surface recognition device based on the improved ShuffleNetV2 as described in claim 9.

Images