Road surface category recognition model construction method based on split-channel deep convolution and application

Abstract

The application provides a road surface type recognition model construction method and application based on a split-channel deep convolution, and comprises the following steps: obtaining multiple road surface images labeled with road surface types as a training sample set; constructing a road surface type recognition architecture, and training the road surface type recognition architecture with the training sample set; the road surface type recognition architecture comprises an initial feature extraction network, a deep feature extraction network and a classifier; when an iteration stop condition is reached, the training is ended, and the optimal parameters of the road surface type recognition architecture are saved to obtain a road surface type recognition model. According to the scheme, the input features are subjected to channel division and differential processing through split-channel deep convolution, invalid calculation is reduced, modulation coefficients are generated based on global context through a light-weight gating network, adaptive modulation of feature channels is realized, the contribution degree of effective features is improved, and the pertinence of feature expression is enhanced.

Description

Technical Field

[0001] This application relates to the field of road surface recognition, and in particular to a method for constructing and applying a road surface type recognition model based on channel-wise depth convolution. Background Technology

[0002] With the rapid development of smart cities and intelligent transportation systems, accurate perception of the vehicle environment has become a core element for the implementation of intelligent driving and the improvement of traffic efficiency. As a key component of the vehicle perception system, road type recognition directly determines the accuracy of vehicle driving safety, the rationality of path planning, and the effectiveness of intelligent control strategies, making it a research focus in the field of intelligent transportation.

[0003] Existing road surface type recognition technologies are mainly divided into three categories: contact measurement, system response-based measurement, and non-contact measurement. Each method has obvious technical shortcomings: Contact measurement relies on physical sensors such as accelerometers to directly contact the road surface, and has high measurement accuracy at low speeds, but the equipment investment cost is high, the real-time performance is poor, and it is difficult to adapt to the dynamic driving needs of vehicles; System response-based measurement methods achieve recognition through indirect data such as vehicle suspension vibration response. Although the real-time performance is improved, there is a problem of response lag in sudden road sections, which can easily cause recognition errors; Non-contact measurement uses visual sensors such as cameras and LiDAR to collect road surface images or point cloud data, and combines them with deep learning models to complete the classification. This is the mainstream direction of current technological development. There are cases in related research that use LSTM networks and SVM combined with EfficientNet neural networks to achieve road surface type classification. However, these models generally have problems such as excessive number of parameters and insufficient inference speed, poor adaptability to vehicle systems, and difficulty in efficient deployment on vehicle embedded devices.

[0004] The lightweight MobileNet series of convolutional neural networks, with its depthwise separable convolutions, significantly reduces computational cost and parameter count, achieving efficient inference. It has been widely applied in computer vision fields such as object detection; for example, research based on MobileNetV3 has effectively improved model generalization ability and prediction accuracy. However, traditional MobileNet models suffer from simple network structures and limited feature representation capabilities. While MobileNetV4 further optimizes the model structure, demonstrating advantages in balancing complexity and performance and adapting to computationally sensitive road classification scenarios, it is prone to increased computational cost and latency as model depth increases, limiting its application in high-precision, high-real-time road recognition scenarios. Currently, there are few studies applying MobileNetV4 to road type classification. Without additional optimization, its native structure struggles to meet the high requirements of feature robustness and fine-grained perception capabilities in complex road scenarios while maintaining extreme lightweight design. It cannot cope with practical problems such as varying road textures and complex lighting and weather conditions. Therefore, exploring road type identification methods based on lightweight MobileNetV4 has become a key requirement for promoting the practical application and implementation of road recognition technology. Summary of the Invention

[0005] This application provides a method and application for constructing a road surface category recognition model based on channel-specific depth convolution. By performing channel-specific depth convolution on the input features, invalid computation is reduced. At the same time, a lightweight gating network is used to generate modulation coefficients based on the global context, thereby achieving adaptive modulation of feature channels, increasing the contribution of effective features and enhancing the relevance of feature representation.

[0006] In a first aspect, embodiments of this application provide a method for constructing a road surface category recognition model based on channel-wise depth convolution, the method comprising:

[0007] Obtain multiple road surface images labeled with road surface type as a training sample set;

[0008] A road surface type recognition architecture is constructed based on MobileNetV4, and the road surface type recognition architecture is trained with a training sample set. The road surface type recognition architecture includes an initial feature extraction network, a deep feature extraction network, and a classifier.

[0009] In the training process of the road surface type training architecture, the initial feature extraction network uses convolutional decomposition to extract features from the road surface image to obtain initial road surface features; the deep feature extraction network uses multiple cascaded general inverted bottleneck blocks to extract features from the initial road surface features to obtain deep road surface features. Within the general inverted bottleneck blocks, channel-wise depth convolution is used to process the input features to obtain an intermediate feature map. Spatial attention is then calculated on the intermediate feature map to obtain the output features of the general inverted bottleneck block. The output feature of the last general inverted bottleneck block is the deep road surface feature. Specifically, the channel-wise depth convolution divides the corresponding input features into a first sub-feature and a second sub-feature in the channel dimension using a preset partitioning factor. A depthwise separable convolution is performed on the first sub-feature to obtain a first deep sub-feature. A gating network is used to obtain the first modulation coefficient corresponding to the first deep sub-feature and the second modulation coefficient corresponding to the second sub-feature. The modulation result of the first modulation coefficient on the first deep sub-feature and the modulation result of the second modulation coefficient on the second sub-feature are concatenated to obtain an intermediate feature map. The deep road surface features are then input into a classifier to obtain the road surface category recognition result.

[0010] The parameters of the road surface type recognition architecture are updated based on the road surface category recognition results and corresponding annotations. The training ends when the iteration stopping condition is met, and the optimal parameters of the road surface type recognition architecture are saved to obtain the road surface type recognition model.

[0011] Secondly, embodiments of this application provide an application method for a road surface category recognition model based on channel-wise depth convolution, including:

[0012] Obtain the road surface image to be identified, and input the road surface image into the road surface category recognition model to obtain the road surface category recognition result.

[0013] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for constructing a road surface category recognition model based on channel-wise depth convolution.

[0014] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements a method for constructing a road surface category recognition model based on channel-wise depth convolution.

[0015] The main contributions and innovations of this invention are as follows:

[0016] This solution builds a road type recognition architecture based on MobileNetV4. Leveraging MobileNetV4's lightweight characteristics, it significantly reduces the model's computational load and parameter count, making it suitable for computationally resource-sensitive scenarios in vehicle-mounted embedded devices. In the intermediate depthwise convolutional units, this solution selects either channel-specific depthwise convolution or depthwise separable convolution based on the stride. With a stride of 1, channel-specific depthwise convolution is used to enhance feature representation capabilities, improving feature robustness while maintaining lightweight design. With a stride greater than 1, depthwise separable convolution is used to achieve rapid downsampling, ensuring downsampling efficiency and balancing feature enhancement with lightweight design requirements. In this solution, channel-specific depthwise convolution divides the input features into two sub-features along the channel dimension. One of these sub-features undergoes depthwise separable convolution, and a gating network is used to generate modulation coefficients to modulate the feature. Through channel partitioning and differential processing, invalid computations are reduced. Simultaneously, the gating network generates modulation coefficients based on the global context, achieving adaptive modulation of the feature channels, increasing the contribution of effective features, and enhancing the specificity of feature representation.

[0017] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is an example diagram of a road surface category recognition model based on channel-wise depth convolution according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of a channel-wise depthwise convolution according to this application;

[0021] Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0023] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0024] Example 1

[0025] This application provides a method for constructing a road surface category recognition model based on channel-specific depthwise convolution. Channel-specific depthwise convolution is used to partition and differentiate input features, reducing unnecessary computation. Simultaneously, a lightweight gating network generates modulation coefficients based on global context, achieving adaptive modulation of feature channels. This enhances the contribution of effective features and improves the relevance of feature representation. Specifically, refer to... Figure 1 The method includes:

[0026] Obtain multiple road surface images labeled with road surface type as a training sample set;

[0027] A road surface type recognition architecture is constructed based on MobileNetV4, and the road surface type recognition architecture is trained with a training sample set. The road surface type recognition architecture includes an initial feature extraction network, a deep feature extraction network, and a classifier.

[0028] In the training process of the road surface type training architecture, the initial feature extraction network uses convolutional decomposition to extract features from the road surface image to obtain initial road surface features; the deep feature extraction network uses multiple cascaded general inverted bottleneck blocks to extract features from the initial road surface features to obtain deep road surface features. Within the general inverted bottleneck blocks, channel-wise depth convolution is used to process the input features to obtain an intermediate feature map. Spatial attention is then calculated on the intermediate feature map to obtain the output features of the general inverted bottleneck block. The output feature of the last general inverted bottleneck block is the deep road surface feature. Specifically, the channel-wise depth convolution divides the corresponding input features into a first sub-feature and a second sub-feature in the channel dimension using a preset partitioning factor. A depthwise separable convolution is performed on the first sub-feature to obtain a first deep sub-feature. A gating network is used to obtain the first modulation coefficient corresponding to the first deep sub-feature and the second modulation coefficient corresponding to the second sub-feature. The modulation result of the first modulation coefficient on the first deep sub-feature and the modulation result of the second modulation coefficient on the second sub-feature are concatenated to obtain an intermediate feature map. The deep road surface features are then input into a classifier to obtain the road surface category recognition result.

[0029] The parameters of the road surface type recognition architecture are updated based on the road surface category recognition results and corresponding annotations. The training ends when the iteration stopping condition is met, and the optimal parameters of the road surface type recognition architecture are saved to obtain the road surface type recognition model.

[0030] In the current embodiment, multiple road surface images are acquired using a camera device and a public dataset, and the road surface type of each road surface image is labeled.

[0031] Specifically, when acquiring multiple road surface images, it is necessary to ensure that the road surface images have different shooting angles, lighting conditions and weather conditions to ensure the diversity of the training sample set. When performing road surface type recognition, this scheme mainly identifies four types of road surfaces: asphalt road, cement road, gravel road and dirt road.

[0032] Specifically, the road surface images in the training sample set are three-channel RGB images.

[0033] In the current embodiment, before using the training sample set for the road surface type recognition architecture, each road surface image is preprocessed. The preprocessing process includes randomly rotating the image, cropping each road surface image to a size of 360×360, and horizontally and vertically flipping the road surface image.

[0034] In the current embodiment, the initial feature extraction network uses multiple convolutional layers with different kernel sizes and strides of 2, concatenated into one layer, to downsample the road surface image and extract initial road surface features.

[0035] Specifically, before inputting the road surface image into the initial feature extraction network, the resolution of the road surface image is adjusted to 224×224. Through multiple cascaded convolutions, the spatial size of the road surface image is gradually reduced to 56×56, while the channel data is increased to 48.

[0036] In this scheme, the deep feature extraction network includes three stages: a first-stage extraction, a second-stage extraction, and a third-stage extraction. In the first-stage extraction, the initial road surface features are downsampled to 28×28 using a few general inverted bottleneck blocks, and the number of channels is expanded to 80. In the second-stage extraction, the feature map is downsampled to 14×14 using a general inverted bottleneck block with a stride of 2, and the number of channels is further increased to 160. Subsequently, at this resolution and number of channels, multiple general inverted bottleneck blocks are stacked to fully extract multi-level contextual information and fine-grained features. In the third-stage extraction, the spatial size is further compressed to a final 7×7 pixels using a general inverted bottleneck block with a stride of 2, and the channel data is expanded to 224. At this resolution, multiple general inverted bottleneck blocks are stacked again to achieve final enhancement of feature extraction.

[0037] In the current embodiment, the general inverted bottleneck block includes a spatial feature extraction unit, an intermediate depth convolution unit, a spatial attention calculation unit, and a residual output unit. The spatial feature extraction unit extracts spatial features from the input features and then inputs them into the intermediate depth convolution unit. The intermediate depth convolution unit performs channel-wise depth convolution on the input features to obtain an intermediate feature map. The spatial attention calculation unit performs spatial attention calculation on the intermediate feature map to obtain a spatial attention result. The residual output unit concatenates the spatial attention result with the input feature residual of the general inverted bottleneck block and then outputs the result.

[0038] Specifically, the spatial feature extraction unit consists of a deep convolutional layer and a dilation layer. The deep convolutional layer extracts spatial features from the initial road surface features and then performs batch normalization before outputting the results to the dilation layer. The dilation layer dilates the number of channels to the desired number of channels through 1×1 convolution. After the dilation layer finishes processing, it is processed by batch normalization and Hardswish activation function before being output to the intermediate deep convolutional unit.

[0039] Furthermore, the stride of each intermediate depth convolutional unit is preset. If the stride is equal to 1, the input features are subjected to channel-separated depth convolution to obtain the intermediate feature map; if the stride is greater than 1, the input features are subjected to depth-separable convolution to obtain the intermediate feature map.

[0040] Specifically, when there are many stacked general inverted bottleneck blocks, not every general inverted bottleneck block needs to be downsampled. Therefore, this scheme uses stride as the judgment condition. When stride is equal to 1, it means that the current general inverted bottleneck block does not need to be downsampled. So, feature enhancement is performed by channel-wise depthwise convolution, thereby improving feature representation ability under the premise of lightweight. When stride is greater than 1, it means that the current general inverted bottleneck block needs to be downsampled. At this time, ordinary degree-separable convolution is used to achieve fast compression of feature map to ensure efficient downsampling effect.

[0041] Specifically, this solution uses stride as a criterion to determine whether to use channel-separate depthwise convolution or depthwise separable convolution. In scenarios where downsampling is required, downsampling is performed using simple depthwise separable convolution, thereby achieving an extremely lightweight road surface category recognition model.

[0042] In the current embodiment, the input feature of the first general inverted bottleneck block in the deep feature extraction network is the initial road surface feature, the input feature of the remaining inverted bottleneck blocks is the output feature of the previous adjacent general inverted bottleneck block, and the output feature of the last general inverted bottleneck block is the deep road surface feature.

[0043] In the current embodiment, the flowchart of channel-wise depthwise convolution is as follows: Figure 2 As shown, the formula for dividing the corresponding input features along the channel dimension using a preset dividing factor is expressed as:

[0044]

[0045] in, C is the dividing factor, and C is the number of channels corresponding to the input feature. The number of channels for the first sub-feature. The number of channels for the second sub-feature.

[0046] The input features are then further split using the number of channels in the first sub-feature and the number of channels in the second sub-feature to obtain the first sub-feature and the second sub-feature. The first sub-feature is represented as follows: The second sub-feature is represented as .

[0047] In this scheme, the formula for performing depthwise separable convolution on the first sub-feature is expressed as:

[0048]

[0049] in, , For pixel coordinates, For the first depth sub-feature, Here, K represents the weights of the depthwise separable convolution kernel, K is the kernel size, and P is the padding. , and is the convolution kernel dimension parameter.

[0050] In the current embodiment, in the step of using a gating network to obtain the first modulation coefficient corresponding to the first deep sub-feature and the second modulation coefficient corresponding to the second sub-feature, the first sub-feature, the second sub-feature, and the first deep sub-feature are respectively subjected to global average pooling and then concatenated to obtain a joint context feature vector. The gating network is then used to generate the first modulation coefficient and the second modulation coefficient based on the joint context feature vector.

[0051] Specifically, the formulas for global average pooling of the first sub-feature, the second sub-feature, and the first deep sub-feature are expressed as follows:

[0052]

[0053] in, As the first sub-feature, For the first depth sub-feature, As the second sub-feature, GAP represents global average pooling. This is the global average pooling result corresponding to the first sub-feature. This corresponds to the global average pooling result of the first depth sub-feature. This is the second sub-feature.

[0054] Then , , By concatenating the features, we obtain the joint context feature vector, expressed by the formula:

[0055]

[0056] in, For joint context feature vectors, This indicates a splicing operation.

[0057] In this scheme, the formula for generating the first modulation coefficient and the second modulation coefficient based on the joint context feature vector using a gated network is expressed as follows:

[0058]

[0059] in, As the gating factor, To modify the activation function of the linear unit, For joint context feature vectors, For batch normalization layer, , All are 1×1 convolutional layers. It is the Sigmoid activation function. The first modulation coefficient, This is the second modulation coefficient.

[0060] Specifically, the gating network used in this scheme is a lightweight gating network. This scheme uses global average pooling to adaptively modulate each feature, so that all channels can dynamically adjust their contributions according to the global context.

[0061] In this scheme, the first modulation coefficient and the second modulation coefficient are multiplied element-wise with the corresponding first depth sub-feature and the second sub-feature, respectively, to obtain the modulation result of the first modulation coefficient on the first depth sub-feature and the modulation result of the second modulation coefficient on the second sub-feature. The formula is expressed as follows:

[0062]

[0063] in, The modulation result of the first modulation coefficient on the first depth sub-feature. For the first depth sub-feature, This represents element-wise multiplication. The first modulation coefficient, This represents the modulation result of the second modulation coefficient on the second sub-feature. This second sub-feature, This is the second modulation coefficient.

[0064] Finally, the modulation results of the first modulation coefficient on the first depth sub-feature and the modulation results of the second modulation coefficient on the second sub-feature are concatenated along the channel dimension to obtain the intermediate feature map, as expressed by the formula:

[0065]

[0066] in, This represents the intermediate feature map. This indicates a splicing operation. The modulation result of the first modulation coefficient on the first depth sub-feature. This represents the modulation result of the second modulation coefficient on the second sub-feature.

[0067] In the current embodiment, in the spatial attention calculation step of the spatial attention calculation unit on the intermediate feature map, the height attention weight in the height dimension and the width attention weight in the width dimension of the intermediate feature map are obtained, and the intermediate feature map, the height attention weight and the width attention weight are multiplied element by element to obtain the spatial attention output feature, which is the output of the spatial attention calculation unit.

[0068] Furthermore, average pooling, max pooling, and pixel variance are calculated on the height dimension of the intermediate feature map, and the results of average pooling, max pooling, and pixel variance calculation on the height dimension are added together to obtain the height attention weight; average pooling, max pooling, and pixel variance are calculated on the width dimension of the intermediate feature map, and the results of average pooling, max pooling, and pixel variance calculation on the width dimension are added together to obtain the width attention weight.

[0069] Specifically, the formulas for average pooling, max pooling, and pixel variance calculation in the height and width dimensions of the intermediate feature map are expressed as follows:

[0070]

[0071] Where W is the width dimension and H is the height dimension. This is an intermediate feature map. For average pooling, For max pooling, Calculation of pixel variance This is the average pooling result along the width dimension. This is the result of max pooling in the width dimension. The result is the pixel variance calculated in the width dimension. This is the average pooling result along the height dimension. This is the result of max pooling in the height dimension. This is the result of calculating the pixel variance in the height dimension.

[0072] Specifically, the formula for obtaining the high attention weight is expressed as:

[0073]

[0074] in, High attention weights are used to generate attention scores between 0 and 1. For activation function, This is the average pooling result along the height dimension. This is the result of max pooling in the height dimension. This is the result of calculating the pixel variance in the height dimension.

[0075] Specifically, the formula for obtaining the width attention weight is expressed as:

[0076]

[0077] in, The activation function is used to generate attention scores between 0 and 1. This is the average pooling result along the width dimension. This is the result of max pooling in the width dimension. This is the result of calculating the pixel variance in the width dimension.

[0078] Specifically, the formula for obtaining the output of the spatial attention computation unit is expressed as:

[0079]

[0080] in, The output of the spatial attention computation unit. This is an intermediate feature map. For high attention weights, For width attention weights, This indicates element-wise multiplication.

[0081] Specifically, the spatial attention calculation in this scheme integrates the mean and maximum values ​​of the intermediate feature map in the height and width dimensions, and introduces pixel variance information in the spatial dimension to generate spatial attention weights that are more sensitive to road texture and environmental changes. Then, it multiplies the weights element by element with the feature map to strengthen the features of key areas, thereby making spatial attention allocation more accurate.

[0082] In the current embodiment, the spatial attention output features are input into the residual output unit in the general inverted bottleneck block. The residual unit first performs a 1×1 convolution on the spatial attention output features using a projection layer to obtain the projection result. Then, the projection result is residually connected with the input features of the corresponding general inverted bottleneck block to serve as the output feature of the general inverted bottleneck block. The spatial attention output features are the corresponding spatial attention calculation results.

[0083] Specifically, after obtaining the spatial attention output features, the number of channels of the spatial attention output features is adjusted to be the same as the input features of the corresponding general inverted bottleneck block through the projection layer, and then output in the form of residual connection.

[0084] In the current embodiment, the classifier can use lightweight structures such as fully connected layers and lightweight convolutional classification layers for classification. The road surface types output by the classifier of this scheme include four types: asphalt road, cement road, gravel road and dirt road.

[0085] In the current embodiment, the optimizer used during training is the SGD optimizer, the learning rate is the cosine annealing learning rate, the image batch size is set to 32, the number of iterations is set to 500, and the loss function is the adaptive regularized loss function. This combines the traditional cross-entropy loss with a dynamic regularization term to improve the model's generalization ability, especially for lightweight models, effectively preventing overfitting and overconfident predictions.

[0086] Furthermore, the weighted sum of the cross-entropy loss and the regularization term is used as the total loss function of this scheme. The formula for calculating the cross-entropy loss is as follows:

[0087]

[0088] in, It is the predicted probability of sample b corresponding to class j. It is the original predicted score of category j corresponding to sample b. It is the sum of the indexed original predicted scores for all categories, to ensure that all The sum is 1. It represents the total number of categories. It is a sample Real category The corresponding predicted probability.

[0089] After obtaining the probability distribution P, an additional regularization term R is calculated to guide the model's predictive behavior. This invention provides two selectable regularization types:

[0090] Entropy regularization: It encourages the model to output a smoother probability distribution by increasing the entropy value of the predicted probability distribution, thereby avoiding overly confident predictions. Its entropy regularization term... The calculation is as follows:

[0091]

[0092] In the formula: It is the predicted probability of sample b corresponding to class j. It represents the total number of categories. It is a preset, extremely small positive value to prevent the value from being unstable in logarithmic operations.

[0093] Confidence penalty: This directly suppresses overconfident predictions by penalizing the sum of squares of high-confidence predictions. Its confidence penalty term... The calculation is as follows:

[0094]

[0095] in, It is the predicted probability of sample b corresponding to class j. This represents the total number of categories.

[0096] Finally, the cross-entropy loss of each sample calculated above is used... With the selected regularization term Perform a weighted summation. We obtain the final combination loss for each sample by assigning weights to the selected regularization terms. The final loss value is obtained by aggregating the losses of all samples in the batch. Used for backpropagation:

[0097]

[0098] In the formula: It is the weight coefficient of the regularization term.

[0099] Example 2

[0100] Based on the same concept, this application also proposes an application method for a road surface category recognition model based on channel-wise depth convolution, including:

[0101] Obtain the road surface image to be identified, and output the road surface image to the road surface category recognition model to obtain the road surface category recognition result.

[0102] Specifically, this solution involves installing a camera at a suitable location on the vehicle and adjusting its position and angle to ensure accurate capture of the road surface image to be identified.

[0103] Example 3

[0104] This embodiment also provides an electronic device, see reference. Figure 3It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0105] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0106] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0107] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0108] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the road surface category recognition model construction methods based on channel-wise depth convolution in the above embodiments.

[0109] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0110] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0111] The input / output device 408 is used to input or output information. In this embodiment, the input information may be a road surface image, and the output information may be the road surface category recognition result of the road surface image, etc.

[0112] Optionally, in this embodiment, the processor 402 can be configured to perform the following steps via a computer program:

[0113] Obtain multiple road surface images labeled with road surface type as a training sample set;

[0114] A road surface type recognition architecture is constructed based on MobileNetV4, and the road surface type recognition architecture is trained with a training sample set. The road surface type recognition architecture includes an initial feature extraction network, a deep feature extraction network, and a classifier.

[0115] In the training process of the road surface type training architecture, the initial feature extraction network uses convolutional decomposition to extract features from the road surface image to obtain initial road surface features; the deep feature extraction network uses multiple cascaded general inverted bottleneck blocks to extract features from the initial road surface features to obtain deep road surface features. Within the general inverted bottleneck blocks, channel-wise depth convolution is used to process the input features to obtain an intermediate feature map. Spatial attention is then calculated on the intermediate feature map to obtain the output features of the general inverted bottleneck block. The output feature of the last general inverted bottleneck block is the deep road surface feature. Specifically, the channel-wise depth convolution divides the corresponding input features into a first sub-feature and a second sub-feature in the channel dimension using a preset partitioning factor. A depthwise separable convolution is performed on the first sub-feature to obtain a first deep sub-feature. A gating network is used to obtain the first modulation coefficient corresponding to the first deep sub-feature and the second modulation coefficient corresponding to the second sub-feature. The modulation result of the first modulation coefficient on the first deep sub-feature and the modulation result of the second modulation coefficient on the second sub-feature are concatenated to obtain an intermediate feature map. The deep road surface features are then input into a classifier to obtain the road surface category recognition result.

[0116] The parameters of the road surface type recognition architecture are updated based on the road surface category recognition results and corresponding annotations. The training ends when the iteration stopping condition is met, and the optimal parameters of the road surface type recognition architecture are saved to obtain the road surface type recognition model.

[0117] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0118] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0119] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 3 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0120] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for constructing a road surface category recognition model based on channel-wise depthwise convolution, characterized in that, Includes the following steps: Obtain multiple road surface images labeled with road surface type as a training sample set; A road surface type recognition architecture is constructed based on MobileNetV4, and the road surface type recognition architecture is trained with a training sample set. The road surface type recognition architecture includes an initial feature extraction network, a deep feature extraction network, and a classifier. During the training process of the road surface type training architecture, the initial feature extraction network uses convolutional decomposition to extract features from the road surface image to obtain initial road surface features. The deep feature extraction network uses multiple cascaded general inverted bottleneck blocks to extract features from the initial road surface features to obtain deep road surface features. The general inverted bottleneck block includes intermediate depth convolutional units. The stride of each intermediate depth convolutional unit is preset. If the stride is equal to 1, the input features are subjected to channel-specific depth convolution to obtain an intermediate feature map. If the stride is greater than 1, the input features are subjected to depthwise separable convolution to obtain an intermediate feature map. Spatial attention is calculated on the intermediate feature map to obtain the output of the general inverted bottleneck block. The output feature of the last general inverted bottleneck block is the depth road surface feature. The channel-separated depth convolution divides the corresponding input feature into a first sub-feature and a second sub-feature in the channel dimension using a preset partitioning factor. A depthwise separable convolution is performed on the first sub-feature to obtain a first depth sub-feature. A gating network is used to obtain the first modulation coefficient corresponding to the first depth sub-feature and the second modulation coefficient corresponding to the second sub-feature. The modulation result of the first modulation coefficient on the first depth sub-feature and the modulation result of the second modulation coefficient on the second sub-feature are concatenated to obtain an intermediate feature map. The depth road surface feature is then input into a classifier to obtain the road surface category recognition result. The parameters of the road surface type recognition architecture are updated based on the road surface category recognition results and corresponding annotations. The training ends when the iteration stopping condition is met, and the optimal parameters of the road surface type recognition architecture are saved to obtain the road surface type recognition model.

2. The method for constructing a road surface category recognition model based on channel-wise depth convolution according to claim 1, characterized in that, The general inverted bottleneck block includes a spatial feature extraction unit, an intermediate depth convolution unit, a spatial attention calculation unit, and a residual output unit. The spatial feature extraction unit extracts spatial features from the input features and then inputs them into the intermediate depth convolution unit. The intermediate depth convolution unit performs channel-wise depth convolution on the input features to obtain an intermediate feature map. The spatial attention calculation unit performs spatial attention calculation on the intermediate feature map to obtain a spatial attention result. The residual output unit concatenates the spatial attention result with the input feature residual of the general inverted bottleneck block and then outputs the result.

3. The method for constructing a road surface category recognition model based on channel-wise depth convolution according to claim 1, characterized in that, The first sub-feature, the second sub-feature, and the first deep sub-feature are respectively subjected to global average pooling and then concatenated to obtain a joint context feature vector. A gating network is then used to generate the first modulation coefficient and the second modulation coefficient based on the joint context feature vector.

4. The method for constructing a road surface category recognition model based on channel-wise depth convolution according to claim 1, characterized in that, In spatial attention computation, the height attention weight in the height dimension and the width attention weight in the width dimension of the intermediate feature map are obtained. The intermediate feature map, the height attention weight, and the width attention weight are multiplied element by element and then output.

5. The method for constructing a road surface category recognition model based on channel-wise depth convolution according to claim 4, characterized in that, Average pooling, max pooling, and pixel variance are calculated on the height dimension of the intermediate feature map. The average pooling result, max pooling result, and pixel variance calculation result on the height dimension are added together to obtain the height attention weight. Average pooling, max pooling, and pixel variance calculation are performed on the width dimension of the intermediate feature map. The average pooling result, max pooling result, and pixel variance calculation result on the width dimension are added together to obtain the width attention weight.

6. The method for constructing a road surface category recognition model based on channel-wise depth convolution according to claim 1, characterized in that, The spatial attention output features are input into the residual output unit in the general inverted bottleneck block. The residual unit first performs a 1×1 convolution on the spatial attention output features using a projection layer to obtain the projection result. Then, the projection result is residually connected with the input features of the corresponding general inverted bottleneck block to serve as the output feature of the general inverted bottleneck block. The spatial attention output features are the corresponding spatial attention calculation results.

7. An application method for a road surface category recognition model based on channel-wise depthwise convolution, characterized in that, include: Obtain a road surface image to be identified, and input the road surface image to be identified into the road surface category recognition model constructed by any one of the methods described in claims 1-6 to obtain the road surface category recognition result.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute a method for constructing a road surface category recognition model based on multichannel depth convolution as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements a method for constructing a road surface category recognition model based on multi-channel depth convolution as described in any one of claims 1-6.

Images