Double-branch lightweight drivable area detection method based on low-level features

Abstract

The invention discloses a double-branch lightweight drivable area detection method based on low-level features, and the method comprises the following steps: obtaining a data set image, inputting the data set image into a high-resolution branch module for processing, obtaining a high-resolution branch feature, carrying out the processing of the data set image through a low-resolution branch module, obtaining a low-resolution branch feature, and carrying out the detection of the low-resolution branch feature. The high-resolution branch features are processed through the detail module to obtain a detail feature map, the low-resolution branch features are processed through the segmentation module to obtain a segmentation feature map, and the lightweight guide fusion module is optimized through the detail feature map and the segmentation feature map to obtain a lightweight guide fusion model; and inputting the high-resolution branch features and the low-resolution branch features into the optimized lightweight guide fusion module for processing to obtain drivable area detection feature information and outputting the drivable area detection feature information. The light-weight guide fusion module is used for fusing the high-resolution branch features and the low-resolution branch features, the fusion effect between the features is optimized, and the detection performance is improved.

Description

Technical Field

[0001] The invention relates to the technical field of road drivable area detection, and is a dual-branch lightweight drivable area detection method based on low-level features. Background Art

[0002] Road drivable area detection is a key link in intelligent driving technology. The main goal is to automatically identify and accurately segment areas suitable for safe vehicle driving from complex road environments. It provides crucial environmental perception data for path planning and dynamic obstacle avoidance of autonomous driving systems, ensuring safe, stable and efficient driving. Clear division of drivable areas helps vehicles adjust their driving paths in real time to avoid collisions with pedestrians, other vehicles and obstacles, which is especially important in complex traffic and dynamic environments. At present, improving the recognition accuracy and reasoning speed of the network is still the core issue in the research of this task. How to maintain high real-time performance and fast reasoning of the detection model while ensuring high accuracy is still a very challenging task.

[0003] In recent years, deep learning-based drivable area detection technology has made significant progress, especially in image understanding and road feature recognition. Many deep learning networks, especially convolutional neural networks (CNNs) and Transformer architectures, have been widely used in this task and have achieved good detection results. However, most current deep learning-based drivable area detection methods, especially those that pursue high accuracy, still face some severe challenges. Due to the large size and slow loading speed of existing model files, it has become a huge obstacle to the deployment of embedded devices and edge devices. Especially on these resource-constrained devices, real-time reasoning has become more difficult. Even on high-performance GPU platforms, the ability to process sensor data in real time is still limited. Secondly, as the scale of deep neural networks increases, the reasoning time and computational complexity of the model also increase exponentially. Especially in autonomous driving applications, real-time performance is crucial. The high computational complexity and complex reasoning process lead to large reasoning delays, which pose a serious threat to real-time decision-making and safety. Many tasks in autonomous driving require responses to sensor data within milliseconds. Any delay may bring safety hazards and affect the vehicle's reaction speed and decision-making efficiency.

[0004] In summary, for the drivable area detection task, this paper starts from the perspective of lightweight and solves the problems of slow computation, accuracy and inference speed when the model is deployed in an embedded system, so as to achieve a balance between accuracy and timeliness. Current deep learning methods and training processes for the drivable area detection task vary, but most methods ignore a key feature of the road, which is that the road exists as a background element rather than a typical "thing". In an image, the road is not as easy to identify as an object with a clear semantic category, it is more like a "thing" (background), that is, part of the environment. Based on the above findings, the primary stage of the current mainstream network model is sufficient to represent most of the pixels of the road for detection, so our network adopts a detection model dominated by low-level features. Specifically, our network adopts a dual-branch structure, namely a high-resolution branch and a low-resolution branch. The high-resolution branch extracts low-level feature representation information of the drivable area in the image, and the low-resolution branch aims to quickly capture the contextual information in the image. The common way to merge two features includes element-level summation and feature concatenation. However, this simple combination ignores the diversity of the two types of feature information and it is difficult to take into account the differences in context in low-level features and high-level features at the same time. Therefore, a lightweight fusion guidance layer is used to balance the spatial details of low-level features with the semantic contextual information of high-level features, and to suppress irrelevant changes through precise feature guidance. Then, the feature information extracted by the two branches is input into the detail module and the segmentation module respectively, and the boundary information is optimized through the detail and segmentation loss functions to improve the detection network's ability to capture small targets and details and the accuracy of pixel-level classification of graphics. These two steps only work during training, so they do not increase the network's inference cost. Finally, the feature information output by the lightweight guidance fusion layer is input into the segmentation head again, and the segmentation head performs stage loss feedback to ensure that the low-resolution branch can learn richer semantic information, and the high-resolution branch can improve the model's perception of boundary information, and finally obtain the output of the drivable area detection image. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a dual-branch lightweight drivable area detection method based on low-level features to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a dual-branch lightweight drivable area detection method based on low-level features, comprising the following steps: Step S1: constructing dataset images; Step S2: construct a dual-branch lightweight model, which consists of a high-resolution branch module, a low-resolution branch module, a detail module, a segmentation module, and a lightweight guided fusion module; the high-resolution branch module consists of a high-resolution feature extraction network and an adaptive enhancement module; the low-resolution branch module consists of a low-resolution feature extraction network and a cross convolution module; the lightweight guided fusion module includes a guided recovery module; Step S3: inputting the dataset image into a high-resolution feature extraction network for processing to obtain feature information, and inputting the feature information into an adaptive enhancement module for processing to obtain high-resolution branch features; Step S4: inputting the dataset image into the low-resolution feature extraction network for processing to obtain the second-layer extracted information, and inputting the second-layer extracted information into the cross convolution module for processing to obtain low-resolution branch features; Step S5: input the high-resolution branch features into the detail module for processing to obtain a detail feature map; Step S6: input the low-resolution branch features into the segmentation module for processing to obtain a segmentation feature map; Step S7: Optimize the lightweight guided fusion module through the detail feature map and the segmentation feature map, input the high-resolution branch features and the low-resolution branch features into the optimized lightweight guided fusion module for processing, obtain the output feature information of the lightweight guided fusion layer, input the high-resolution branch features, the low-resolution branch features and the output feature information of the lightweight guided fusion layer into the guided recovery module for processing, and obtain the output of the drivable area detection feature information.

[0007] Further, the data set image in step S1, the specific process is: A road segmentation data set is obtained, and the road segmentation data set includes training images and test images; the training images and test images are processed to obtain data set images.

[0008] Furthermore, the high-resolution feature extraction network is a ResNet18 module, which includes an L1 layer; the L1 layer is composed of a high-resolution feature residual block, which includes two high-resolution feature residual units, and the residual unit includes two 3×3 convolutional layers; The dataset image is input into the high-resolution feature extraction network for processing to obtain feature information F1, which is expressed as: ; In the formula, Represents the dataset image; represents batch normalization; represents a 3×3 convolutional layer; Represents the activation function.

[0009] Further, in step S3, high-resolution branch features are obtained, and the specific process is as follows: The feature information F1 is input into the adaptive enhancement module for processing to obtain high-resolution branch features; The adaptive enhancement module consists of a structure of four dilated convolution branches; the first dilated convolution branch consists of a 1×1 standard convolution; the second dilated convolution branch consists of a 1×1 standard convolution, a 3×1 standard convolution and a 3×3 dilated convolution; the third dilated convolution branch consists of a 1×1 standard convolution, a 1×3 standard convolution and a 3×3 dilated convolution; the fourth dilated convolution branch consists of a 1×1 standard convolution and a 3×3 standard convolution; The feature information F1 only undergoes a 1×1 standard convolution operation in the first dilated convolution branch to obtain the output of the first dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the second dilated convolution branch, a 3×1 standard convolution and a 3×3 dilated convolution operation are performed to obtain the output of the second dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the third dilated convolution branch, a 1×3 standard convolution and a 3×3 dilated convolution operation are performed in sequence to obtain the output of the third dilated convolution branch; the feature information F1 undergoes a 1×1 standard convolution in the fourth dilated convolution branch, and then a 3×3 standard convolution operation is performed to obtain the output of the fourth dilated convolution branch; the output of the second dilated convolution branch and the output of the third dilated convolution branch are added to obtain the added output, and the added output is cascaded with the output of the first dilated convolution branch and the output of the fourth dilated convolution branch to obtain a high-resolution branch feature. .

[0010] Furthermore, the low-resolution feature extraction network is a ResNet18 module, which includes an L1 layer and an L2 layer; the ResNet18 module is composed of a low-resolution feature residual block, which includes two low-resolution feature residual units, and the residual unit includes two 3×3 convolutional layers; The dataset image is input into the low-resolution feature extraction network for processing to obtain the second-layer extracted information F2, which is expressed as: ; ; In the formula, Represents the output of the first low-resolution feature residual unit in the low-resolution feature extraction network.

[0011] Furthermore, the low-resolution branch features in step S4 are specifically: The extracted information F2 of the second layer is input into the cross convolution module for processing to obtain low-resolution branch features; The cross convolution module consists of a 1×5 depthwise row convolution, a 5×1 depthwise column convolution, and a 1×1 convolution layer, a batch normalization, and an activation function; The extracted information F2 of the second layer is first subjected to a 1×5 depth row convolution operation in the cross convolution module to capture the context information of the extracted information F2 of the second layer in the horizontal direction, and then a 5×1 depth column convolution operation is performed to extract the context information of the extracted information F2 of the second layer in the vertical direction. A 1×1 convolution layer is used to fuse the context information of the extracted information F2 of the second layer in the horizontal direction and the context information of the extracted information F2 of the second layer in the vertical direction, and a nonlinear transformation is completed through batch normalization and activation function to obtain the output of the nonlinear transformation. The extracted information F2 of the second layer and the output of the nonlinear transformation are added to obtain the low-resolution branch features. , expressed as: ; In the formula, represents a 1×1 convolutional layer; Represents 1×5 depthwise row convolution; Represents a 5×1 depthwise column convolution.

[0012] Further, in step S5, the detailed feature map is specifically processed as follows: High-resolution branch features The input is processed in the detail module, which includes a detail head; the detail head consists of a 3×3 convolution layer and a 1×1 convolution layer; high-resolution branch features In the detail head, it first passes through a 3×3 convolution layer and a 1×1 convolution layer, and then operates through batch normalization and activation functions to generate a detail feature map. , expressed as: ; In the formula, Represents an upsampling operation.

[0013] Further, in step S6, the feature map is segmented, and the specific process is as follows: The low-resolution branch features In the input segmentation module, the segmentation module includes a segmentation head, which consists of a 3×3 convolutional layer and a 1×1 convolutional layer. The low-resolution branch feature In the segmentation head, the 3×3 convolution layer and the 1×1 convolution layer are first passed, and then the segmentation feature map is generated through batch normalization and activation function operations. , expressed as: .

[0014] Furthermore, the lightweight guided fusion module consists of a high-resolution branch, a low-resolution branch, and a 3×3 convolutional layer; The high-resolution branch consists of the first branch and the second branch; the low-resolution branch consists of the third branch and the fourth branch; the first branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; the second branch consists of a 3×3 convolution layer and a downsampling; the third branch consists of a 3×3 convolution layer and an upsampling; the fourth branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; Based on detail feature map and segmentation feature map Optimizing the lightweight guidance fusion module to obtain an optimized lightweight guidance fusion module; High-resolution branch features and low-resolution branch features Input into the optimized lightweight guidance fusion module; High-resolution branch features The 3×3 depth-wise separable convolution of the first branch and the 3×3 convolution layer of the second branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 depth-wise separable convolution of the first branch and the output of the 3×3 convolution layer of the second branch respectively; Low-resolution branch features The 3×3 convolution layer of the third branch and the 3×3 depth-separable convolution of the fourth branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 convolution layer of the third branch and the output of the 3×3 depth-separable convolution of the fourth branch respectively; Input the output of the 3×3 depth-separable convolution of the first branch to the 1×1 convolution layer of the first branch; input the output of the 3×3 depth-separable convolution of the fourth branch to the 1×1 convolution layer of the fourth branch, and obtain the output of the 1×1 convolution layer of the first branch and the output of the 1×1 convolution layer of the fourth branch respectively; Downsample the output of the second branch 3×3 convolutional layer to obtain a downsampled feature map, and upsample the output of the third branch 3×3 convolutional layer to obtain an upsampled feature map; The downsampled feature map is multiplied by the output of the fourth branch 1×1 convolution layer to obtain the low-resolution branch weight feature. The upsampled feature map is multiplied by the output of the first branch 1×1 convolution layer to obtain the high-resolution branch weight feature. The high-resolution branch weight feature and the low-resolution branch weight feature are added element by element to obtain the fusion feature. The fusion feature is then integrated through a 3×3 convolution layer to generate the output feature information of the lightweight guided fusion layer. .

[0015] Further, in step S7, the drivable area detection feature information is output, and the specific process is as follows: The output feature information of the lightweight guide fusion layer , high-resolution branch features and low-resolution branch features The input is processed in the guidance recovery module to obtain the output of the drivable area detection feature information; The guided recovery module consists of an upper branch, a middle branch, and a lower branch; the upper branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; The middle branch consists of a global average pooling layer, three 3×3 convolutional layers, and a 1×1 convolutional layer; The lower branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; High-resolution branch features In the upper branch, the upsampled data is first input to obtain the upsampled high-resolution branch features. , upsample the high-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the high-resolution branch features of the 3×3 convolution layer, and combine the high-resolution branch features of the 3×3 convolution layer with the upsampled high-resolution branch features. Element-by-element addition to obtain high-resolution branch features of the upper branch ; Low-resolution branch features In the lower branch, the upsampled data is first input to obtain the upsampled low-resolution branch features. , upsample the low-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the low-resolution branch features of the 3×3 convolution layer, and combine the low-resolution branch features of the 3×3 convolution layer with the upsampled low-resolution branch features. Element-by-element addition to obtain the low-resolution branch features of the lower branch ; Lightweight guide fusion layer output feature information In the middle branch, the global average pooling layer is first input to obtain the output feature information of the lightweight guided fusion layer of the global average pooling layer. , lightweight the global average pooling layer to guide the output feature information of the fusion layer Input the 3×3 convolutional layer to obtain the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, multiply the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, the 3×3 convolutional layer low-resolution branch features and the 3×3 convolutional layer high-resolution branch features to obtain the output feature information of the multiplied lightweight guided fusion layer, and then pass the multiplied information through a 3×3 convolutional layer to obtain the output feature information of the middle branch lightweight guided fusion layer. ; The high-resolution branch features of the upper branch , low-resolution branch features of the lower branch The output feature information of the light-weight guide fusion layer in the middle branch Perform cascade operation to obtain the spliced ​​feature map , concatenate the feature maps Input the 3×3 convolution layer and the 1×1 convolution layer in sequence to obtain the 1×1 convolution layer splicing feature map, which is the output feature map of the guided recovery module. ; The output feature map of the boot recovery module The input is processed into the segmentation head of the segmentation module to obtain the drivable area detection feature information output F, which is expressed as: .

[0016] Compared with the existing technology, the present invention has the following beneficial effects: (1) From the perspective of the dual-branch lightweight model, the high-resolution branch module of the present invention only uses the first layer of the ResNet18 module to extract high-resolution branch features, while the low-resolution branch module uses the first two layers of the ResNet18 module to extract high-level semantic context information; this design greatly reduces the amount of calculation and improves the inference speed of the dual-branch lightweight model; by introducing the adaptive enhancement module, the expressive power of the high-resolution branch features is enhanced, noise and background interference are effectively suppressed, misidentification is reduced, and the adaptability and robustness of the dual-branch lightweight model to the task are improved.

[0017] (2) The present invention replaces the third layer of the ResNet18 module in the low-resolution branch module with a cross convolution module, thereby significantly reducing the number of parameters and computational overhead, and successfully achieving a receptive field equivalent to that of the third layer of the ResNet18 module, providing a stronger feature extraction capability for the dual-branch lightweight model.

[0018] (3) The present invention uses a lightweight guided fusion module to fuse high-resolution branch features and low-resolution branch features, balance the spatial details of low-resolution branch features with the high-level semantic context information of high-resolution branch features, guide the suppression of irrelevant changes, and optimize the fusion effect between features to improve detection performance.

[0019] (4) The present invention designs a detail module and a segmentation module to generate a detail feature map and a segmentation feature map, which help the lightweight guided fusion module learn richer spatial detail features and enhance the ability to capture high-level semantic context information of high-resolution branch features and low-resolution branch features. The detail module and the segmentation module only work during the training of the dual-branch lightweight model and do not work during the inference stage. Therefore, this dual-branch lightweight model does not increase the computational complexity of the entire model.

[0020] (5) The present invention designs a guided recovery module to guide the recovery of the output feature information of the high-resolution branch features, the low-resolution branch features and the lightweight guided fusion layer, so as to help capture the details of the output of the drivable area detection feature information in the dataset image, especially in the recognition of edges, small targets and complex boundaries, while enhancing the adaptability of the dual-branch lightweight model to different input data. It helps the dual-branch lightweight model cope with environmental changes (such as lighting changes, road conditions, etc.), and improves the adaptability of the dual-branch lightweight model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the present invention.

[0022] Figure 2 It is a schematic diagram of the structure of the adaptive enhancement module of the present invention.

[0023] Figure 3 It is a schematic diagram of the structure of the lightweight guidance fusion module of the present invention.

[0024] Figure 4 It is a schematic diagram of the structure of the guide recovery module of the present invention. DETAILED DESCRIPTION

[0025] like Figure 1 As shown, the present invention provides a technical solution: a dual-branch lightweight drivable area detection method based on low-level features, comprising the following steps: Step S1: constructing dataset images; Step S2: construct a dual-branch lightweight model, which consists of a high-resolution branch module, a low-resolution branch module, a detail module, a segmentation module, and a lightweight guided fusion module; the high-resolution branch module consists of a high-resolution feature extraction network and an adaptive enhancement module; the low-resolution branch module consists of a low-resolution feature extraction network and a cross convolution module; the lightweight guided fusion module includes a guided recovery module; Step S3: inputting the dataset image into a high-resolution feature extraction network for processing to obtain feature information, and inputting the feature information into an adaptive enhancement module for processing to obtain high-resolution branch features; Step S4: inputting the dataset image into the low-resolution feature extraction network for processing to obtain the second-layer extracted information, and inputting the second-layer extracted information into the cross convolution module for processing to obtain low-resolution branch features; Step S5: input the high-resolution branch features into the detail module for processing to obtain a detail feature map; Step S6: input the low-resolution branch features into the segmentation module for processing to obtain a segmentation feature map; Step S7: Optimize the lightweight guided fusion module through the detail feature map and the segmentation feature map, input the high-resolution branch features and the low-resolution branch features into the optimized lightweight guided fusion module for processing, obtain the output feature information of the lightweight guided fusion layer, input the high-resolution branch features, the low-resolution branch features and the output feature information of the lightweight guided fusion layer into the guided recovery module for processing, and obtain the output of the drivable area detection feature information.

[0026] Among them, the data set image in step S1, the specific process is: This paper uses the KITTI-Road dataset, which is a road segmentation dataset. The road segmentation dataset contains 289 training images and 290 test images; The resolution range of the training images and the test images is 370×1224 to 375×1242. The training images and the test images with a resolution range of 370×1224 to 375×1242 are uniformly padded. The training images and the test images are adjusted to 375×1240 through the padding operation to obtain the dataset images after the padding operation.

[0027] Among them, the high-resolution feature extraction network is a ResNet18 module, which includes an L1 layer; the L1 layer contains 64 channels; the L1 layer consists of a high-resolution feature residual block, which contains two high-resolution feature residual units, and the residual unit contains two 3×3 convolutional layers; The dataset image is input into the high-resolution feature extraction network for processing to obtain feature information F1, which is expressed as: ; In the formula, Represents feature information; Represents the dataset image; represents batch normalization; represents a 3×3 convolutional layer; Represents the activation function.

[0028] like Figure 2 As shown, in step S3, high-resolution branch features are obtained, and the specific process is as follows: The feature information F1 is input into the adaptive enhancement module for processing to obtain high-resolution branch features; The adaptive enhancement module consists of a structure of four dilated convolution branches; the first dilated convolution branch consists of a 1×1 standard convolution; the second dilated convolution branch consists of a 1×1 standard convolution, a 3×1 standard convolution and a 3×3 dilated convolution; the third dilated convolution branch consists of a 1×1 standard convolution, a 1×3 standard convolution and a 3×3 dilated convolution; the fourth dilated convolution branch consists of a 1×1 standard convolution and a 3×3 standard convolution; The feature information F1 only undergoes a 1×1 standard convolution operation in the first dilated convolution branch to obtain the output of the first dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the second dilated convolution branch, a 3×1 standard convolution and a 3×3 dilated convolution operation are performed to obtain the output of the second dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the third dilated convolution branch, a 1×3 standard convolution and a 3×3 dilated convolution operation are performed in sequence to obtain the output of the third dilated convolution branch; the feature information F1 undergoes a 1×1 standard convolution in the fourth dilated convolution branch, and then a 3×3 standard convolution operation is performed to obtain the output of the fourth dilated convolution branch; the output of the second dilated convolution branch and the output of the third dilated convolution branch are added to obtain the added output, and the added output is cascaded with the output of the first dilated convolution branch and the output of the fourth dilated convolution branch to obtain a high-resolution branch feature. , expressed as: ; ; ; ; In the formula, represents the output of the first dilated convolution branch; Represents the output after addition; represents the output of the fourth dilated convolution branch; Represents 3×3 standard convolution; Represents 1×1 standard convolution; Represents 3×3 dilated convolution; Represents 3×1 standard convolution; Represents 1×3 standard convolution; represents a cascade operation; ⊕ represents an addition operation.

[0029] Among them, the low-resolution feature extraction network is a ResNet18 module, which includes L1 and L2 layers; the ResNet18 module is composed of a low-resolution feature residual block, which includes two low-resolution feature residual units, and the residual unit includes two 3×3 convolutional layers; The dataset image is input into the low-resolution feature extraction network for processing to obtain the second-layer extracted information F2, which is expressed as: ; ; In the formula, Represents the output of the first low-resolution feature residual unit in the low-resolution feature extraction network.

[0030] The low-resolution branch features in step S4 are specifically: The extracted information F2 of the second layer is input into the cross convolution module for processing to obtain low-resolution branch features; The cross convolution module consists of a 1×5 depthwise row convolution, a 5×1 depthwise column convolution, and a 1×1 convolution layer, a batch normalization, and an activation function; The extracted information F2 of the second layer is first subjected to a 1×5 depth row convolution operation in the cross convolution module to capture the context information of the extracted information F2 of the second layer in the horizontal direction, and then a 5×1 depth column convolution operation is performed to extract the context information of the extracted information F2 of the second layer in the vertical direction. A 1×1 convolution layer is used to fuse the context information of the extracted information F2 of the second layer in the horizontal direction and the context information of the extracted information F2 of the second layer in the vertical direction, and a nonlinear transformation is completed through batch normalization and activation function to obtain the output of the nonlinear transformation. The extracted information F2 of the second layer and the output of the nonlinear transformation are added to obtain the low-resolution branch features. , expressed as: ; In the formula, represents a 1×1 convolutional layer; Represents 1×5 depthwise row convolution; Represents a 5×1 depthwise column convolution.

[0031] Among them, the detailed feature map in step S5, the specific process is: High-resolution branch features The input is processed in the detail module, which includes a detail head; the detail head consists of a 3×3 convolution layer and a 1×1 convolution layer; high-resolution branch features In the detail head, it first passes through a 3×3 convolution layer and a 1×1 convolution layer, and then operates through batch normalization and activation functions to generate a detail feature map. , expressed as: ; In the formula, Represents an upsampling operation.

[0032] The specific process of segmenting the feature map in step S6 is as follows: The low-resolution branch features In the input segmentation module, the segmentation module includes a segmentation head; the segmentation head consists of a 3×3 convolution layer and a 1×1 convolution layer, and the low-resolution branch feature In the segmentation head, the 3×3 convolution layer and the 1×1 convolution layer are first passed, and then the segmentation feature map is generated through batch normalization and activation function operations. , expressed as: .

[0033] like Figure 3 As shown, the lightweight guided fusion module consists of a high-resolution branch, a low-resolution branch and a 3×3 convolutional layer; The high-resolution branch consists of the first branch and the second branch; the low-resolution branch consists of the third branch and the fourth branch; the first branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; the second branch consists of a 3×3 convolution layer and a downsampling; the third branch consists of a 3×3 convolution layer and an upsampling; the fourth branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; Based on detail feature map and segmentation feature map Optimizing the lightweight guidance fusion module to obtain an optimized lightweight guidance fusion module; High-resolution branch features and low-resolution branch features Input into the optimized lightweight guidance fusion module; High-resolution branch features The 3×3 depth-wise separable convolution of the first branch and the 3×3 convolution layer of the second branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 depth-wise separable convolution of the first branch and the output of the 3×3 convolution layer of the second branch respectively; Low-resolution branch features The 3×3 convolution layer of the third branch and the 3×3 depth-separable convolution of the fourth branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 convolution layer of the third branch and the output of the 3×3 depth-separable convolution of the fourth branch respectively; The output of the 3×3 depth-separable convolution of the first branch is input into the 1×1 convolution layer of the first branch; the output of the 3×3 depth-separable convolution of the fourth branch is input into the 1×1 convolution layer of the fourth branch, and the output of the 1×1 convolution layer of the first branch and the output of the 1×1 convolution layer of the fourth branch are obtained respectively, which are expressed as: ; ; In the formula, Represents the output of the first branch 3×3 depth-wise separable convolution; Represents 3×3 depth-wise separable convolution; Represents the output of the fourth branch 3×3 depth-separable convolution; Represents the output of the first branch 1×1 convolutional layer; Represents the output of the fourth branch 1×1 convolutional layer; The output of the second branch 3×3 convolutional layer is downsampled to obtain the downsampled feature map, and the output of the third branch 3×3 convolutional layer is upsampled to obtain the upsampled feature map, which is expressed as: ; ; In the formula, Represents the downsampled feature map; Represents the upsampled feature map; represents downsampling; The downsampled feature map is multiplied by the output of the fourth branch 1×1 convolution layer to obtain the low-resolution branch weight feature. The upsampled feature map is multiplied by the output of the first branch 1×1 convolution layer to obtain the high-resolution branch weight feature. The high-resolution branch weight feature and the low-resolution branch weight feature are added element by element to obtain the fusion feature. The fusion feature is then integrated through a 3×3 convolution layer to generate the output feature information of the lightweight guided fusion layer. , expressed as: ; In the formula, represents the activation function; Represents the dot product operation.

[0034] like Figure 4 As shown, the drivable area detection feature information output in step S7 is specifically processed as follows: The output feature information of the lightweight guide fusion layer , high-resolution branch features and low-resolution branch features The input is processed in the guidance recovery module to obtain the output of the drivable area detection feature information; The guided recovery module consists of an upper branch, a middle branch, and a lower branch; the upper branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; The middle branch consists of a global average pooling layer, three 3×3 convolutional layers, and a 1×1 convolutional layer; The lower branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; High-resolution branch features In the upper branch, the upsampled data is first input to obtain the upsampled high-resolution branch features. , upsample the high-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the high-resolution branch features of the 3×3 convolution layer, and combine the high-resolution branch features of the 3×3 convolution layer with the upsampled high-resolution branch features. Element-by-element addition to obtain high-resolution branch features of the upper branch , expressed as; ; ; Low-resolution branch features In the lower branch, the upsampled data is first input to obtain the upsampled low-resolution branch features. , upsample the low-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the low-resolution branch features of the 3×3 convolution layer, and combine the low-resolution branch features of the 3×3 convolution layer with the upsampled low-resolution branch features. Element-by-element addition to obtain the low-resolution branch features of the lower branch , expressed as; ; ; Lightweight guide fusion layer output feature information In the middle branch, the global average pooling layer is first input to obtain the output feature information of the lightweight guided fusion layer of the global average pooling layer. , lightweight the global average pooling layer to guide the output feature information of the fusion layer Input the 3×3 convolutional layer to obtain the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, multiply the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, the 3×3 convolutional layer low-resolution branch features and the 3×3 convolutional layer high-resolution branch features to obtain the output feature information of the multiplied lightweight guided fusion layer, and then pass the multiplied information through a 3×3 convolutional layer to obtain the output feature information of the middle branch lightweight guided fusion layer. , expressed as; ; ; In the formula, represents the global average pooling layer; The high-resolution branch features of the upper branch , low-resolution branch features of the lower branch The output feature information of the light-weight guide fusion layer in the middle branch Perform the cascade operation Concat to obtain the concatenated feature map , concatenate the feature maps Input the 3×3 convolution layer and the 1×1 convolution layer in sequence to obtain the 1×1 convolution layer splicing feature map, which is the output feature map of the guided recovery module. , expressed as: ; );The output feature map of the boot recovery module The input is processed into the segmentation head of the segmentation module to obtain the drivable area detection feature information output F, which is expressed as: .

[0035] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dual-branch lightweight drivable area detection method based on low-level features, characterized in that: The following steps are involved: Step S1: constructing dataset images; Step S2: construct a dual-branch lightweight model, which consists of a high-resolution branch module, a low-resolution branch module, a detail module, a segmentation module, and a lightweight guided fusion module; the high-resolution branch module consists of a high-resolution feature extraction network and an adaptive enhancement module; the low-resolution branch module consists of a low-resolution feature extraction network and a cross convolution module; the lightweight guided fusion module includes a guided recovery module; Step S3: input the dataset image into the high-resolution feature extraction network for processing to obtain feature information, and input the feature information into the adaptive enhancement module for processing to obtain high-resolution branch features; Step S4: inputting the dataset image into the low-resolution feature extraction network for processing to obtain the second-layer extracted information, and inputting the second-layer extracted information into the cross convolution module for processing to obtain low-resolution branch features; Step S5: input the high-resolution branch features into the detail module for processing to obtain a detail feature map; Step S6: input the low-resolution branch features into the segmentation module for processing to obtain a segmentation feature map; Step S7: Optimize the lightweight guided fusion module through the detail feature map and the segmentation feature map, input the high-resolution branch features and the low-resolution branch features into the optimized lightweight guided fusion module for processing, obtain the output feature information of the lightweight guided fusion layer, input the high-resolution branch features, the low-resolution branch features and the output feature information of the lightweight guided fusion layer into the guided recovery module for processing, and obtain the output of the drivable area detection feature information.

2. The dual-branch lightweight drivable area detection method based on low-level features according to claim 1 is characterized in that: The specific process of the dataset image in step S1 is as follows: A road segmentation data set is obtained, and the road segmentation data set includes training images and test images; the training images and test images are processed to obtain data set images.

3. The dual-branch lightweight drivable area detection method based on low-level features according to claim 2 is characterized in that: The high-resolution feature extraction network is a ResNet18 module, which includes an L1 layer. The L1 layer consists of a high-resolution feature residual block, which includes two high-resolution feature residual units, and the residual unit includes two 3×3 convolutional layers. The dataset image is input into the high-resolution feature extraction network for processing to obtain feature information F1, which is expressed as: ； In the formula, Represents the dataset image; represents batch normalization; represents a 3×3 convolutional layer; Represents the activation function.

4. The dual-branch lightweight drivable area detection method based on low-level features according to claim 3 is characterized in that: In step S3, high-resolution branch features are obtained. The specific process is as follows: The feature information F1 is input into the adaptive enhancement module for processing to obtain high-resolution branch features; The adaptive enhancement module consists of a structure of four dilated convolution branches; the first dilated convolution branch consists of a 1×1 standard convolution; the second dilated convolution branch consists of a 1×1 standard convolution, a 3×1 standard convolution and a 3×3 dilated convolution; the third dilated convolution branch consists of a 1×1 standard convolution, a 1×3 standard convolution and a 3×3 dilated convolution; the fourth dilated convolution branch consists of a 1×1 standard convolution and a 3×3 standard convolution; The feature information F1 only undergoes a 1×1 standard convolution operation in the first dilated convolution branch to obtain the output of the first dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the second dilated convolution branch, a 3×1 standard convolution and a 3×3 dilated convolution operation are performed to obtain the output of the second dilated convolution branch; after the feature information F1 undergoes a 1×1 standard convolution in the third dilated convolution branch, a 1×3 standard convolution and a 3×3 dilated convolution operation are performed in sequence to obtain the output of the third dilated convolution branch; the feature information F1 undergoes a 1×1 standard convolution in the fourth dilated convolution branch, and then a 3×3 standard convolution operation is performed to obtain the output of the fourth dilated convolution branch; the output of the second dilated convolution branch and the output of the third dilated convolution branch are added to obtain the added output, and the added output is cascaded with the output of the first dilated convolution branch and the output of the fourth dilated convolution branch to obtain a high-resolution branch feature. .

5. The dual-branch lightweight drivable area detection method based on low-level features according to claim 4 is characterized in that: The low-resolution feature extraction network is a ResNet18 module, which includes an L1 layer and an L2 layer. The ResNet18 module consists of a low-resolution feature residual block, which includes two low-resolution feature residual units, and the residual unit includes two 3×3 convolutional layers. The dataset image is input into the low-resolution feature extraction network for processing to obtain the second-layer extracted information F2, which is expressed as: ； ； In the formula, Represents the output of the first low-resolution feature residual unit in the low-resolution feature extraction network.

6. The dual-branch lightweight drivable area detection method based on low-level features according to claim 5 is characterized in that: The low-resolution branch features in step S4 are specifically: The extracted information F2 of the second layer is input into the cross convolution module for processing to obtain low-resolution branch features; The cross convolution module consists of a 1×5 depthwise row convolution, a 5×1 depthwise column convolution, and a 1×1 convolution layer, a batch normalization, and an activation function; The extracted information F2 of the second layer is first subjected to a 1×5 depth row convolution operation in the cross convolution module to capture the context information of the extracted information F2 of the second layer in the horizontal direction, and then a 5×1 depth column convolution operation is performed to extract the context information of the extracted information F2 of the second layer in the vertical direction. A 1×1 convolution layer is used to fuse the context information of the extracted information F2 of the second layer in the horizontal direction and the context information of the extracted information F2 of the second layer in the vertical direction, and a nonlinear transformation is completed through batch normalization and activation function to obtain the output of the nonlinear transformation. The extracted information F2 of the second layer and the output of the nonlinear transformation are added to obtain the low-resolution branch features. , expressed as: ； In the formula, represents a 1×1 convolutional layer; Represents 1×5 depthwise row convolution; Represents a 5×1 depthwise column convolution.

7. The dual-branch lightweight drivable area detection method based on low-level features according to claim 6 is characterized in that: The detailed feature map in step S5, the specific process is: High-resolution branch features The input is processed in the detail module, which includes a detail head; the detail head consists of a 3×3 convolution layer and a 1×1 convolution layer; high-resolution branch features In the detail head, it first passes through a 3×3 convolution layer and a 1×1 convolution layer, and then operates through batch normalization and activation functions to generate a detail feature map. , expressed as: ； In the formula, Represents an upsampling operation.

8. The dual-branch lightweight drivable area detection method based on low-level features according to claim 7 is characterized in that: In step S6, the feature map is segmented, and the specific process is as follows: The low-resolution branch features In the input segmentation module, the segmentation module includes a segmentation head, which consists of a 3×3 convolutional layer and a 1×1 convolutional layer. The low-resolution branch feature In the segmentation head, the 3×3 convolution layer and the 1×1 convolution layer are first passed, and then the segmentation feature map is generated through batch normalization and activation function operations. , expressed as: 。 9. The dual-branch lightweight drivable area detection method based on low-level features according to claim 8, characterized in that: The lightweight guided fusion module consists of a high-resolution branch, a low-resolution branch, and a 3×3 convolutional layer; The high-resolution branch consists of the first branch and the second branch; the low-resolution branch consists of the third branch and the fourth branch; the first branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; the second branch consists of a 3×3 convolution layer and a downsampling; the third branch consists of a 3×3 convolution layer and an upsampling; the fourth branch consists of a 3×3 depth-separable convolution and a 1×1 convolution layer; Based on detail feature map and segmentation feature map Optimizing the lightweight guidance fusion module to obtain an optimized lightweight guidance fusion module; High-resolution branch features and low-resolution branch features Input into the optimized lightweight guidance fusion module; High-resolution branch features The 3×3 depth-separable convolution of the first branch and the 3×3 convolution layer of the second branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 depth-separable convolution of the first branch and the output of the 3×3 convolution layer of the second branch respectively; Low-resolution branch features The 3×3 convolution layer of the third branch and the 3×3 depth-separable convolution of the fourth branch are respectively input into the optimized lightweight guided fusion module to obtain the output of the 3×3 convolution layer of the third branch and the output of the 3×3 depth-separable convolution of the fourth branch respectively; Input the output of the 3×3 depth-separable convolution of the first branch to the 1×1 convolution layer of the first branch; input the output of the 3×3 depth-separable convolution of the fourth branch to the 1×1 convolution layer of the fourth branch, and obtain the output of the 1×1 convolution layer of the first branch and the output of the 1×1 convolution layer of the fourth branch respectively; Downsample the output of the second branch 3×3 convolutional layer to obtain a downsampled feature map, and upsample the output of the third branch 3×3 convolutional layer to obtain an upsampled feature map; The downsampled feature map is multiplied by the output of the fourth branch 1×1 convolution layer to obtain the low-resolution branch weight feature. The upsampled feature map is multiplied by the output of the first branch 1×1 convolution layer to obtain the high-resolution branch weight feature. The high-resolution branch weight feature and the low-resolution branch weight feature are added element by element to obtain the fusion feature. The fusion feature is then integrated through a 3×3 convolution layer to generate the output feature information of the lightweight guided fusion layer. .

10. The dual-branch lightweight drivable area detection method based on low-level features according to claim 9, characterized in that: In step S7, the drivable area detection feature information is output, and the specific process is as follows: The output feature information of the lightweight guide fusion layer , high-resolution branch features and low-resolution branch features The input is processed in the guidance recovery module to obtain the output of the drivable area detection feature information; The guided recovery module consists of an upper branch, a middle branch, and a lower branch; the upper branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; The middle branch consists of a global average pooling layer, three 3×3 convolutional layers, and a 1×1 convolutional layer; The lower branch consists of an upsampling layer, a 1×1 convolution layer, and a 3×3 convolution layer; High-resolution branch features In the upper branch, the upsampled data is first input to obtain the upsampled high-resolution branch features. , upsample the high-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the high-resolution branch features of the 3×3 convolution layer, and combine the high-resolution branch features of the 3×3 convolution layer with the upsampled high-resolution branch features. Element-by-element addition to obtain high-resolution branch features of the upper branch ; Low-resolution branch features In the lower branch, the upsampled data is first input to obtain the upsampled low-resolution branch features. , upsample the low-resolution branch features Input the 1×1 convolution layer and the 3×3 convolution layer in sequence to obtain the low-resolution branch features of the 3×3 convolution layer, and combine the low-resolution branch features of the 3×3 convolution layer with the upsampled low-resolution branch features. Element-by-element addition to obtain the low-resolution branch features of the lower branch ; Lightweight guide fusion layer output feature information In the middle branch, the global average pooling layer is first input to obtain the output feature information of the lightweight guided fusion layer of the global average pooling layer. , lightweight the global average pooling layer to guide the output feature information of the fusion layer Input the 3×3 convolutional layer to obtain the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, multiply the output feature information of the 3×3 convolutional layer lightweight guided fusion layer, the 3×3 convolutional layer low-resolution branch features and the 3×3 convolutional layer high-resolution branch features to obtain the output feature information of the multiplied lightweight guided fusion layer, and then pass the multiplied information through a 3×3 convolutional layer to obtain the output feature information of the middle branch lightweight guided fusion layer. ; The high-resolution branch features of the upper branch , low-resolution branch features of the lower branch The output feature information of the light-weight guide fusion layer in the middle branch Perform cascade operation to obtain the spliced ​​feature map , concatenate the feature maps Input the 3×3 convolution layer and the 1×1 convolution layer in sequence to obtain the 1×1 convolution layer splicing feature map, which is the output feature map of the guided recovery module. ; The output feature map of the boot recovery module The input is processed into the segmentation head of the segmentation module to obtain the drivable area detection feature information output F, which is expressed as: 。

Images