A road segmentation method based on bimodal feature uncertainty fusion

Abstract

The application discloses a road segmentation method based on dual-modal feature uncertainty fusion, aiming at solving the problems of modal feature conflict, poor adaptability of fixed weight fusion and insufficient utilization of modal uncertainty in existing depth camera road segmentation; the method constructs an integrated framework of dual-modal feature extraction and uncertainty perception fusion: for color images, a pure thinking method module is adopted to extract global semantic features, and for depth data, a convolution network with fused asymmetric dilated convolution is adopted to enhance spatial geometric perception; through a multi-scale dilated attention module, feature interaction is optimized, and pixel-level classification evidence is generated by using evidence collection; based on the Dempster-Sheffer evidence theory, dual-modal uncertainty is quantified and dynamically fused, the fusion weight is adaptively adjusted, and feature conflict and information redundancy are effectively inhibited; the application significantly improves the road segmentation precision and robustness in complex scenes, and provides reliable drivable area perception information for an automatic driving system.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and autonomous driving environmental perception technology, specifically a road segmentation method based on dual-modal feature uncertainty fusion. Background Technology

[0002] Accurate segmentation of road scenes is a crucial step in achieving environmental perception and behavioral decision-making in autonomous driving technology. Its core task is to precisely define drivable areas from visual input, providing a reliable basis for vehicle path planning and safety control. With the development of multimodal sensing technology, fusing color images and depth information for road segmentation has become an important direction for improving perception performance. Color images contain rich color and texture semantics, while depth data provides accurate spatial geometry. The two are naturally complementary in terms of information dimensions, and the hardware cost is relatively controllable, thus attracting extensive research and exploration.

[0003] However, existing depth camera-based road segmentation methods still face many challenges in practical applications. Firstly, regarding computational efficiency, most methods employ complex cross-connections or dense fusion structures to achieve deep feature interaction between modalities, leading to a significant increase in model parameters and computational overhead, making it difficult to meet the stringent real-time inference requirements of automotive embedded platforms. Especially under high-resolution input conditions, existing methods generally have low frame rates, often failing to exceed 20 frames per second, limiting their deployment feasibility in real-time systems.

[0004] Secondly, regarding feature fusion strategies, color modalities and depth modalities differ fundamentally in their physical meaning and expressive dimensions. The former focuses on apparent semantics, while the latter reflects spatial layout. Existing fusion mechanisms often employ preset fixed weights or simple stitching methods, failing to dynamically adjust the contribution of each modality according to the scene. For example, under conditions of strong light, shadow occlusion, or severe weather, the quality of color images deteriorates, significantly reducing their reliability. In such cases, greater reliance on depth information is necessary. However, traditional methods lack the ability to perceive modal reliability, leading to severe noise interference in the fusion results, resulting in blurred or even erroneous segmentation boundaries.

[0005] Furthermore, real-world road scenarios contain numerous dynamic obstacles and complex road surface materials, which can easily cause local distortion or decreased confidence of single-modal features. Existing models generally lack explicit modeling and utilization mechanisms for feature uncertainty, making it impossible to make robust decisions when perception is unreliable, further limiting their applicability and robustness in complex real-world environments.

[0006] In summary, existing depth camera-based road segmentation methods have not effectively addressed the feature conflicts caused by modal differences during bimodal feature fusion, nor have they achieved dynamic perception and adaptive fusion of modal uncertainties. Furthermore, they struggle to achieve a good balance between accuracy and efficiency. Therefore, there is an urgent need for a road segmentation method that can perceive and utilize bimodal uncertainties, possesses dynamic fusion capabilities, and balances segmentation accuracy with inference efficiency, in order to improve the drivable area recognition capability and overall reliability of autonomous driving systems in complex scenarios. Summary of the Invention

[0007] In view of the above situation, this invention proposes a road segmentation method based on dual-modal feature uncertainty fusion. By constructing an integrated segmentation framework, it uses a color branch of a pure thinking model architecture to extract global semantic features, and combines a deep branch incorporating dilated convolution to enhance spatial geometric perception. It utilizes a multi-scale dilated attention module to optimize feature interaction, and achieves uncertainty quantification and dynamic weight fusion through evidence theory. This effectively overcomes modal differences and feature conflicts, significantly improves the accuracy and robustness of road segmentation in complex scenarios, and provides reliable drivable area information for autonomous driving, thereby solving the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a road segmentation method based on dual-modal feature uncertainty fusion, comprising the following steps:

[0009] Step S1: Read the depth image and color image from the benchmark dataset, and perform normalization and size cropping on the depth image and color image to obtain bimodal input data with consistent resolution. The bimodal input data includes a depth branch and a color branch.

[0010] Step S2: Input the bimodal input data into the bimodal feature extraction module; the depth branch is processed sequentially through the depth normal vector extraction component of the bimodal feature extraction module and the first three layers of the depth residual network to obtain the depth feature maps of the first three stages; the fourth layer of the depth residual network is replaced with a directional dilated convolution block, and the depth feature map of the third stage is processed through the replaced fourth layer directional dilated convolution block to obtain the depth feature map of the fourth stage;

[0011] Step S3: The color branch is processed sequentially through the image embedding module in the dual-modal feature extraction module and four base layers containing proxy attention to obtain color feature maps in four stages;

[0012] Step S4: Input the depth feature map of the fourth stage obtained in step S2 and the color feature map of the fourth stage obtained in step S3 into the multi-dilation rate attention enhancement module for feature enhancement to obtain depth enhancement features and color enhancement features;

[0013] Step S5: Input the depth enhancement features and color enhancement features into the corresponding feature decoding modules for processing to obtain the depth branch fine features and color branch fine features;

[0014] Step S6: Input the deep branch fine features and color branch fine features into the dual-modal uncertainty fusion module for processing to obtain the comprehensive classification criteria; calculate the category probability of the comprehensive classification criteria to obtain the final road segmentation map;

[0015] Step S7: Construct a road segmentation model based on the dual-modal feature extraction module, multi-expansion rate attention enhancement module, feature decoding module, and dual-modal uncertainty fusion module; construct a total loss function based on the road segmentation model for hierarchical supervision.

[0016] Further, in step S2, the depth branch is processed sequentially by the depth normal vector extraction component of the dual-modal feature extraction module and the first three layers of the deep residual network to obtain the depth feature maps of the first three stages; the fourth layer of the deep residual network is replaced by a directional dilation convolution block, and the depth feature map of the third stage is processed by the replaced fourth layer directional dilation convolution block to obtain the depth feature map of the fourth stage; wherein, the deep residual network is ResNet18; specifically:

[0017] Step S21: The depth branch input is a depth image. The pixel values ​​of each pixel in the depth image are obtained through the depth normal vector extraction component. The pixel values ​​are divided by 1000 to obtain the actual depth values. Using the camera intrinsic matrix parameters in the calibration file provided by the benchmark dataset KITTI, the focal length in the horizontal direction and the focal length in the vertical direction, as well as the principal point coordinates in the horizontal direction and the principal point coordinates in the vertical direction are obtained. The pixel coordinates of the depth image are mapped to three-dimensional coordinates in the camera coordinate system.

[0018] Step S22: Multiply the rate of change of each pixel in the depth image in the horizontal and vertical directions with the corresponding focal length in the horizontal and vertical directions to obtain the initial normal components in the horizontal and vertical directions of the three-dimensional coordinates. Calculate the initial normal components in the horizontal and vertical directions with the depth value to obtain the normal component in the depth value direction. Normalize the normal component in the depth value direction to obtain the unit normal vector.

[0019] Step S23: The unit normal vector is fused and corrected in 8 directions to obtain the three-dimensional surface normal feature map;

[0020] Step S24: The 3D surface normal feature map is input into the input preprocessing layer of the deep residual network of the dual-modal feature extraction module; the 3D surface normal feature map is processed sequentially through a 7×7 convolutional layer with a stride of 2 and a padding of 3, a batch normalization layer, a modified linear unit activation layer, and a 3×3 max pooling layer, compressing the spatial size of the 3D surface normal feature map to 1 / 4 of the original size, and obtaining the initial feature map adapted to the subsequent residual blocks;

[0021] Step S25: The initial feature map is input into the first layer of the deep residual network. The first layer consists of two residual blocks, each with the same structure. The first residual block operates on the input initial feature map: using a 1 x 1 convolutional layer, the number of channels of the initial feature map is compressed to 1 / 4 of the original number of channels. The compressed initial feature map is then processed by a batch normalization layer and a modified linear unit activation layer to obtain a first intermediate feature map. The first intermediate feature map is then processed by a 3 x 3 convolutional layer to capture the normal vector correlation of local regions. The normal vector correlation of local regions is then processed by a batch normalization layer and a modified linear unit activation layer to obtain a second intermediate feature map. The second intermediate feature map is then restored to the size of the initial feature map using a 1 x 1 convolutional layer. The restored second intermediate feature map is then residually connected to the initial feature map to obtain a residual feature map.

[0022] Step S26: After the residual feature map passes through the first layer of the deep residual network, the first stage deep feature map is obtained; after the first stage deep feature map is input into the second layer of the deep residual network, the second stage deep feature map is obtained; after the second stage deep feature map is input into the third layer of the deep residual network, the third stage deep feature map is obtained.

[0023] Step S27: The depth feature map of the third stage is input into the directional dilation convolution block; the directional dilation convolution block is composed of 4 layers of directional dilation convolution units connected sequentially; in the first layer of the directional dilation convolution unit, the depth feature map of the third stage uses a 3×1 convolution with a vertical dilation rate of 1 to capture the vertical abrupt change of the near-distance road edge normal, and uses a 1×3 convolution with a horizontal dilation rate of 5 to extract the continuity of the local lane line normal, and after activation by a batch normalization layer and a modified linear unit activation layer, the first feature map is obtained; the second layer of the directional dilation convolution unit takes the first feature map, uses a 3×1 convolution with a vertical dilation rate of 3, and uses horizontal dilation... The first layer uses a 1×3 convolution with a vertical dilation rate of 8. After batch normalization and activation by a modified linear unit activation layer, the second feature map is obtained. The third layer uses the same parameters as the second layer to obtain the third feature map. The fourth layer uses a 3×1 convolution with a vertical dilation rate of 7 and a 1×3 convolution with a horizontal dilation rate of 21. After batch normalization and activation by a modified linear unit activation layer, the fourth feature map is obtained. The depth feature map of the third stage of the initial input is residually concatenated with the fourth feature map to obtain the fifth feature map. The fifth feature map is used as the depth feature map of the fourth stage of the depth branch.

[0024] Furthermore, in step S3, the color branch is processed sequentially through the image embedding module in the dual-modal feature extraction module and four base layers containing proxy attention to obtain color feature maps in four stages; specifically:

[0025] Step S31: The input to the color branch is the original image. The original image is processed by a convolutional layer with a kernel size of 4, a stride of 4, and 96 output channels to perform image patch embedding. The original image is divided into 4x4 pixel non-overlapping local blocks and mapped to 96-dimensional feature vectors. The 96-dimensional feature vectors are normalized and arranged according to their original spatial positions to form the original feature map. The original feature map is flattened to obtain the feature sequence.

[0026] Step S32: The feature sequence is input into a four-layer surrogate-shift window transformer network; each layer contains several surrogate-shift window transformer blocks; each surrogate-shift window transformer block uses three attention heads, and each surrogate-shift window transformer block processes the input feature sequence as follows:

[0027] Step S321: Normalize the input feature sequence by calculating the mean and variance along the feature dimension to obtain the first feature sequence;

[0028] Step S322: The first feature sequence is reshaped through a reshaping operation to obtain a two-dimensional feature map;

[0029] Step S323: Divide the two-dimensional feature map into multiple sub-feature maps according to a 7x7 window, and generate query features, key features and value features by performing a linear transformation on each sub-feature map;

[0030] Step S324: Based on the query features, a query feature map is obtained. A 7x7 adaptive average pooling operation is performed on the query feature map, dividing it evenly into 7 parts according to the height and width of the two-dimensional feature map, resulting in 49 region representative features. The 49 region representative features are flattened to obtain 49 proxy token sequences. The feature dimension of each proxy token sequence is 96. Three attention heads split the proxy token sequences, query features, key features, and value features equally according to the feature dimensions, that is, each attention head is responsible for 32 feature dimensions.

[0031] Step S325: Assign attention to sub-feature maps by using the key features of all sub-feature maps through the proxy token; then calculate the attention of sub-feature maps to the proxy token by using the query features of the sub-feature maps; calculate the attention weight from the proxy token to the sub-feature map and the attention weight from the sub-feature map to the proxy token.

[0032] Step S326: Perform matrix multiplication on the value features of the sub-feature maps using the attention weights from the proxy token to the sub-feature maps, and aggregate the features of all sub-feature maps to obtain the global association features guided by the proxy token; perform reverse aggregation on the global association features using the attention weights from the sub-feature maps to the proxy token, so that the local details of each sub-feature map are fed back into the global association features to obtain aggregated features; use depthwise separable convolution on the aggregated features to extract local association information in the neighborhood of the sub-feature maps to obtain aggregated enhanced features; perform residual connection between the aggregated enhanced features and the input feature sequence to obtain the first comprehensive feature;

[0033] Step S327: Introduce a multilayer perceptron layer to enhance the nonlinear expressive power of the first comprehensive feature. The multilayer perceptron layer includes two fully connected layers, a Gaussian error linear unit activation function, and random deactivation regularization processing to obtain the second comprehensive feature.

[0034] Step S328: Perform a residual connection between the second synthetic feature and the first synthetic feature to obtain the final synthetic feature of the surrogate-shift window converter block;

[0035] Step S33: After the final synthesized features are iteratively processed by two proxy-shift window transformer blocks in the first layer, the first-stage color feature map output by the first layer is obtained; after the first-stage color feature map is iteratively processed by two proxy-shift window transformer blocks in the second layer, the second-stage color feature map output by the second layer is obtained; after the second-stage color feature map is iteratively processed by six proxy-shift window transformer blocks in the third layer, the third-stage color feature map output by the third layer is obtained; after the third-stage color feature map is iteratively processed by two proxy-shift window transformer blocks in the fourth layer, the fourth-stage color feature map output by the fourth layer is obtained.

[0036] Furthermore, in step S4, the depth feature map obtained in step S2 and the color feature map obtained in step S3 are input into the multi-dilation rate attention enhancement module for feature enhancement, resulting in depth-enhanced features and color-enhanced features; specifically:

[0037] Step S41: The dimensional order of the input fourth-stage depth feature map is changed from batch size, number of channels, height, width to batch size, height, width, number of channels, and then transposed. A 1x1 convolutional layer is used to process the transposed fourth-stage depth feature map, expanding the number of output channels to 512×3, so that each of the 512 channels generates three sets of features: query features, key features, and value features.

[0038] Step S42: Divide the three groups of features with 512 channels—query features, key features, and value features—into two groups according to the average number of channels. The two groups correspond to two predefined expansion rate parameters, which are set to 2 and 3, to generate a first group of sub-features with an expansion rate of 2 and a second group of sub-features with an expansion rate of 3.

[0039] Step S43: Perform multi-head expanded attention calculation in parallel on the first group of sub-features and the second group of sub-features;

[0040] Step S431: The first group of sub-features is further split according to the attention head to obtain 8 32-channel head branch features. The head branch features include head branch query features, head branch key features and head branch value features.

[0041] Step S432: Extract local region features from the head branch key features and head branch value features using a sliding window expansion operation with an expansion rate of 2, to obtain local key features and local value features.

[0042] Step S433: Perform matrix multiplication on the head branch query features and the expanded local key features to calculate the original similarity score, and after adjusting the scaling factor, normalize it through the normalized exponential function to obtain the attention weight.

[0043] Step S434: Use attention weights to perform weighted fusion of the expanded local value features to obtain fused local features;

[0044] Step S435: The fused local features output from the 8 head branches are spliced ​​together in the channel dimension and merged into a 256-channel feature map to obtain an attention-enhanced feature with an expansion rate of 2.

[0045] Step S436: The second set of sub-features is based on steps S431 to S435 to obtain attention-enhanced features with an expansion rate of 3;

[0046] Step S44: The attention enhancement feature with an expansion rate of 2 and the attention enhancement feature with an expansion rate of 3 are concatenated along the channel dimension and fused through a projection layer to obtain the final feature;

[0047] Step S45: Obtain the depth enhancement feature corresponding to the depth branch based on the output of the final feature;

[0048] Step S46: The color feature map of the fourth stage obtains the color enhancement features corresponding to the color branches based on steps S41 to S45.

[0049] Furthermore, in step S5, the depth enhancement features and color enhancement features are respectively input to the corresponding feature decoding modules for processing to obtain the depth branch fine features and the color branch fine features; specifically:

[0050] Step S51: Use a 1x1 convolutional layer to compress the number of channels of the input depth enhancement features and color enhancement features from 512 to 64, to obtain the basic decoding features of the depth branch and the basic decoding features of the color branch.

[0051] Step S52: Perform a three-level upsampling operation on the basic decoding features of the depth branch and the basic decoding features of the color branch through the upsampling module. The upsampling module consists of bilinear interpolation operation, effective feature enhancement unit and residual fusion operation.

[0052] Step S53: Perform the first upsampling operation on the basic decoding features of the depth branch: fuse the depth enhancement features with the depth feature map of the third stage of the depth branch to obtain the first depth branch upsampling result. The specific steps are as follows:

[0053] Step S531: Upsample the depth enhancement features to the same spatial size as the depth feature map in the third stage using bilinear interpolation to obtain the depth enhancement feature map;

[0054] Step S532: Use 1x1 convolution to compress the channel of the depth feature map in the third stage to obtain the directional dilated convolution block; input the directional dilated convolution block into the effective feature enhancement unit for fine calibration. The effective feature enhancement unit first divides the directional dilated convolution block into K groups according to the channel, performs adaptive pooling in the height and width directions on each group of features to capture the spatial context, and then activates them through 1×1 convolution and activation function to obtain height attention weights and width attention weights.

[0055] Step S533: Multiply the height attention weight and width attention weight element-wise with the corresponding k-th group of features, and then obtain the weighted features through group normalization. At the same time, perform a 3x3 convolution operation on the original grouped features to obtain local detail features.

[0056] Step S534: Perform global adaptive pooling to accelerate graphics port operations and normalized exponential function activation on the weighted features, then flatten the local detail features, perform matrix multiplication on the weighted features and local detail features, and then obtain the channel-space interaction weights through an activation function.

[0057] Step S535: For the k-th feature group, first multiply the height attention weight and the width attention weight element by element to obtain the comprehensive spatial attention weight; add the comprehensive spatial attention weight to the above channel-space interaction weight to obtain the final fusion weight; use the final fusion weight to perform element-wise weighting on the k-th feature group of the original group features to obtain the k-th calibrated features; then concatenate the calibrated features to obtain the final calibrated features.

[0058] Step S536: Add the upsampled depth enhancement feature map to the final calibration feature element by element to obtain the first depth branch upsampling result;

[0059] Step S54: The second and third upsampling operations are logically the same as the first upsampling operation; the second upsampling operation is to fuse the first depth branch upsampling result with the depth feature map of the second stage of the depth branch to obtain the second depth branch upsampling result; the third upsampling operation is to fuse the second depth branch upsampling result with the depth feature map of the first stage of the depth branch to obtain the third depth branch upsampling result.

[0060] Step S55: Based on steps S53 to S54, the basic decoding features of the color branch in step S52 are used to obtain the first color branch upsampling result, the second color branch upsampling result, and the third color branch upsampling result.

[0061] Step S56: Use the third depth branch upsampling result and the third color branch upsampling result as the depth branch fine features and color branch fine features, respectively.

[0062] Furthermore, in step S6, the deep branch fine features and the color branch fine features are input into the dual-modal uncertainty fusion module for processing to obtain a comprehensive classification basis; the category probability of the comprehensive classification basis is calculated to obtain the final road segmentation map; specifically:

[0063] Step S61: Input the deep branch fine features and color branch fine features into the multi-scale evidence collection component to perform three convolution operations: 1x1 convolution, convolution with a kernel of 3x3 and a dilation rate of 3, and convolution with a kernel of 3x3 and a dilation rate of 6, to generate feature maps with three different receptive fields.

[0064] Step S62: The feature maps of the three different receptive fields are upsampled to restore the size of the initially read depth or color image, and activated by the activation function to obtain three single-scale evidences of the depth branch and three single-scale evidences of the color branch, respectively.

[0065] Step S63: Calculate the average of the three single-scale evidences of the depth branch and the three single-scale evidences of the color branch to obtain the fused evidence of the depth branch and the fused evidence of the color branch.

[0066] Step S64: Convert the fusion evidence of the deep branch and the fusion evidence of the color branch into parameters of the Dirichlet distribution. The conversion relationship is: the parameter of the Dirichlet distribution is equal to the evidence value plus 1, to obtain the deep branch parameters and the color branch parameters. Then fuse the deep branch parameters and the color branch parameters to obtain the fusion parameters of the deep branch parameters and the color branch parameters.

[0067] Step S65: For the fusion parameters of the deep branch parameters and the color branch parameters, calculate the total confidence, relative evidence, and uncertainty of the deep branch parameters and the color branch parameters respectively, and calculate the evidence conflict coefficient between the deep branch parameters and the color branch parameters.

[0068] Step S66: Based on the conflict coefficient, calculate the joint relative evidence, joint uncertainty, and joint total confidence of the depth branch parameters and the color branch parameters;

[0069] Step S67: Calculate the fused bimodal joint evidence based on joint relative evidence, joint uncertainty, and joint total confidence.

[0070] Step S68: Based on dual-modal joint evidence, calculate the probability of each pixel belonging to the road category, and determine the pixels with a probability value greater than 0.5 as road areas to generate the final road segmentation mask.

[0071] Furthermore, in step S7, a total loss function is constructed based on the road segmentation model. This total loss function includes an evidence cross-entropy loss function, which provides hierarchical supervision of the multiple levels of evidence generated by the dual-modal uncertainty fusion module. Specifically:

[0072] Step S71: For given depth branch parameters and color branch parameters, the evidence cross-entropy loss function consists of a classification error term and an uncertainty regularization term, as shown in the following formula:

[0073] ;

[0074] ;

[0075] in, This indicates the adjusted parameters for the depth branch and color branch. include , , The label represents the actual road label, with a value of 1 for road areas and a value of 0 for non-road areas. Represents the evidence cross-entropy loss function. Represents pixels, Let S represent the logarithmic derivative of the gamma function, and let S represent the total confidence level of the evidence. Indicates the annealing coefficient. Denotes KL divergence, Represents a vector whose elements are all 1s;

[0076] Step S72: Perform hierarchical supervision on the three single-scale evidences of the depth branch and color branch, the fusion parameter of the depth branch parameters and the color branch parameters, and the dual-modal joint evidence using the evidence cross-entropy loss function; then obtain the total loss function by calculating the loss function one by one and weighted summing, as shown in the following formula:

[0077] ;

[0078] in, Represents the total loss function. Three single-scale pieces of evidence representing the depth branch and the color branch. This represents the fusion parameter between the depth branch parameter and the color branch parameter. This represents the bimodal joint evidence; a loss weight of 2 is applied to the bimodal joint evidence to enhance the optimization of the final fusion result.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] Achieving adaptive fusion to effectively overcome modal conflicts: By dynamically evaluating the reliability of color and depth modal features and adaptively adjusting the fusion weights, the feature conflict problem caused by modal differences is effectively solved, overcoming the shortcomings of traditional fixed-weight fusion strategies in complex scenarios with poor adaptability, and realizing efficient complementarity and synergistic utilization of dual-modal information.

[0081] Enhancing the comprehensiveness and discriminative power of feature representation: An asymmetric dual-branch feature extraction architecture is adopted. The color branch uses an agent-Swain converter to enhance the capture of global semantic details, while the deep branch enhances the perception of spatial geometric structure by combining the improved deep residual network ResNet18 with directional dilated convolution. This fully leverages the information advantages of different modalities and significantly improves the comprehensive representation capability of road features.

[0082] Enhance the ability to extract and focus key road features: By using a multi-expansion rate attention enhancement module, a differentiated receptive field is constructed using multiple sets of expansion rates, taking into account both local details and global contextual information. Furthermore, the attention mechanism is used to effectively enhance the features of key road areas and suppress non-road interference, significantly improving the model's recognition accuracy and feature completeness for road areas.

[0083] To ensure the spatial detail and edge accuracy of the segmentation results: The feature decoding module adopts a progressive fusion strategy of three-level upsampling and multi-stage features to achieve multi-scale detail restoration and enhancement from road contours and road surface textures to image edges. The final output is a fine feature map with the same resolution as the input, which effectively ensures the high resolution and high detail of the road segmentation results.

[0084] Provides accurate segmentation results with reliability assessment: The dual-modal uncertainty fusion module achieves quantitative assessment of uncertainty through multi-scale evidence collection and Dirichlet distribution parameterization, and completes dynamic weight fusion based on the Durmst-Schaffer evidence theory. It not only outputs accurate road segmentation maps, but also provides predictive reliability metrics, which greatly improves the credibility of decision-making basis for autonomous driving and other systems in complex scenarios. Attached Figure Description

[0085] Figure 1 This is an overall flowchart of the present invention;

[0086] Figure 2 This is a schematic diagram of the overall model framework of the present invention;

[0087] Figure 3 This is a flowchart of the multi-expansion rate attention enhancement module of the present invention. Detailed Implementation

[0088] Referring to the following description and accompanying drawings, some specific embodiments of the present invention are disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention. However, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0089] like Figures 1 to 3 As shown, this embodiment proposes the following technical solution: a road segmentation method based on dual-modal feature uncertainty fusion, which includes the following steps:

[0090] Step S1: Read the depth image and color image from the benchmark dataset, and perform normalization and size cropping on the depth image and color image to obtain bimodal input data with consistent resolution. The bimodal input data includes a depth branch and a color branch.

[0091] Step S2: Input the bimodal input data into the bimodal feature extraction module; the depth branch is processed sequentially through the depth normal vector extraction component of the bimodal feature extraction module and the first three layers of the depth residual network to obtain the depth feature maps of the first three stages; the fourth layer of the depth residual network is replaced with a directional dilated convolution block, and the depth feature map of the third stage is processed through the replaced fourth layer directional dilated convolution block to obtain the depth feature map of the fourth stage;

[0092] Step S3: The color branch is processed sequentially through the image embedding module in the dual-modal feature extraction module and four base layers containing proxy attention to obtain color feature maps in four stages;

[0093] Step S4: Input the depth feature map of the fourth stage obtained in step S2 and the color feature map of the fourth stage obtained in step S3 into the multi-dilation rate attention enhancement module for feature enhancement to obtain depth enhancement features and color enhancement features;

[0094] Step S5: Input the depth enhancement features and color enhancement features into the corresponding feature decoding modules for processing to obtain the depth branch fine features and color branch fine features;

[0095] Step S6: Input the deep branch fine features and color branch fine features into the dual-modal uncertainty fusion module for processing to obtain the comprehensive classification criteria; calculate the category probability of the comprehensive classification criteria to obtain the final road segmentation map;

[0096] Step S7: Construct a road segmentation model based on the dual-modal feature extraction module, multi-expansion rate attention enhancement module, feature decoding module, and dual-modal uncertainty fusion module; construct a total loss function based on the road segmentation model for hierarchical supervision.

[0097] Further, in step S2, the depth branch is processed by the depth normal vector extraction component of the dual-modal feature extraction module to obtain a 3D surface normal feature map; the 3D surface normal feature map is input to the first three layers of the depth residual network of the dual-modal feature extraction module for processing to obtain the depth feature maps of the first three stages; the fourth layer of the depth residual network is replaced by a directional dilation convolution block, and the depth feature map of the third stage is processed by the fourth layer replaced by the directional dilation convolution block to obtain the depth feature map of the fourth stage; wherein, the depth residual network is ResNet18; specifically:

[0098] Step S21: The depth branch input is a depth image. The depth normal vector extraction component obtains the pixel values ​​of each pixel in the depth image, and divides these pixel values ​​by 1000 to obtain the actual depth values. Using the camera intrinsic parameter matrix parameters in the calibration file provided by the KITTI benchmark dataset, the horizontal and vertical focal lengths, as well as the horizontal and vertical principal point coordinates, are obtained. The pixel coordinates of the depth image are then mapped to three-dimensional coordinates in the camera coordinate system, as shown in the following formula:

[0099] ;

[0100] ;

[0101] in, Represents the horizontal direction of three-dimensional coordinates. Represents pixels in the horizontal direction. Indicates the coordinates of the principal point in the horizontal direction. Indicates the depth value. The focal length indicates the horizontal direction. Represents the vertical direction of the three-dimensional coordinates. Represents pixels in the vertical direction. Represents the coordinates of the principal point in the vertical direction. Indicates the focal length in the vertical direction;

[0102] Step S22: Multiply the rate of change of each pixel in the depth image in the horizontal and vertical directions by the corresponding focal lengths in the horizontal and vertical directions to obtain the initial normal components in the horizontal and vertical directions of the three-dimensional coordinates. Calculate the initial normal components in the horizontal and vertical directions with the depth value to obtain the normal components in the depth value direction. Normalize the normal components in the depth value direction to obtain the unit normal vector, as shown in the following formula:

[0103] · ; ;

[0104] ;

[0105] ;

[0106] in, This represents the initial normal component in the horizontal direction. Indicates the rate of change in the horizontal direction. This represents the initial normal component in the vertical direction. Indicates the rate of change in the vertical direction. The normal component representing the direction of the depth value. Indicates the amount of change in the horizontal direction. Indicates the change in the vertical direction. Indicates the amount of change in the depth direction. Represents the unit normal vector. The square of the variable representing the initial normal component in the horizontal direction. The square of the variable representing the initial normal component in the vertical direction. The square of the normal component representing the direction of the depth value;

[0107] Step S23: The unit normal vector is fused and corrected in eight directions to obtain the 3D surface normal feature map, as shown in the following formula:

[0108] ;

[0109] in, This represents a 3D surface normal feature map. This represents the direction correction parameter, which takes the value of -1 or 1. When the initial normal component ny in the vertical direction is greater than 0, it is -1, and otherwise it is 1. It indicates one of the eight directions;

[0110] Step S24: Input the 3D surface normal feature map into the input preprocessing layer of the depth residual network of the dual-modal feature extraction module;

[0111] The 3D surface normal feature map is processed sequentially through a 7×7 convolutional layer with a stride of 2 and padding of 3, a batch normalization layer, a modified linear unit activation layer, and a 3×3 max pooling layer. This process compresses the spatial size of the 3D surface normal feature map to 1 / 4 of its original size, resulting in an initial feature map adapted for subsequent residual blocks. The formula is as follows:

[0112] ;

[0113] in, Represents the initial feature map. Indicates the max pooling layer. This indicates a modified linear unit activation layer. Indicates the batch normalization layer. This represents a 7x7 convolutional layer;

[0114] Step S25: Input the initial feature map into the first layer of the deep residual network. The first layer consists of two residual blocks, each with the same structure. The first residual block operates on the input initial feature map:

[0115] Using a 1x1 convolutional layer, the number of channels in the initial feature map is compressed to 1 / 4 of the original number of channels. A batch normalization layer and a modified linear unit activation layer are then applied to the compressed initial feature map to obtain the first intermediate feature map. The first intermediate feature map is then passed through a 3x3 convolutional layer to capture the normal vector correlations of local regions. These local normal vector correlations are then passed through another batch normalization layer and a modified linear unit activation layer to obtain the second intermediate feature map. The second intermediate feature map is then restored to the size of the initial feature map using a 1x1 convolutional layer. The restored second intermediate feature map is then residually concatenated with the initial feature map to obtain the residual feature map, as shown in the following formula:

[0116] ;

[0117] ;

[0118] ;

[0119] in, This represents the first intermediate feature map. This represents a 1x1 convolution operation. This represents the second intermediate feature map. This represents a 3x3 convolution operation. Represents the residual feature map;

[0120] Step S26: After the residual feature map passes through the first layer of the deep residual network, a first-stage deep feature map with 64 channels is obtained; after the first-stage deep feature map is input into the second layer of the deep residual network, a second-stage deep feature map with 128 channels is obtained; after the second-stage deep feature map is input into the third layer of the deep residual network, a third-stage deep feature map with 256 channels is obtained.

[0121] Step S27: Input the depth feature map of the third stage into the directional dilated convolution block; the directional dilated convolution block is composed of 4 layers of directional dilated convolution units connected in sequence;

[0122] The third-stage depth feature map is obtained in the first layer of the directional dilated convolutional unit. A 3×1 convolution with a vertical dilation rate of 1 captures abrupt vertical changes in the near-distance roadside normal, and a 1×3 convolution with a horizontal dilation rate of 5 extracts the continuity of local lane line normals. After activation by a batch normalization layer and a modified linear unit activation layer, a first feature map with 512 channels is obtained, as shown in the following formula:

[0123] ;

[0124] in, Represents the first feature map. This indicates a convolution operation with a 1x3 kernel. This indicates a convolution operation with a 3x1 kernel. This represents the depth feature map of the third stage. Represents the expansion rate, where Indicates the vertical expansion rate. Indicates the horizontal expansion rate;

[0125] The second layer of directional dilated convolutional units applies a 3×1 convolution with a vertical dilation rate of 3 and a 1×3 convolution with a horizontal dilation rate of 8 to the first feature map. After batch normalization and activation by a modified linear unit activation layer, the second feature map still has 512 channels. The formula is as follows:

[0126] ;

[0127] in, Represents the second feature map;

[0128] The parameters of the third layer's directional dilated convolutional units are the same as those of the second layer, resulting in the third feature map, as shown in the following formula:

[0129] ;

[0130] in, Represents the third feature map;

[0131] The fourth layer of directional dilated convolutional units first applies a 3×1 convolution with a vertical dilation rate of 7 to the third feature map, followed by a 1×3 convolution with a horizontal dilation rate of 21. After batch normalization and activation by a modified linear unit activation layer, the fourth feature map is obtained, as shown in the following formula:

[0132] ;

[0133] in, Represents the fourth feature map;

[0134] After expanding the number of channels of the initial input third-stage depth feature map to 512, it is residually connected with the fourth feature map to obtain the fifth feature map. The fifth feature map is then used as the fourth-stage depth feature map of the depth branch, as shown in the following formula:

[0135] + ;

[0136] in, This represents the fifth feature map.

[0137] Further, in step S3, the color branch is segmented into image patches by the PatchEmbed module of the bimodal feature extraction module. The image patches are converted into embedded features. The embedded features are processed using the four base layers in the bimodal feature extraction module that include agent attention to generate agent tokens to strengthen feature associations, resulting in four stages of color feature maps; specifically:

[0138] Step S31: The input to the color branch is the original image. The original image is processed by image patch embedding through a convolutional layer with a kernel size of 4, a stride of 4, and 96 output channels. The original image is segmented into 4x4 pixel non-overlapping local blocks and mapped to 96-dimensional feature vectors. The 96-dimensional feature vectors are normalized and arranged according to their original spatial positions to form the original feature map. The original feature map is flattened to obtain the feature sequence, as shown in the following formula:

[0139] ;

[0140] in, Represents the characteristic sequence, This indicates an operation to arrange the elements according to their original spatial positions. This indicates a normalization operation. This represents a 4x4 convolution operation. This represents the original image input to the color branch;

[0141] Step S32: The feature sequence is input into a four-layer surrogate-shift window transformer network; each layer contains several surrogate-shift window transformer blocks; each surrogate-shift window transformer block uses three attention heads, and each surrogate-shift window transformer block processes the input feature sequence as follows:

[0142] Step S321: Normalize the input feature sequence by calculating the mean and variance along the feature dimension to obtain the first feature sequence, as shown in the following formula:

[0143] =LayerNorm(S) ;

[0144] in, Indicates the first characteristic sequence;

[0145] Step S322: The first feature sequence is reshaped into a two-dimensional feature map through a reshaping operation, as shown in the following formula:

[0146] = ReShape( );

[0147] in, Represents a two-dimensional feature map; ReShape represents the reshaping operation.

[0148] Step S323: Divide the two-dimensional feature map into multiple sub-feature maps according to a 7x7 window, and generate query features, key features and value features by performing a linear transformation on each sub-feature map;

[0149] Step S324: Obtain a query feature map based on the query features, perform a 7x7 adaptive average pooling operation on the query feature map, and divide the query feature map into 7 parts evenly according to the height and width of the two-dimensional feature map to obtain 49 regional representative features. Flatten the 49 regional representative features to obtain 49 proxy token sequences.

[0150] Each proxy token sequence has 96 feature dimensions. The three attention heads split the proxy token sequence, query features, key features, and value features equally according to the feature dimensions, that is, each attention head is responsible for 32 feature dimensions.

[0151] Step S325: Assign attention to sub-feature maps based on the key features of all sub-feature maps using the proxy token; then, calculate the attention of sub-feature maps to the proxy token using the query features of the sub-feature maps; calculate the attention weight from the proxy token to the sub-feature map and the attention weight from the sub-feature map to the proxy token, using the following formulas:

[0152] ;

[0153] ;

[0154] in, This represents the attention weights from the proxy token to the sub-feature map. This represents the attention weights from the sub-feature map to the proxy token. Indicates attention weights, Indicates a proxy token. Represents sub-feature maps, Represents the normalized exponential function, This represents the key features in the sub-feature map. and Indicates bias. This represents the query feature in the sub-feature map, and T represents the transpose.

[0155] Step S326: Perform matrix multiplication on the value features of the sub-feature maps using the attention weights from the proxy token to the sub-feature maps, and aggregate the features of all sub-feature maps to obtain the global association features guided by the proxy token, as shown in the following formula:

[0156] · ;

[0157] in, Indicates global correlation features. Represents the value features in the sub-feature graph;

[0158] The global associated features are back-aggregated using the attention weights from the sub-feature maps to the proxy token. This allows the local details of each sub-feature map to be fed back into the global associated features, resulting in aggregated features. The formula is as follows:

[0159] · ;

[0160] in, Indicates aggregation features;

[0161] Depthwise separable convolution is used on the aggregated features to extract local correlation information within the neighborhood of the sub-feature maps, resulting in aggregated enhanced features, as shown in the following formula:

[0162] DepthwiseConv( );

[0163] in, DepthwiseConv represents depthwise separable convolution, which represents aggregated enhanced features.

[0164] The aggregated enhanced features are residually concatenated with the input feature sequence to obtain the first comprehensive feature, as shown in the following formula:

[0165] = ;

[0166] in, Indicates the first comprehensive feature;

[0167] Step S327: Introducing a multilayer perceptron layer to enhance the nonlinear expressive power of the first comprehensive feature. The multilayer perceptron layer includes two fully connected layers, a Gaussian error linear unit activation function, and random deactivation regularization processing to obtain the second comprehensive feature, as shown in the following formula:

[0168] ;

[0169] in, Indicates the second comprehensive feature, and This represents two fully connected layers. This indicates random deactivation regularization. This represents the activation function of the Gaussian error linear unit;

[0170] Step S328: Perform a residual connection between the second synthesized feature and the first synthesized feature to obtain the final synthesized feature of the surrogate-shift window transformer block, as shown in the following formula:

[0171] = + ;

[0172] in, Indicates the final composite characteristics;

[0173] Step S33: After iterative operation of the two proxy-shift window transformer blocks in the first layer, the final integrated features are processed to obtain the first stage color feature map of the first layer output. The number of channels of the first stage color feature map is changed to 64 by 1x1 convolution.

[0174] The first-stage color feature map is iteratively processed by two surrogate-shift window transformer blocks in the second layer to obtain the second-stage color feature map output by the second layer. The number of channels of the second-stage color feature map is changed to 128 by 1x1 convolution.

[0175] The color feature map of the second stage is iteratively processed by the six proxy-shift window transformer blocks of the third layer to obtain the color feature map of the third stage output by the third layer. The number of channels of the color feature map of the third stage is changed to 256 by 1x1 convolution.

[0176] The color feature map of the third stage is iteratively processed by two surrogate-shift window transformer blocks in the fourth layer to obtain the color feature map of the fourth stage output by the fourth layer. The number of channels of the color feature map of the fourth stage is changed to 512 by 1x1 convolution.

[0177] Furthermore, in step S4, the depth feature map obtained in step S2 and the color feature map obtained in step S3 are input into the multi-dilation rate attention enhancement module for feature enhancement, resulting in depth-enhanced features and color-enhanced features; specifically:

[0178] Step S41: Transpose the dimensional order of the input fourth-stage depth feature map from batch size, number of channels, height, width to batch size, height, width, number of channels.

[0179] A 1x1 convolutional layer is used to process the transposed fourth-stage depth feature map, expanding the number of output channels to 512×3. Each 512 channel generates three sets of features: query features, key features, and value features, as shown in the following formula:

[0180] ;

[0181] in, This indicates query features, key features, and value features. This indicates the transpose operation. This represents the depth feature map of the fourth stage. Indicates batch size, height, width, and number of channels;

[0182] Step S42: Divide the three sets of features (query feature, key feature, and value feature) with 512 channels into two groups based on the average number of channels. For each group, use predefined expansion rate parameters, set to 2 and 3, to generate a first set of sub-features with an expansion rate of 2 and a second set of sub-features with an expansion rate of 3. Both the first and second sets of sub-features have 256 channels, as shown in the following formula:

[0183] = Split(Q,K,V) ;

[0184] in, Indicates the first group of sub-features, This represents the second set of sub-features; Split represents the segmentation operation.

[0185] Step S43: Perform multi-head expanded attention calculation in parallel on the first group of sub-features and the second group of sub-features;

[0186] Step S431: The first group of sub-features is further split according to the attention head to obtain 8 32-channel head branch features. The head branch features include head branch query features, head branch key features and head branch value features.

[0187] Step S432: Extract local region features from the head branch key features and head branch value features using a sliding window expansion operation with an expansion rate of 2 to obtain local key features and local value features; wherein, for an expansion rate of 2, sample the 3×3 neighborhood of each pixel at intervals of 2 to obtain 9 local key features and local value features for each pixel; for an expansion rate of 3, sample the neighborhood of the same size at intervals of 3.

[0188] Step S433: Perform matrix multiplication between the head branch query features and the expanded local key features to calculate the original similarity score. After adjusting for scaling factors, normalize the score using a normalized exponential function to obtain the attention weight, as shown in the following formula:

[0189] (h=1,2,...,8) ;

[0190] in, Indicates attention weights, Represents the normalized exponential function, This indicates the query characteristics of the header branch. This represents the local bond features after transposition. Indicates the scaling factor. =32; h represents the head dimension;

[0191] Step S434: Use attention weights to perform weighted fusion of the expanded local value features to obtain the fused local features, as shown in the following formula:

[0192] ;

[0193] in, Indicates the fusion of local features, Represents local value characteristics;

[0194] Step S435: The fused local features output from the 8 head branches are concatenated along the channel dimension to form a 256-channel feature map, resulting in an attention enhancement feature with an expansion rate of 2. The formula for the expansion rate of 2 is as follows:

[0195] ;

[0196] in, This indicates an attention enhancement feature with an expansion rate of 2. This indicates a splicing operation. This represents the fused local features output from the eight head branches;

[0197] Step S436: The second set of sub-features is based on steps S431 to S435 to obtain attention-enhanced features with an expansion rate of 3;

[0198] Step S44: The attention enhancement feature with an expansion rate of 2 and the attention enhancement feature with an expansion rate of 3 are concatenated along the channel dimension and fused through a projection layer to obtain the final feature, as shown in the following formula:

[0199] ;

[0200] in, Indicates the final feature, Indicates the projection layer. This indicates an attention-enhancing feature with an expansion rate of 3;

[0201] Step S45: Obtain the depth enhancement feature corresponding to the depth branch based on the output of the final feature;

[0202] Step S46: The color feature map of the fourth stage obtains the color enhancement features corresponding to the color branches based on steps S41 to S45.

[0203] Furthermore, in step S5, the depth enhancement features and color enhancement features are respectively input to the corresponding feature decoding modules for processing to obtain the depth branch fine features and the color branch fine features; specifically:

[0204] Step S51: Use a 1x1 convolutional layer to compress the number of channels of the input depth enhancement features and color enhancement features from 512 to 64, to obtain the basic decoding features of the depth branch and the basic decoding features of the color branch.

[0205] Step S52: Perform a three-level upsampling operation on the basic decoding features of the depth branch and the basic decoding features of the color branch through the upsampling module. The upsampling module consists of bilinear interpolation operation, effective feature enhancement unit and residual fusion operation.

[0206] Step S53: Perform the first upsampling operation on the basic decoding features of the depth branch: fuse the depth enhancement features with the depth feature map of the third stage of the depth branch to obtain the first depth branch upsampling result. The specific steps are as follows:

[0207] Step S531: Upsample the depth enhancement features to the same spatial size as the depth feature map in the third stage using bilinear interpolation to obtain the depth enhancement feature map;

[0208] Step S532: Use 1x1 convolution to compress the channels of the depth feature map in the third stage to obtain directional dilated convolution blocks; input the directional dilated convolution blocks into the effective feature enhancement unit for fine calibration. The effective feature enhancement unit first divides the directional dilated convolution blocks into K groups according to channels, performs adaptive pooling in the height and width directions on each group of features to capture the spatial context, and then activates them through 1×1 convolution and activation functions to obtain height attention weights and width attention weights, as shown in the following formula:

[0209] ;

[0210] ;

[0211] ;

[0212] in, This represents the k-th feature group. Indicates grouping operation. This represents a directionally dilated convolution block. This indicates that the group is divided into K groups. Indicates high attention weight, This represents the activation function. This represents a 1x1 convolution operation. This represents an adaptive pooling operation in the height direction. Indicates the width attention weight. This indicates an adaptive pooling operation in the width direction;

[0213] Step S533: Multiply the height attention weight and width attention weight element-wise with their corresponding k-th feature group, and then perform group normalization to obtain the weighted features. At the same time, perform a 3x3 convolution operation on the original grouped features to obtain local detail features, as shown in the following formula:

[0214] ;

[0215] ;

[0216] in, This represents the weighted characteristics. This indicates a group normalization operation. Indicates local details, This represents a 3x3 convolution operation;

[0217] Step S534: Perform global adaptive pooling to accelerate graphics port operations and normalized exponential activation on the weighted features, then flatten the local detail features, multiply the weighted features and local detail features by matrix, and finally obtain the channel-space interaction weights through an activation function, as shown in the following formula:

[0218] ;

[0219] in, Indicates the interaction weights between the channel and space. This represents the activation function. Represents the normalized exponential function, This indicates accelerated graphics port operation. Indicates the flattening operation;

[0220] Step S535: For the k-th feature group, first multiply the height attention weight and the width attention weight element by element to obtain the comprehensive spatial attention weight;

[0221] The final fusion weight is obtained by adding the integrated spatial attention weight to the aforementioned channel-space interaction weight.

[0222] The k-th feature group of the original grouped features is weighted element-wise using the final fusion weights to obtain the calibrated features of the k-th group. The calibrated features are then concatenated to obtain the final calibrated features, as shown in the following formula:

[0223] ;

[0224] ;

[0225] in, This represents the features of the k-th group after calibration. This represents the k-th feature group. Indicates the final calibration characteristics. Indicates splicing, This represents the features of each group after calibration;

[0226] Step S536: Add the upsampled depth enhancement feature map to the final calibration feature map element by element to obtain the first depth branch upsampling result, as shown in the following formula:

[0227] ;

[0228] in, This represents the result of the first deep branch upsampling. Represents a depth-enhanced feature map;

[0229] Step S54: The second and third upsampling operations are logically the same as the first upsampling operation;

[0230] The second upsampling operation is to fuse the first depth branch upsampling result with the depth feature map of the second stage of the depth branch to obtain the second depth branch upsampling result;

[0231] The third upsampling operation is to fuse the second depth branch upsampling result with the depth feature map of the first stage of the depth branch to obtain the third depth branch upsampling result;

[0232] Step S55: Based on steps S53 to S54, the basic decoding features of the color branch in step S52 are used to obtain the first color branch upsampling result, the second color branch upsampling result, and the third color branch upsampling result.

[0233] Step S56: Use the third depth branch upsampling result and the third color branch upsampling result as the depth branch fine features and color branch fine features, respectively.

[0234] Further, in step S6, the fine features of the deep branch and the fine features of the color branch are input into the dual-modal uncertainty fusion module; through the multi-scale confidence evidence component of the dual-modal uncertainty fusion module, fused evidence of the deep branch and fused evidence of the color branch are obtained; the fused evidence of the deep branch and the fused evidence of the color branch are converted into parameters of a Dirichlet distribution to obtain deep branch parameters and color branch parameters; the deep branch parameters and color branch parameters are fused using the Durmst-Schaffer evidence theory to obtain a comprehensive classification basis; the category probability of the comprehensive classification basis is calculated to obtain the final road segmentation map, specifically:

[0235] Step S61: Input the deep branch fine features and color branch fine features into the multi-scale evidence collection component to perform three convolution operations: 1x1 convolution, convolution with a kernel of 3x3 and a dilation rate of 3, and convolution with a kernel of 3x3 and a dilation rate of 6, to generate feature maps with three different receptive fields.

[0236] Step S62: The feature maps of the three different receptive fields are upsampled to restore the size of the initially read depth or color image, and activated by the activation function to obtain three single-scale evidences of the depth branch and three single-scale evidences of the color branch, respectively.

[0237] Step S63: Calculate the average of the three single-scale evidences from the depth branch and the three single-scale evidences from the color branch to obtain the fusion evidence for the depth branch and the fusion evidence for the color branch, as shown in the following formula:

[0238] ;

[0239] ;

[0240] in, This indicates evidence of fusion in deep branches. This represents the first single-scale evidence of the deep branch. This represents the second single-scale evidence indicating the deep branch. This represents the third single-scale piece of evidence representing the deep branch. Evidence of fusion of colored branches, This represents the first single-scale evidence of the color branch. This represents the second single-scale evidence representing the colored branch. This represents the third single-scale evidence representing the colored branch;

[0241] Step S64: Convert the fusion evidence of the deep branch and the fusion evidence of the color branch into parameters of a Dirichlet distribution. The conversion relationship is: the parameter of the Dirichlet distribution equals the evidence value plus 1, thus obtaining the deep branch parameters and the color branch parameters. Fuse the deep branch parameters and the color branch parameters to obtain the fusion parameters of the deep branch parameters and the color branch parameters, as shown in the following formula:

[0242] ;

[0243] in, This represents the fusion parameter between the depth branch parameter and the color branch parameter. Indicates the value of evidence;

[0244] Step S65: For the fusion parameters of the depth branch parameters and the color branch parameters, calculate the total confidence, relative evidence, and uncertainty of the depth branch parameters and the color branch parameters respectively, and calculate the evidence conflict coefficient between the depth branch parameters and the color branch parameters, as follows:

[0245] );

[0246] in, Indicates the conflict coefficient. This represents relative evidence for the depth branch parameters. This indicates relative evidence for the color branch parameters. This indicates element-wise multiplication. This indicates the extraction of diagonal elements from a matrix; Between depth branch parameters and color branch parameters ;

[0247] Step S66: Based on the conflict coefficient, calculate the joint relative evidence, joint uncertainty, and joint total confidence of the depth branch parameters and the color branch parameters, using the following formula:

[0248] ;

[0249] ;

[0250] ;

[0251] in, Indicates joint relative evidence, This indicates the uncertainty of the deep branch parameters. This indicates the uncertainty of the colored branch parameters. Indicating joint uncertainty, Indicates the joint total confidence level. This indicates the number of road categories, set to 2.

[0252] Step S67: Based on the joint relative evidence, joint uncertainty, and joint total confidence, calculate the fused bimodal joint evidence, as shown in the following formula:

[0253] +1;

[0254] in, This indicates joint evidence from two modalities;

[0255] Step S68: Based on dual-modal joint evidence, calculate the probability of each pixel belonging to the road category, and determine the pixels with a probability value greater than 0.5 as road areas to generate the final road segmentation mask.

[0256] Furthermore, in step S7, a total loss function is constructed based on the road segmentation model. This total loss function includes an evidence cross-entropy loss function, which provides hierarchical supervision of the multiple levels of evidence generated by the dual-modal uncertainty fusion module. Specifically:

[0257] Step S71: For given depth branch parameters and color branch parameters, the evidence cross-entropy loss function consists of a classification error term and an uncertainty regularization term, as shown in the following formula:

[0258] ;

[0259] ;

[0260] in, This indicates the adjusted parameters for the depth branch and color branch. include , , The label represents the actual road label, with a value of 1 for road areas and a value of 0 for non-road areas. Represents the evidence cross-entropy loss function. Represents pixels, Let S represent the logarithmic derivative of the gamma function, and let S represent the total confidence level of the evidence. Indicates the annealing coefficient. Denotes KL divergence, Represents a vector whose elements are all 1s;

[0261] Step S72: Perform hierarchical supervision on the three single-scale evidences of the depth branch and color branch, the fusion parameter of the depth branch parameters and the color branch parameters, and the dual-modal joint evidence using the evidence cross-entropy loss function; then obtain the total loss function by calculating the loss function one by one and weighted summing, as shown in the following formula:

[0262] ;

[0263] in, Represents the total loss function. Three single-scale pieces of evidence representing the depth branch and the color branch. This represents the fusion parameter between the depth branch parameter and the color branch parameter; the loss weight applied to the dual-modal joint evidence is 2, which is used to enhance the optimization of the final fusion result.

[0264] Example: Step 1: Data preparation and preprocessing;

[0265] Color and depth images are simultaneously acquired from publicly available autonomous driving road segmentation datasets. Both types of images undergo uniform standardization, and during the training phase, their centers are cropped to a fixed size of 1248×384 pixels to ensure that the input data for the color and depth modalities are completely consistent in spatial resolution. After preprocessing, aligned depth and color branch input data are generated and fed into the subsequent bimodal feature extraction module.

[0266] Step 2: Bimodal Feature Extraction; This step extracts features of the color modality and depth modality separately using an asymmetric dual-branch network structure;

[0267] Deep branch feature extraction: The input of the deep branch is first processed by a depth normal vector extraction component. This component converts the depth map into a 3D surface normal feature map that more intuitively reflects the road surface geometry based on the pixel coordinates of the depth image and camera intrinsic parameters. Subsequently, this normal feature map is input into an improved residual network. The first three layers of this network consist of standard residual blocks used to extract basic to intermediate geometric features. Its fourth layer is replaced by a custom directional dilated convolution block. This module uses 3x1 and 1x3 convolutions with different dilation rates to capture abrupt changes in the normals of road edges and obstacles in the vertical direction, and captures the long-distance continuity of the road surface normals in the horizontal direction. Finally, it outputs a feature map containing rich geometric structure information in four stages.

[0268] Color branch feature extraction: The input of the color branch first passes through a patch embedding module, which segments the image into 4x4 pixel blocks and maps them to 96-dimensional feature embeddings. Subsequently, the feature sequence passes through a four-layer agent-Swain transformer network. This network utilizes its unique agent attention mechanism to achieve global association modeling through agent tokens, efficiently capturing semantic context information across the entire image. Finally, the number of feature map channels of the four layers is adjusted to 64, 128, 256, and 512 respectively through 1×1 convolutions to align with the corresponding feature map channels of the deep branch.

[0269] Step 3: Enhance attention features;

[0270] The fourth-stage feature maps extracted from the deep and color branches, each with 512 channels and a spatial size of 39×12, are input together into the at most dilation rate attention enhancement module. This module first reshapes the feature maps and generates query, key, and value features using 1x1 convolutions. Then, based on preset dilation rate parameter groups, such as 2 and 3, the query, key, and value features are grouped and multi-head attention is calculated in parallel. Different dilation rates determine the range of the sampling neighborhood when calculating attention, thus simultaneously taking into account local details and global context. The module finally outputs optimized deep attention enhancement features and color attention enhancement features with enhanced road key region features.

[0271] Step 4: Feature Decoding and Detail Restoration;

[0272] The enhanced deep attention features and color attention features are input into two identical feature decoding modules. The module first compresses the number of channels to 64 using a 1x1 convolution. Then, it performs a three-level upsampling operation. Each upsampling level includes bilinear interpolation and an effective feature enhancement unit. This unit generates calibration features that combine spatial attention and channel-space interaction weights by fusing shallow features from the corresponding stages of the color branch, such as features from the first, second, and third stages. These features are then residually fused with the upsampled deep features. This process gradually restores the spatial resolution of the feature map and supplements multi-scale details from road contours and road surface textures to image edges. Finally, it outputs fine features of the deep branch and fine features of the color branch with the same size as the original input image.

[0273] Step 5: Uncertainty Fusion and Segmentation Decision Making;

[0274] The decoded deep branch fine features and color branch fine features are input into the bimodal uncertainty fusion module. This module first generates multi-scale evidence for each modality through a multi-scale confidence evidence collection component and averages the fused evidence for each modality. Then, based on evidence theory, the evidence is converted into Dirichlet distribution parameters to quantify the prediction uncertainty of each modality. Next, the core Durmst-Schaffer evidence fusion process is activated: it quantifies the evidence conflict between modalities by calculating the conflict coefficient and dynamically adjusts the fusion weights to calculate the joint bimodal evidence. Finally, based on the joint bimodal evidence, the probability of each pixel belonging to the road category is calculated, and the final binary road segmentation mask is generated with a threshold of 0.5.

[0275] Step Six: Model Training and Inference;

[0276] During the training phase, a custom evidence cross-entropy loss function is used. This loss function provides hierarchical supervision for the multiple types of evidence generated in step five—including unimodal multiscale evidence, unimodal fusion evidence, and bimodal joint evidence. By imposing constraints on these pieces of evidence, especially assigning higher weights to the final bimodal joint evidence, the road segmentation model is driven to simultaneously optimize the accuracy of road segmentation and its ability to quantify its own prediction uncertainty. The network parameters are iteratively updated by minimizing the total loss function. During the testing or application phase, the optimal model weights obtained from training are loaded, and steps one through five are performed on a new input image to obtain the final road segmentation result map.

[0277] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A road segmentation method based on bimodal feature uncertainty fusion, characterized in that, The method comprises the following steps: Step S1: reading a depth image and a color image from a reference data set, performing standardization and size cropping on the depth image and the color image, and obtaining dual-modality input data with consistent resolution, the dual-modality input data comprising a depth branch and a color branch; Step S2: inputting the dual-modality input data into a dual-modality feature extraction module; The depth branch is sequentially processed by a depth normal vector extraction component of the dual-modality feature extraction module and the first three layers of a depth residual network to obtain depth feature maps of the first three stages; The fourth layer of the depth residual network is replaced by a split-direction dilated convolution block, and the depth feature map of the third stage is processed by the split-direction dilated convolution block of the replaced fourth layer to obtain a depth feature map of the fourth stage; Step S3: the color branch is sequentially processed by an image embedding module and four basic layers containing proxy attention in the dual-modality feature extraction module to obtain color feature maps of the four stages; Step S4: inputting the depth feature map of the fourth stage obtained in step S2 and the color feature map of the fourth stage obtained in step S3 into a multi-dilation-rate attention enhancement module for feature enhancement to obtain depth enhanced features and color enhanced features; Step S5: inputting the depth enhanced features and the color enhanced features into corresponding feature decoding modules for processing to obtain fine depth branch features and fine color branch features; Step S6: inputting the fine depth branch features and the fine color branch features into a dual-modality uncertainty fusion module for processing to obtain a comprehensive classification basis; calculating the class probability of the comprehensive classification basis to obtain a final road segmentation map; Step S7: constructing a road segmentation model based on the dual-modality feature extraction module, the multi-dilation-rate attention enhancement module, the feature decoding module, and the dual-modality uncertainty fusion module; Constructing a total loss function based on the road segmentation model for hierarchical supervision; In step S7, the total loss function is constructed based on the road segmentation model, and the total loss function comprises an evidence cross-entropy loss function, which performs hierarchical supervision on multiple levels of evidence generated by the dual-modality uncertainty fusion module; specifically: Step S71: for given depth branch parameters and color branch parameters, the evidence cross-entropy loss function is composed of a classification error term and an uncertainty regularization term, and the formula is as follows: ； ； wherein, denotes the parameters of the depth branch and the color branch after adjustment, comprises , , , denotes three single-scale evidences of the depth branch and the color branch, denotes the fusion parameters of the depth branch parameters and the color branch parameters, denotes the bimodal joint evidence; label denotes the real label of the road, and the value of the road region is 1 and the value of the non-road region is 0; denotes the evidence cross-entropy loss function, denotes a pixel point, denotes the logarithmic derivative of the gamma function, and S denotes the total confidence of the evidence, denotes the annealing coefficient, denotes the KL divergence, denotes a vector with all elements being 1; Step S72: the evidence cross-entropy loss function is used to perform hierarchical supervision on three single-scale evidences of the depth branch and the color branch, fusion parameters of the depth branch parameters and the color branch parameters, and a dual-modality joint evidence; and then the total loss function is obtained by calculating the loss function one by one and weighted summation, and the formula is as follows: ； wherein, represents the total loss function, represents three single-scale evidences of the depth branch and the color branch, represents the fusion parameters of the depth branch parameters and the color branch parameters, represents the bimodal joint evidence; the loss weight imposed on the bimodal joint evidence is 2. 2.The road segmentation method based on dual-modal feature uncertainty fusion according to claim 1, characterized in that: In step S2, the depth branch is sequentially processed by a depth normal vector extraction component of the dual-modality feature extraction module and the first three layers of a depth residual network to obtain depth feature maps of the first three stages; the fourth layer of the depth residual network is replaced by a split-direction dilated convolution block, and the depth feature map of the third stage is processed by the split-direction dilated convolution block of the replaced fourth layer to obtain a depth feature map of the fourth stage; wherein the depth residual network is ResNet18; specifically: Step S21, the depth branch input is a depth image, the pixel value of each pixel point in the depth image is obtained through a depth normal vector extraction component, and the pixel value is divided by 1000 to obtain an actual depth value; the focal length in the horizontal direction and the focal length in the vertical direction and the principal point coordinates in the horizontal direction and the principal point coordinates in the vertical direction are obtained by using the camera intrinsic matrix parameters in the calibration file provided by the benchmark data set KITTI, and the pixel coordinates of the depth image are mapped into three-dimensional coordinates in the camera coordinate system; Step S22, the initial normal component in the horizontal direction and the initial normal component in the vertical direction of the three-dimensional coordinates are obtained by multiplying the change rate of each pixel point in the depth image in the horizontal direction and the vertical direction with the corresponding focal length in the horizontal direction and the vertical direction, and the initial normal component in the horizontal direction and the initial normal component in the vertical direction are calculated with the depth value to obtain the normal component in the depth value direction, and the normal component in the depth value direction is normalized to obtain a unit normal vector; Step S23, the unit normal vector is fused and direction corrected in 8 directions to obtain a three-dimensional surface normal feature map; Step S24, the three-dimensional surface normal feature map is input into the input preprocessing layer of the depth residual network of the dual-modal feature extraction module; The three-dimensional surface normal feature map is processed in turn through a 7x7 convolution layer with a step of 2 and padding of 3, a batch normalization layer, a rectified linear unit activation layer and a 3x3 maximum pooling layer, and the spatial size of the three-dimensional surface normal feature map is compressed to 1 / 4 of the original size to obtain an initial feature map adapted to the subsequent residual block; Step S25, the initial feature map is input into the first layer of the depth residual network, and the first layer is composed of 2 residual blocks, and the structure in each residual block is the same; the first residual block operates on the input initial feature map: The channel number of the initial feature map is compressed to 1 / 4 of the original channel number by using a 1x1 convolution layer, and the compressed initial feature map is subjected to a batch normalization layer and a rectified linear unit activation layer to obtain a first intermediate feature map; the first intermediate feature map is subjected to a 3x3 convolution layer to capture the local normal vector correlation, and the local normal vector correlation is subjected to a batch normalization layer and a rectified linear unit activation layer to obtain a second intermediate feature map; the second intermediate feature map is restored to the size of the initial feature map by using a 1x1 convolution layer, and the restored second intermediate feature map is connected with the initial feature map in residual to obtain a residual feature map; Step S26, after the residual feature map passes through the first layer of the depth residual network, a first-stage depth feature map is obtained; after the first-stage depth feature map is input into the second layer of the depth residual network, a second-stage depth feature map is obtained; after the second-stage depth feature map is input into the third layer of the depth residual network, a third-stage depth feature map is obtained; Step S27, the third-stage depth feature map is input into a directional dilated convolution block; the directional dilated convolution block is composed of 4 layers of directional dilated convolution units connected in turn. The third stage deep feature map is processed in the first layer of the split-branch dilated convolution unit, 3x1 convolution with a vertical dilation rate of 1 is used to capture the vertical abrupt change of the lane along the normal, and 1x3 convolution with a horizontal dilation rate of 5 is used to extract the continuity of the local lane normal, and after being activated by the batch normalization layer and the rectified linear unit activation layer, the first feature map is obtained; The second layer of the split-branch dilated convolution unit processes the first feature map by using 3x1 convolution with a vertical dilation rate of 3 and 1x3 convolution with a horizontal dilation rate of 8, and after being activated by the batch normalization layer and the rectified linear unit activation layer, the second feature map is obtained; The third layer of the split-branch dilated convolution unit has the same parameters as the second layer, and the third feature map is obtained; The fourth layer of the split-branch dilated convolution unit processes the third feature map by using 3x1 convolution with a vertical dilation rate of 7 and 1x3 convolution with a horizontal dilation rate of 21, and after being activated by the batch normalization layer and the rectified linear unit activation layer, the fourth feature map is obtained; The third stage deep feature map and the fourth feature map of the initial input are connected in residual, and the fifth feature map is obtained as the fourth stage deep feature map of the depth branch.

3. The road segmentation method based on bimodal feature uncertainty fusion according to claim 2, characterized in that: In step S3, the color branch is processed in sequence by the image embedding module in the dual-modal feature extraction module and four basic layers containing agent attention, and four stages of color feature maps are obtained; specifically: In step S31, the color branch inputs the original image, and the original image is processed by a convolution layer with a convolution kernel size of 4, a step size of 4, and an output channel number of 96 for image block embedding, the original image is divided into non-overlapping local blocks of 4x4 pixels and mapped into 96-dimensional feature vectors, the 96-dimensional feature vectors are normalized and arranged according to the original spatial position to form an original feature map, the original feature map is flattened to obtain a feature sequence; In step S32, the feature sequence is input into a four-layer proxy-shift window transformer network; each layer contains a plurality of proxy-shift window transformer blocks; each proxy-shift window transformer block uses 3 attention heads, and each proxy-shift window transformer block processes the input feature sequence, specifically: In step S321, the input feature sequence is normalized to calculate the mean and variance along the feature dimension to obtain a first feature sequence; In step S322, the first feature sequence is reshaped to obtain a two-dimensional feature map; In step S323, the two-dimensional feature map is divided into a plurality of sub-feature maps according to a 7x7 window, and each sub-feature map is processed by linear transformation to obtain query features, key features and value features; In step S324, a query feature map is obtained based on the query features, and a 7x7 adaptive average pooling operation is performed on the query feature map, which is evenly divided into 7 parts according to the height and width of the two-dimensional feature map to obtain 49 regional representative features, and the 49 regional representative features are flattened to obtain 49 proxy token sequences; The feature dimension of each proxy token sequence is 96, and the proxy token sequence, the query features, the key features and the value features are evenly split along the feature dimension by the 3 attention heads, i.e., each attention head is responsible for 32 feature dimensions. Step S325, assigning attention to the sub-feature maps by proxy tokens to the key features of all sub-feature maps; then calculating the attention of the sub-feature maps to the proxy tokens by the query features of the sub-feature maps; calculating the attention weight from the proxy token to the sub-feature map and calculating the attention weight from the sub-feature map to the proxy token; Step S326, using the attention weight from the proxy token to the sub-feature map to do matrix multiplication on the value features of the sub-feature map, and aggregating the features of all sub-feature maps to obtain proxy token guided global correlation features; Using the attention weight from the sub-feature map to the proxy token to perform reverse aggregation on the global correlation features, so that the local details of each sub-feature map are fed back to the global correlation features, and the aggregated features are obtained; Using a depth separable convolution on the aggregated features to extract local correlation information within the sub-feature map neighborhood, and obtaining aggregated enhanced features; Residual connection is performed between the aggregated enhanced features and the input feature sequence to obtain the first comprehensive feature; Step S327, introducing a multi-layer perceptron layer to improve the nonlinear expression ability of the first comprehensive feature, and the multi-layer perceptron layer includes two fully connected layers, a Gaussian error linear unit activation function, and a random deactivation regularization process, to obtain the second comprehensive feature; Step S328, performing residual connection between the second comprehensive feature and the first comprehensive feature to obtain the final comprehensive feature of the proxy-shift window transformer block; Step S33, after the final comprehensive feature is iteratively operated through the two proxy-shift window transformer blocks of the first layer, the first-stage color feature map output by the first layer is obtained; After the first-stage color feature map is iteratively operated through the two proxy-shift window transformer blocks of the second layer, the second-stage color feature map output by the second layer is obtained; After the second-stage color feature map is iteratively operated through the six proxy-shift window transformer blocks of the third layer, the third-stage color feature map output by the third layer is obtained; After the third-stage color feature map is iteratively operated through the two proxy-shift window transformer blocks of the fourth layer, the fourth-stage color feature map output by the fourth layer is obtained.

4. The road segmentation method based on bimodal feature uncertainty fusion according to claim 3, characterized in that: In step S4, the fourth-stage depth feature map obtained in step S2 and the fourth-stage color feature map obtained in step S3 are input into a multi-dilation rate attention enhancement module for feature enhancement to obtain depth enhanced features and color enhanced features; Specifically: Step S41, performing a transpose operation on the input fourth-stage depth feature map by changing the dimension order from batch size, channel number, height, and width to batch size, height, width, and channel number; Using a 1x1 convolution layer to process the transposed fourth-stage depth feature map, expanding the output channel number to 512x3, and obtaining three groups of query features, key features, and value features corresponding to each 512 channels; Step S42, dividing the three groups of query features, key features, and value features with a channel number of 512 into two groups according to the channel average, corresponding to two groups of predefined dilation rate parameters, and setting the parameters to 2 and 3 to generate a first group of sub-features corresponding to a dilation rate of 2 and a second group of sub-features corresponding to a dilation rate of 3; Step S43, multi-head expansion attention calculation is performed on the first group of sub-features and the second group of sub-features in parallel; Step S431, the first group of sub-features is further split by attention heads to obtain 8 head branch features each having 32 channels, the head branch features including a head branch query feature, a head branch key feature and a head branch value feature; Step S432, local region features are extracted from the head branch key feature and the head branch value feature by using a sliding window expansion operation with an expansion rate of 2 to obtain local key features and local value features; Step S433, the head branch query feature is multiplied by the expanded local key feature to calculate an original similarity score, which is adjusted by a scaling factor and normalized by a normalization exponential function to obtain an attention weight; Step S434, the expanded local value feature is weighted and fused by using the attention weight to obtain a fused local feature; Step S435, the fused local features output by the 8 head branches are spliced in the channel dimension to merge into a 256-channel feature map to obtain an attention enhanced feature with an expansion rate of 2; Step S436, the second group of sub-features obtains an attention enhanced feature with an expansion rate of 3 based on steps S431 to S435; Step S44, the attention enhanced feature with an expansion rate of 2 and the attention enhanced feature with an expansion rate of 3 are spliced in the channel dimension, and fused by a projection layer to obtain a final feature; Step S45, a depth enhanced feature corresponding to the depth branch is obtained according to the output of the final feature; Step S46, the color feature map of the fourth stage obtains a color enhanced feature corresponding to the color branch based on steps S41 to S45.

5. The road segmentation method based on bimodal feature uncertainty fusion according to claim 4, characterized in that: In step S5, the depth enhanced feature and the color enhanced feature are respectively input into the corresponding feature decoding module for processing to obtain the depth branch fine feature and the color branch fine feature; specifically: Step S51: using a 1x1 convolutional layer, the channel number of the input depth enhanced feature and color enhanced feature is compressed from 512 to 64 to obtain the depth branch basic decoding feature and the color branch basic decoding feature; Step S52: the depth branch basic decoding feature and the color branch basic decoding feature are subjected to three-level up-sampling operations by an up-sampling module composed of a bilinear interpolation operation, an effective feature enhancement unit and a residual fusion operation; Step S53: the depth branch basic decoding feature is subjected to a first up-sampling operation: the depth enhanced feature is fused with the depth feature map of the third stage of the depth branch to obtain a first depth branch up-sampling result, and the specific steps are as follows: Step S531: the depth enhanced feature is up-sampled to the same spatial size as the depth feature map of the third stage by bilinear interpolation to obtain a depth enhanced feature map; Step S532: Channel compression is performed on the third-stage depth feature map using a 1x1 convolution to obtain a split-direction dilated convolution block; the split-direction dilated convolution block is input into an effective feature enhancement unit for fine calibration, and the effective feature enhancement unit first divides the split-direction dilated convolution block into K groups according to channels, performs adaptive pooling in the height and width directions on each group of features to capture spatial context, and then respectively performs 1x1 convolution and activation function activation to obtain height attention weights and width attention weights; Step S533: The height attention weights and the width attention weights are multiplied element by element with the corresponding kth group of features, and then group normalization is performed to obtain weighted features, and 3x3 convolution is performed on the original feature group to obtain local detail features; Step S534: Global adaptive pooling acceleration port operation and normalized exponential function activation are performed on the weighted features, the local detail features are flattened, the weighted features and the local detail features are multiplied in matrix form, and then an activation function is used to obtain channel-space interaction weights; Step S535: For the kth group of feature groups, the height attention weights and the width attention weights are multiplied element by element to obtain comprehensive spatial attention weights; The comprehensive spatial attention weights and the above channel-space interaction weights are added to obtain final fusion weights; The final fusion weights are used for element-wise weighting operation on the kth group of feature groups of the original grouped features to obtain the kth group of calibrated features, and the calibrated features of each group are spliced to obtain final calibrated features; Step S536: The up-sampled depth enhancement feature map and the final calibrated features are added element by element to obtain the first depth branch up-sampling result; Step S54: The second and third up-sampling operations have the same logic as the first up-sampling operation; The second up-sampling operation is to fuse the first depth branch up-sampling result with the depth feature map of the second stage of the depth branch to obtain the second depth branch up-sampling result; The third up-sampling operation is to fuse the second depth branch up-sampling result with the depth feature map of the first stage of the depth branch to obtain the third depth branch up-sampling result; Step S55: The first, second, and third color branch up-sampling results are obtained based on the color branch basic decoding features in step S52 and steps S53 to S54; Step S56: The third depth branch up-sampling result and the third color branch up-sampling result are respectively taken as the depth branch fine feature and the color branch fine feature.

6. The road segmentation method based on bimodal feature uncertainty fusion according to claim 5, characterized in that: In step S6, the depth branch fine feature and the color branch fine feature are input into a dual-modal uncertainty fusion module for processing to obtain a comprehensive classification basis; the class probability of the comprehensive classification basis is calculated to obtain a final road segmentation map, which is specifically: Step S61: input the depth branch fine features and the color branch fine features into the multi-scale evidence collection component for three convolution operations: 1x1 convolution, convolution with a 3x3 kernel and a 3 rate of void, and convolution with a 3x3 kernel and a 6 rate of void, to generate feature maps of three different receptive fields; Step S62: restore the feature maps of three different receptive fields to the size of the originally read depth or color image through upsampling, and activate through an activation function to obtain three single-scale evidences of the depth branch and three single-scale evidences of the color branch respectively; Step S63: average the three single-scale evidences of the depth branch and the three single-scale evidences of the color branch respectively to obtain the fusion evidence of the depth branch and the fusion evidence of the color branch; Step S64: convert the fusion evidence of the depth branch and the fusion evidence of the color branch into the parameters of Dirichlet distribution, and the conversion relationship is: the parameter of Dirichlet distribution is equal to the evidence value plus 1, to obtain the depth branch parameters and the color branch parameters, fuse the depth branch parameters and the color branch parameters to obtain the fusion parameters of the depth branch parameters and the color branch parameters; Step S65: calculate the total confidence, relative evidence and uncertainty of the depth branch parameters and the color branch parameters respectively based on the fusion parameters of the depth branch parameters and the color branch parameters, and calculate the evidence conflict coefficient between the depth branch parameters and the color branch parameters; Step S66: calculate the joint relative evidence, joint uncertainty and joint total confidence of the depth branch parameters and the color branch parameters based on the conflict coefficient; Step S67: calculate the fused bimodal joint evidence based on the joint relative evidence, joint uncertainty and joint total confidence; Step S68: calculate the probability of each pixel belonging to the road category based on the bimodal joint evidence, and judge the pixels with a probability value greater than 0.5 as road regions to generate the final road segmentation mask.

Images