Surface terrain segmentation method and system based on double-trunk coding network model

Abstract

The invention provides a surface topography segmentation method and system based on a double-trunk coding network model, and relates to the technical field of image processing algorithms, and the method comprises the steps: determining a training sample set according to a Mars surface image; a double-trunk coding network model is constructed, the double-trunk coding network model is composed of a convolutional neural network branch and a Swin Transform branch in parallel, the multiple image samples are input into the double-trunk coding network model, and a Mars terrain semantic segmentation prediction map corresponding to each image sample is obtained; and training the double-trunk coding network model, inputting a to-be-detected Mars surface image into the trained double-trunk coding network model, and outputting a pixel-level semantic tag of the to-be-detected Mars surface image. According to the method, the terrain recognition precision and the boundary definition can be remarkably improved, the geological consistency of the segmentation result is enhanced, and technical support is provided for reliable environment perception and safe path planning of the Mars patroller.

Description

Technical Field

[0001] This invention relates to the field of image processing algorithm technology, and in particular to a method and system for surface terrain segmentation based on a dual-backbone coding network model. Background Technology

[0002] In deep space exploration missions, the autonomous environmental perception and safe path planning of Mars rovers heavily rely on the accurate identification of Martian surface terrain. Pixel-level semantic segmentation, as a core technology for terrain information extraction, needs to efficiently distinguish between various landform units such as Martian soil, sand, gravel, and bedrock, while meeting the requirements of real-time performance and robustness in complex environments. As Mars exploration missions expand towards higher resolution and more diverse scenarios, the surface images acquired by rovers exhibit characteristics such as diverse landform structures, significant scale differences, and intertwined visually similar categories, such as gravel and rock, sand and Martian soil. Traditional single-feature extraction frameworks are no longer suitable for such complex scenarios and cannot support the rover's reliable judgment of unknown terrain.

[0003] To address the needs of Martian terrain segmentation, existing technologies typically employ segmentation methods centered on convolutional neural networks (CNNs). These methods extract terrain texture and edge features through local convolutional kernels and multi-scale pooling operations. For example, the U-Net series models, which use ResNet as their backbone, retain shallow details through skip connections. However, due to the limitations of the local receptive field of convolutional kernels, while they can preserve detailed textures, they cannot effectively establish cross-regional terrain structure relationships. This results in shortcomings in global semantic modeling of large-scale landforms (such as continuous gravel areas and irregular bedrock zones). More critically, existing technologies generally treat Martian terrain categories as independent labels, failing to consider the inherent geological continuity between different landforms. This leads to significant confusion between visually similar categories, making it difficult to meet the terrain segmentation requirements of Mars rovers and severely restricting the reliability of autonomous navigation and environmental perception. Summary of the Invention

[0004] This invention provides a method and system for terrain segmentation based on a dual-backbone coding network model, which addresses the problems of incomplete preservation of local features, insufficient modeling of global dependencies, and severe confusion between similar categories in existing technologies. This invention significantly improves the accuracy and boundary clarity of terrain recognition by collaboratively extracting local detailed features and global contextual information, and introduces a feature relationship constraint mechanism to enhance category discrimination capabilities. This enhances the geological consistency of the segmentation results and provides technical support for reliable environmental perception and safe path planning for Mars rovers.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: Firstly, a method for terrain segmentation based on a dual-backbone coding network model is provided. This method includes: determining a training sample set based on Martian surface images, the training sample set including multiple image samples and pixel-level semantic labels corresponding to each image sample, the pixel-level semantic labels being used to identify the terrain category of each pixel in the image sample; constructing a dual-backbone coding network model, the dual-backbone coding network model consisting of a convolutional neural network branch and a Swin Transformer branch in parallel, the convolutional neural network branch being composed of multi-level asymptotic residual modules connected in series, the dilation rate of each asymptotic residual module increasing layer by layer with increasing network depth; and Swin... The Transformer branch alternately stacks window multi-head self-attention submodules and shift window multi-head self-attention submodules. The dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-structure decoding unit is used to restore the image spatial resolution through upsampling. Multiple image samples are input into the dual-backbone coding network model and processed by the multi-scale feature fusion unit and the U-structure decoding unit to obtain the Martian terrain semantic segmentation prediction map corresponding to each image sample. The Martian terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information that each pixel belongs to a different terrain category. Based on the objective loss function, the dual-backbone coding network model is trained according to the Martian terrain semantic segmentation prediction maps and pixel-level semantic labels corresponding to multiple image samples to obtain the trained dual-backbone coding network model. The objective loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The Martian surface image to be detected is input into the trained dual-backbone coding network model, and the pixel-level semantic labels of the Martian surface image to be detected are output.

[0006] The beneficial effects of this invention are as follows: The method provided by this invention, through the design of a dual-backbone coding network model, can perform complete and accurate segmentation of Martian terrain, solving the core defects of traditional single-branch networks: although traditional CNN branches can extract local terrain details, they are difficult to capture global semantic relationships; although traditional Transformer branches can model global semantics, their ability to extract local details is weak and their computational cost is high. The parallel design of the dual backbone allows the CNN branch to focus on local details and the Swing Transformer branch to focus on global semantics. The multi-scale fusion unit integrates the features of the two branches into dual-dimensional features of details and semantics. The U-shaped decoding unit restores the resolution consistent with the original image through upsampling. The composite loss function optimizes training from multiple dimensions such as category discrimination, semantic consistency, and boundary accuracy. Ultimately, the method can achieve high-detail localization and high-semantic-accuracy segmentation effects in complex Martian terrain scenes, meeting the core requirements of Mars rovers for surface environment perception.

[0007] In one possible implementation of the first aspect, the Martian surface images are derived from high-resolution images acquired by the onboard camera system of a Mars rover. Terrain categories include Martian soil, sand, gravel, bedrock, rock, shadows, rover tracks, background, and unknowns. A training sample set is determined based on the Martian surface images, including: cropping the Martian surface images to obtain multiple image samples of a preset size, the preset size being determined based on the input size of a dual-backbone coding network model; determining pixel-level semantic labels for each image sample among the multiple image samples; and performing data augmentation operations on the multiple image samples and the pixel-level semantic labels of each image sample to obtain the training sample set. The data augmentation operations include random horizontal flipping, random rotation within a preset angle range, random perturbation within a preset brightness range or a preset contrast range, and random scaling within a preset magnification range.

[0008] The method provided by this invention solves the key problems of low sample quality, insufficient diversity, and inconsistent size in Martian terrain segmentation by limiting the source of Martian surface images, nine core terrain categories, unified cropping processing, and training sample set construction rules with multiple types of data augmentation. High-resolution image sources ensure that the samples contain sufficient terrain details, providing high-quality raw data for subsequent feature extraction; the nine terrain categories fully cover typical scenes on the Martian surface, avoiding model bias caused by missing sample categories; unified cropping processing adapts the image sample size to the input requirements of the dual backbone network, eliminating training errors caused by inconsistent sizes; data augmentation operations such as random flipping, rotation, and brightness / contrast perturbation effectively expand sample diversity, alleviating the model overfitting problem caused by high cost and limited quantity of samples in Martian exploration. The final constructed training sample set has high quality, high coverage, and high diversity, laying a data foundation for the effective training of the dual backbone network and effectively improving the model's generalization ability.

[0009] In one possible implementation of the first aspect, the convolutional neural network branch contains 5 progressive residual modules. The dilation rate of the i-th progressive residual module, connected sequentially from shallow to deep according to network depth, is i, where i is a positive integer from 1 to 5. The i-th progressive residual module processes the (i-1)-th level detail feature map output from the (i-1)-th progressive residual module, outputting the i-th level detail feature map. The i-th progressive residual module includes a 3×3 dilated convolutional path, a 1×1 channel mapping path, and a feature fusion layer. A 3×3 dilated convolution path and a 1×1 channel mapping path are set in parallel. The 3×3 dilated convolution path is used to extract the terrain features within the receptive field corresponding to the input i-1 level detail feature map to obtain the first feature information. The 1×1 channel mapping path is used to perform channel dimension transformation and information supplementation on the input i-1 level detail feature map to obtain the second feature information. The feature fusion layer is used to add the first feature information and the second feature information element by element, and then perform feature fusion through 3×3 depthwise separable convolution to obtain the i-th level detail feature map.

[0010] The method provided by this invention solves the problems of poor scale adaptability, vanishing gradient, and high computational cost in traditional CNNs for Martian terrain feature extraction by refining the specific structure of the five progressive residual modules of the CNN branch. The five modules have an increasing dilation rate according to the network depth, so that the shallow modules (dilation rate 1-2) capture local details such as sand texture and gravel edges with a small receptive field, while the deep modules (dilation rate 3-5) capture macroscopic structures such as bedrock distribution and rock formations with a large receptive field, achieving full coverage of local and macroscopic multi-scale terrain features. In the dual parallel path, the 3×3 dilated convolution focuses on feature extraction, and the 1×1 channel mapping realizes channel dimension adaptation and information supplementation, avoiding information loss caused by a single path. The residual connection retains the basic information of the input features by adding elements one by one, effectively alleviating the gradient vanishing problem in deep network training. The 3×3 depthwise separable convolution significantly reduces the number of convolution parameters and computation while maintaining the feature fusion effect. In the end, the CNN branch can not only efficiently extract multi-scale terrain details, but also adapt to the limited computing resources of Mars exploration equipment, taking into account both feature extraction capability and engineering practicality.

[0011] In one possible implementation of the first aspect, in the Swin Transformer branch, the number of alternating stacked groups of window multi-head self-attention submodules and shifted window multi-head self-attention submodules is 5. The j-th stacked module, connected sequentially from shallow to deep according to network depth, is used to process the (j-1)-th level semantic feature map output by the input stacked module, outputting the j-th level semantic feature map, where j is a positive integer from 1 to 5. The j-th stacked module includes window multi-head self-attention submodules and shifted window multi-head self-attention submodules connected sequentially, and the window size of the window multi-head self-attention submodule and the shifted window multi-head self-attention submodule is 7×7. Among them, the window multi-head self-attention submodule is used to process the input (j-1)-th level semantic feature map. The semantic feature map is divided into non-overlapping 7×7 windows. Multi-head self-attention is independently calculated within each window to extract local semantic associations of pixels within the window, thus obtaining local semantic features. The shifted window multi-head self-attention submodule is used to perform window shifting processing on the local semantic features output by the window multi-head self-attention submodule, so that there is a partial overlap between adjacent windows. Multi-head self-attention is then calculated within the shifted window to establish global semantic associations across windows, thus obtaining global semantic features. The j-th stacking module is used to output the j-th level semantic feature map containing local semantic features and global semantic associations through the collaborative processing of the window multi-head self-attention submodule and the shifted window multi-head self-attention submodule.

[0012] The method provided by this invention addresses the problems of high computational cost and weak local associations in Martian terrain semantic extraction by clearly defining the specific structure of the five stacked modules in the Swing Transformer branch. The multi-head self-attention submodule calculates multi-head self-attention within a 7×7 non-overlapping window, focusing on capturing local semantic associations. Compared to the global attention of the traditional Transformer, the computational cost decreases with the square of the window size, significantly improving efficiency. The shifted window multi-head self-attention submodule creates overlapping areas between adjacent windows through window shifting, establishing cross-window global semantic associations based on local attention, thus overcoming the limitation of the traditional Transformer where computational power and global associations cannot be simultaneously achieved. The five stacked modules progressively enhance semantic extraction capabilities layer by layer according to depth, ensuring coherent modeling from shallow local semantics to deep global semantics. Ultimately, the Swing Transformer branch can efficiently extract global semantic associations of Martian terrain with lower computational cost, providing a macroscopic distribution basis for terrain category judgment and helping to solve the problem of weak semantic discrimination in single CNN branches.

[0013] In one possible implementation of the first aspect, the multi-scale feature fusion unit is specifically used to: receive five levels of detail feature maps output by a convolutional neural network branch and five levels of semantic feature maps output by a Swing Transformer branch, wherein the levels of the convolutional neural network branch and the Swing Transformer branch correspond one-to-one; adjust the number of channels of each level of detail feature map and semantic feature map to a preset number of channels through a channel dimension alignment operation; then perform element-level concatenation on the channel-aligned detail feature map and semantic feature map to obtain an initial fused feature map; process the initial fused feature map sequentially with a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation layer to output a level fused feature map corresponding to each of the five levels, wherein the level fused feature map contains terrain detail information and semantic association information at the corresponding scale.

[0014] The method provided by this invention solves the problems of scale mismatch, channel inconsistency, and feature redundancy in dual-branch feature fusion by refining the process of hierarchical one-to-one correspondence reception, channel dimension alignment, concatenation, 1×1 convolution, batch normalization, and ReLU activation of multi-scale feature fusion units. Specifically, it ensures that the fused features come from the same scale, such as the CNN's level 3 detail features corresponding to the Swin Transformer's level 3 semantic features, avoiding information misalignment caused by cross-scale fusion. The channel dimension alignment operation adjusts the features of the two branches to a preset number of channels, eliminating fusion conflicts caused by differences in channel dimensions. Element-level concatenation directly integrates the terrain detail information of the CNN with the semantic association information of the Swin, achieving preliminary fusion of detail and semantics. 1×1 convolution filters out feature redundancy after concatenation, batch normalization unifies the feature value distribution, and ReLU activation strengthens the effective feature response. The final output hierarchical fusion features possess both high detail resolution and strong semantic association, providing high-quality feature input for subsequent U-shaped structure decoding units, thus improving the accuracy and reliability of Mars terrain segmentation from the source.

[0015] In one possible implementation of the first aspect, the U-shaped decoding unit includes 5 levels of decoding modules, corresponding sequentially from deep to shallow network depth to the 5th to 1st level fusion feature maps output by the multi-scale feature fusion unit; the k-th level decoding module is used to: receive the k-th level fusion feature map, first upsample it by passing it through a deconvolution layer to increase its resolution to the (k-1)-th level feature map resolution; and then introduce the (k-1)-th level fusion feature map output by the multi-scale feature fusion unit through a skip connection, and then upsample it. The feature maps are then element-wise concatenated to obtain the concatenated feature map. After feature optimization by processing the concatenated feature map through a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation layer, the (k-1)th level decoded feature map is output. The upsampling, element-wise concatenation, and feature optimization operations are repeated to determine the first-level decoded feature map output by the first-level decoding module as the Mars terrain semantic segmentation prediction map. The size of the first-level decoded feature map is consistent with that of the input image sample, k is a positive integer from 2 to 5, and the decoding process starts from k=5 and decreases sequentially to k=2.

[0016] The method provided by this invention solves the problems of poor resolution recovery, loss of shallow details, and mismatch of predicted image size in traditional decoding processes by refining the specific workflow of the 5-level decoding module of the U-shaped structure decoding unit: the 5-level decoding module corresponds one-to-one with the 5-level features of the multi-scale fusion unit, decoding from deep low-resolution features (level 5) to shallow high-resolution features (level 1) step by step, ensuring the continuity of resolution recovery; 2x deconvolution upsampling accurately amplifies the feature map resolution, avoiding resolution distortion caused by traditional interpolation upsampling; skip connections introduce fusion features of the corresponding level, such as introducing level 4 fusion features when decoding level 4, to supplement the shallow details lost during upsampling; the combination of 3×3 convolution, batch normalization, and ReLU activation optimizes the stitched features, strengthening the effective signal and suppressing noise; the final output level 1 decoded feature map has the same size as the input image sample, ensuring that the segmentation prediction result can accurately correspond to each pixel of the original image, solving the core problem of pixel misalignment between the predicted image and the original image, and significantly improving the positioning accuracy of Mars terrain segmentation.

[0017] In one possible implementation of the first aspect, the target loss function for: ; in, As the first weighting coefficient, For relation-aware prototype regularization terms. This is the second weighting coefficient. For hierarchical feature aggregation terms, This is the third weighting coefficient. .

[0018] The method provided in this invention addresses the challenge of a single loss function simultaneously achieving class discrimination, multi-scale semantic consistency, and boundary segmentation accuracy by explicitly defining the weighted combination formula of the target loss function. The relation-aware prototype regularization term strengthens the model's ability to distinguish similar terrain categories by constraining the spacing of terrain category feature prototypes. The hierarchical feature aggregation term ensures semantic consistency across different levels of features by aggregating multi-scale features from the encoder and decoder. The boundary-weighted Dice term improves the model's segmentation accuracy for terrain boundary regions by weighting the terrain boundary pixels. Three weight coefficients balance the influence of each loss term, preventing any single loss term from excessively dominating training. Ultimately, the target loss function guides model optimization from three core dimensions: class discrimination, semantic consistency, and boundary accuracy, significantly reducing the misclassification rate and boundary error rate in Martian terrain segmentation.

[0019] Secondly, this invention provides a surface terrain segmentation system based on a dual-backbone coding network model. The system includes: a sample generation module for determining a training sample set based on Martian surface images. The training sample set includes multiple image samples and pixel-level semantic labels corresponding to each image sample. The pixel-level semantic labels are used to identify the terrain category of each pixel in the image sample. A model construction module is used to construct a dual-backbone coding network model, which consists of a convolutional neural network branch and a Swin Transformer branch in parallel. The convolutional neural network branch is composed of multi-level asymptotic residual modules connected in series, and the dilation rate of each asymptotic residual module increases layer by layer according to the network depth. The Transformer branch alternately stacks window multi-head self-attention submodules and shift-window multi-head self-attention submodules; the dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-structure decoding unit is used to restore the image spatial resolution through upsampling; the terrain segmentation module is used to input multiple image samples into the dual-backbone coding network model, and through the multi-scale feature fusion unit and the U-structure decoding unit, it processes them to obtain the Mars terrain semantic segmentation prediction map corresponding to each image sample. The mapping and corresponding image sample sizes are consistent, and each pixel contains prediction information indicating that it belongs to a different terrain category. The model training module is used to train the dual-backbone coding network model based on the target loss function, using the Martian terrain semantic segmentation prediction map and pixel-level semantic labels corresponding to multiple image samples. The target loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The terrain segmentation module is also used to input the Martian surface image to be detected into the trained dual-backbone coding network model and output the pixel-level semantic labels of the Martian surface image to be detected.

[0020] Thirdly, an electronic device is provided, the electronic device including a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any implementation of the first aspect.

[0021] Fourthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform a method as described in any implementation of the first aspect.

[0022] Fifthly, a computer program product is provided that, when run on a computer, causes the computer to perform the method in any implementation of the first aspect.

[0023] Understandably, the beneficial effects achieved by the system of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to with reference to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; Figure 2 A flowchart of a surface terrain segmentation method based on a dual-backbone coding network model provided in an embodiment of the present invention; Figure 3 A flowchart of another surface terrain segmentation method based on a dual-backbone coding network model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a dual-backbone coding network model provided in an embodiment of the present invention; Figure 5 An experimental result diagram of an ablation experiment with different weighting coefficients provided in an embodiment of the present invention; Figure 6 This is a display diagram showing the feature map results of each layer in the ablation experiment provided in the embodiments of the present invention; Figure 7 A schematic diagram of the results of a comparative experiment provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a terrain segmentation system provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the present invention, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. The "or" in the present invention is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Furthermore, in the description of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items.

[0026] Furthermore, to facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0027] In this embodiment of the invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this embodiment of the invention should not be construed as superior or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0028] In deep space exploration missions, the autonomous environmental perception and safe path planning of Mars rovers heavily rely on the accurate identification of Martian surface terrain. Pixel-level semantic segmentation, as a core technology for terrain information extraction, needs to efficiently distinguish between various landform units such as Martian soil, sand, gravel, and bedrock, while meeting the requirements of real-time performance and robustness in complex environments. As Mars exploration missions expand towards higher resolution and more diverse scenarios, the surface images acquired by rovers exhibit characteristics such as diverse landform structures, significant scale differences, and intertwined visually similar categories, such as gravel and rock, sand and Martian soil. Traditional single-feature extraction frameworks are no longer suitable for such complex scenarios and cannot support the rover's reliable judgment of unknown terrain.

[0029] To address the needs of Martian terrain segmentation, existing technologies typically employ segmentation methods centered on convolutional neural networks (CNNs). These methods extract terrain texture and edge features through local convolutional kernels and multi-scale pooling operations. For example, the U-Net series models, which use ResNet as their backbone, retain shallow details through skip connections. However, due to the limitations of the local receptive field of convolutional kernels, while they can preserve detailed textures, they cannot effectively establish cross-regional terrain structure relationships. This results in shortcomings in global semantic modeling of large-scale landforms (such as continuous gravel areas and irregular bedrock zones). More critically, existing technologies generally treat Martian terrain categories as independent labels, failing to consider the inherent geological continuity between different landforms. This leads to significant confusion between visually similar categories, making it difficult to meet the terrain segmentation requirements of Mars rovers and severely restricting the reliability of autonomous navigation and environmental perception.

[0030] In view of this, embodiments of the present invention provide a method and system for surface terrain segmentation based on a dual-backbone coding network model. The method includes: determining a training sample set based on Martian surface images, the training sample set including multiple image samples and pixel-level semantic labels corresponding to each image sample, the pixel-level semantic labels being used to identify the terrain category of each pixel in the image sample; constructing a dual-backbone coding network model, the dual-backbone coding network model consisting of a convolutional neural network branch and a SwinTransformer branch in parallel, the convolutional neural network branch being composed of multi-level asymptotic residual modules connected in series, the dilation rate of each asymptotic residual module increasing layer by layer with increasing network depth; Swin... The Transformer branch alternately stacks window multi-head self-attention submodules and shift window multi-head self-attention submodules. The dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-structure decoding unit is used to restore the image spatial resolution through upsampling. Multiple image samples are input into the dual-backbone coding network model and processed by the multi-scale feature fusion unit and the U-structure decoding unit to obtain the Martian terrain semantic segmentation prediction map corresponding to each image sample. The Martian terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information that each pixel belongs to a different terrain category. Based on the objective loss function, the dual-backbone coding network model is trained according to the Martian terrain semantic segmentation prediction maps and pixel-level semantic labels corresponding to multiple image samples to obtain the trained dual-backbone coding network model. The objective loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The Martian surface image to be detected is input into the trained dual-backbone coding network model, and the pixel-level semantic labels of the Martian surface image to be detected are output.

[0031] The method provided by this invention, through the design of a dual-backbone coding network model, can achieve complete and accurate segmentation of Martian terrain, solving the core defects of traditional single-branch networks: while traditional CNN branches can extract local terrain details, they struggle to capture global semantic relationships; while traditional Transformer branches can model global semantics, their ability to extract local details is weak and computationally intensive. The parallel design of the dual-backbone network allows the CNN branch to focus on local details and the Swing Transformer branch to focus on global semantics. A multi-scale fusion unit integrates the features from both branches into dual-dimensional features of detail and semantics. A U-shaped decoding unit restores the resolution consistent with the original image through upsampling. A composite loss function optimizes training from multiple dimensions, including category discrimination, semantic consistency, and boundary accuracy. Ultimately, this method enables high-detail localization and high-semantic-accuracy segmentation in complex Martian terrain scenes, meeting the core requirements of Mars rovers for surface environment perception.

[0032] In some embodiments, the surface terrain segmentation method based on a dual-backbone coding network model provided by the present invention can be executed by a surface terrain segmentation system 100 based on a dual-backbone coding network model (hereinafter referred to as terrain segmentation system 100).

[0033] As an example, the terrain segmentation system 100 can be any electronic device 200 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer. The specific implementation of the terrain segmentation system 100 is not limited here.

[0034] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown. The electronic device 200 includes a processor 210, a memory 220, and a communication interface 230.

[0035] Processor 210 may include one or more processing cores. Processor 210 connects to various parts within electronic device 200 using various interfaces and lines, and performs various functions and processes data of electronic device 200 by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one of the following hardware forms: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0036] The memory 220 may include random access memory (RAI) or read-only memory (ROI). Optionally, the memory 220 may include non-transitory computer-readable storage ledger. The memory 220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a stored program area. The stored program area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc.

[0037] Communication interface 230 is used to communicate with other devices, equipment or communication networks, such as data storage devices, image processing devices or Ethernet, wireless access network (RAN), wireless local area network (WLAN), etc.

[0038] In terms of physical implementation, the aforementioned devices (such as processor 210, memory 220, and communication interface 230) can each be devices within the same device (such as a laptop computer). Alternatively, at least two of these devices can be located within the same device, i.e., as different devices within the same device, similar to the deployment of devices or components in a distributed system.

[0039] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments of the present invention, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0040] The following description, in conjunction with the accompanying drawings, illustrates a method for surface terrain segmentation based on a dual-backbone coding network model provided by an embodiment of the present invention.

[0041] Figure 2 This is a flowchart illustrating a surface terrain segmentation method based on a dual-backbone coding network model, provided as an embodiment of the present invention. Optionally, this method can be... Figure 1 The illustrated electronic device 200 performs this function. The method may include the following steps: S1. Determine the training sample set based on the Martian surface images. The training sample set includes multiple image samples and pixel-level semantic labels corresponding to each image sample.

[0042] Among them, pixel-level semantic labels are used to identify the terrain category of each pixel in the image sample; In some embodiments, the Martian surface images are derived from high-resolution images acquired by the onboard camera system of the Mars rover, and the terrain categories include Martian soil, sand, gravel, bedrock, rock, shadows, rover tracks, background, and unknowns.

[0043] Further, see Figure 3 The aforementioned S1 specifically includes: S11. Perform cropping processing on the Mars surface image to obtain multiple image samples of a preset size. The preset size is determined based on the input size of the dual backbone coding network model.

[0044] In one example, the default size is 256×256 pixels.

[0045] S12. Determine the pixel-level semantic label for each image sample among multiple image samples.

[0046] Specifically, pixel-level semantic labels are generated by a combination of manual annotation and geological feature verification. First, geological technicians annotate the terrain categories of the image samples pixel by pixel according to the Martian terrain and geological standards. Then, the annotation results are verified and corrected by environmental parameters transmitted back by the Mars rover, such as surface hardness and spectral reflectance, to ensure that the matching degree between the pixel-level semantic labels and the actual terrain is not less than 95%.

[0047] S13. Perform data augmentation operations on multiple image samples and the pixel-level semantic labels of each image sample to obtain the training sample set.

[0048] The data enhancement operations include random horizontal flipping, random rotation within a preset angle range, random perturbation within a preset brightness range or preset contrast range, and random scaling within a preset magnification range.

[0049] In one example, the preset angle range is ±15°, the preset brightness range is ±20%, the preset contrast range is ±15%, and the preset magnification range is 0.8-1.2 times.

[0050] It should be noted that the above data augmentation operations are merely illustrative examples, and the specific implementation methods of the data augmentation operations in this embodiment of the invention are not particularly limited.

[0051] The method provided by this invention solves the key problems of low sample quality, insufficient diversity, and inconsistent size in Martian terrain segmentation by limiting the source of Martian surface images, nine core terrain categories, unified cropping processing, and training sample set construction rules with multiple types of data augmentation. High-resolution image sources ensure that the samples contain sufficient terrain details, providing high-quality raw data for subsequent feature extraction; the nine terrain categories fully cover typical scenes on the Martian surface, avoiding model bias caused by missing sample categories; unified cropping processing adapts the image sample size to the input requirements of the dual backbone network, eliminating training errors caused by inconsistent sizes; data augmentation operations such as random flipping, rotation, and brightness / contrast perturbation effectively expand sample diversity, alleviating the model overfitting problem caused by high cost and limited quantity of samples in Martian exploration. The final constructed training sample set has high quality, high coverage, and high diversity, laying a data foundation for the effective training of the dual backbone network and effectively improving the model's generalization ability.

[0052] S2. Construct a dual-backbone coding network model, which consists of a convolutional neural network branch and a SwinTransformer branch in parallel.

[0053] Specifically, the convolutional neural network branch is composed of multi-level asymptotic residual modules connected in series, and the inflation rate of each asymptotic residual module increases layer by layer with network depth; the Swing Transformer branch uses window multi-head self-attention sub-modules and shift window multi-head self-attention sub-modules stacked alternately; the dual backbone coding network model also includes a multi-scale feature fusion unit and a U-shaped structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-shaped structure decoding unit is used to restore the spatial resolution of the image through upsampling.

[0054] In the network implementation, the convolutional branch uses PRB as the backbone, and the Transformer branch uses Swin-Transformer as the main branch. The multi-scale features output by both are aligned at each coding layer for subsequent fusion.

[0055] In some embodiments, the number of progressive residual modules in the convolutional neural network branch is 5. The dilation rate of the i-th progressive residual module, which is connected sequentially from shallow to deep according to the network depth, is i, where i is a positive integer from 1 to 5. For the i-th progressive residual module, it is used to process the i-1 level detail feature map output by the input i-1 progressive residual module to output the i-th level detail feature map. The i-th progressive residual module includes a 3×3 dilated convolution path, a 1×1 channel mapping path, and a feature fusion layer. The 3×3 dilated convolution path and the 1×1 channel mapping path are set in parallel. The 3×3 dilated convolution path is used to extract the terrain features within the receptive field corresponding to the input i-1 level detail feature map to obtain the first feature information. The 1×1 channel mapping path is used to perform channel dimension transformation and information supplementation on the input i-1 level detail feature map to obtain the second feature information. The feature fusion layer is used to add the first feature information and the second feature information element by element, and then perform feature fusion through 3×3 depthwise separable convolution to obtain the i-th level detail feature map.

[0056] Specifically, the convolutional neural network branch uses progressive residual blocks (PRBs) cascaded together to form the encoder. Each PRB contains two parallel paths: the first path is a 3×3 dilated convolution for feature extraction; the second path is a 1×1 convolution for channel transformation and feature supplementation. The convolution result is added to the input residual and then fused using depthwise separable convolution, thereby enhancing expressive power while maintaining feature map resolution. Different layers of PRBs use progressively increasing dilation rates, with the five PRBs having dilation rates of 1, 2, 3, 4, and 5 respectively. This progressive dilation design allows the network's receptive field to gradually expand with encoding depth: shallow layers focus on capturing the edges and rock texture details of the Martian surface, while deeper layers focus on larger-scale terrain structures and spatial distribution, thus achieving a progressive fusion of local features and global structure, improving the model's feature representation ability and stability in complex terrain.

[0057] In one example, the structure and processing flow of each progressive residual module are as follows: The progressive residual module receives the detail feature map output by the previous module (or the detail feature map generated after preprocessing the initial image sample, which is equivalent to the detail feature map corresponding to the image sample after preprocessing received by the first-level residual module) as input. The input detail feature map contains the basic terrain features of the corresponding level. The progressive residual module has two parallel processing paths. The first path is a 3×3 dilated convolution path: using a 3×3 convolution kernel with a dilation rate matching the current module, the convolution operation is performed on the input detail feature map to extract the terrain features within the corresponding receptive field. Among them, the shallow progressive residual module (dilation rate 1, 2) captures detailed features such as sand texture and gravel edges through a small receptive field, while the deep progressive residual module (dilation rate 3, 4, 5) captures macroscopic geomorphic structural features such as bedrock distribution and rock swarms through a large receptive field. The second path is a 1×1 channel mapping path: a 1×1 convolution kernel is used to perform convolution operations on the input detailed feature map, compressing or expanding the channel dimension of the input features (such as mapping the input 64-channel features to 32-channel or 128-channel), realizing channel dimension adaptation and feature information supplementation, and avoiding information loss caused by a single convolution path. After the two parallel paths output features (first feature information and second feature information) respectively, the output features of the two paths are first added element-wise (i.e., residual connection, retaining the basic information of the input features and alleviating gradient vanishing), and then the added features are input to a 3×3 depth separable convolutional layer. The 3×3 depthwise separable convolutional layer is an inherent component of the progressive residual module. It is used to perform spatial dimension feature fusion and redundant information filtering on the features after element-wise addition. While maintaining the spatial resolution of the feature map, it reduces the number of parameters and computational complexity of the convolution operation, while enhancing the effective representation of morphological features. Finally, each progressive residual module outputs a detailed feature map that has undergone parallel convolution extraction, residual addition, and depthwise separable fusion. The detailed feature map is used to be transmitted to the next progressive residual module for further processing, or to be transmitted to the multi-scale feature fusion unit and the corresponding level of the semantic feature map of the Swin Transformer branch for fusion.

[0058] The method provided by this invention solves the problems of poor scale adaptability, vanishing gradient, and high computational cost in traditional CNNs for Martian terrain feature extraction by refining the specific structure of the five progressive residual modules of the CNN branch. The five modules have an increasing dilation rate according to the network depth, so that the shallow modules (dilation rate 1-2) capture local details such as sand texture and gravel edges with a small receptive field, while the deep modules (dilation rate 3-5) capture macroscopic structures such as bedrock distribution and rock formations with a large receptive field, achieving full coverage of local and macroscopic multi-scale terrain features. In the dual parallel path, the 3×3 dilated convolution focuses on feature extraction, and the 1×1 channel mapping realizes channel dimension adaptation and information supplementation, avoiding information loss caused by a single path. The residual connection retains the basic information of the input features by adding elements one by one, effectively alleviating the gradient vanishing problem in deep network training. The 3×3 depthwise separable convolution significantly reduces the number of convolution parameters and computation while maintaining the feature fusion effect. In the end, the CNN branch can not only efficiently extract multi-scale terrain details, but also adapt to the limited computing resources of Mars exploration equipment, taking into account both feature extraction capability and engineering practicality.

[0059] In other embodiments, in the Swing Transformer branch, the number of alternating stacked window multi-head self-attention submodules and shifted window multi-head self-attention submodules is 5. The j-th stacked module, connected sequentially from shallow to deep network depth, is used to process the (j-1)-th level semantic feature map output by the (j-1)-th stacked module, outputting the j-th level semantic feature map, where j is a positive integer from 1 to 5. The j-th stacked module includes sequentially connected window multi-head self-attention submodules and shifted window multi-head self-attention submodules, and the window size of the window multi-head self-attention submodule and the shifted window multi-head self-attention submodule is 7×7. Among them, the window multi-head self-attention submodule is used to process the input (j-1)-th level semantic feature map. The semantic feature map is divided into non-overlapping 7×7 windows. Multi-head self-attention is independently calculated within each window to extract local semantic associations of pixels within the window, thus obtaining local semantic features. The shifted window multi-head self-attention submodule is used to perform window shifting processing on the local semantic features output by the window multi-head self-attention submodule, so that there is a partial overlap between adjacent windows. Multi-head self-attention is then calculated within the shifted window to establish global semantic associations across windows, thus obtaining global semantic features. The j-th stacking module is used to output the j-th level semantic feature map containing local semantic features and global semantic associations through the collaborative processing of the window multi-head self-attention submodule and the shifted window multi-head self-attention submodule.

[0060] Specifically, the Swing Transformer branch employs a sliding window self-attention mechanism (Swin TransformerBlock), alternately stacking window multi-head self-attention (W-MSA) and shifted window multi-head self-attention (SW-MSA) modules. This structure enables cross-regional global information interaction while maintaining local sensitivity, thereby obtaining the spatial layout and semantic relationships of the terrain.

[0061] In one example, the structure and processing flow of each stacked module group are as follows: The j-th stacked module receives the (j-1)-th level semantic feature map output by the previous stacked module (j-1) as input. The input feature map contains preliminary semantic association information of the terrain at the corresponding level (such as the category correlation of local pixels). The j-th stacked module is connected in sequence according to the order of window multi-head self-attention (W-MSA) sub-module → shift window multi-head self-attention (SW-MSA) sub-module. The window size of the two sub-modules is fixed at 7×7, and the number of multi-head attention heads is set to 4 to ensure the consistency and efficiency of feature extraction.

[0062] First, the input (j-1)th level semantic feature map enters the W-MSA submodule: The W-MSA submodule first divides the feature map into non-overlapping rectangular windows with a window size of 7×7 (e.g., when the input feature map resolution is 128×128, it is divided into 16×16 independent windows). Multi-head self-attention is calculated independently in each window. By operating on the similarity matrix of pixel features within the window, local semantic associations within the same window are captured. For example, in a window containing gravel and sand, the category consistency between gravel pixels is strengthened, and the semantic boundaries between gravel and sand are distinguished. Finally, an intermediate feature map containing local semantic associations is output.

[0063] Next, the intermediate feature map output by the W-MSA submodule enters the SW-MSA submodule: the submodule first performs window shifting on the intermediate feature map, with the shifting method set to shift 3 pixels horizontally and 3 pixels vertically, so that the originally non-overlapping adjacent windows produce a 3×3 pixel overlapping area; then, the multi-head self-attention is recalculated in the shifted window, and pixel associations across windows are established using the overlapping area. For example, the semantic features of the gravel group window are associated with the adjacent bedrock window to capture the global distribution pattern of gravel around the bedrock, filter out semantic misjudgments in isolated windows, and output an intermediate feature map containing global semantic associations.

[0064] The output features of both sub-modules are processed by layer normalization (LN) and residual connections: the input features of each sub-module are first normalized to unify the numerical distribution, and then enter the attention calculation stage. The calculation result is added element-wise with the original input of the sub-module (residual connection) to preserve basic semantic information and alleviate the gradient vanishing problem in deep network training. Finally, the j-th stacked module outputs a j-th level semantic feature map that has undergone local semantic extraction, cross-window global association, and normalized residual optimization. The resolution of the layer feature map is consistent with that of the input j-1 level semantic feature map (e.g., 128×128→128×128), which is used to transmit to the next stacked module to further enhance the semantic features, or to transmit to the multi-scale feature fusion unit to align and fuse with the j-th level feature map of the convolutional neural network branch.

[0065] The method provided by this invention addresses the problems of high computational cost and weak local associations in Martian terrain semantic extraction by clearly defining the specific structure of the five stacked modules in the Swing Transformer branch. The multi-head self-attention submodule calculates multi-head self-attention within a 7×7 non-overlapping window, focusing on capturing local semantic associations. Compared to the global attention of the traditional Transformer, the computational cost decreases with the square of the window size, significantly improving efficiency. The shifted window multi-head self-attention submodule creates overlapping areas between adjacent windows through window shifting, establishing cross-window global semantic associations based on local attention, thus overcoming the limitation of the traditional Transformer where computational power and global associations cannot be simultaneously achieved. The five stacked modules progressively enhance semantic extraction capabilities layer by layer according to depth, ensuring coherent modeling from shallow local semantics to deep global semantics. Ultimately, the Swing Transformer branch can efficiently extract global semantic associations of Martian terrain with lower computational cost, providing a macroscopic distribution basis for terrain category judgment and helping to solve the problem of weak semantic discrimination in single CNN branches.

[0066] In some other embodiments, the multi-scale feature fusion unit is specifically used to: receive five levels of detail feature maps output by a convolutional neural network branch and five levels of semantic feature maps output by a Swing Transformer branch, wherein the levels of the convolutional neural network branch and the Swing Transformer branch correspond one-to-one; adjust the number of channels of each level of detail feature map and semantic feature map to a preset number of channels through a channel dimension alignment operation; then perform element-level concatenation on the channel-aligned detail feature map and semantic feature map to obtain an initial fused feature map; process the initial fused feature map sequentially with a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation layer to output a level fused feature map corresponding to each of the five levels, wherein the level fused feature map contains terrain detail information and semantic association information at the corresponding scale.

[0067] The method provided by this invention solves the problems of scale mismatch, channel inconsistency, and feature redundancy in dual-branch feature fusion by refining the process of hierarchical one-to-one correspondence reception, channel dimension alignment, concatenation, 1×1 convolution, batch normalization, and ReLU activation of multi-scale feature fusion units. Specifically, it ensures that the fused features come from the same scale, such as the CNN's level 3 detail features corresponding to the Swin Transformer's level 3 semantic features, avoiding information misalignment caused by cross-scale fusion. The channel dimension alignment operation adjusts the features of the two branches to a preset number of channels, eliminating fusion conflicts caused by differences in channel dimensions. Element-level concatenation directly integrates the terrain detail information of the CNN with the semantic association information of the Swin, achieving preliminary fusion of detail and semantics. 1×1 convolution filters out feature redundancy after concatenation, batch normalization unifies the feature value distribution, and ReLU activation strengthens the effective feature response. The final output hierarchical fusion features possess both high detail resolution and strong semantic association, providing high-quality feature input for subsequent U-shaped structure decoding units, thus improving the accuracy and reliability of Mars terrain segmentation from the source.

[0068] Furthermore, the U-shaped decoding unit contains 5 levels of decoding modules, corresponding sequentially from the deepest to the shallowest network depth to the 5th to 1st level fusion feature maps output by the multi-scale feature fusion unit. The k-th level decoding module is used to: receive the k-th level fusion feature map, first upsample it by a deconvolutional layer to increase its resolution to the k-1 level feature map resolution; introduce the k-1 level fusion feature map output by the multi-scale feature fusion unit through a skip connection, and perform element-wise concatenation with the upsampled feature map to obtain the concatenated feature map; after the concatenated feature map is processed by a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation layer for feature optimization, the k-1 level decoded feature map is output; repeat the upsampling, element-wise concatenation, and feature optimization operations to determine the 1st level decoded feature map output by the 1st level decoding module as the Mars terrain semantic segmentation prediction map. The 1st level decoded feature map has the same size as the input image sample, k is a positive integer from 2 to 5, and the decoding process starts from k=5 and decreases sequentially to k=2.

[0069] It should be noted that the specific workflow of the multi-scale feature fusion unit and the U-shaped decoding unit can be understood as follows: features from two main branches are cascaded and fused with upsampled features via skip connections. After fusion, spatial alignment and channel calibration are completed through 1×1 convolution, batch normalization, and ReLU activation. Subsequently, the decoder upsamples stepwise to restore spatial resolution, forming an output segmentation map of the same size as the input image. The decoder adopts a U-shaped structure and consists of five decoding modules. Each decoding module includes a deconvolution layer, a feature stitching layer, and a 3×3 convolutional layer. Through stepwise feature fusion and upsampling, the model can effectively recover the boundary details and terrain morphology of the Martian surface.

[0070] The method provided by this invention solves the problems of poor resolution recovery, loss of shallow details, and mismatch of predicted image size in traditional decoding processes by refining the specific workflow of the 5-level decoding module of the U-shaped structure decoding unit: the 5-level decoding module corresponds one-to-one with the 5-level features of the multi-scale fusion unit, decoding from deep low-resolution features (level 5) to shallow high-resolution features (level 1) step by step, ensuring the continuity of resolution recovery; 2x deconvolution upsampling accurately amplifies the feature map resolution, avoiding resolution distortion caused by traditional interpolation upsampling; skip connections introduce fusion features of the corresponding level, such as introducing level 4 fusion features when decoding level 4, to supplement the shallow details lost during upsampling; the combination of 3×3 convolution, batch normalization, and ReLU activation optimizes the stitched features, strengthening the effective signal and suppressing noise; the final output level 1 decoded feature map has the same size as the input image sample, ensuring that the segmentation prediction result can accurately correspond to each pixel of the original image, solving the core problem of pixel misalignment between the predicted image and the original image, and significantly improving the positioning accuracy of Mars terrain segmentation.

[0071] S3. Input multiple image samples into the dual backbone coding network model, and process them through the multi-scale feature fusion unit and the U-shaped structure decoding unit to obtain the Martian terrain semantic segmentation prediction map corresponding to each image sample.

[0072] Among them, the Mars terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information of each pixel belonging to different terrain categories; For a better understanding of this solution, please refer to [link / reference]. Figure 4 The following example will be used to explain the dual-backbone coding network model provided in the embodiments of the present invention.

[0073] Specifically, image samples of RGB images (3 channels) with a size of 256×256 pixels are input into the CNN branch and the Swin Transformer branch respectively. The CNN branch consists of 5 levels of progressive residual modules connected in series. The dilation rates of each module are 1, 2, 3, 4, and 5 according to the network depth from shallow to deep (i=1 to 5). Each module has a 3×3 dilated convolution path to extract the terrain features of the corresponding receptive field. The shallow layer captures details such as sand texture and gravel edges, while the deep layer captures macroscopic structures such as bedrock distribution and rock formations. A 1×1 channel mapping path is used to perform channel dimension transformation and information supplementation on the input features. The output features of the two paths are added element-wise (residual connection) and then fused through a 3×3 depthwise separable convolution. Finally, each module outputs a detailed feature map of the corresponding level, which is then transmitted to the multi-scale feature fusion unit. The SwinTransformer branch consists of five stacked modules connected in series. Each module, arranged from shallow to deep (j=1 to 5), is composed of alternating stacked Window Multi-Head Self-Attention (W-MSA) and Shifted Window Multi-Head Self-Attention (SW-MSA) sub-modules, with a window size of 7×7. The W-MSA sub-module first divides the input semantic feature map into non-overlapping 7×7 windows, independently calculating multi-head self-attention within each window to extract local semantic associations of pixels within the window. The SW-MSA sub-module performs window shifting on the local semantic features output by W-MSA, creating partial overlap between adjacent windows. It then calculates multi-head self-attention within the shifted windows to establish global semantic associations across windows. Finally, each module outputs a semantic feature map corresponding to its level, which is then sequentially transmitted to the multi-scale feature fusion unit.

[0074] The multi-scale feature fusion unit receives five detail feature maps output from the CNN branch and five semantic feature maps output from the Swing Transformer branch, with the two branches corresponding one-to-one in terms of level (the first-level residual module corresponds to the first stacked module, and so on up to the fifth-level residual module corresponding to the fifth stacked module). First, the number of channels in the corresponding level's detail feature map and semantic feature map is adjusted to the preset number of channels through channel dimension alignment. Then, element-wise concatenation is performed on the channel-aligned feature maps to obtain the initial fused feature map. The initial fused feature map is then processed sequentially through a 1×1 convolutional layer (to filter channel redundancy), a batch normalization layer (to unify the distribution of feature values), and a ReLU activation layer (to enhance effective feature responses), finally outputting five hierarchical fused feature maps, which are transmitted to the U-shaped structure decoding unit in hierarchical order.

[0075] The U-shaped decoding unit contains five decoding modules, corresponding sequentially from the deepest to the shallowest network depth to the level 5 to level 1 fusion feature maps output by the multi-scale fusion module. The level 5 decoding module receives the level 5 fusion feature map and first upsamples it to the level 4 feature map resolution using a deconvolution layer. Then, it introduces the level 4 fusion feature map output by the multi-scale fusion module through a skip connection, performing element-wise concatenation with the upsampled feature map to obtain the concatenated feature map. This concatenated feature map is then processed by a 3×3 convolutional layer (to strengthen feature associations), a batch normalization layer (to unify numerical distribution), and a ReLU activation layer (to filter invalid features) to output the level 4 decoded feature map. Subsequent level 4 to level 2 decoding modules repeat the upsampling, skip connection concatenation, and feature optimization operations sequentially. Finally, the level 1 decoding module outputs a Mars terrain semantic segmentation prediction map with the same size as the input image sample, completing the forward propagation process of the entire model.

[0076] S4. Based on the target loss function, the dual-backbone coding network model is trained according to the Martian terrain semantic segmentation prediction map and pixel-level semantic labels corresponding to multiple image samples, and the trained dual-backbone coding network model is obtained.

[0077] The target loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. In one possible implementation, the target loss function for: ; in, As the first weighting coefficient, For relation-aware prototype regularization terms. This is the second weighting coefficient. For hierarchical feature aggregation terms, This is the third weighting coefficient. .

[0078] To address the confusion between different terrain categories and improve boundary recognition capabilities, this embodiment designs a Feature-Guided Loss function, also known as the target loss function mentioned above. This target loss function consists of three parts: a Relation-Aware Prototype Regularization term, which constrains the distance between category feature centers (prototypes) based on the geological similarity matrix, ensuring that similar categories remain moderately close while heterogeneous categories maintain a larger gap; a Hierarchical Aggregation Loss term, which weights and aggregates multi-layer features from the encoding and decoding stages, ensuring consistency of features at different scales in the semantic space; and a Boundary-Weighted Dice Loss term, which uses distance transformation to generate a boundary weight map, enhancing the model's focus on boundary pixels and reducing boundary blurring.

[0079] It should be understood that the relation-aware prototype regularization term is used to constrain the semantic relationships of landform-related categories based on the distance between category feature centers; the hierarchical feature aggregation term is used to enhance semantic consistency in multi-layer feature spaces; and the boundary-weighted Dice term generates a spatial weight map through distance transformation, which is used to improve the model's accuracy in recognizing terrain boundary regions.

[0080] To facilitate understanding, the target loss function provided by the present invention will be further explained below with reference to an embodiment.

[0081] In the semantic segmentation task of Martian terrain, different landform categories are not completely independent, but rather exhibit complex semantic and physical relationships. For example, Martian soil, sand, and gravel are continuous-grained surface sediments with often blurred and gradual boundaries; rocks and bedrock, although sharing similar lithological origins, need to be distinguished by morphology and spatial distribution; while shadows are highly correlated with rocks due to illumination geometry, making them easily misclassified by the model. In contrast, categories such as rover tracks, background, and unknown areas have weaker relationships with other landform types and are more semantically isolated. To address this characteristic, we designed a joint supervision mechanism consisting of three complementary loss terms, aiming to guide the network to explicitly model these structured inter-category relationships during training, thereby improving its ability to discriminate subtle landform differences.

[0082] First, we introduce a relation-aware prototype regularization loss Lrpl to construct learnable prototype vectors for all C=9 terrain categories. This serves as the semantic center representation for each category. Unlike traditional contrastive learning, which forces all prototypes to be uniformly distributed, this loss uses pre-defined relation weights w. ij(Representing the weights of the relationship between category i and category j), differentiating separation constraints are imposed on different category pairs. Specifically, for landform pairs with continuous grain size, the weight is set to 1.0, allowing their prototypes to remain close together in the feature space; for easily confused but physically different category pairs, the weight is set to 0.7 to enhance their discriminative power; and for semantically isolated categories combined with other categories, the weight is uniformly set to 0.2 to impose stronger separation constraints; the default weight for other category pairs is 0.4. These initial weights are based on geological priors and are dynamically updated as learnable parameters during training, enabling the model to adaptively adjust category boundaries while preserving domain knowledge. The loss function is the sum of weighted cosine similarities. By minimizing this loss function, the model can expand the feature gaps between key category pairs in a relationship-aware manner, avoiding the imposition of rigid uniform separation on all categories.

[0083] Therefore, the formula for determining the relation-aware prototype regularization term is: Secondly, to enhance the consistency between intermediate feature representations and semantic prototypes, we propose a hierarchical feature aggregation term. Considering that deep features have high semantic abstraction and are not conducive to alignment with specific prototypes, we only utilize the first three shallow feature layers in the encoder-decoder structure, which retain richer spatial details. We sum the outputs of each encoder and corresponding decoder layer, perform global average pooling, and then perform weighted fusion using learnable weights to obtain a unified high-level feature representation f. Subsequently, based on the model's prediction confidence for each pixel, we divide f into 27 regions (corresponding to 9 categories, each category divided into high, medium, and low confidence intervals), and take the average of the feature vectors within each interval to obtain representative vectors. These vectors are then required to be aligned with the prototype p of their corresponding category. i Alignment is performed using both Euclidean distance and cosine similarity as the alignment objective to ensure consistency in direction and amplitude. This confidence-based grouping strategy helps the model balance the stability of high-confidence samples with the learning signals of low-confidence regions during the alignment process, thereby improving the overall feature discriminative power.

[0084] The formula for determining the hierarchical feature aggregation term is: , ; Finally, to address the common problem of blurred boundaries in Mars images, we employ boundary-weighted Dice loss. To enhance oversight of uncertain regions, this loss function generates a spatial weighted map based on truth-labeled boundaries. ,in Let x be the Euclidean distance from pixel x to the nearest class boundary. Controlling boundaries emphasizes intensity. To control the scope of influence, Pc and Gc represent the predicted mask and the true mask for class c, respectively. Considering that the network backbone used (Swin Transformer combined with asymptotic residual blocks) already possesses strong fine-grained structure awareness, we set... As a moderate enhancement, it highlights the contribution of boundary pixels while avoiding excessive suppression of the internal region, thus achieving a stable and balanced optimization process.

[0085] The final total loss function is the weighted sum of the three losses mentioned above: In one example, see Figure 5 and Figure 6 , Figure 5 An experimental result diagram of an ablation experiment with different weighting coefficients provided in an embodiment of the present invention is shown below. Figure 5 As shown, the optimal performance is achieved when the first weight coefficient is 1.0, the second weight coefficient is 0.5, and the third weight coefficient is 0.5. This joint loss mechanism works synergistically from three levels: semantic structure, feature alignment, and boundary optimization, effectively improving the model's segmentation accuracy for complex Martian terrain, especially showing significant advantages when dealing with semantically similar or spatially strongly correlated categories.

[0086] Furthermore, Figure 6 This is a display diagram showing the feature map results of each layer in the ablation experiment provided in this embodiment of the invention. Wherein, Figure 6 (a) illustrates a complex scenario: rocks are tightly surrounded by bedrock, and the boundaries between them are very difficult to distinguish, but the deeper stages of the network show a strong response in the contact areas between the two classes, indicating that the model has learned to focus on the fuzzy boundaries where misclassification is most likely. Figure 6 In (b), the rock surface in the lower right corner is relatively flat and visually resembles bedrock. Without guidance from inter-category relationships and contextual information, such areas are often incorrectly merged into a single bedrock region. However, our designed network accurately identifies the outer edge as rock and the occluded inner areas as shadows. Particularly, the distribution of Martian soil and sand at the boundaries with other landforms, with their irregular orientation and diffuse transition characteristics, often leads to category confusion. Feature maps show that the network continuously focuses on these areas at multiple stages, gradually strengthening the separation of Martian soil from neighboring categories. This behavior reflects the role of a relationship-aware prototype mechanism—it reasonably expands the decision boundary in the feature space by explicitly modeling the affinity between geologically continuous categories; simultaneously, the boundary weighting strategy further enhances the model's attention to uncertain interfaces, and the two work together to improve the ability to discriminate fuzzy transition regions. Figure 6In (c), a bedrock protrudes from the lower right corner, and its shadow indicates that it has a certain height. This stepped bedrock could potentially obstruct the rover's safe movement, so it's crucial to correctly identify this structure. Thanks to modeling geometric dependencies, the network successfully distinguishes the stepped bedrock from the surrounding rocks and accurately delineates the shadow boundaries. In the lower left corner, a small patch of Martian soil is sandwiched between two bedrock rocks. Despite its small size and being surrounded by strong neighbors, the model retains it as a separate category. Above it is a widespread gravel deposit; the rough texture could easily mask fine-grained differences, but the network still maintains clear category boundaries. Figure 6 (d) Containing the rover itself, it is labeled "Unknown". This category is semantically relatively isolated and less likely to be confused with other geological categories. Therefore, the model can reliably identify "Unknown" areas while focusing its main attention on the boundaries between different terrains, demonstrating the effectiveness of the strong separation constraint imposed on isolated categories in the loss function.

[0087] The method provided in this invention addresses the challenge of a single loss function simultaneously achieving class discrimination, multi-scale semantic consistency, and boundary segmentation accuracy by explicitly defining the weighted combination formula of the target loss function. The relation-aware prototype regularization term strengthens the model's ability to distinguish similar terrain categories by constraining the spacing of terrain category feature prototypes. The hierarchical feature aggregation term ensures semantic consistency across different levels of features by aggregating multi-scale features from the encoder and decoder. The boundary-weighted Dice term improves the model's segmentation accuracy for terrain boundary regions by weighting the terrain boundary pixels. Three weight coefficients balance the influence of each loss term, preventing any single loss term from excessively dominating training. Ultimately, the target loss function guides model optimization from three core dimensions: class discrimination, semantic consistency, and boundary accuracy, significantly reducing the misclassification rate and boundary error rate in Martian terrain segmentation.

[0088] S5. Input the Martian surface image to be detected into the trained dual-backbone coding network model, and output the pixel-level semantic labels of the Martian surface image to be detected.

[0089] In one example, see Figure 7 , Figure 7 This is a schematic diagram of the results of a comparative experiment provided in an embodiment of the present invention. Figure 7 In (a), the trajectory is embedded within a large area of ​​sand, making the boundaries rather blurry. DeepLabv3+ and U-Net3+ methods often merge the trajectory with the surrounding sand, while Transformer-based models generate more consistent boundaries, but still fail to capture these narrow structures. MarsTerrNet ​​more clearly delineates the trajectory boundaries, reducing the possibility of misclassification. Figure 7In (b), the smooth surfaces of rocks and bedrock have high visual similarity, making them difficult for many models to distinguish. The elongated strips of Martian soil between sand and other categories present additional segmentation challenges: DeepLabv3+ and U-Net3+ largely miss this region, while Transformer-based models retain some of the Martian soil distribution, but with blurred boundaries. MarsTerrNet ​​generates more continuous and clearly defined segmentations, closer to reality. Figure 7 (c) Contains a large number of rocks of varying sizes, interspersed with small pieces of bedrock. DeepLabv3+ and U-Net3+ models tend to over-segment, producing fragmented predictions, while Transformer-based methods often fail to capture small details effectively. MarsTerrNet ​​maintains a balance between large-scale structure and small-scale detail, reducing segmentation fragmentation and omissions. Figure 7 In (d), small patches of Martian soil and visually similar bedrock and rocks in the lower right corner are difficult to distinguish. DeepLabv3+, U-Net3+, and SegFormer exhibit significant blurring in these areas, while MarsTerrNet ​​achieves clearer separation and sharper boundaries. Figure 7 In (e), the distant gravel area appears in a gray tone similar to the rock, leading to misclassification. DeepLabv3+ and U-Net3+ misclassified this area, while Transformer-based models tended to over-smooth the area, reducing class discriminability. MarsTerrNet ​​generated relatively more consistent results, mitigating some of the confusion.

[0090] As shown in S1-S5 above, the method provided by this embodiment of the invention, through the design of a dual-backbone coding network model, can perform complete and accurate segmentation of Martian terrain, solving the core defects of traditional single-branch networks: although traditional CNN branches can extract local terrain details, they are difficult to capture global semantic relationships; although traditional Transformer branches can model global semantics, their ability to extract local details is weak and their computational cost is high. The parallel design of the dual backbone allows the CNN branch to focus on local details and the SwingTransformer branch to focus on global semantics. The multi-scale fusion unit integrates the features of the two branches into dual-dimensional features of details and semantics. The U-shaped decoding unit restores the resolution consistent with the original image through upsampling. The composite loss function optimizes training from multiple dimensions such as category discrimination, semantic consistency, and boundary accuracy. Ultimately, the method can achieve high-detail localization and high-semantic accuracy segmentation in complex Martian terrain scenes, meeting the core requirements of Mars rovers for surface environment perception.

[0091] The method provided in this invention significantly enhances the model's multi-scale information representation and global semantic modeling capabilities in complex Martian terrain scenarios by introducing a co-coding mechanism of convolutional and Transformer backbones within a unified U-shaped structure. The progressive residual module effectively strengthens the preservation of local texture and edge details, ensuring the spatial continuity and morphological integrity of landform units such as rocks, gravel, and sand. Furthermore, this invention proposes a feature-guided loss function (target loss function) that incorporates geological relevance into the network training process, improving the discriminative power between similar categories while maintaining landform continuity, making the segmentation results more reasonable and reliable in geological interpretation. The boundary weighting mechanism further improves the model's accuracy in identifying complex terrain boundaries, resulting in clear boundaries and consistent structure in the output terrain distribution map. This method overcomes the limitations of a single feature extraction framework, achieving efficient complementarity between convolutional local features and Transformer global features, significantly improving the accuracy and generalization ability of the segmentation results. Overall, this invention has stronger physical consistency and geological interpretability, and can provide highly reliable terrain segmentation results in tasks such as environmental perception, path planning and geological analysis of Mars rovers, providing advanced technical support for automatic surface identification and navigation decision-making in deep space exploration.

[0092] In one possible implementation, the method provided by the embodiments of the present invention further includes: The pixel-level semantic labels of the output Martian surface image to be detected are visualized, and a terrain classification map and landform structure distribution map corresponding to the Martian surface image to be detected are generated.

[0093] The foregoing mainly describes the solutions of the embodiments of the present invention from a methodological perspective. It is understood that, in order to achieve the above-mentioned functions, the terrain segmentation system 100 includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present invention.

[0094] In this embodiment of the invention, the terrain segmentation system 100 can be divided into functional units according to the above method example. For example, the terrain segmentation system 100 can be divided into functional units corresponding to various functions, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0095] For example, Figure 8 This diagram illustrates the hardware structure of a terrain segmentation system according to an embodiment of the present invention. The terrain segmentation system 100 includes: a sample generation module 110, used to determine a training sample set based on Martian surface images; the training sample set includes multiple image samples and pixel-level semantic labels corresponding to each image sample; the pixel-level semantic labels are used to identify the terrain category of each pixel in the image sample; and a model building module 120, used to build a dual-backbone coding network model, the dual-backbone coding network model consisting of convolutional neural network branches and Swin... The Transformer branches are composed of parallel structures, and the convolutional neural network branches are composed of multi-level asymptotic residual modules connected in series. The dilation rate of each asymptotic residual module increases layer by layer with increasing network depth. The SwinTransformer branches are constructed by alternately stacking window multi-head self-attention sub-modules and shift window multi-head self-attention sub-modules. The dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-structure decoding unit is used to restore the image spatial resolution through upsampling. The terrain segmentation module 130 is used to input multiple image samples into the dual-backbone coding network model, which are then processed by the multi-scale feature fusion unit and the U-structure decoding unit to obtain... The system generates a Martian terrain semantic segmentation prediction map for each image sample. The Martian terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information indicating that each pixel belongs to a different terrain category. The model training module 140 is used to train the dual-backbone coding network model based on the Martian terrain semantic segmentation prediction maps and pixel-level semantic labels corresponding to multiple image samples, according to the objective loss function. The objective loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The terrain segmentation module 130 is also used to input the Martian surface image to be detected into the trained dual-backbone coding network model and output the pixel-level semantic labels of the Martian surface image to be detected.

[0096] It should be understood that specific descriptions of the above-mentioned optional methods can be found in the foregoing method embodiments, and will not be repeated here. Furthermore, explanations of any of the terrain segmentation systems 100 provided above, as well as descriptions of their beneficial effects, can be found in the corresponding method embodiments described above, and will not be repeated here.

[0097] This invention also provides a computer-readable storage medium storing at least one computer instruction, which is loaded and executed by a processor to implement the methods of the various embodiments described above. Explanations of the relevant content and descriptions of the beneficial effects of any of the computer-readable storage media provided above can be found in the corresponding embodiments described above, and will not be repeated here.

[0098] This invention also provides a chip. This chip integrates a control circuit for implementing the functions of the terrain segmentation system 100 described above, and one or more ports. Optionally, the functions supported by this chip are as described above, and will not be repeated here.

[0099] Those skilled in the art will understand that the program for implementing all or part of the steps of the above embodiments, which can be executed by a program instructing related hardware, can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0100] This invention also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this invention is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD), etc.

[0101] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of the present invention, such as, but not limited to, the aforementioned memory, computer-readable storage medium, and communication chip, are all non-transitory. Those skilled in the art should recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0102] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for surface terrain segmentation based on a dual-backbone coding network model, characterized in that, The method includes: A training sample set is determined based on images of the Martian surface. The training sample set includes multiple image samples and pixel-level semantic labels corresponding to each image sample. The pixel-level semantic labels are used to identify the terrain category of each pixel in the image sample. A dual-backbone coding network model is constructed, which consists of a convolutional neural network branch and a Swing Transformer branch in parallel. The convolutional neural network branch is composed of multi-level asymptotic residual modules connected in series, and the dilation rate of each asymptotic residual module increases layer by layer with network depth. The Swing Transformer branch uses window multi-head self-attention sub-modules and shift window multi-head self-attention sub-modules stacked alternately. The dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-shaped structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-shaped structure decoding unit is used to restore the image spatial resolution through upsampling. The multiple image samples are input into the dual backbone coding network model and processed by the multi-scale feature fusion unit and the U-shaped structure decoding unit to obtain the Mars terrain semantic segmentation prediction map corresponding to each image sample. The Mars terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information of each pixel belonging to different terrain categories. Based on the objective loss function, the dual-backbone coding network model is trained according to the Mars terrain semantic segmentation prediction map and pixel-level semantic labels corresponding to the multiple image samples to obtain the trained dual-backbone coding network model. The objective loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The Martian surface image to be detected is input into the trained dual-backbone coding network model, which outputs pixel-level semantic labels for the Martian surface image.

2. The method according to claim 1, characterized in that, The Martian surface images are high-resolution images acquired by the onboard camera system of the Mars rover. The terrain categories include Martian soil, sand, gravel, bedrock, rock, shadows, rover tracks, background, and unknowns. The process of determining the training sample set based on Martian surface images includes: The Martian surface image is cropped to obtain multiple image samples of a preset size, which is determined based on the input size of the dual backbone coding network model. Determine the pixel-level semantic label for each of the plurality of image samples; Data augmentation operations are performed on the plurality of image samples and the pixel-level semantic labels of each image sample to obtain the training sample set. The data augmentation operations include random horizontal flipping, random rotation within a preset angle range, random perturbation within a preset brightness range or a preset contrast range, and random scaling within a preset magnification range.

3. The method according to claim 2, characterized in that, In the convolutional neural network branch, there are 5 asymptotic residual modules. The inflation rate of the i-th asymptotic residual module, which is connected sequentially from shallow to deep according to the network depth, is i, where i is a positive integer from 1 to 5. For the i-th asymptotic residual module, it is used to process the i-1 level detail feature map output by the (i-1)-th asymptotic residual module, and output the i-th level detail feature map. The i-th asymptotic residual module includes a 3×3 dilated convolution path, a 1×1 channel mapping path, and a feature fusion layer. The 3×3 dilated convolution path and the 1×1 channel mapping path are set in parallel. The 3×3 dilated convolution path is used to extract the terrain features within the receptive field corresponding to the input i-1-th level detail feature map to obtain the first feature information. The 1×1 channel mapping path is used to perform channel dimension transformation and information supplementation on the input i-1-th level detail feature map to obtain the second feature information. The feature fusion layer is used to add the first feature information and the second feature information element by element, and then perform feature fusion through 3×3 depthwise separable convolution to obtain the i-th level detail feature map.

4. The method according to claim 3, characterized in that, In the Swing Transformer branch, the number of alternating stacks of the window multi-head self-attention submodule and the shift window multi-head self-attention submodule is 5. The j-th stacked module, which is connected sequentially from shallow to deep according to the network depth, is used to process the j-1 level semantic feature map output by the j-1-th stacked module and output the j-th level semantic feature map, where j is a positive integer from 1 to 5. The j-th stacked module includes a window multi-head self-attention submodule and a shifted window multi-head self-attention submodule connected in sequence, with the window size of both submodules being 7×7. Specifically: the window multi-head self-attention submodule divides the input (j-1)-th level semantic feature map into non-overlapping 7×7 windows, independently calculates multi-head self-attention within each window, extracts local semantic associations of pixels within the window, and obtains local semantic features; the shifted window multi-head self-attention submodule performs window shifting processing on the local semantic features output by the window multi-head self-attention submodule, creating partial overlap between adjacent windows, and then calculates multi-head self-attention within the shifted window to establish global semantic associations across windows, obtaining global semantic features; the j-th stacked module outputs a j-th level semantic feature map containing local semantic features and global semantic associations through the collaborative processing of the window multi-head self-attention submodule and the shifted window multi-head self-attention submodule.

5. The method according to claim 4, characterized in that, The multi-scale feature fusion unit is specifically used for: It receives detailed feature maps of 5 levels from the output of the convolutional neural network branch and semantic feature maps of 5 levels from the output of the Swing Transformer branch, and the levels of the convolutional neural network branch and the Swing Transformer branch correspond one-to-one. By performing channel-dimensional alignment, the number of channels in the detail feature map and semantic feature map at each level is adjusted to the preset number of channels; then, element-level concatenation is performed on the channel-aligned detail feature map and semantic feature map to obtain the initial fused feature map. The initial fused feature map is processed sequentially by a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation layer to output a hierarchical fused feature map corresponding to each of the five levels. The hierarchical fused feature map contains terrain detail information and semantic association information at the corresponding scale.

6. The method according to claim 5, characterized in that, The U-shaped structure decoding unit contains 5 levels of decoding modules, which correspond to the level 5 to level 1 hierarchical fusion feature maps output by the multi-scale feature fusion unit in descending order of network depth. The k-th level decoding module is used to: receive the k-th level fused feature map, first upsample it by a deconvolutional layer to increase its resolution to that of the (k-1)-th level feature map; introduce the (k-1)-th level fused feature map output by the multi-scale feature fusion unit through skip connections, and perform element-wise concatenation with the upsampled feature map to obtain the concatenated feature map; after the concatenated feature map is processed by a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation layer for feature optimization, output the (k-1)-th level decoded feature map; repeat the upsampling, element-wise concatenation, and feature optimization operations to determine the first-level decoded feature map output by the first-level decoding module as the Mars terrain semantic segmentation prediction map. The size of the first-level decoded feature map is the same as that of the input image sample, k is a positive integer from 2 to 5, and the decoding process starts from k=5 and decreases sequentially to k=2.

7. The method according to claim 6, characterized in that, The target loss function for: ； in, As the first weighting coefficient, For relation-aware prototype regularization terms. This is the second weighting coefficient. For hierarchical feature aggregation terms, The third weighting coefficient, .

8. A surface terrain segmentation system based on a dual-backbone coding network model, characterized in that, The system includes: A sample generation module is used to determine a training sample set based on Martian surface images. The training sample set includes multiple image samples and pixel-level semantic labels corresponding to each image sample. The pixel-level semantic labels are used to identify the terrain category of each pixel in the image sample. The model building module is used to construct a dual-backbone coding network model, which consists of a convolutional neural network branch and a Swing Transformer branch in parallel. The convolutional neural network branch is composed of multi-level asymptotic residual modules connected in series, and the dilation rate of each asymptotic residual module increases layer by layer with network depth. The Swing Transformer branch uses windowed multi-head self-attention sub-modules and shifted windowed multi-head self-attention sub-modules stacked alternately. The dual-backbone coding network model also includes a multi-scale feature fusion unit and a U-shaped structure decoding unit. The multi-scale feature fusion unit is used to align and fuse the features output from each level of the two branches, and the U-shaped structure decoding unit is used to restore the image spatial resolution through upsampling. The terrain segmentation module is used to input the multiple image samples into the dual backbone coding network model, and process them through the multi-scale feature fusion unit and the U-shaped structure decoding unit to obtain the Mars terrain semantic segmentation prediction map corresponding to each image sample. The Mars terrain semantic segmentation prediction map has the same size as the corresponding image sample and contains prediction information of each pixel belonging to different terrain categories. The model training module is used to train the dual-backbone coding network model based on the target loss function, according to the Mars terrain semantic segmentation prediction map and pixel-level semantic labels corresponding to the multiple image samples, to obtain the trained dual-backbone coding network model. The target loss function includes a relation-aware prototype regularization term, a hierarchical feature aggregation term, and a boundary-weighted Dice term. The terrain segmentation module is also used to input the Martian surface image to be detected into the trained dual-backbone coding network model and output pixel-level semantic labels for the Martian surface image to be detected.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the surface terrain segmentation method based on the dual-backbone coding network model as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the surface terrain segmentation method based on the dual-backbone coding network model as described in any one of claims 1 to 7.

Images