An automatic rock extraction method and system based on deep learning and Mars rover camera imagery

Abstract

This invention discloses an automatic rock extraction method and system based on deep learning and Mars rover camera imagery. The invention aims to address the problems of insufficient generalization ability, high computational load, and difficulty in deployment on spaceborne edge devices in existing Mars rock segmentation algorithms under complex surface backgrounds. Its core lies in constructing a lightweight encoder-decoder network that integrates three key modules: a frequency-assisted Mamba enhancement module, which accurately captures rock texture and contours by fusing Mamba's global sequence modeling capabilities with wavelet transform frequency domain enhancement; a multi-scale feature enhancer, which adaptively fuses multi-level features using a parallel dual attention mechanism; and a boundary-aware auxiliary branch, which improves the integrity of the segmented contour through explicit edge supervision and feature decoupling mechanisms. This invention significantly improves rock extraction accuracy while greatly reducing model complexity and computational overhead, making it suitable for outer space exploration scenarios with limited communication bandwidth and computing resources, such as those involving Mars rovers, achieving an effective balance between high accuracy and lightweight design.

Description

Technical Field

[0001] This invention relates to the field of deep space exploration technology, specifically to an automatic rock extraction method and system based on deep learning and Mars rover camera images. Background Technology

[0002] Extracting rocks from the Martian surface is of great significance for planetary geological research and space engineering safety. Early methods mainly relied on manual annotation or machine learning methods based on human-designed features, such as edge detection, shadow analysis, and superpixel segmentation. These methods are time-consuming, labor-intensive, have poor generalization ability, and are difficult to adapt to the complex and ever-changing Martian surface environment.

[0003] In recent years, deep learning-based semantic segmentation methods have been gradually applied to Martian rock extraction. While early CNN-based methods could extract local features, they struggled to model long-range dependencies. Subsequent Transformer-based methods, while enhancing global perception capabilities, suffered from structural feature loss, particularly performing poorly in small object segmentation. Furthermore, their high model complexity and computational and storage burdens made them difficult to deploy in Mars exploration scenarios with limited communication bandwidth and computational resources.

[0004] Furthermore, most existing methods rely solely on spatial domain features for analysis, leading to a significant decrease in segmentation accuracy when rocks are similar in color to the background or when there is interference from lighting and shadows. Therefore, there is an urgent need for an intelligent Martian rock extraction method that can consider both global and local features, possess lightweight characteristics, and effectively cope with complex background interference. Summary of the Invention

[0005] To address this, embodiments of the present invention provide an automatic rock extraction method and system based on deep learning and Mars rover camera images, in order to solve the technical problems of existing technologies that are difficult to balance global and local features, lack lightweight characteristics, and are easily affected by complex backgrounds.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] According to a first aspect of the present invention, an automatic rock extraction method based on deep learning and Mars rover camera imagery is provided, the method comprising:

[0008] The Mars rover's camera captures images of the Martian surface as input images.

[0009] The input image is fed into a lightweight encoder-decoder segmentation network. The encoder of this network consists of four frequency-assisted Mamba modules stacked together to simultaneously extract and fuse global sequence dependency features of the image with local frequency domain texture features enhanced by wavelet transform.

[0010] The multi-scale features output by the encoder are fed into a multi-scale feature enhancer, and the multi-scale features are adaptively enhanced and fused through a parallel spatial attention mechanism and a channel attention mechanism.

[0011] The enhanced multi-scale features are input into a main decoding branch and a boundary-aware auxiliary branch, respectively.

[0012] The main decoding branch restores the feature map resolution through upsampling and skip connections to generate a rock body segmentation mask;

[0013] The boundary-aware auxiliary branch decouples edge features by performing edge enhancement and optical flow-guided resampling on deep features, and then fuses them with high-resolution encoder features to generate a rock edge prediction map.

[0014] By fusing the rock body segmentation mask and the rock edge prediction map, the final rock extraction result at the pixel level is output.

[0015] Furthermore, the frequency-assisted enhancement Mamba module executes the following two sub-processes in parallel:

[0016] Global space modeling sub-process: After serializing the input features, long-range dependency modeling is performed using the Mamba state space model based on a selective scanning mechanism;

[0017] Wavelet frequency enhancement sub-process: The input features are decomposed into two-dimensional discrete wavelet to obtain low-frequency and high-frequency sub-bands in multiple directions. The convolutional processing of the spliced ​​sub-bands is then performed followed by inverse wavelet transform to enhance edge and texture information.

[0018] The output features of the two sub-processes are added element by element to form the output of this module.

[0019] Furthermore, the execution process of the multi-scale feature enhancer includes:

[0020] Spatial attention weighting and channel attention weighting are applied to the input features in parallel to obtain two sets of enhanced features;

[0021] A dynamic weight estimation network is used to adaptively generate fusion weights for the two sets of enhancement features;

[0022] The two sets of features are weighted and summed according to the fusion weights, and then residually connected with the original input features to output the final enhanced features.

[0023] Furthermore, the specific steps of the boundary-aware auxiliary branch include:

[0024] Laplacian convolution is applied to the deepest features output by the encoder to perform initial edge enhancement.

[0025] The enhanced features are blurred to regress the optical flow field, and the original features are resampled using the optical flow field to obtain the main structural features.

[0026] Subtracting the original feature from the main structural feature yields the explicit residual edge feature;

[0027] The residual edge features are concatenated and fused with higher-resolution features from the encoder to improve edge localization accuracy.

[0028] Furthermore, when training the segmentation network, the method uses a combined loss function, the expression of which is:

[0029] L total =L seg +αL bound

[0030] Among them, L seg L is the sum of the dice loss and cross-entropy loss used to supervise the segmentation of the rock mass. bound The sum of the binary cross-entropy loss and the dice loss used to supervise rock edge prediction is α, where α is the weighting coefficient balancing the two losses.

[0031] Furthermore, the encoder-decoder segmentation network adopts a four-level downsampling and upsampling structure, with its basic channel number set to 8, 16, 24, and 32. It also reduces the number of parameters through depthwise separable convolution to adapt to the onboard computing resources of the Mars rover.

[0032] Furthermore, the final rock extraction results include binarized rock region masks and rock contour vector information, which are used to support the statistical analysis of Martian surface geological composition and rover path planning.

[0033] Furthermore, the method is deployed on the Mars rover's embedded computing platform to enable online, real-time detection and extraction of rocks on the Martian surface.

[0034] According to a second aspect of the present invention, an automated rock extraction system based on deep learning and Mars rover camera imagery is provided, the system comprising:

[0035] Image input interface, used to receive images from the Mars rover's camera;

[0036] The processor and memory, wherein the memory stores a computer program that can run on the processor, and when the program is executed by the processor, implements the steps of the automatic rock extraction method;

[0037] The results output interface is used to output the rock extraction results.

[0038] Furthermore, the processor is an edge computing AI chip integrated inside the Mars rover, and the system constitutes part of the Mars rover's autonomous visual perception subsystem.

[0039] The embodiments of the present invention have the following advantages:

[0040] This invention aims to address the shortcomings of existing Martian rock segmentation algorithms, such as insufficient generalization ability in complex surface backgrounds, high model computational load, and difficulty in deployment on spaceborne edge devices. Its core lies in constructing a lightweight encoder-decoder network that integrates three key modules: a frequency-assisted Mamba enhancement module, which accurately captures rock texture and contours by fusing Mamba's global sequence modeling capabilities with wavelet transform frequency domain enhancement; a multi-scale feature enhancer, which adaptively fuses multi-level features using a parallel dual attention mechanism; and a boundary-aware auxiliary branch, which improves the integrity of the segmented contour through explicit edge supervision and feature decoupling mechanisms. This invention significantly improves rock extraction accuracy while drastically reducing model complexity and computational overhead, making it suitable for outer space exploration scenarios with limited communication bandwidth and computing resources, such as those involving Mars rovers, achieving an effective balance between high accuracy and lightweight design. Attached Figure Description

[0041] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0042] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0043] Figure 1 A flowchart illustrating an automatic rock extraction method based on deep learning and Mars rover camera images, provided in an embodiment of the present invention;

[0044] Figure 2 M is a method for automatic rock extraction based on deep learning and Mars rover camera imagery provided in this embodiment of the invention. 2 Schematic diagram of the overall structure of Rock-Mamba;

[0045] Figure 3A schematic diagram of the FAEM structure in an automatic rock extraction method based on deep learning and Mars rover camera images provided in an embodiment of the present invention;

[0046] Figure 4 This diagram illustrates the parallel dual attention mechanism in MSFE of an automatic rock extraction method based on deep learning and Mars rover camera images provided in an embodiment of the present invention. Detailed Implementation

[0047] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Rocks are typical exposed geological units on the Martian surface, and they play an important role in planetary science research and space engineering safety.

[0049] Early Martian rock extraction primarily relied on manual annotation or human-computer interaction techniques. While this method provided some support for scientific research on early Martian rocks, manually counting large numbers of rocks was a time-consuming and tedious task. Furthermore, the accuracy of this method was easily affected by subjective factors such as perspective, experience, and proficiency, making the process even more challenging. Subsequently, many machine learning methods based on human-designed features were proposed. These methods rely on expert knowledge to design discriminative features and can be categorized into edge-based methods, illumination and shadow-based methods, and superpixel-based methods. These machine learning methods improved extraction efficiency to some extent, but their generalization ability was significantly insufficient in the complex Martian surface environment due to limitations in the representational power of human-designed features.

[0050] In recent years, deep learning has demonstrated remarkable potential in multiple fields due to its powerful feature representation capabilities and excellent generalization ability. In Martian rock science research, rock extraction is often considered a semantic segmentation task, primarily because pixel-by-pixel classification can effectively distinguish rock boundaries from surrounding features. Early applications of deep learning in rock segmentation mainly relied on CNN-based methods. CNN-based methods primarily capture texture and edge features through hierarchical convolutions, but are limited by local receptive fields, making it difficult to model long-range semantic relationships between rocks and surrounding features. The Transformer architecture, with its global self-attention mechanism, has shown revolutionary breakthroughs in long-range dependency modeling and demonstrated significant potential in multiple fields. Some studies have attempted Transformer-based methods for Martian rock extraction. Although Transformer-based methods can improve rock segmentation performance to some extent, they suffer from structural feature loss, especially performing poorly in small-sized rock segmentation. Therefore, developing a rock segmentation method that can simultaneously focus on global and local features is crucial, and this need has driven the development of hybrid CNN-Transformer based methods. However, CNN-transformer-based methods often have high model complexity, leading to increased computational and storage costs, making them unsuitable for scenarios with limited communication bandwidth in outer space and limited edge computing resources. Furthermore, it's worth noting that the aforementioned methods are all limited to spatial domain feature analysis; when rocks have high color similarity to the background or are subject to shadow occlusion, segmentation accuracy drops significantly.

[0051] In summary, current research on Martian rock segmentation has the following limitations: 1) Existing algorithms mainly rely on spatial domain feature extraction, which makes it difficult to effectively capture detailed information of weakly textured rocks under complex lighting and topography on the Martian surface; 2) Existing models have complex architectures and a large number of parameters, which pose serious challenges to data transmission efficiency and real-time processing capabilities in scenarios where communication bandwidth in outer space is limited and edge computing resources are limited.

[0052] To overcome the above problems, this application proposes the first Mamba-based Martian rock segmentation algorithm - M 2 Rock-Mamba. This method first fuses the spatial awareness capabilities of CNNs with the sequence modeling advantages of Mamba through FAEM, while suppressing complex background noise on Mars through a frequency-domain-assisted enhancement mechanism, achieving accurate capture of the geometric contours and texture features of rocks. Second, MSFE effectively enhances the multi-scale feature representation of the encoder output through a parallel dual attention mechanism and an adaptive feature selection strategy. Finally, BAAB, as an auxiliary decoding branch, models the semantics of rock edges through explicit boundary supervision signals, improving the contour integrity and detail accuracy of the segmentation results.

[0053] refer to Figure 1 This invention discloses an automatic rock extraction method based on deep learning and Mars rover camera images, which obtains Martian surface images captured by the Mars rover camera as input images;

[0054] The input image is fed into a lightweight encoder-decoder segmentation network. The encoder of this network consists of four frequency-assisted Mamba modules stacked together to simultaneously extract and fuse global sequence dependency features of the image with local frequency domain texture features enhanced by wavelet transform.

[0055] The multi-scale features output by the encoder are fed into a multi-scale feature enhancer, and the multi-scale features are adaptively enhanced and fused through a parallel spatial attention mechanism and a channel attention mechanism.

[0056] The enhanced multi-scale features are input into a main decoding branch and a boundary-aware auxiliary branch, respectively.

[0057] The main decoding branch restores the feature map resolution through upsampling and skip connections to generate a rock body segmentation mask;

[0058] The boundary-aware auxiliary branch decouples edge features by performing edge enhancement and optical flow-guided resampling on deep features, and then fuses them with high-resolution encoder features to generate a rock edge prediction map.

[0059] By fusing the rock body segmentation mask and the rock edge prediction map, the final rock extraction result at the pixel level is output.

[0060] refer to Figure 2 The proposed M²Rock-Mamba overall architecture design. The network adopts a four-layer encoder-decoder structure, with the core consisting of an encoder, dual decoders (main decoder and edge decoder) and skip connections. The number of channels in the four-layer structure is set to [8, 16, 24, 32].

[0061] For the input image, it first goes through four cascaded encoder stages, each containing a FAEM module. As the encoding process progresses, the spatial resolution of the feature map is halved at each stage, ultimately generating multi-level feature representations at four scales: (H / 2, W / 2), (H / 4, W / 4), (H / 8, W / 8), and (H / 16, W / 16).

[0062] Next, the multi-scale features output by the encoder are fed into the MSFE module, which effectively enhances the multi-scale features through a hybrid attention mechanism, providing support for subsequent decoding stages. Then, the enhanced features are fed into the main decoder for decoding. The main decoder also consists of four stages, each consisting of two convolutional layers, normalization, and upsampling operations. It is fused with the corresponding layer features from the encoder through skip connections to achieve efficient recovery of detailed information.

[0063] To effectively separate Martian rocks from the complex background, this application also incorporates an edge decoder. Specifically, the features of the last layer of the encoder are first processed by a Laplacian operator to obtain an edge-enhanced feature map. This enhanced feature map is then passed through two 3×3 depthwise separable convolutional layers and an upsampling operation to achieve blurring, resulting in low-pass features. These low-pass features are then concatenated with the original features and directly regressed using a 3×3 convolutional layer. This optical flow is then used to sample the original features to obtain the main body segmentation features; the residual is the edge feature. Next, to compensate for edge localization errors caused by the loss of resolution in deep features, the separated edge features are upsampled to a resolution four times lower using bilinear interpolation. Subsequently, the second-stage output features of the encoder, containing more refined spatial details, are introduced and adjusted using a 1×1 convolutional layer to adjust the channel dimensions. Finally, the upsampled edge features and the adjusted high-resolution features are concatenated along the channel dimension, and then adjusted through a 1×1 convolutional layer to obtain the final output features. Ultimately, the output of the edge decoder generates pixel-level edge predictions through the edge output module and is fused with the main branch to obtain the final mask segmentation result. This dual-decoder design effectively extracts and utilizes edge information, ensuring not only pixel-level accuracy and region integrity but also enhancing the model's perception of global structure and boundaries. This results in superior segmentation performance at both the detail and structural levels. During training, the network uses both segmentation and edge loss functions to simultaneously supervise the main segmentation and edge prediction results.

[0064] FAEM: Frequency-assisted Enhanced Mamba Module

[0065] refer to Figure 3 The proposed FAEM employs an innovative four-layer parallel architecture, with each layer integrating a parallelized Global Spatial Modeling Branch (GSM) and a Wavelet Frequency Transform Branch (WFT). The GSM branch achieves efficient modeling of sequence information through a selective scanning mechanism, while the WFT branch extracts fine-grained high-frequency information such as edges and textures using a multi-scale wavelet decomposition algorithm.

[0066] Within each layer, the output features of the GSM and WFT branches are fused to form the final feature representation of that layer. This dual-branch collaborative design ensures both the semantic integrity and contextual coherence of the features, and significantly enhances the model's ability to extract and preserve fine-grained details. Finally, by merging the features output from the four-layer architecture, the final feature output of FAEM is obtained.

[0067] Figure 3 In the diagram, (b) represents the Global Spatial Modeling Branch (GSM), and (c) represents the Wavelet Frequency Transform Branch (WFT).

[0068] Specifically, from Figure 3 It can be seen that, for input features First, divide the channel into four identical sub-features. Each sub-feature is then processed simultaneously through both the GSM branch and the WFT branch. Specifically, in the GSM branch, the input features... First, serialization is performed, followed by channel alignment, then feature normalization. Next, long-range dependencies are captured using a Mamba state-space model. Finally, feature normalization is performed again, and the result is mapped to the target dimension via a linear projection layer. The above process can be described as follows:

[0069]

[0070]

[0071] in This indicates a feature normalization operation. This represents a linear projection mapping operation. This indicates a channel dimension adjustment operation. Indicates Mamba network operations. Figure 3 (b) shows the structure of the Mamba network.

[0072] In the wavelet transform branch, for input features First, four subbands are obtained through two-dimensional discrete wavelet decomposition: a low-frequency structure LL and three high-frequency structures LH, HL, and HH in three directions. Then, the four subbands are concatenated along the channel dimension and sequentially processed through depthwise separable convolution and point convolution, multiplied by a learnable scalar coefficient. This is done to adjust the frequency domain processing intensity at this level. Then, the convolutional splicing result is adjusted according to subbands to obtain the processed LL and three high frequencies. Finally, the processed frequency domain features are reconstructed back to the original spatial resolution using inverse wavelet transform to obtain the wavelet branch output. The above process can be represented as:

[0073]

[0074]

[0075]

[0076]

[0077] in Used to capture low-frequency information, Responsible for extracting horizontal edge features, It is responsible for extracting vertical edge features, and Extract diagonal edge features. This indicates a depthwise separable convolution operation. This represents a point convolution operation. This represents a learnable scalar.

[0078] The output feature map corresponding to each sub-feature By using the output feature maps from various GSM branches Output feature maps from each WFT branch The features are obtained by adding them together, and finally, the final output features are obtained by adding each sub-output feature together. .

[0079]

[0080] in, and These represent the first in FAEM. Output characteristics of the GSM and WFT branches of the layer.

[0081] For the Multi-Scale Feature Enhancer (MSFE), this application proposes a composite attention bridging architecture - MSFE, which aims to effectively enhance the four-layer multi-scale feature representation output by the encoder through a parallel dual attention mechanism and a selective feature fusion strategy, and then feed the optimized representation into the decoder for further processing.

[0082] Specifically, for input features First, it goes through a dual attention mechanism of parallel spatial attention and channel attention. Figure 4 The structure of spatial attention is demonstrated. First, the input features are... Global average pooling and max pooling operations are performed simultaneously along the channel dimension. The resulting two single-channel feature maps are then concatenated along the channel dimension to form a dual-channel feature map. Next, regarding A shared dilated convolutional layer is applied, and a spatial attention map is generated using the sigmoid activation function. Finally, the spatial attention map Compared with the original feature map Perform element-wise multiplication and add it back to the original features in the form of residual connections to obtain the enhanced features. The propagation process of input features in SAB can be described as follows:

[0083]

[0084]

[0085]

[0086] in This represents the input features of the MSFE, derived from the output feature maps of different stages of the encoder; and These represent average pooling and max pooling operations, respectively. Indicates feature concatenation operation; This represents a 3×3 convolutional layer; It is the sigmoid activation function; It is element-wise multiplication.

[0087] Figure 4 The structure of channel attention is shown. It can be seen that CAB first processes the input features. Global average pooling is performed; then, a cross-stage global channel representation is formed by cascading along the channel axis. Then, the global channel representation D is sequentially fed into the convolutional layer and the fully connected layer, and after passing through the Sigmoid activation function, it is obtained as follows: Finally and original feature map Perform element-wise multiplication and add it back to the original features in the form of residual connections to obtain the enhanced features. The propagation process of input features in CAB can be described as follows:

[0088]

[0089]

[0090]

[0091] in This refers to the global average pooling operation.

[0092] Subsequently, the features enhanced by the dual attention mechanism were concatenated to obtain... Next, a dynamic weight estimation mechanism adaptively balances the importance of spatial attention and channel attention, and two weight values ​​are generated through a 1×1 convolutional layer. and The final output is obtained through weighted fusion and residual connection. The above process can be described as follows:

[0093]

[0094]

[0095]

[0096] in, Indicates spatial and channel attention weights. . Indicates original features. Figure 4 Parallel dual attention mechanism in MSFE.

[0097] To improve the model's segmentation accuracy of target edge details, this application designs the BAAB component, which constructs an explicit edge information learning path by embedding an edge-aware module in the deep feature flow, helping the model to more accurately capture and represent target edge detail features. Figure 2 As shown in (a), the feature map of the last layer of the encoder is first processed. The Laplacian operator is applied to obtain the feature map after edge enhancement. The Laplacian operator enhances the edge information of the input features through learnable convolution operations and a fixed Laplacian operator, aiming to highlight the boundary difference between the target and the background and lay the foundation for subsequent edge feature extraction. The above calculation process can be represented as:

[0098]

[0099] in, This represents the Laplacian operator convolution operation. This represents the feature map after edge enhancement.

[0100] Next, the enhanced feature map is blurred by passing it through two 3×3 depthwise separable convolutional layers and an upsampling operation to obtain low-pass features. Then, after concatenating with the original features, a 3×3 convolutional layer is used to directly regress the optical flow; this optical flow is then used to sample the original features to obtain the main body segmentation features. The residual is the edge feature. The main features preserve the overall structural information, while the edge features focus on the detailed representation of the target contour. This design robustly guides optical flow estimation with a shallow low-pass prior and achieves main body alignment through differentiable resampling; subsequently, edges are explicitly extracted in residual form, achieving structure-edge decoupling and complementary representation. The above calculation process can be expressed as:

[0101]

[0102]

[0103]

[0104] in, This indicates bilinear interpolation upsampling. This represents a differentiable spatial resampling operation based on optical flow. Represents the main segmentation features. This represents edge segmentation features.

[0105] Then, to compensate for the edge localization error caused by the loss of deep feature resolution, the separated edge features are upsampled to 4 times the downsampled resolution through bilinear interpolation; subsequently, the second-stage output features of the encoder, which have finer spatial detail information, are introduced. The channel dimensions are adjusted using a 1×1 convolutional layer. Finally, the upsampled edge features and the adjusted high-resolution features are concatenated along the channel dimension, followed by channel adjustment using a 1×1 convolutional layer to obtain the final output features. This design can utilize The high-resolution characteristics of the feature assist in the spatial transformation of edge features, further calibrating the spatial location accuracy of edge information and avoiding boundary blurring caused by single-scale features. The above calculation process can be expressed as:

[0106]

[0107] in, This represents the feature map of the second stage of the encoder. This indicates that a splicing operation is performed along the channel dimension. This represents the edge features after fusion.

[0108] Because the rock foreground category accounts for a smaller proportion, while the background dominates, this imbalance can lead to learning bias during training, causing the network to easily overlook smaller categories. To alleviate the semantic category imbalance problem, this application introduces a dice loss function. It is highly effective for tasks with unevenly distributed classes. Combining it with cross-entropy loss... Combined, this forms the total segmentation loss. To more effectively optimize the model, this application addresses the issue that the number of pixels at object edges is significantly less than those at non-edges, leading to an imbalance in the distribution of edge prediction categories. To resolve this, the application utilizes binary cross-entropy loss, which is suitable for the binary properties of edge classification, and jointly optimizes dice loss during edge learning to construct a boundary detection loss. Total loss This can be explicitly expressed by the following formula:

[0109]

[0110]

[0111]

[0112] in and These represent the segmentation and boundary detection losses, respectively. These are hyperparameters, used to adjust... In this embodiment of the invention, the proportion of the total loss function is set to 0.5.

[0113] The embodiments of the present invention have the following beneficial effects:

[0114] 1. This invention proposes a Martian rock segmentation algorithm M. 2 Rock-Mamba is an algorithm consisting of three core components: FAEM, MSFE, and BAAB. These modules work together to achieve an optimal balance between segmentation accuracy and model lightweighting.

[0115] 2. FAEM combines the spatial awareness capabilities of CNN with the sequence modeling advantages of Mamba, and at the same time suppresses complex background noise on Mars through a frequency domain-assisted enhancement mechanism, thereby achieving accurate capture of the geometric contours and texture features of rocks.

[0116] 3. MSFE effectively enhances the multi-scale feature representation of the encoder output through a parallel dual attention mechanism and an adaptive feature selection strategy;

[0117] 4. BAAB, as an auxiliary decoding branch, models the semantics of rock edges through explicit boundary supervision signals, thereby improving the contour integrity and detail accuracy of the segmentation results.

[0118] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for automatic rock extraction based on deep learning and Mars rover camera imagery, characterized in that, The method includes: The Mars rover's camera captures images of the Martian surface as input images. The input image is fed into a lightweight encoder-decoder segmentation network. The encoder of this network consists of four frequency-assisted Mamba modules stacked together to simultaneously extract and fuse global sequence dependency features of the image with local frequency domain texture features enhanced by wavelet transform. The multi-scale features output by the encoder are fed into a multi-scale feature enhancer, and the multi-scale features are adaptively enhanced and fused through a parallel spatial attention mechanism and a channel attention mechanism. The enhanced multi-scale features are input into a main decoding branch and a boundary-aware auxiliary branch, respectively. The main decoding branch restores the feature map resolution through upsampling and skip connections to generate a rock body segmentation mask; The boundary-aware auxiliary branch decouples edge features by performing edge enhancement and optical flow-guided resampling on deep features, and then fuses them with high-resolution encoder features to generate a rock edge prediction map. By fusing the rock body segmentation mask with the rock edge prediction map, the final rock extraction result at the pixel level is output; The frequency-assisted enhancement Mamba module executes the following two sub-processes in parallel: Global space modeling sub-process: After serializing the input features, long-range dependency modeling is performed using the Mamba state space model based on a selective scanning mechanism; Wavelet frequency enhancement sub-process: The input features are decomposed into two-dimensional discrete wavelet to obtain low-frequency and high-frequency sub-bands in multiple directions. The convolutional processing of the spliced ​​sub-bands is then performed followed by inverse wavelet transform to enhance edge and texture information. The output features of the two sub-processes are added element by element to form the output of this module; The execution process of the multi-scale feature enhancer includes: Spatial attention weighting and channel attention weighting are applied to the input features in parallel to obtain two sets of enhanced features; A dynamic weight estimation network is used to adaptively generate fusion weights for the two sets of enhancement features; The two sets of features are weighted and summed according to the fusion weights, and then residually connected with the original input features to output the final enhanced features; The specific steps of the boundary-aware auxiliary branch include: Laplacian convolution is applied to the deepest features output by the encoder to perform initial edge enhancement. The enhanced features are blurred to regress the optical flow field, and the original features are resampled using the optical flow field to obtain the main structural features. Subtracting the original feature from the main structural feature yields the explicit residual edge feature; The residual edge features are concatenated and fused with higher-resolution features from the encoder to improve edge localization accuracy.

2. The method for automatic rock extraction based on deep learning and Mars rover camera images as described in claim 1, characterized in that, The method uses a combined loss function when training the segmentation network, and its expression is: L total =L seg +αL bound Among them, L seg L is the sum of the dice loss and cross-entropy loss used to supervise the segmentation of the rock mass. bound The sum of the binary cross-entropy loss and the dice loss used to supervise rock edge prediction is α, where α is the weighting coefficient balancing the two losses.

3. The method for automatic rock extraction based on deep learning and Mars rover camera images as described in claim 1, characterized in that, The encoder-decoder segmentation network adopts a four-level downsampling and upsampling structure, with the basic number of channels set to 8, 16, 24, and 32. It also reduces the number of parameters through depthwise separable convolution to adapt to the onboard computing resources of the Mars rover.

4. The method for automatic rock extraction based on deep learning and Mars rover camera images as described in claim 1, characterized in that, The final rock extraction results include binarized rock region masks and rock contour vector information, which are used to support the statistical analysis of geological composition on the Martian surface and the planning of the rover's path.

5. The method for automatic rock extraction based on deep learning and Mars rover camera images as described in claim 1, characterized in that, The method is deployed on the Mars rover's embedded computing platform to enable online, real-time detection and extraction of rocks on the Martian surface.

6. An automated rock extraction system based on deep learning and Mars rover camera imagery, for implementing the method according to any one of claims 1-5, characterized in that, The system includes: Image input interface, used to receive images from the Mars rover's camera; A processor and a memory, wherein the memory stores a computer program that can run on the processor, and when the program is executed by the processor, implements the steps of the automatic rock extraction method as described in any one of claims 1-5; The results output interface is used to output the rock extraction results.

7. The automatic rock extraction system based on deep learning and Mars rover camera imagery as described in claim 6, characterized in that, The processor is an edge computing AI chip integrated inside the Mars rover, and the system constitutes part of the Mars rover's autonomous visual perception subsystem.

Images