Lightweight network-based Mars surface stone extraction method and system

Abstract

The invention relates to the technical field of mars surface rocks, and particularly discloses a mars surface stone extraction method and system based on a lightweight network, and the method comprises the steps: obtaining a mars surface image shot by a mars patroller, and screening a rock image; making labels based on the rock images, performing image processing on the labels and the rock images to expand the number of the images, and dividing the rock images into a training set, a test set and a verification set; dynamically adjusting the sampling shape and size of a convolution kernel of the training set through adaptive kernel convolution so as to extract rock features in the rock image; separating the rock image from the surrounding environment by using an improved lightweight Transform model; the attention of rocks covered by sand and dust is improved through an enhanced multi-dimensional convolution attention mechanism; and evaluating the lightweight Transform model based on the pre-training index, if the evaluation passes, obtaining a final model, and based on the final model, carrying out rock surface extraction on the input Mars rock image.

Description

Technical Field

[0001] The present invention relates to the field of Martian surface rock technology, and in particular to a Martian surface rock extraction method and system based on a lightweight network. Background Art

[0002] The rocks on Mars are both important objects of scientific research and the main obstacle for the Mars rover. As the eyes of the Mars rover, the navigation camera needs to obtain a large amount of image information for path planning and subsequent scientific tasks. Therefore, segmenting the rocks from the image is the key to subsequent exploration tasks. Image segmentation methods in Mars exploration can also be divided into traditional methods and deep learning methods. The most classic practical application of traditional methods is the rock image segmentation algorithm Rockster (Rock Segmentation Through Edge Regrouping) deployed on the AEGIS system on the Mars rover launched by NASA, but it is not used much due to the limitations of the system itself. The Rockster algorithm is mainly based on the Canny edge detector.

[0003] With the introduction of deep learning and visual transformers, many CNN and transformer-based methods have been proposed and used in practical tasks. Kiri et al. from NASA's Jet Propulsion Laboratory used transfer learning to adjust the convolutional neural network AlexNet for Mars images to achieve the classification of Mars images. In addition, NASA and ESA also applied deep convolutional neural networks to the Mars terrain classification task. NASA's Soil Property and Object Classification (SPOC) task is used to process MRO HiRISE orbital images and MSL Navcam Mars surface images, and is applied to the landing site traversability analysis of the Mars 2020 rover mission and the slip prediction analysis of the Mars Science Laboratory (MSL) mission. ESA's NOAH-H (Noveltyor Anomaly Hunter – HiRISE) terrain classification system uses deep neural networks to perform semantic segmentation of HiRISE orbital images to serve the subsequent ExoMars landing mission. DeLatte et al. used the classic U-Net network to segment craters. Furlán et al. optimized U-Net to reduce processing time. Another classic network structure, the Deeplab series of algorithms, has also been applied to Mars image segmentation. Ebadi et al. used the DeepLabV3+ framework to perform semantic segmentation of the skyline of Mars vista images to achieve high-precision positioning of the Mars rover in extreme and GPS-denied environments. Liu et al. designed global and local attention modules to process homogeneous and heterogeneous pixels and aggregate them respectively to achieve good results in panoramic terrain segmentation on Mars, and proposed an MRISNet that combines generative adversarial networks, attention mechanisms, and simple linear iterative clustering (SLIC) algorithms to improve rock segmentation accuracy.

[0004] However, current Mars rock segmentation networks focus on using complex classification networks as encoders to extract multi-scale features, and designing complex decoder networks to establish connections between multiple scales. These improvements come at the expense of complex hardware equipment and a large number of parameters, and do not take into account dust coverage and the difficulty of actual deployment. Therefore, this application proposes a lightweight Mars rock semantic segmentation network.

[0005] Problems found: There are still many problems to be solved in the segmentation of Martian rocks: First, rocks have various shapes, and the network does not take into account the impact of the fixed shape of ordinary convolution on the recognition of different rocks.

[0006] Second, most current networks use a U-shaped structure, and the Mars exploration mission needs to quickly process a large amount of image data. At the same time, the hardware resources it carries are extremely limited. The U-shaped structure is not conducive to improving computing efficiency.

[0007] Third, the robustness of the network structure is weak, and the rock recognition rate is significantly reduced when the rock is covered by fine sand. Summary of the invention

[0008] To achieve the purpose of the present invention, the present application provides a method for extracting rocks from the surface of Mars based on a lightweight network, comprising: Step S1: Acquire the Martian surface image and filter the rock image; Step S2: creating labels based on the rock images, performing image processing on the labels and rock images to expand the number of images, and dividing the rock images into a training set, a test set, and a validation set; Step S3: dynamically adjusting the sampling shape and size of the convolution kernel of the training set through adaptive kernel convolution to extract rock features in the rock image; Step S4: Separating the rock image from the surrounding environment using an improved lightweight Transformer model; Step S5: Improve the attention of rocks covered by sand and dust through the enhanced multi-dimensional convolutional attention mechanism; Step S6: Evaluate the lightweight Transformer model based on the pre-training indicators. If the evaluation passes, a final model is obtained. Based on the final model, rock surface extraction is performed on the input Martian rock image.

[0009] In some specific embodiments, step S3 includes: Step S31: taking the received image as input, randomly generating initial coordinates and defining initial sampling positions, and obtaining an initial convolution kernel size; Step S32: Calculating a base integer based on the initial convolution kernel size; Step S33: determining the position of the two-dimensional grid based on the initial convolution kernel size and the base integer; Step S34: performing a convolution operation on the received image to obtain an offset; Step S35: determining new sampling coordinates based on the offset, the initial sampling position, and the position of the two-dimensional grid; Step S36: resampling the rock image based on the new sampling coordinates; Step S37: The resampled rock image is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

[0010] In some specific embodiments, step S4 includes: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

[0011] In some specific embodiments, step S41 includes: Step S411: low-pass filtering the information based on average pooling and channel grouping to ensure that the dimension and fine details of the regenerated image are consistent with the original rock image; Step S412: performing high-pass filtering on the information based on convolution kernels of different dimensions; Step S413: selecting frequency information related to semantic segmentation based on the frequency similarity kernel; Step S414: superimpose the low-pass information, the high-pass information and the selected frequency information to obtain spatial domain information of different frequency bands.

[0012] In some specific embodiments, step S5 includes: Step S51: The input rock image is divided into three branches. The first branch is rotated 90° counterclockwise along channel H to obtain F 1 The second branch rotates 90° counterclockwise along channel W to obtain F 2 , the third branch keeps the original features unchanged to obtain F 3 ; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: generating new values ​​by using an adaptive combination mechanism for the results of the three branches after the squeeze transformation to determine the results of average pooling, standard pooling, and maximum pooling; Step S54: The three new values ​​are respectively passed through the activation function and averaged to determine the feature map of the rock image covered by sand and dust; Step S55: classify the feature map through depthwise separable convolution to obtain a label map of the rock image covered by sand and dust.

[0013] In some specific embodiments, in step S6, the pre-training indicators include: intersection over union (IoU), precision (Precision), pixel accuracy (PA), recall (Recall), and harmonic mean (F) of precision and recall. 1 ;in, ; ; ; ; ; Where TP is the number of positive instances correctly identified as positive examples, FP is the number of negative instances misclassified as positive examples, TN is the number of instances correctly identified as negative examples, and FN is the number of positive instances misclassified as negative examples.

[0014] To achieve the same invention purpose, the present application also provides a Mars surface stone extraction system based on a lightweight network, comprising: Image screening module: used for Mars surface images and screening rock images; Image classification module: used for creating labels based on the rock images, performing image processing on the labels and rock images to expand the number of images, and dividing the rock images into a training set, a test set, and a validation set; Feature extraction module: used to dynamically adjust the sampling shape and size of the convolution kernel of the training set through adaptive kernel convolution to extract rock features in the rock image; Image separation module: used for separating the rock image from the surrounding environment by using an improved lightweight Transformer model; Feature weighting module: used to increase the weight of local features of covered rocks through an enhanced multi-dimensional convolutional attention mechanism; Model evaluation module: used to evaluate the lightweight Transformer model based on pre-training indicators. If the evaluation passes, a final model is obtained, and rock surface extraction is performed on the input Martian rock image based on the final model.

[0015] In some specific embodiments, the feature extraction module is used to perform the following steps: Step S31: taking the received image as input, randomly generating initial coordinates and defining initial sampling positions, and obtaining an initial convolution kernel size; Step S32: Calculating a base integer based on the initial convolution kernel size; Step S33: determining the position of the two-dimensional grid based on the initial convolution kernel size and the base integer; Step S34: performing a convolution operation on the received image to obtain an offset; Step S35: determining new sampling coordinates based on the offset, the initial sampling position, and the position of the two-dimensional grid; Step S36: resampling the rock image based on the new sampling coordinates; Step S37: The resampled rock image is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

[0016] In some specific embodiments, the image separation module is used to perform the following steps: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

[0017] In some specific embodiments, the feature weighting module is used to perform the following steps: Step S51: The input rock image is divided into three branches. The first branch is rotated 90° counterclockwise along channel H to obtain F 1 The second branch rotates 90° counterclockwise along channel W to obtain F 2 , the third branch keeps the original features unchanged to obtain F 3 ; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: generating new values ​​by using an adaptive combination mechanism for the results of the three branches after the squeeze transformation to determine the results of average pooling, standard pooling, and maximum pooling; Step S54: The three new values ​​are respectively passed through the activation function and averaged to determine the feature map of the rock image covered by sand and dust; Step S55: classify the feature map through depthwise separable convolution to obtain a label map of the rock image covered by sand and dust.

[0018] Beneficial effects: (1) In the preprocessing process, this application proposes a variable kernel convolution, which provides a convolution kernel with arbitrary sampling shape and size, replaces the ordinary convolution, and makes up for the shortcomings of the conventional convolution, so that it can effectively extract features for rocks of different scales.

[0019] (2) This application uses an improved Transformer model, AFFormer, as the backbone network for Martian rock semantic segmentation. This model distinguishes rocks from background by learning the frequency information of different feature categories, and achieves excellent performance using fewer parameters.

[0020] (3) This application proposes an improved spatial attention mechanism (MCA) that uses a new multi-dimensional attention mechanism to learn three types of attention for all convolution kernels in the convolution kernel space, thereby providing higher attention to rocks covered by dust. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic flow chart of a method for extracting rocks from the surface of Mars based on a lightweight network provided by an embodiment of the present invention; Figure 2 A flowchart of an attention mechanism for a method for extracting rocks from the surface of Mars based on a lightweight network provided by an embodiment of the present invention; Figure 3 An overall network architecture diagram of a method for extracting rocks from the surface of Mars based on a lightweight network provided by an embodiment of the present invention; Figure 4 A comparison chart of experimental results of a method for extracting rocks from the surface of Mars based on a lightweight network provided by an embodiment of the present invention; Figure 5 A schematic structural diagram of a lightweight network-based Martian surface stone extraction system provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0024] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0025] Embodiment 1 An embodiment of the present invention provides a method for extracting rocks from the surface of Mars based on a lightweight network. Figure 1-Figure 3 As shown, including: Step S1: Obtain the Martian surface images taken by the Mars rover and filter the rock images.

[0026] Step S2: creating labels based on the rock images, performing image processing on the labels and rock images to expand the number of images, and dividing the rock images into a training set, a test set, and a validation set.

[0027] Step S3: dynamically adjust the sampling shape and size of the convolution kernel of the training set through adaptive kernel convolution to extract rock features in the rock image.

[0028] In a specific embodiment of the present invention, step S3 includes: Step S31: Take the received image as input x, randomly generate initial coordinates to define the initial sampling position P 0 , the initial convolution kernel size is obtained as k; Step S32: Calculate the base integer B based on the initial convolution kernel size k, the formula is: ; Step S33: Based on The value of Calculate the new 2D grid P n , that is, P n for( , ); Step S34: performing a convolution operation on the received image to obtain an offset Offset; Step S35: Based on the offset Offset and the initial sampling position P 0 , the new two-dimensional grid P n Get the new sampling coordinate P, the formula is ; Step S36: resampling the feature map based on the new sampling coordinates P; Step S37: The resampled feature map is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

[0029] Step S4: Separate the rock image from the surrounding environment using an improved lightweight Transformer model. The present application uses an adaptive frequency filter with linear complexity to replace the standard adaptive mechanism to reduce the number of model parameters and reduce computational complexity.

[0030] In a specific embodiment of the present invention, step S4 includes: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

[0031] In a specific embodiment of the present invention, step S41 includes: Step S411: low-pass filtering the information based on average pooling and channel grouping to ensure that the dimension and fine details of the regenerated image are consistent with the original rock image; Step S412: performing high-pass filtering on the information based on convolution kernels of different dimensions; Step S413: selecting frequency information related to semantic segmentation based on the frequency similarity kernel; Step S414: superimpose the low-pass information, the high-pass information and the selected frequency information to obtain spatial domain information of different frequency bands.

[0032] Step S5: Increase the weight of the local features of the covered rocks through the enhanced multi-dimensional convolutional attention mechanism.

[0033] In a specific embodiment of the present invention, step S5 includes: Step S51: The input rock image F is divided into three branches. The first branch F is rotated 90° counterclockwise along the channel H to obtain F 1 , the second branch F rotates 90° counterclockwise along channel W to obtain F 2 , the third branch F keeps the original features unchanged to obtain F 3 ; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: The results of the three pooling operations are sent to the adaptive combination mechanism to generate a new value, the formula of which is: ,in is the result of average pooling, is the result of standard pooling, is the result of maximum pooling; Step S54: After the three new values ​​are respectively passed through the activation function, they are averaged to obtain the final enhanced output feature map; Step S55: The feature map is classified through depth-wise separable convolution to obtain the final label map.

[0034] Step S6: Evaluate the lightweight Transformer model based on the pre-training indicators. If the evaluation passes, a final model is obtained. Based on the final model, rock surface extraction is performed on the input Martian rock image.

[0035] In a specific embodiment of the present invention, referring to Figure 4 As shown in the figure, among the latest Mars rock segmentation models, this application selected the classic NI-U-Net++ and the relatively new MarsNet as representatives for comparison. Specifically, in the TWMARS-V2 dataset, the accuracy of NI-U-Net++ reached 88.5%, while the accuracy of MarsNet was 92.58%. It can be clearly seen that MarsNet does have higher accuracy and precision than some mainstream segmentation models.

[0036] The Swin Transformer also showed some performance, with a pixel accuracy (PA) of 88.5%, while MarsNet had an accuracy of 90.2%. Compared with traditional convolution-based models, such as the Unet-based FastFCN model, which has a pixel accuracy of only 85.36%, the Unet model has a pixel accuracy of 80.54%. However, the model proposed in this application showed a more superior performance in this task.

[0037] The model of this application performs well in the Mars rock segmentation task. On the MarsData-V2 dataset, the pixel accuracy of the model reaches 98.18%, which is 4.89% higher than MarsNet and 5.51% higher than NI. Further in-depth comparison of the IOU (Intersection over Union, IoU) indicator with other models shows that the model of this application reaches 96.62%. In comparison, the IOU of MarsNet is 90.26% and the IoU of NI-U-Net++ is 89.05%. Among the traditional convolution-based models, the IOU of Deeplabv3+ is 94.3%, the IoU of EMO is 90.24%, and the IoU of FastFCN is 87.56%; among the unet-based models, the IOU of PspNet is 92.11%; among the transformer-based models, the IOU of SegFormer is 92.8%, the IOU of VisionTransformer is 91.3%, and the IOU of Swin Transformer is 93.38%. The model of this application outperforms many comparative models in terms of IOU metrics, which fully demonstrates that the model of this application can not only accurately identify Martian rock pixels, but also well define the boundaries of rock areas. This further proves the advantage of the model of this application in segmentation accuracy. It can not only accurately identify Martian rock pixels, but also more accurately define the boundaries of rock areas. In terms of accuracy, for the main categories of Martian rocks, the model of this application achieved an extremely high accuracy of 98.64%. In comparison, MarsNet has an accuracy of 91.53% for this category, and NI-U-Net++ has an accuracy of 91.20%. This shows that the model of this application has a significant advantage in the classification accuracy of Martian rocks, and can more effectively distinguish rock pixels from the complex Martian background.

[0038] Through comparative experiments with various advanced models, this application fully demonstrates the effectiveness and superiority of the proposed model in the Martian rock segmentation task, providing a more powerful technical means for rock identification and analysis in Mars exploration. In subsequent research, this application will further explore the optimization direction of the model, further improve its performance, and expand its scope of application.

[0039] Embodiment 2 This application also provides a Mars surface stone extraction system based on a lightweight network, referring to Figure 5 As shown, including: Image screening module 10: used to obtain the Martian surface images taken by the Mars rover and screen the rock images; Image classification module 20: used to create labels based on the rock images, perform image processing on the labels and rock images to expand the number of images, and divide the rock images into a training set, a test set, and a validation set; Feature extraction module 30: used to dynamically adjust the sampling shape and size of the convolution kernel through adaptive kernel convolution of the training set to extract rock features in the rock image; Image separation module 40: used for separating the rock image from the surrounding environment by using an improved lightweight Transformer model; Feature weighting module 50: used to increase the weight of local features of covered rocks through an enhanced multi-dimensional convolutional attention mechanism; Model evaluation module 60: used to evaluate the lightweight Transformer model based on pre-training indicators. If the evaluation passes, a final model is obtained, and based on the final model, rock surface extraction is performed on the input Martian rock image.

[0040] In some specific embodiments, the feature extraction module 30 is used to perform the following steps: Step S31: taking the received image as input, randomly generating initial coordinates and defining initial sampling positions, and obtaining an initial convolution kernel size; Step S32: Calculating a base integer based on the initial convolution kernel size; Step S33: determining the position of the two-dimensional grid based on the initial convolution kernel size and the base integer; Step S34: performing a convolution operation on the received image to obtain an offset; Step S35: determining new sampling coordinates based on the offset, the initial sampling position, and the position of the two-dimensional grid; Step S36: resampling the rock image based on the new sampling coordinates; Step S37: The resampled rock image is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

[0041] In some specific embodiments, the image separation module 40 is used to perform the following steps: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

[0042] In a specific embodiment of the present invention, step S41 includes: Step S411: low-pass filtering the information based on average pooling and channel grouping to ensure that the dimension and fine details of the regenerated image are consistent with the original rock image; Step S412: performing high-pass filtering on the information based on convolution kernels of different dimensions; Step S413: selecting frequency information related to semantic segmentation based on the frequency similarity kernel; Step S414: superimpose the low-pass information, the high-pass information and the selected frequency information to obtain spatial domain information of different frequency bands.

[0043] Step S5: Increase the weight of the local features of the covered rocks through the enhanced multi-dimensional convolutional attention mechanism.

[0044] In some specific embodiments, the feature weighting module 50 is used to perform the following steps: Step S51: The input rock image is divided into three branches. The first branch is rotated 90° counterclockwise along channel H to obtain F 1 The second branch rotates 90° counterclockwise along channel W to obtain F 2 , the third branch keeps the original features unchanged to obtain F 3 ; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: generating new values ​​by using an adaptive combination mechanism for the results of the three branches after the squeeze transformation to determine the results of average pooling, standard pooling, and maximum pooling; Step S54: The three new values ​​are respectively passed through the activation function and averaged to determine the feature map of the rock image covered by sand and dust; Step S55: classify the feature map through depthwise separable convolution to obtain a label map of the rock image covered by sand and dust.

[0045] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0046] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other. The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention. Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0047] The method and device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0048] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "one specific embodiment" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for extracting rocks from the surface of Mars based on a lightweight network, characterized in that: include: Step S1: Acquire the Martian surface image and filter the rock image; Step S2: creating labels based on the rock images, performing image processing on the labels and the rock images to expand the number of images, and dividing the rock images into a training set, a test set, and a validation set; Step S3: dynamically adjusting the sampling shape and size of the convolution kernel of the training set through adaptive kernel convolution to extract rock features in the rock image; Step S4: Separating the rock image from the surrounding environment using an improved lightweight Transformer model; Step S5: Improve the attention of rocks covered by sand and dust through the enhanced multi-dimensional convolutional attention mechanism; Step S6: Evaluate the lightweight Transformer model based on the pre-training indicators. If the evaluation passes, a final model is obtained. Based on the final model, rock surface extraction is performed on the input Martian rock image.

2. The method for extracting rocks from the surface of Mars based on a lightweight network according to claim 1, characterized in that: Step S3 includes: Step S31: taking the received image as input, randomly generating initial coordinates and defining initial sampling positions, and obtaining an initial convolution kernel size; Step S32: Calculating a base integer based on the initial convolution kernel size; Step S33: determining the position of the two-dimensional grid based on the initial convolution kernel size and the base integer; Step S34: performing a convolution operation on the received image to obtain an offset; Step S35: determining new sampling coordinates based on the offset, the initial sampling position, and the position of the two-dimensional grid; Step S36: resampling the rock image based on the new sampling coordinates; Step S37: The resampled rock image is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

3. The method for extracting rocks from the surface of Mars based on a lightweight network according to claim 2, characterized in that: Step S4 includes: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

4. The method for extracting rocks from the surface of Mars based on a lightweight network according to claim 3, characterized in that: Step S41 includes: Step S411: low-pass filtering the information based on average pooling and channel grouping to ensure that the dimension and fine details of the regenerated image are consistent with the original rock image; Step S412: performing high-pass filtering on the information based on convolution kernels of different dimensions; Step S413: selecting frequency information related to semantic segmentation based on the frequency similarity kernel; Step S414: superimpose the low-pass information, the high-pass information and the selected frequency information to obtain spatial domain information of different frequency bands.

5. The method for extracting rocks from the surface of Mars based on a lightweight network according to claim 1, characterized in that: Step S5 includes: Step S51: the input rock image is divided into three branches, the first branch is rotated 90° counterclockwise along channel H to obtain F1, the second branch is rotated 90° counterclockwise along channel W to obtain F2, and the third branch keeps the original features unchanged to obtain F3; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: generating new values ​​by using an adaptive combination mechanism for the results of the three branches after the squeeze transformation to determine the results of average pooling, standard pooling, and maximum pooling; Step S54: The three new values ​​are respectively passed through the activation function and averaged to determine the feature map of the rock image covered by sand and dust; Step S55: classify the feature map through depthwise separable convolution to obtain a label map of the rock image covered by sand and dust.

6. The method for extracting rocks from the surface of Mars based on a lightweight network according to claim 1, characterized in that: In step S6, the pre-training indicators include: intersection over union (IoU), precision (Precision), pixel accuracy (PA), recall (Recall), and harmonic mean (F1) of precision and recall; ； ； ； ； ； Where TP is the number of positive instances correctly identified as positive examples, FP is the number of negative instances misclassified as positive examples, TN is the number of instances correctly identified as negative examples, and FN is the number of positive instances misclassified as negative examples.

7. A Mars surface stone extraction system based on lightweight network, characterized in that: include: Image screening module: used to obtain images of the Martian surface and screen rock images; Image classification module: used for creating labels based on the rock images, performing image processing on the labels and rock images to expand the number of images, and dividing the rock images into a training set, a test set, and a validation set; Feature extraction module: used to dynamically adjust the sampling shape and size of the convolution kernel of the training set through adaptive kernel convolution to extract rock features in the rock image; Image separation module: used for separating the rock image from the surrounding environment by using an improved lightweight Transformer model; Feature weighting module: used to increase the weight of local features of covered rocks through an enhanced multi-dimensional convolutional attention mechanism; Model evaluation module: used to evaluate the lightweight Transformer model based on pre-training indicators. If the evaluation passes, a final model is obtained, and rock surface extraction is performed on the input Martian rock image based on the final model.

8. The Mars surface stone extraction system based on lightweight network according to claim 7 is characterized in that: The feature extraction module is used to perform the following steps: Step S31: taking the received image as input, randomly generating initial coordinates and defining initial sampling positions, and obtaining an initial convolution kernel size; Step S32: Calculating a base integer based on the initial convolution kernel size; Step S33: determining the position of the two-dimensional grid based on the initial convolution kernel size and the base integer; Step S34: performing a convolution operation on the received image to obtain an offset; Step S35: determining new sampling coordinates based on the offset, the initial sampling position, and the position of the two-dimensional grid; Step S36: resampling the rock image based on the new sampling coordinates; Step S37: The resampled rock image is reshaped, convolved again, and standardized, and finally the original pixel information is output through the activation function.

9. The Mars surface stone extraction system based on lightweight network according to claim 7, characterized in that: The image separation module is used to perform the following steps: Step S41: obtaining spatial domain information of different frequency bands of original pixel information through an improved lightweight Transformer model; Step S42: Integrate the spatial domain information with the original non-clustered pixel information.

10. The Mars surface stone extraction system based on lightweight network according to claim 7, characterized in that: The feature weighting module is used to perform the following steps: Step S51: the input rock image is divided into three branches, the first branch is rotated 90° counterclockwise along channel H to obtain F1, the second branch is rotated 90° counterclockwise along channel W to obtain F2, and the third branch keeps the original features unchanged to obtain F3; Step S52: The three branches are respectively subjected to squeezing transformation, wherein the squeezing transformation includes average pooling, standard pooling, and maximum pooling; Step S53: generating new values ​​by using an adaptive combination mechanism for the results of the three branches after the squeeze transformation to determine the results of average pooling, standard pooling, and maximum pooling; Step S54: The three new values ​​are respectively passed through the activation function and averaged to determine the feature map of the rock image covered by sand and dust; Step S55: classify the feature map through depthwise separable convolution to obtain a label map of the rock image covered by sand and dust.

Images