Mars terrain sensing method, device and equipment and storage medium

Abstract

The invention discloses a Mars terrain perception method, device and equipment and a storage medium, and relates to the field of environment perception, and the method comprises the steps: obtaining an initial environment image of a target Mars terrain, and extracting a target local feature in the initial environment image through a preset CBlock module in a target detection model; projecting the initial environment image into a query vector, a key vector and a value vector by using a preset attention mechanism of a preset Mblock module in the target detection model, adjusting the key vector and the value vector to extract feature information of each level in the initial environment image, and fusing the feature information of each level in the initial environment image based on the query vector to obtain a fused feature information of each level in the initial environment image; obtaining a target global feature; and inputting the target local feature and the target global feature into a preset semantic segmentation algorithm to obtain a target candidate frame containing the target object in the target Mars terrain so as to complete perception of the target Mars terrain. And the multi-scale feature representation and the terrain global feature information extraction capability are improved.

Description

Technical Field

[0001] The present invention relates to the field of environmental perception, and in particular to a Martian terrain perception method, device, equipment and storage medium. Background Art

[0002] With the development of artificial intelligence, deep learning technology has made breakthrough progress, providing important technical support for solving perception problems in various complex environments. Currently, perception methods in complex environments mainly include object detection based on convolutional neural networks and object detection based on Transformers.

[0003] Among them, methods based on convolutional neural networks mainly use convolutional layers to extract features at different levels in the output image. During the convolution calculation process, the convolution kernel moves on the two-dimensional features in a sliding window manner, which can perform fine-grained feature extraction of the Martian terrain. Transformer-based methods are mainly composed of multiple encoder and decoder structures. The input and output dimensions of each structure are the same. Therefore, the network depth can be stacked to deepen for different practical problems, and the self-attention mechanism is introduced to improve the network's feature extraction capabilities. Currently, most researchers are using the Transformer model or its variants to improve the efficiency of Martian global environment modeling, and have achieved good results in image classification, object detection, and semantic segmentation.

[0004] In summary, how to improve the representation of multi-scale features and the ability to extract global feature information of terrain is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the present invention aims to provide a Martian terrain perception method, apparatus, device, and storage medium that can improve the representation of multi-scale features and the ability to extract global feature information from terrain. The specific solution is as follows:

[0006] In a first aspect, the present application provides a method for Martian terrain perception, comprising:

[0007] Obtain an initial environment image of the target Martian terrain, and use a preset CBlock module in the target detection model to extract the target local features in the initial environment image;

[0008] Projecting the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model, adjusting the key vector and the value vector to extract feature information at each level in the initial environment image, and then fusing the feature information at each level in the initial environment image based on the query vector to obtain a global feature of the target;

[0009] The obtained target local features and the target global features are input into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

[0010] Optionally, projecting the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model includes:

[0011] The preset number of projections is determined using the preset multi-head attention mechanism in the preset Mblock module;

[0012] Determining a corresponding preset linear projection weight according to the preset projection quantity;

[0013] The initial environment image is projected into a query vector, a key vector, and a value vector based on the preset linear projection weights.

[0014] Optionally, projecting the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model includes:

[0015] Dividing the initial environment image into a preset number of divided environment images based on a preset image preprocessing technology;

[0016] Performing a preset linear projection operation on the divided environment image based on a first preset linear projection weight to obtain a query vector;

[0017] Performing a preset linear projection operation on the divided environment image based on a second preset linear projection weight to obtain a key vector;

[0018] A preset linear projection operation is performed on the divided environment image based on a second preset linear projection weight to obtain a value vector.

[0019] Optionally, adjusting the key vector and the value vector to extract feature information of each level in the initial environment image includes:

[0020] Directly performing a preset shape transformation operation on the key vector and the value vector to generate first-level feature information in the initial environment image;

[0021] Performing a preset shape transformation operation on the key vector and the value vector based on a preset convolution kernel size and step size to generate non-first-level feature information in the initial environment image;

[0022] The feature information of each level in the initial environment image is acquired according to the feature information of the first level and the feature information of the non-first level.

[0023] Optionally, performing a preset shape transformation operation on the key vector and the value vector based on a preset convolution kernel size and step size to generate feature information other than the first level in the initial environment image includes:

[0024] Determine the first convolution kernel size and first step length corresponding to the preset coarse granularity;

[0025] Determine the second convolution kernel size and the second step size corresponding to the preset fine granularity;

[0026] Performing a preset shape transformation operation on the key vector and the value vector based on the first convolution kernel size and the first step length to generate coarse-grained features containing global structural information;

[0027] Performing a preset shape transformation operation on the key vector and the value vector based on the second convolution kernel size and the second step size to generate fine-grained features containing local detail information;

[0028] The coarse-grained features are fused with the fine-grained features to obtain feature information of a non-first level in the initial environment image.

[0029] Optionally, fusing feature information of various levels in the initial environment image based on the query vector to obtain a target global feature includes:

[0030] Merging the feature information of each level in the initial environment image based on the query vector to obtain merged feature information;

[0031] Determining correlation information between feature information at each level in the initial environment image using a deformable convolution algorithm;

[0032] The merged feature information is optimized based on the association information to obtain a target global feature.

[0033] Optionally, the method further includes:

[0034] Divide the input image into a preset number of small blocks of the same size, and convert the small blocks of the same size into corresponding target low-dimensional vectors;

[0035] Based on the preset accuracy requirements and the obtained target low-dimensional vector, a small detection model, a medium detection model, and a basic detection model are constructed; wherein the small detection model corresponds to the preset low-precision requirement, the medium detection model corresponds to the preset medium-precision requirement, and the basic detection model corresponds to the preset high-precision requirement;

[0036] According to the accuracy requirements corresponding to the target Mars terrain perception mission, the corresponding target detection model is determined from the small detection model, the medium detection model and the basic detection model, and the target detection model is used to perform the target Mars terrain perception mission.

[0037] In a second aspect, the present application provides a Mars terrain sensing device, comprising:

[0038] A local feature extraction module is used to obtain an initial environment image of the target Martian terrain and extract the target local features in the initial environment image using a preset CBlock module in the target detection model;

[0039] a global feature acquisition module, configured to project the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model, adjust the key vector and the value vector to extract feature information at each level in the initial environment image, and then fuse the feature information at each level in the initial environment image based on the query vector to obtain a target global feature;

[0040] The Martian terrain perception completion module is used to input the obtained target local features and the target global features into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

[0041] In a third aspect, the present application provides an electronic device, comprising:

[0042] Memory, used to store computer programs;

[0043] A processor is used to execute the computer program to implement the aforementioned Mars terrain perception method.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned Mars terrain perception method.

[0045] In summary, the present application first obtains an initial environmental image of the target Martian terrain, and uses the preset CBlock module in the target detection model to extract the target local features in the initial environmental image; uses the preset attention mechanism of the preset Mblock module in the target detection model to project the initial environmental image into a query vector, a key vector and a value vector, adjusts the key vector and the value vector to extract the feature information of each level in the initial environmental image, and then fuses the feature information of each level in the initial environmental image based on the query vector to obtain the target global features; inputs the obtained target local features and the target global features into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain. As can be seen from the above, this application first obtains the initial environment image of the target Martian terrain, and then uses the preset CBlock module in the target detection model to extract the target local features therein, and then uses the preset attention mechanism in the preset Mblock module to project the initial environment image into a query vector, a key vector and a value vector, and then adjusts the second sum vector to extract the feature information of each level in the initial environment image, and then fuses the feature information of each level based on the query vector to obtain the target global features, and finally inputs the target local features and the target global features into the preset semantic segmentation algorithm to obtain the target candidate box containing the target object in the target Martian terrain, thereby completing the perception of the target Martian terrain. In this way, the target local features are extracted by the CBlock module, and the Mblock module uses the attention mechanism to extract and fuse features in layers, and finally combines the instance segmentation algorithm to accurately capture image details and global semantics, which is suitable for Martian terrain perception in complex scenes. At the same time, the overall computational complexity is effectively controlled. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0047] Figure 1 This is a flow chart of a Mars terrain perception method disclosed in this application;

[0048] Figure 2 This is a schematic diagram of a specific attention mechanism disclosed in this application;

[0049] Figure 3 This is a schematic structural diagram of a specific feature fusion disclosed in this application;

[0050] Figure 4This is a flowchart of a specific Mars terrain perception method disclosed in this application;

[0051] Figure 5 A schematic diagram of image segmentation of a different model disclosed in this application;

[0052] Figure 6 This is a diagram showing the overall framework of a Mars terrain perception method disclosed in this application;

[0053] Figure 7 This is a schematic structural diagram of a Mars terrain sensing device disclosed in this application;

[0054] Figure 8 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0056] At present, the perception methods in complex environments mainly include target detection based on convolutional neural networks and target detection methods based on Transformer. Among them, the method based on convolutional neural networks mainly uses convolution layers to extract features of different levels in the output image. During the convolution calculation, the convolution kernel moves on the two-dimensional features in a sliding window manner, which can perform fine-grained feature extraction on the Martian terrain. The method based on Transformer is mainly composed of a combination of multiple Encoder structures and Decoder structures. The input dimension and output dimension of each structure are the same. Therefore, the network depth can be stacked and deepened for different practical problems, and the self-attention mechanism is introduced to improve the network feature extraction capability. At present, most researchers are using Transformer models or their variants to improve the efficiency of Martian global environment modeling, and have achieved good results in image classification, target detection and semantic segmentation. In order to solve the above technical problems, the present application discloses a Martian terrain perception method, device, equipment and storage medium, which can improve the representation of multi-scale features and the ability to extract global feature information of the terrain.

[0057] See also Figure 1 As shown, an embodiment of the present invention discloses a method for sensing Martian terrain, including:

[0058] Step S11: Acquire an initial environment image of the target Martian terrain, and use a preset CBlock module in the target detection model to extract target local features in the initial environment image.

[0059] In this example, an initial environment image X∈R^(H×W×C) of the target Martian terrain of a preset size is first obtained. This image X is then fed into the preset CBlock module in the target detection model. The preset Cblock module uses ConvAttention and CMLP modules to extract local features of the target in the initial environment image. Here, H is the image length, W is the image width, and C is the number of channels.

[0060] Step S12: Use the preset attention mechanism of the preset Mblock module in the target detection model to project the initial environment image into a query vector, a key vector and a value vector, adjust the key vector and the value vector to extract the feature information of each level in the initial environment image, and then fuse the feature information of each level in the initial environment image based on the query vector to obtain the target global feature.

[0061] In this embodiment, after extracting the local features of the target from the initial environmental image, the initial environmental image is input into a preset Mblock module in the target detection model. The preset multi-head attention mechanism in the preset Mblock module can be used to determine a preset number of projections; corresponding preset linear projection weights are determined based on the preset number of projections; and based on the preset linear projection weights, the initial environmental image is projected into a query vector, a key vector, and a value vector. Specifically, the preset number of projections required is first determined. Then, based on the preset number of projections, the initial environmental image X is input using multiple sets of different attention weights. This is projected into three attention vectors: a query vector Q, a key vector K, and a value vector V.

[0062] Furthermore, based on a preset image preprocessing technique, the initial environment image is divided into a preset number of divided environment images; a preset linear projection operation is performed on the divided environment images based on a first preset linear projection weight to obtain a query vector; a preset linear projection operation is performed on the divided environment images based on a second preset linear projection weight to obtain a key vector; and a preset linear projection operation is performed on the divided environment images based on the second preset linear projection weight to obtain a value vector. Specifically, for a given initial environment image X, it is first divided into a preset number of divided parts H, and a query vector Q, a key vector K, and a value vector V are obtained by linearly projecting X based on the preset linear projection weights:

[0063] ;

[0064] ;

[0065] ;

[0066] Among them, Q is the query vector; K is the key vector; V is the value vector; X is the initial environment image; is the first preset linear projection weight; Preset linear projection weights for the second; Preset linear projection weights for the third; is a linear function.

[0067] It should be noted that the key vector and the value vector are directly subjected to a preset shape transformation operation to generate the first-level feature information in the initial environment image; the key vector and the value vector are subjected to a preset shape transformation operation based on the preset convolution kernel size and step size to generate the non-first-level feature information in the initial environment image; and the feature information of each level in the initial environment image is obtained based on the first-level feature information and the non-first-level feature information. Specifically, Figure 2 As shown in the figure, under the premise of keeping the query vector Q unchanged, the key vector K and the value vector V are divided into multiple scales respectively. First, the key vector K and the value vector V are directly transformed to obtain the key vector of the first level. Sum value vector For non-first-level key vectors Sum value vector , it is necessary to further introduce convolution operations with different convolution kernel sizes and step sizes to generate coarse-grained and fine-grained and .

[0068] ;

[0069] ;

[0070] Where i is a natural number, which represents the level in the formula; X is the initial environment image; is a linear function; is the convolution kernel; is the step length; is the second preset linear projection weight corresponding to the key vector K in a certain level; The third preset linear projection weight corresponding to the median vector V at a certain level; is the convolutional layer of the convolutional neural network; is the key vector of each level; is the value vector of each level.

[0071] It is understandable that in order to obtain coarse-grained and fine-grained and , it is necessary to determine the first convolution kernel size and the first step length corresponding to the preset coarse granularity; determine the second convolution kernel size and the second step length corresponding to the preset fine granularity; perform a preset shape transformation operation on the key vector and the value vector based on the first convolution kernel size and the first step length to generate a coarse-grained feature containing global structural information; perform a preset shape transformation operation on the key vector and the value vector based on the second convolution kernel size and the second step length to generate a fine-grained feature containing local detail information; fuse the coarse-grained feature with the fine-grained feature to obtain feature information of the non-first level in the initial environment image. For example, introducing different convolution kernel sizes and step length The convolution operation, that is, when the first convolution kernel size is 4 and the first step length is 4, obtains the coarse-grained key vector Sum value vector ; When the second convolution kernel size is 2 and the second step length is 2, a fine-grained key vector is obtained Sum value vector . The coarse-grained and fine-grained key vectors obtained Sum vector and the key vector at the first level Sum value vector Integrate to obtain feature information at each level:

[0072] ;

[0073] Then, if Figure 3 As shown, the feature information of each level in the initial environment image is merged based on the query vector to obtain merged feature information; the correlation information between the feature information of each level in the initial environment image is determined using a deformable convolution algorithm; and the merged feature information is optimized based on the correlation information to obtain the target global feature. Specifically, Q and K are first converted into probability distributions using the Softmax function, and Q and K are scaled as weights of V, which can effectively extract the most relevant information, thereby forming a representative merged feature information:

[0074] ;

[0075] in, is the dimension of the initial environment image; Q is the query vector; is the key vector of the first level; is the value vector of the first level; is the key vector of each level; is the value vector of each level; is the query vector at each level; Softmax() is the normalized exponential function; is the merged feature information.

[0076] Next, since the feature fusion process may lead to the loss or weakening of position information, a local position encoding path can be introduced. The DCN (Deformable Convolution) algorithm is used to enhance the correlation between features of different scales, obtain the correlation information between feature information at each level, and optimize the merged feature information based on the correlation information to obtain the target global feature:

[0077] ;

[0078] Among them, F is the target global feature; X is the initial environment image; DCN() is the deformable convolution algorithm; is the merged feature information.

[0079] Step S13: input the obtained target local features and the target global features into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

[0080] In this embodiment, after obtaining the target local features and the target global features, the final target local features and the target global features are further input into Mask R-CNN. Mask R-CNN detects the target object, obtains the target position and contour of the target object, generates an accurate target contour mask based on the target position and contour of the target object, and then generates a series of target candidate boxes containing the target object, thereby completing the recognition and positioning of the target object in the target Martian terrain.

[0081] As can be seen from the above, the embodiment of the present application first obtains the initial environment image of the target Martian terrain, and then uses the preset CBlock module in the target detection model to extract the target local features therein, and then uses the preset attention mechanism in the preset Mblock module to project the initial environment image into a query vector, a key vector and a value vector, and then adjusts the second sum value vector to extract the feature information of each level in the initial environment image, and then fuses the feature information of each level based on the query vector to obtain the target global features, and finally inputs the target local features and the target global features into the preset semantic segmentation algorithm to obtain the target candidate box containing the target object in the target Martian terrain, thereby completing the perception of the target Martian terrain. In this way, the target local features are extracted by the CBlock module, and the Mblock module uses the attention mechanism to extract and fuse features in layers, and finally combines the instance segmentation algorithm to accurately capture image details and global semantics, which is suitable for Martian terrain perception in complex scenes. At the same time, the overall computational complexity is effectively controlled.

[0082] See also Figure 4As shown, in order to obtain good environmental perception performance in different mission scenarios, an embodiment of the present invention discloses a Martian terrain perception method, including:

[0083] Step S21: Divide the input image into a preset number of small blocks of the same size, and convert the small blocks of the same size into corresponding target low-dimensional vectors.

[0084] In this embodiment, the image is divided into a preset number of small blocks by patch embedding, and each small block is converted into a target low-dimensional vector. Figure 5 As shown in the figure, the image is divided into small blocks of corresponding sizes at different stages. Based on parameters such as image block P, number of image blocks m, number of channels C, number of attention heads, and number of images N, the image is divided into 56×56 blocks in stage 1, 14×14 blocks in stage 2, and 7×7 blocks in stage 3. Each small block is flattened and mapped to a corresponding low-dimensional vector through a linear projection layer.

[0085] Step S22: Based on the preset accuracy requirements and the obtained target low-dimensional vector, a small detection model, a medium detection model and a basic detection model are constructed; wherein the small detection model corresponds to the preset low-precision requirements, the medium-sized detection model corresponds to the preset medium-precision requirements, and the basic detection model corresponds to the preset high-precision requirements.

[0086] In this example, three detection models are constructed based on the generated low-dimensional target vectors: a small detection model, a medium detection model, and a basic detection model. The small detection model achieves fast inference by reducing model complexity; the medium detection model strikes a balance between accuracy and efficiency; and the basic detection model generates more complex feature representations through deep network learning.

[0087] Step S23: According to the accuracy requirements corresponding to the target Mars terrain perception mission, determine the corresponding target detection model from the small detection model, the medium detection model and the basic detection model, and use the target detection model to perform the target Mars terrain perception mission.

[0088] In this embodiment, a target detection model is dynamically selected from a small, medium, and basic detection model based on the precision requirements of the target Mars terrain perception mission. For scenarios requiring high real-time performance, such as drone inspections, the small detection model is selected as the target detection model; for scenarios requiring balanced performance, such as in-vehicle assisted driving, the medium detection model is selected as the target detection model; and for scenarios requiring high precision, such as detailed geographic information annotation, the basic detection model is selected as the target detection model.

[0089] From the above, it can be seen that this application constructs three detection models based on different accuracy requirements, breaking the contradiction between accuracy and efficiency of a single model, forming a differentiated solution, which can flexibly adapt to different scenarios and equipment resources, while reducing hardware costs.

[0090] The technical solution in this application is suitable for identifying the landform type of a certain environment. Figure 6 Assume that the target detection model is stored in a quadruped robot, and then the quadruped robot containing the target detection model is required to perceive the Martian environment.

[0091] First, the first stage acquires an initial environment image of Mars. This image, X, is fed into the preset CBlock module within the object detection model to extract local features of the target within the initial environment image. Next, in the second stage, this image is fed into the preset Mblock module within the object detection model. This image is projected into three attention vectors: a query vector Q, a key vector K, and a value vector V. While maintaining the query vector Q constant, the key vector K and the value vector V are each divided into multiple scales to obtain feature information at each level. The third stage integrates the feature information at each level to form a combined feature information. To avoid loss or attenuation of position information, a local position encoding path is introduced to obtain global features of the target. Finally, the obtained local and global target features are fed into Mask R-CNN to generate a series of candidate boxes containing the Martian rock, completing the identification and location of the Martian rock.

[0092] See also Figure 7 As shown, an embodiment of the present invention discloses a Martian terrain sensing device, comprising:

[0093] A local feature extraction module 11 is used to obtain an initial environment image of the target Martian terrain and extract the target local features in the initial environment image using a preset CBlock module in the target detection model;

[0094] a global feature acquisition module 12, configured to project the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model, adjust the key vector and the value vector to extract feature information at each level in the initial environment image, and then fuse the feature information at each level in the initial environment image based on the query vector to obtain a target global feature;

[0095] The Martian terrain perception completion module 13 is used to input the obtained target local features and the target global features into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

[0096] As can be seen from the above, this application first obtains the initial environment image of the target Martian terrain, and then uses the preset CBlock module in the target detection model to extract the target local features therein, and then uses the preset attention mechanism in the preset Mblock module to project the initial environment image into a query vector, a key vector and a value vector, and then adjusts the second sum vector to extract the feature information of each level in the initial environment image, and then fuses the feature information of each level based on the query vector to obtain the target global features, and finally inputs the target local features and the target global features into the preset semantic segmentation algorithm to obtain the target candidate box containing the target object in the target Martian terrain, thereby completing the perception of the target Martian terrain. In this way, the target local features are extracted by the CBlock module, and the Mblock module uses the attention mechanism to extract and fuse features in layers, and finally combines the instance segmentation algorithm to accurately capture image details and global semantics, which is suitable for Martian terrain perception in complex scenes. At the same time, the overall computational complexity is effectively controlled.

[0097] In some specific implementations, the global feature acquisition module 12 may specifically include:

[0098] A preset projection number determining unit, configured to determine a preset projection number using a preset multi-head attention mechanism in a preset Mblock module;

[0099] a preset linear projection weight determining unit, configured to determine a corresponding preset linear projection weight according to the preset number of projections;

[0100] An image projection unit is configured to project the initial environment image into a query vector, a key vector, and a value vector based on the preset linear projection weights.

[0101] In some specific implementations, the global feature acquisition module 12 may specifically include:

[0102] An image division unit, configured to divide the initial environment image into a preset number of divided environment images based on a preset image preprocessing technology;

[0103] a query vector acquiring unit, configured to perform a preset linear projection operation on the divided environment image based on a first preset linear projection weight to obtain a query vector;

[0104] a key vector acquiring unit, configured to perform a preset linear projection operation on the divided environment image based on a second preset linear projection weight to obtain a key vector;

[0105] The value vector obtaining unit is configured to perform a preset linear projection operation on the divided environment image based on a second preset linear projection weight to obtain a value vector.

[0106] In some specific implementations, the global feature acquisition module 12 may specifically include:

[0107] A first-level feature information acquisition unit, configured to directly perform a preset shape transformation operation on the key vector and the value vector to generate first-level feature information in the initial environment image;

[0108] a non-first-level feature information acquisition unit, configured to perform a preset shape transformation operation on the key vector and the value vector based on a preset convolution kernel size and step size, to generate non-first-level feature information in the initial environment image;

[0109] The hierarchical feature information acquisition unit is configured to acquire feature information of each level in the initial environment image according to the feature information of the first level and the feature information of the non-first level.

[0110] In some specific implementations, the non-first-level feature information acquisition unit includes:

[0111] A first convolution kernel size and first step length determination subunit, used to determine the first convolution kernel size and first step length corresponding to a preset coarse granularity;

[0112] A second convolution kernel size and second step size determining subunit, configured to determine a second convolution kernel size and a second step size corresponding to a preset fine granularity;

[0113] a coarse-grained feature generation subunit, configured to perform a preset shape transformation operation on the key vector and the value vector based on the first convolution kernel size and the first step length, so as to generate a coarse-grained feature containing global structural information;

[0114] a fine-grained feature generation subunit, configured to perform a preset shape transformation operation on the key vector and the value vector based on the second convolution kernel size and the second step size, so as to generate fine-grained features containing local detail information;

[0115] The non-first-level feature information acquisition subunit is used to fuse the coarse-grained features with the fine-grained features to obtain the non-first-level feature information in the initial environment image.

[0116] In some specific implementations, the global feature acquisition module 12 may specifically include:

[0117] a merged feature information acquisition unit, configured to merge the feature information of each layer in the initial environment image based on the query vector to obtain merged feature information;

[0118] a correlation information acquisition unit, configured to determine correlation information between feature information at each level in the initial environment image using a deformable convolution algorithm;

[0119] A target global feature acquisition unit is used to optimize the merged feature information based on the association information to obtain a target global feature.

[0120] In some specific embodiments, the Mars terrain sensing device may further include:

[0121] A target low-dimensional vector conversion module is used to divide the input image into a preset number of small blocks of the same size, and convert the small blocks of the same size into corresponding target low-dimensional vectors;

[0122] A model construction module is used to construct a small detection model, a medium detection model, and a basic detection model based on the preset accuracy requirements and the obtained target low-dimensional vector; wherein the small detection model corresponds to the preset low-accuracy requirements, the medium detection model corresponds to the preset medium-accuracy requirements, and the basic detection model corresponds to the preset high-accuracy requirements;

[0123] The task execution module is used to determine the corresponding target detection model from the small detection model, the medium detection model and the basic detection model according to the accuracy requirements corresponding to the target Mars terrain perception task, and use the target detection model to perform the target Mars terrain perception task.

[0124] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0125] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. This electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the Martian terrain perception method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0126] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0127] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0128] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222. The operating system 221 can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the Martian terrain perception method performed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs capable of performing other specific tasks.

[0129] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned Martian terrain perception method. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0131] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0133] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0134] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for sensing Martian terrain, characterized in that: include: Obtain an initial environment image of the target Martian terrain, and use a preset CBlock module in the target detection model to extract the target local features in the initial environment image; Projecting the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model, adjusting the key vector and the value vector to extract feature information at each level in the initial environment image, and then fusing the feature information at each level in the initial environment image based on the query vector to obtain a global feature of the target; The obtained target local features and the target global features are input into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

2. The Martian terrain perception method according to claim 1, characterized in that: The projecting of the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model includes: The preset number of projections is determined using the preset multi-head attention mechanism in the preset Mblock module; Determining a corresponding preset linear projection weight according to the preset projection quantity; The initial environment image is projected into a query vector, a key vector, and a value vector based on the preset linear projection weights.

3. The Martian terrain perception method according to claim 2, characterized in that: The projecting of the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model includes: Dividing the initial environment image into a preset number of divided environment images based on a preset image preprocessing technology; Performing a preset linear projection operation on the divided environment image based on a first preset linear projection weight to obtain a query vector; Performing a preset linear projection operation on the divided environment image based on a second preset linear projection weight to obtain a key vector; A preset linear projection operation is performed on the divided environment image based on a second preset linear projection weight to obtain a value vector.

4. The Martian terrain perception method according to any one of claims 1 to 3, characterized in that: The step of adjusting the key vector and the value vector to extract feature information of each level in the initial environment image includes: Directly performing a preset shape transformation operation on the key vector and the value vector to generate first-level feature information in the initial environment image; Performing a preset shape transformation operation on the key vector and the value vector based on a preset convolution kernel size and step size to generate non-first-level feature information in the initial environment image; The feature information of each level in the initial environment image is acquired according to the feature information of the first level and the feature information of the non-first level.

5. The Martian terrain perception method according to claim 4, characterized in that: The performing a preset shape transformation operation on the key vector and the value vector based on a preset convolution kernel size and step size to generate non-first-level feature information in the initial environment image includes: Determine the first convolution kernel size and first step length corresponding to the preset coarse granularity; Determine the second convolution kernel size and the second step size corresponding to the preset fine granularity; Performing a preset shape transformation operation on the key vector and the value vector based on the first convolution kernel size and the first step length to generate coarse-grained features containing global structural information; Performing a preset shape transformation operation on the key vector and the value vector based on the second convolution kernel size and the second step size to generate fine-grained features containing local detail information; The coarse-grained features are fused with the fine-grained features to obtain feature information of a non-first level in the initial environment image.

6. The Martian terrain perception method according to claim 4, characterized in that: The fusing of feature information at each level in the initial environment image based on the query vector to obtain a target global feature includes: Merging feature information of each level in the initial environment image based on the query vector to obtain merged feature information; Determining correlation information between feature information at each level in the initial environment image using a deformable convolution algorithm; The merged feature information is optimized based on the association information to obtain a target global feature.

7. The Martian terrain perception method according to claim 1, characterized in that: Also includes: Divide the input image into a preset number of small blocks of the same size, and convert the small blocks of the same size into corresponding target low-dimensional vectors; Based on the preset accuracy requirements and the obtained target low-dimensional vector, a small detection model, a medium detection model, and a basic detection model are constructed; wherein the small detection model corresponds to the preset low-precision requirement, the medium detection model corresponds to the preset medium-precision requirement, and the basic detection model corresponds to the preset high-precision requirement; According to the accuracy requirements corresponding to the target Mars terrain perception mission, the corresponding target detection model is determined from the small detection model, the medium detection model and the basic detection model, and the target detection model is used to perform the target Mars terrain perception mission.

8. A Martian terrain sensing device, characterized in that: include: A local feature extraction module is used to obtain an initial environment image of the target Martian terrain and extract the target local features in the initial environment image using a preset CBlock module in the target detection model; a global feature acquisition module, configured to project the initial environment image into a query vector, a key vector, and a value vector using a preset attention mechanism of a preset Mblock module in the target detection model, adjust the key vector and the value vector to extract feature information at each level in the initial environment image, and then fuse the feature information at each level in the initial environment image based on the query vector to obtain a target global feature; The Martian terrain perception completion module is used to input the obtained target local features and the target global features into a preset semantic segmentation algorithm to obtain a target candidate box containing the target object in the target Martian terrain, so as to complete the perception of the target Martian terrain.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the Mars terrain perception method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the Mars terrain perception method according to any one of claims 1 to 7 is implemented.

Images