Mars surface rock classification system and method based on element attention mechanism

Abstract

The invention provides a Mars surface rock classification system and method based on a meta attention mechanism, and relates to the technical field of image classification and recognition. According to the system provided by the invention, the element attention mechanism is configured for the neural network model, and the attention mask is allocated for each image block of the Mars image, so that the self-attention value of each transformer unit of the neural network model is adjusted. Meanwhile, the meta attention mechanism can also determine a weight value corresponding to each neuron, and adjust a weight matrix of a feedforward neural network of the neural network model through the weight values, so that the classification precision can be effectively improved in the face of different Mars rock classification tasks, and due to limited resources of computing equipment deployed in Mars, the classification efficiency is improved. According to the system, large-scale parameter adjustment and model retraining can be avoided when the model faces different classification tasks and different types of rock areas, and computing resources and time are saved.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification and recognition, and in particular to a Martian surface rock classification system and method based on a meta-attention mechanism. Background Art

[0002] Currently, researchers usually use remote sensing images and deep learning technology to classify rocks on the surface of Mars. Remote sensing images provide large-scale, high-resolution information on the surface of Mars, while deep learning technology, especially convolutional neural networks, has shown excellent performance in image recognition and classification tasks.

[0003] In the related technology, the Transformer model based on the self-attention mechanism has gradually attracted attention, which can capture global information and improve classification accuracy. However, the Transformer model still has some shortcomings when processing complex geological images. For example, the rock types on the surface of Mars are diverse, and their texture, color and morphological characteristics are complex, and the Transformer model has the problem of low classification accuracy. In addition, when facing different image classification tasks, the Transformer model needs to perform a large number of parameter adjustments and model retraining, which requires a lot of computing resources and time. However, due to the limited computing resources configured by the computing equipment set up on Mars, the adaptability and generalization ability of the Transformer model are weak, and it cannot meet the needs of accurately classifying rocks on the surface of Mars.

[0004] Therefore, there is an urgent need for a Martian surface rock classification system and method based on the meta-attention mechanism, which can improve the classification accuracy, avoid large-scale parameter adjustment and model retraining, save computing resources and time, and meet the needs of accurate classification of rocks on the Martian surface. Summary of the invention

[0005] The embodiments of the present invention provide a system and method for classifying rocks on the surface of Mars based on a meta-attention mechanism. When faced with different Martian rock classification tasks, the system and method can effectively improve the classification accuracy, avoid large-scale parameter adjustment and model retraining of the model, save computing resources and time, and meet the needs of accurately classifying rocks on the surface of Mars.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0007] In a first aspect, a Martian surface rock classification system based on a meta-attention mechanism is provided, the system comprising: an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; the image acquisition module is used to acquire a Martian image, the Martian image includes a rock area; the neural network model is used to process the Martian image to obtain a classification result corresponding to the rock area; wherein the neural network model comprises an embedding unit, a plurality of transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine a feature vector corresponding to each of a plurality of image blocks included in the Martian image; the plurality of transformer units and the feedforward neural network are used to obtain a classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0008] In a possible implementation manner of the first aspect, the adjusted attention value corresponding to the i-th image block is The formula for determining is:

[0009]

[0010] Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

[0011] In a possible implementation of the first aspect, the weight value w of each neuron in the feedforward neural network is i The formula for determining is:

[0012]

[0013] Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0;

[0014] The formula for determining the weight matrix of the feedforward neural network is ω i,j for:

[0015] W2=w i,j W1;

[0016] Among them, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0017] In a possible implementation of the first aspect, the image acquisition module is further used to acquire a first number of training samples, each training sample including a Mars image of a rock area and a classification result corresponding to the rock area; the image acquisition module is further used to perform rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set, the training sample set including a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock area for each classification result included in the second number of training samples is the same; the above-mentioned system also includes a model training module; the model training module is used to construct an objective function; based on the objective function, the neural network model is iteratively trained according to the training sample set to obtain a trained neural network model; wherein the objective function is a cross entropy function L ce and the discard control function L dc The sum of

[0018] In a possible implementation manner of the first aspect, the cross entropy function is:

[0019] L ce (ρ1, ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2);

[0020] Among them, ρ1 is the true probability and ρ2 is the predicted probability;

[0021] The discard control function is:

[0022]

[0023] Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

[0024] The beneficial effects of the present invention are as follows: the system provided by the present invention configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. Moreover, since the computing resources of the computing equipment deployed on Mars are limited, the system provided by the present invention can avoid large-scale parameter adjustment and model retraining when the model faces different classification tasks and different types of rock areas, saving computing resources and time, and meeting the needs of accurate classification of rocks on the surface of Mars.

[0025] In a second aspect, an embodiment of the present invention provides a method for classifying rocks on the surface of Mars based on a meta-attention mechanism, which is applied to a rock classification system on the surface of Mars, wherein the system comprises an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; the method comprises: the image acquisition module acquires a Mars image, which includes a rock area; the neural network model processes the Mars image to obtain a classification result corresponding to the rock area; wherein the neural network model comprises an embedding unit, a plurality of transformer units and a feedforward neural network connected in sequence, the embedding unit being used to determine a feature vector corresponding to each of a plurality of image blocks included in the Mars image; the plurality of transformer units and the feedforward neural network being used to obtain a classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, wherein for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine a weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0026] In a possible implementation manner of the second aspect, the adjusted attention value corresponding to the i-th image block is The formula for determining is:

[0027]

[0028] Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m sis the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

[0029] In a possible implementation of the second aspect, the weight value w of each neuron in the feedforward neural network is i The formula for determining is:

[0030]

[0031] Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0;

[0032] The formula for determining the weight matrix of the feedforward neural network is ω i,j for:

[0033] W2=w i,j W1;

[0034] Among them, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0035] In a possible implementation manner of the second aspect, the system further includes a model training module, and the method further includes: the image acquisition module acquires a first number of training samples, each training sample includes a Mars image of a rock area and a classification result corresponding to the rock area; the image acquisition module performs rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set, and the training sample set includes a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock area included in each classification result in the second number of training samples is the same; the model training module constructs an objective function; the model training module iteratively trains the neural network model based on the objective function and the training sample set to obtain a trained neural network model; wherein the objective function is a cross entropy function L ce and the discard control function L dc The sum of .

[0036] In a possible implementation manner of the second aspect, the cross entropy function is:

[0037] L ce (ρ1, ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2);

[0038] Among them, ρ1 is the true probability and ρ2 is the predicted probability;

[0039] The discard control function is:

[0040]

[0041] Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

[0042] It can be understood that the beneficial effects that can be achieved by the method of the second aspect provided above can refer to the beneficial effects in the first aspect and any possible design thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the hardware structure of a Martian surface rock classification system based on a meta-attention mechanism is shown in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the hardware structure of a neural network model shown in an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a process flow of a method for classifying rocks on the surface of Mars based on a meta-attention mechanism according to an embodiment of the present invention;

[0046] Figure 4 A schematic flowchart of another method for classifying rocks on the surface of Mars based on a meta-attention mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution in the embodiment of the present invention will be described below in conjunction with the accompanying drawings in the embodiment of the present invention. Among them, in the description of the present invention, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present invention is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the present invention, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items.

[0048] In addition, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0049] Meanwhile, in the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way for easy understanding.

[0050] Currently, researchers usually use remote sensing images and deep learning technology to classify rocks on the surface of Mars. Remote sensing images provide large-scale, high-resolution information on the surface of Mars, while deep learning technology, especially convolutional neural networks, has shown excellent performance in image recognition and classification tasks.

[0051] In the related art, the Transformer model based on the self-attention mechanism has gradually attracted attention, which can capture global information and improve classification accuracy. The Transformer model based on the self-attention mechanism usually divides the input Mars image into multiple image blocks, and then maps each image block to a corresponding feature vector. Each feature vector is passed to multiple transformer units of the Transformer model as part of the input sequence. The multi-head self-attention (MHSA) layer and the feed-forward network (FFN) block included in each transformer unit process each feature vector in turn, and each transformer unit contains these blocks and their residual connections. However, the Transformer model still has some shortcomings when processing complex geological images. For example, the rock types on the surface of Mars are diverse, and their texture, color and morphological characteristics are complex, and the Transformer model has the problem of low classification accuracy. In addition, when facing different image classification tasks, the Transformer model needs to perform a large number of parameter adjustments and model retraining, which requires a lot of computing resources and time. However, due to the limited computing resources configured by the computing equipment set up on Mars, the adaptability and generalization ability of the Transformer model are weak, and it cannot meet the needs of accurately classifying rocks on the surface of Mars.

[0052] Therefore, there is an urgent need for a Martian surface rock classification system and method based on the meta-attention mechanism, which can improve the classification accuracy, avoid large-scale parameter adjustment and model retraining, save computing resources and time, and meet the needs of accurate classification of rocks on the Martian surface.

[0053] In view of this, an embodiment of the present invention provides a Martian surface rock classification system based on a meta-attention mechanism, the system comprising: an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; the image acquisition module is used to acquire a Martian image, the Martian image includes a rock area; the neural network model is used to process the Martian image to obtain a classification result corresponding to the rock area; wherein the neural network model comprises an embedding unit, a plurality of transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine a feature vector corresponding to each of a plurality of image blocks included in the Martian image; the plurality of transformer units and the feedforward neural network are used to obtain a classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0054] The system provided by the embodiment of the present invention configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. In addition, since the computing resources of the computing equipment deployed on Mars are limited, the system provided by the embodiment of the present invention can avoid large-scale parameter adjustment and model retraining when the model faces different classification tasks and different types of rock areas, saving computing resources and time, and meeting the needs of accurate classification of rocks on the surface of Mars.

[0055] See also Figure 1 , Figure 1The hardware structure diagram of a Martian surface rock classification system based on a meta-attention mechanism is shown in an embodiment of the present invention. The Martian surface rock classification system 100 based on a meta-attention mechanism (hereinafter referred to as classification system 100) includes: an image acquisition module 110 and a neural network model 120; the output end of the image acquisition module 110 is connected to the input end of the neural network model 120;

[0056] The image acquisition module 110 is used to acquire a Martian image, which includes a rock region. Specifically, the Martian image is an image of the Martian surface acquired by an image acquisition device, and each Martian surface image includes at least one rock region.

[0057] The neural network model 120 is used to process the Mars image and obtain the classification result corresponding to the rock area.

[0058] In some embodiments, see Figure 2 , Figure 2 The schematic diagram of the hardware structure of a neural network model shown in an embodiment of the present invention is a schematic diagram of the hardware structure of a neural network model, wherein the neural network model 120 includes an embedding unit 121, a plurality of transformer units 122 and a feedforward neural network 123 which are sequentially connected in layers.

[0059] The embedding unit 121 is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units 122 and the feedforward neural network 123 are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block;

[0060] In one embodiment, in combination Figure 2 The neural network model 120 is configured with a meta-attention mechanism. For each transformer unit 122, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block.

[0061] Specifically, the meta-attention mechanism MEta-ATtention (MEAT) dynamically assigns an attention mask to each image block of each Mars image in each classification task to adjust the pre-set self-attention value to adapt to the new classification task. It should be understood that the attention mask is a vector of continuous values ​​used to adjust the interaction pattern between each image block included in the input Mars image. Specifically, for each transformer unit 122, the attention mask is used to modify the self-attention value associated with the i-th image block token i of the Mars image in each transformer unit 122.

[0062] Among them, the adjusted attention value corresponding to the i-th image block is The formula for determining is:

[0063]

[0064] Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

[0065] In one embodiment, the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network 123, and to adjust the weight matrix of the feedforward neural network 123 based on the weight value of each neuron in the feedforward neural network 123.

[0066] Specifically, the weight value w of each neuron in the feedforward neural network is i The formula for determining is:

[0067]

[0068] Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0;

[0069] The formula for determining the weight matrix of the feedforward neural network is i,j for:

[0070] W2=w i,j W1;

[0071] Among them, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0072] From the above, it can be seen that the system provided by the embodiment of the present invention configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. In addition, since the computing resources of the computing equipment deployed on Mars are limited, the system provided by the embodiment of the present invention can avoid large-scale parameter adjustment and model retraining when the model faces different classification tasks and different types of rock areas, saving computing resources and time, and meeting the needs of accurate classification of rocks on the surface of Mars.

[0073] In some embodiments, the image acquisition module 110 is further used to acquire a first number of training samples, each training sample including a Mars image of a rock region and a classification result corresponding to the rock region; the image acquisition module 110 is further used to perform rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set, the training sample set including a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock region including each classification result in the second number of training samples is the same;

[0074] In this way, by rotating, flipping and / or mirroring the first number of training samples, a training sample set with the same number of classification results can be obtained, and then the neural network model can be trained according to the obtained training sample set, thereby improving the precision and accuracy of the neural network model.

[0075] In one possible implementation, the classification system 100 further includes a model training module 130;

[0076] The model training module 130 is used to construct an objective function; based on the objective function, the neural network model is iteratively trained according to the training sample set to obtain a trained neural network model;

[0077] Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of

[0078] In a possible implementation, the cross entropy function is:

[0079] L ce (ρ1, ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2);

[0080] Among them, ρ1 is the true probability and ρ2 is the predicted probability.

[0081] The discard control function is:

[0082]

[0083] Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

[0084] It is understood that the structure shown in this embodiment does not constitute a specific limitation on the classification system 100. In other embodiments of the present invention, the classification system 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0085] The following describes the Martian surface rock classification method based on the meta-attention mechanism provided by an embodiment of the present invention in conjunction with the accompanying drawings.

[0086] Figure 3 A flowchart of a method for classifying rocks on the surface of Mars based on a meta-attention mechanism is provided in an embodiment of the present invention. Optionally, the method can be Figure 1 or Figure 2 The classification system 100 shown is applied to the Martian surface rock classification system. The system includes an image acquisition module and a neural network model. The output end of the image acquisition module is connected to the input end of the neural network model. The method may include the following steps:

[0087] S1, the image acquisition module acquires a Martian image, where the Martian image includes a rocky area;

[0088] S2. The neural network model processes the Mars image and obtains the classification results corresponding to the rock area.

[0089] Among them, the neural network model includes an embedding unit, multiple transformer units and a feedforward neural network connected in sequence. The embedding unit is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units and the feedforward neural network are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism. For each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain the adjusted attention value corresponding to each image block.

[0090] In one possible implementation, the adjusted attention value corresponding to the i-th image block is The formula for determining is:

[0091]

[0092] Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

[0093] In one possible implementation, the meta-attention mechanism is also used to determine a weight value of each neuron in the feedforward neural network, and to adjust a weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0094] Among them, the weight value w of each neuron in the feedforward neural network is i The formula for determining is:

[0095]

[0096] Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0;

[0097] The formula for determining the weight matrix of the feedforward neural network is ω i,j for:

[0098] W2=w i,j W1;

[0099] Among them, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0100] In some embodiments, see Figure 4 , the method provided by the present invention further comprises the following steps:

[0101] S41. The image acquisition module acquires a first number of training samples.

[0102] Each training sample includes a Martian image of a rocky area and the corresponding classification result of the rocky area;

[0103] S42: The image acquisition module further performs rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set.

[0104] The training sample set includes a second number of training samples, wherein the first number is smaller than the second number, and the second number of training samples includes the same number of Mars images of rock regions for each classification result;

[0105] S43, the model training module constructs the objective function;

[0106] S44, the model training module iteratively trains the neural network model based on the objective function and the training sample set to obtain a trained neural network model;

[0107] Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of

[0108] In some embodiments, the cross entropy function is:

[0109] L ce (ρ1, ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2);

[0110] Among them, ρ1 is the true probability and ρ2 is the predicted probability;

[0111] The discard control function is:

[0112]

[0113] Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

[0114] From the above S1-S2, it can be seen that the method provided by the embodiment of the present invention configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. In addition, due to the limited computing resources of the computing equipment deployed on Mars, the method provided by the embodiment of the present invention can avoid large-scale parameter adjustment and model retraining when the model faces different classification tasks and different types of rock areas, saving computing resources and time, and meeting the needs of accurate classification of rocks on the surface of Mars.

[0115] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A Martian surface rock classification system based on meta-attention mechanism, characterized in that: The system comprises: An image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; The image acquisition module is used to acquire a Martian image, wherein the Martian image includes a rocky area; The neural network model is used to process the Mars image to obtain a classification result corresponding to the rock area; wherein the neural network model includes an embedding unit, multiple transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units and the feedforward neural network are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain the adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

2. The system according to claim 1, characterized in that The adjusted attention value corresponding to the i-th image block The formula for determining is: Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

3. The system according to claim 2, characterized in that The weight value w of each neuron in the feedforward neural network is i The formula for determining is: Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0; The formula for determining the weight matrix of the feedforward neural network is ω i,j for: W2=w i,j W1; Wherein, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

4. The system according to claim 3, characterized in that The image acquisition module is further used to acquire a first number of training samples, each training sample including a Mars image of a rock region and a classification result corresponding to the rock region; the image acquisition module is further used to perform rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set, the training sample set including a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock region including each classification result in the second number of training samples is the same; The system also includes a model training module; The model training module is used to construct an objective function; based on the objective function, the neural network model is iteratively trained according to the training sample set to obtain a trained neural network model; Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of .

5. The system according to claim 4, characterized in that The cross entropy function is: L ce (ρ1,ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2); Among them, ρ1 is the true probability and ρ2 is the predicted probability; The discard control function is: Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

6. A method for classifying rocks on the Martian surface based on a meta-attention mechanism, characterized in that: Applied to a Martian surface rock classification system, the system comprises an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; The method comprises: The image acquisition module acquires a Martian image, wherein the Martian image includes a rocky area; The neural network model processes the Mars image to obtain a classification result corresponding to the rock area; wherein the neural network model includes an embedding unit, multiple transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units and the feedforward neural network are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain the adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

7. The system according to claim 6, characterized in that The adjusted attention value corresponding to the i-th image block The formula for determining is: Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the ith image block to the sth image block.

8. The system according to claim 7, characterized in that The weight value w of each neuron in the feedforward neural network is i The formula for determining is: Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0; The formula for determining the weight matrix of the feedforward neural network is ω i,j for: W2=w i,j W1; Wherein, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

9. The system according to claim 8, characterized in that The system further includes a model training module, and the method further includes: The image acquisition module acquires a first number of training samples, each training sample including a Martian image of a rock region and a classification result corresponding to the rock region; The image acquisition module performs rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set, wherein the training sample set includes a second number of training samples, wherein the first number is smaller than the second number, and the second number of training samples includes the same number of Mars images of rock areas of each classification result; The model training module constructs an objective function; The model training module iteratively trains the neural network model based on the objective function and the training sample set to obtain a trained neural network model; Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of .

10. The system according to claim 9, characterized in that The cross entropy function is: L ce (ρ1,ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2); Among them, ρ1 is the true probability and ρ2 is the predicted probability; The discard control function is: Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

Images