A mars surface rock classification system and method based on meta-attention mechanism

Abstract

The application provides a Mars surface rock classification system and method based on a meta-attention mechanism, and relates to the technical field of image classification and identification.The system is configured with a meta-attention mechanism for a neural network model, so as to assign an attention mask to each image block of a Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model.Meanwhile, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value, so that the classification accuracy can be effectively improved when facing different Mars rock classification tasks, and since the computing device deployed on Mars has limited resources, the system can avoid large-scale parameter adjustment and model retraining of the model when facing different classification tasks and different types of rock regions, thereby saving computing resources and time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image classification and recognition, and particularly relates to a Mars surface rock classification system and method based on a meta-attention mechanism. BACKGROUND

[0002] Currently, researchers usually use remote sensing images and deep learning techniques to classify rocks on the surface of Mars. Remote sensing images provide a wide range of high-resolution information about the surface of Mars, while deep learning techniques, especially convolutional neural networks, have shown excellent performance in image recognition and classification tasks.

[0003] In related technologies, Transformer models based on self-attention mechanisms have gradually attracted attention and can capture global information and improve classification accuracy. However, when dealing with complex geological images, the Transformer model still has some shortcomings. For example, the types of rocks on the surface of Mars are diverse, and their texture, color, and morphological features are complex, and the Transformer model has the problem of low classification accuracy. Moreover, when facing different image classification tasks, the Transformer model needs to be adjusted and retrained due to a large number of parameters, which requires a large amount of computing resources and time, but since the computing resources of the computing device set on Mars are limited, the adaptability and generalization ability of the Transformer model are weak, and it cannot meet the demand for accurate classification of rocks on the surface of Mars.

[0004] Therefore, there is an urgent need for a Mars surface rock classification system and method based on a meta-attention mechanism that can improve classification accuracy and avoid large-scale parameter adjustment and model retraining of the model, saving computing resources and time, and meeting the demand for accurate classification of rocks on the surface of Mars. SUMMARY

[0005] The present application provides a Mars surface rock classification system and method based on a meta-attention mechanism, which can effectively improve the classification accuracy and avoid large-scale parameter adjustment and model retraining of the model, saving computing resources and time, and meeting the demand for accurate classification of rocks on the surface of Mars.

[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a Mars surface rock classification system based on a meta-attention mechanism is provided, and the system comprises: an image acquisition module and a neural network model; an output end of the image acquisition module is connected to an input end of the neural network model; the image acquisition module is configured to acquire a Mars image, the Mars image comprising a rock region; the neural network model is configured to process the Mars image to obtain a classification result corresponding to the rock region; wherein the neural network model comprises an embedding unit, a plurality of transformer units and a feedforward neural network connected in turn, the embedding unit being configured to determine a feature vector corresponding to each image block in a plurality of image blocks included in the Mars image; the plurality of transformer units and the feedforward neural network being configured to obtain the classification result corresponding to the rock region according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is configured to assign an attention mask to each image block, and to adjust a self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is further configured to determine a weight value of each neuron in the feedforward neural network, and to adjust a weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0008] In a possible implementation manner of the first aspect, the adjusted attention value corresponding to the ith image block is determined according to the following formula:

[0009]

[0010] wherein m j is the attention mask of the ith image block to the jth image block, is the self-attention value of the ith image block to the jth image block, m s is the attention mask of the ith image block to the sth image block, is the self-attention value of the ith image block to the sth image block.

[0011] In a possible implementation manner of the first aspect, the weight value w i of each neuron in the feedforward neural network is determined according to the following formula:

[0012]

[0013] wherein t i,l is a relationship value corresponding to the ith image block in the lth classification task, t i,k is a relationship value corresponding to the ith image block in the kth classification task, g i is a random sampling value in a Gumbel distribution, and τ is a preset parameter and greater than 0. ​

[0014] The formula for determining the weight matrix of the feedforward neural network is ω i,j

[0015] W2=w i,j W1;

[0016] wherein W2 is the adjusted weight matrix of the feedforward neural network, W1 is the weight matrix of the feedforward neural network before adjustment, w i,j is the weight matrix determined according to the weight value of each neuron.

[0017] In a possible implementation of the first aspect, the image acquisition module is further configured to acquire a first number of training samples, each of which includes a Mars image of a rock region and a corresponding classification result of the rock region; the image acquisition module is further configured to perform rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set including a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock region of each classification result in the second number of training samples is the same; the system further includes a model training module; the model training module is configured to construct a target function; based on the target function, the neural network model is iteratively trained according to the training sample set to obtain a trained neural network model; wherein the target function is the sum of a cross-entropy function L ce and a dropout control function L dc .

[0018] In a possible implementation of the first aspect, the cross-entropy function is:

[0019] L ce (ρ1, ρ2) = -(ρ1 log(ρ2)) + (1-ρ1) log(1-ρ2).

[0020] wherein ρ1 is a real probability and ρ2 is a predicted probability.

[0021] The dropout control function is:

[0022]

[0023] wherein L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is a preset coefficient.

[0024] ​The system provided by the present application can assign an attention mask to each image block of the Mars image by configuring the meta-attention mechanism for the neural network model, so as to adjust the self-attention value of each transformer unit of the neural network model. Meanwhile, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. Moreover, since the computing resources of the computing device deployed on Mars are limited, the system provided by the present application can avoid large-scale parameter adjustment and model retraining of the model when facing different classification tasks and different types of rock regions, thereby saving computing resources and time, and meeting the demand for accurate classification of rocks on the surface of Mars.

[0025] In a second aspect, an embodiment of the present application provides a Mars surface rock classification method based on a meta-attention mechanism, applied to a Mars surface rock classification system, the system comprising an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; the method comprises: the image acquisition module acquires a Mars image, the Mars image comprising a rock region; the neural network model processes the Mars image to obtain a classification result corresponding to the rock region; wherein the neural network model comprises an embedding unit, a plurality of transformer units and a feedforward neural network connected in turn, the embedding unit being used to determine a feature vector corresponding to each image block in a plurality of image blocks included in the Mars image; the plurality of transformer units and the feedforward neural network being used to obtain the classification result corresponding to the rock region according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and is used to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and is used to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0026] In a possible implementation manner of the second aspect, the adjusted attention value of the i th image block is determined according to the following formula:

[0027]

[0028] wherein, m j is the attention mask of the i th image block to the j th image block, is the self-attention value of the i th image block to the j th image block, m s ​an attention mask of the i-th image block to the s-th image block, a self-attention value of the i-th image block to the s-th image block.

[0029] In a possible implementation of the second aspect, the weight value w i of each neuron in the feedforward neural network is determined according to the following formula:

[0030]

[0031] where t i,l is a relationship value corresponding to the i-th image block in the l-th classification task, t i,k is a relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in a Gumbel distribution, and τ is a preset parameter and greater than 0.

[0032] The weight matrix ω i,j of the feedforward neural network is determined according to the following formula:

[0033] W2=w i,j W1;

[0034] where W2 is the weight matrix of the adjusted feedforward neural network, W1 is the weight matrix of the unadjusted feedforward neural network, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0035] In a possible implementation of the second aspect, the system further includes a model training module, and the method further includes: the image acquisition module acquires a first number of training samples, each training sample including a Mars image of a rock region and a classification result corresponding to the rock region; the image acquisition module performs rotation processing, flip processing, and / or mirror processing on the first number of training samples to obtain a training sample set including a second number of training samples, where the first number is less than the second number, and the number of Mars images of the rock region of each classification result included in the second number of training samples is the same; the model training module constructs an objective function; the model training module iteratively trains the neural network model based on the objective function according to the training sample set to obtain a trained neural network model; and the objective function is a sum of a cross-entropy function L ce and a dropout control function L dc .

[0036] In a possible implementation of the second aspect, the cross-entropy function is:

[0037] L ce (ρ1, ρ2) = -(ρ1 log(ρ2)) + (1-ρ1) log(1-ρ2).

[0038] wherein ρ1 is a real probability, and ρ2 is a predicted probability;

[0039] The discard control function is:

[0040]

[0041] wherein L is a second quantity, n is a quantity of image blocks included in each training sample, is a binary mask of the i-th image block of the l-th training sample; and λ is a preset coefficient.

[0042] It can be understood that the beneficial effects achieved by the method of the second aspect provided above can refer to the beneficial effects in the first aspect and any possible design manner thereof, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 Fig. 1 is a hardware structure schematic diagram of a Mars surface rock classification system based on a meta-attention mechanism according to an embodiment of the present application;

[0044] Figure 2 Fig. 2 is a hardware structure schematic diagram of a neural network model according to an embodiment of the present application;

[0045] Figure 3 Fig. 3 is a flow schematic diagram of a Mars surface rock classification method based on a meta-attention mechanism according to an embodiment of the present application;

[0046] Figure 4 Fig. 4 is a flow schematic diagram of another Mars surface rock classification method based on a meta-attention mechanism according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. In the description of the present application, unless otherwise specified, “ / ” represents an “or” relationship between the objects before and after it, for example, A / B can represent A or B; “and / or” in the present application is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. In addition, in the description of the present application, unless otherwise specified, “multiple” means two or more than two. “At least one of the following (one)” or similar expressions means any combination of these items, including any combination of single (one) or multiple items.

[0048] In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same function and role. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. also do not necessarily mean different.

[0049] Meanwhile, in the embodiments of the present application, the words "exemplary" or "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. On the contrary, the words "exemplary" or "for example" are used to present the relevant concept in a specific way, for the purpose of understanding.

[0050] Currently, researchers usually use remote sensing images and deep learning techniques to classify the rocks on the surface of Mars. Remote sensing images provide a wide range of high-resolution information about the surface of Mars, while deep learning techniques, especially convolutional neural networks, have shown excellent performance in image recognition and classification tasks.

[0051] In the related art, the Transformer model based on self-attention mechanism has gradually attracted attention, which can capture global information and improve classification accuracy. The Transformer model based on self-attention mechanism usually divides the input Mars image into multiple image blocks, and then maps each image block to a corresponding feature vector. Each feature vector is passed as part of the input sequence to multiple transformer units of the Transformer model. Each transformer unit includes a multi-head self-attention (MHSA) layer and a feed-forward network (FFN) block that sequentially processes each feature vector, and each transformer unit contains these blocks and their residual connections. However, the Transformer model still has some shortcomings when processing complex geological images. For example, the rock types on the surface of Mars are diverse, and their texture, color and morphological features are complex, and the Transformer model has the problem of low classification accuracy. Moreover, when facing different image classification tasks, the Transformer model needs to be adjusted and retrained due to a large number of parameters, which requires a large amount of computing resources and time, but since the computing resources of the computing device set on Mars are limited, the adaptability and generalization ability of the Transformer model are weak, and it cannot meet the demand for accurate classification of rocks on the surface of Mars.

[0052] Therefore, there is an urgent need for a Martian surface rock classification system and method based on the meta-attention mechanism, which can improve the classification accuracy, avoid large-scale parameter adjustment and model retraining, save computing resources and time, and meet the needs of accurate classification of rocks on the Martian surface.

[0053] In view of this, an embodiment of the present invention provides a Martian surface rock classification system based on a meta-attention mechanism, the system comprising: an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; the image acquisition module is used to acquire a Martian image, which includes a rock area; the neural network model is used to process the Martian image to obtain a classification result corresponding to the rock area; wherein the neural network model comprises an embedding unit, multiple transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine a feature vector corresponding to each of the multiple image blocks included in the Martian image; the multiple transformer units and the feedforward neural network are used to obtain a classification result corresponding to the rock area based on the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0054] The system provided by the embodiment of the present invention configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. Moreover, since the computing resources of the computing equipment deployed on Mars are limited, the system provided by the embodiment of the present invention can avoid the model from performing large-scale parameter adjustments and model retraining when facing different classification tasks and different types of rock areas, saving computing resources and time, and meeting the needs of accurately classifying rocks on the surface of Mars.

[0055] See also Figure 1 , Figure 1A hardware structure schematic diagram of a Mars surface rock classification system based on a meta-attention mechanism is shown in an embodiment of the present application. The Mars surface rock classification system 100 (hereinafter referred to as the classification system 100) based on the meta-attention mechanism comprises an image acquisition module 110 and a neural network model 120. The output end of the image acquisition module 110 is connected to the input end of the neural network model 120.

[0056] The image acquisition module 110 is configured to acquire a Mars image. The Mars image comprises a rock region. Specifically, the Mars image is a Mars surface image acquired by an image acquisition device. Each Mars surface image comprises at least one rock region.

[0057] The neural network model 120 is configured to process the Mars image to obtain a classification result corresponding to the rock region.

[0058] In some embodiments, referring to Figure 2 , Figure 2 A hardware structure schematic diagram of a neural network model is shown in an embodiment of the present application. The neural network model 120 comprises an embedding unit 121, a plurality of transformer units 122 and a feedforward neural network 123 connected in sequence.

[0059] The embedding unit 121 is configured to determine a feature vector corresponding to each image block in a plurality of image blocks included in the Mars image. The plurality of transformer units 122 and the feedforward neural network 123 are configured to obtain a classification result corresponding to the rock region according to the feature vector corresponding to each image block.

[0060] In one embodiment, in combination Figure 2 , the neural network model 120 is configured with a meta-attention mechanism. For each transformer unit 122, the meta-attention mechanism is configured to assign an attention mask to each image block, and to adjust a self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block.

[0061] Specifically, the meta-attention mechanism MEta-ATtention (MEAT) is to dynamically assign an attention mask to each image block of each Mars image in each classification task to adjust a pre-set self-attention value to adapt to a new classification task. It should be understood that the attention mask is a vector of continuous values, which is used to adjust the interaction mode between each image block included in the input Mars image. Specifically, for each transformer unit 122, the attention mask is used to modify the self-attention value related to the i-th image block token i of the Mars image in each transformer unit 122.

[0062] Among them, the adjusted attention value corresponding to the i-th image block is The formula for determining is:

[0063]

[0064] Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the i-th image block to the s-th image block.

[0065] In one embodiment, the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network 123, and to adjust the weight matrix of the feedforward neural network 123 based on the weight value of each neuron in the feedforward neural network 123.

[0066] Specifically, the weight value w of each neuron in the feedforward neural network is i The formula for determining is:

[0067]

[0068] Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0;

[0069] The formula for determining the weight matrix of the feedforward neural network is w i,j for:

[0070] W2=w i,j W1;

[0071] Among them, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

[0072] From the above, the system provided by the embodiment of the application can assign an attention mask to each image block of the Mars image by configuring the meta-attention mechanism for the neural network model, so as to adjust the self-attention value of each transformer unit of the neural network model. Meanwhile, the meta-attention mechanism can also determine the weight value corresponding to each neuron and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. Moreover, since the computing resources of the computing device deployed on Mars are limited, the system provided by the embodiment of the application can avoid large-scale parameter adjustment and model retraining of the model when facing different classification tasks and different types of rock regions, thereby saving computing resources and time and meeting the demand for accurate classification of rocks on the surface of Mars.

[0073] In some embodiments, the image acquisition module 110 is further configured to acquire a first number of training samples, each of which includes a Mars image of a rock region and a classification result corresponding to the rock region; and the image acquisition module 110 is further configured to perform rotation processing, flip processing and / or mirror processing on the first number of training samples to obtain a training sample set including a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of the rock region of each classification result in the second number of training samples is the same.

[0074] In this way, by performing rotation processing, flip processing and / or mirror processing on the first number of training samples, a training sample set with the same number of classification results can be obtained, and then the neural network model can be trained according to the obtained training sample set, so as to improve the accuracy and precision of the neural network model.

[0075] In a possible implementation, the classification system 100 further includes a model training module 130.

[0076] The model training module 130 is configured to construct a target function, and iteratively train the neural network model according to the training sample set based on the target function to obtain a trained neural network model.

[0077] The target function is a sum of a cross-entropy function L ce and a dropout control function L dc .

[0078] In a possible implementation, the cross-entropy function is:

[0079] L ce (p1, p2) = -(p1log(p2)) + (1-p1)log(1-p2).

[0080] Wherein, p1 is a real probability, p2 is a predicted probability.

[0081] The discard control function is:

[0082]

[0083] Wherein, L is a second quantity, n is a quantity of image blocks included in each training sample, is a binary mask of the i-th image block of the l-th training sample; and λ is a preset coefficient.

[0084] It can be understood that the structure illustrated in the embodiment does not constitute a specific limitation on the classification system 100. In other embodiments of the application, the classification system 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0085] The Mars surface rock classification method based on the meta-attention mechanism provided by the embodiment of the application is described below in conjunction with the accompanying drawings of the specification.

[0086] Figure 3 A flowchart of the Mars surface rock classification method based on the meta-attention mechanism provided by the embodiment of the application. Optionally, the method can be executed by the classification system 100 shown in Figure 1 or Figure 2 The classification system 100 shown in the application is applied to a Mars surface rock classification system, and the system includes an image acquisition module and a neural network model. The output end of the image acquisition module is connected to the input end of the neural network model. The method can include the following steps:

[0087] S1, the image acquisition module acquires a Mars image, and the Mars image includes a rock region;

[0088] S2, the neural network model processes the Mars image to obtain a classification result corresponding to the rock region.

[0089] Wherein, the neural network model includes an embedding unit, a plurality of transformer units and a feedforward neural network connected in turn, the embedding unit is used to determine a feature vector corresponding to each image block in a plurality of image blocks included in the Mars image; the plurality of transformer units and the feedforward neural network are used to obtain a classification result corresponding to the rock region according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to allocate an attention mask for each image block, and is used to adjust a self-attention value corresponding to each image block according to the attention mask to obtain an adjusted attention value corresponding to each image block.

[0090] In a possible implementation, the adjusted attention value corresponding to the ith image block is determined according to the following formula:

[0091]

[0092] wherein m j is an attention mask of the ith image block to the jth image block, is a self-attention value of the ith image block to the jth image block, m s is an attention mask of the ith image block to the sth image block, is a self-attention value of the ith image block to the sth image block.

[0093] In a possible implementation, the meta-attention mechanism is further used to determine a weight value of each neuron in the feedforward neural network, and used to adjust a weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

[0094] wherein the weight value w i of each neuron in the feedforward neural network is determined according to the following formula:

[0095]

[0096] wherein t i,l is a relationship value corresponding to the ith image block in the lth classification task, t i,k is a relationship value corresponding to the ith image block in the kth classification task, g i is a random sampling value in a Gumbel distribution, and τ is a preset parameter and greater than 0;

[0097] The determination formula ω i,j of the weight matrix of the feedforward neural network is as follows:

[0098] W2=w i,j W1;

[0099] wherein W2 is the weight matrix of the adjusted feedforward neural network, W1 is the weight matrix of the unadjusted feedforward neural network, w i,j is the weight matrix determined according to the weight value of each neuron.

[0100] In some embodiments, referring to Figure 4 , the method provided by the present application further includes the following steps:

[0101] S41, an image acquisition module acquires a first number of training samples.

[0102] Each training sample includes a Mars image of a rock region and a classification result corresponding to the rock region; ​

[0103] S42, the image acquisition module also performs rotation processing, flipping processing and / or mirroring processing on the first number of training samples to obtain a training sample set.

[0104] The training sample set includes a second number of training samples, wherein the first number is less than the second number, and the number of Mars images of each classification result of the rock region included in the second number of training samples is the same;

[0105] S43, the model training module constructs a target function;

[0106] S44, the model training module iteratively trains the neural network model based on the target function according to the training sample set to obtain a trained neural network model;

[0107] The target function is the sum of a cross-entropy function L ce and a dropout control function L dc .

[0108] In some embodiments, the cross-entropy function is:

[0109] L ce (p1, p2) = -(p1log(p2)) + (1-p1)log(1-p2);

[0110] Wherein, p1 is the true probability, and p2 is the predicted probability;

[0111] The dropout control function is:

[0112]

[0113] Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; and λ is a preset coefficient.

[0114] As can be seen from the above S1-S2, the method provided by the embodiments of the present application configures a meta-attention mechanism for the neural network model to assign an attention mask to each image block of the Mars image, so as to adjust the self-attention value of each transformer unit of the neural network model. At the same time, the meta-attention mechanism can also determine the weight value corresponding to each neuron, and adjust the weight matrix of the feedforward neural network of the neural network model through the weight value. In this way, when facing different Mars rock classification tasks, the classification accuracy can be effectively improved. Moreover, since the computing resources of the computing device deployed on Mars are limited, the method provided by the embodiments of the present application can avoid large-scale parameter adjustment and model retraining of the model when facing different classification tasks and different types of rock regions, thereby saving computing resources and time and meeting the demand for accurate classification of rocks on the surface of Mars.

[0115] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that variations, modifications, substitutions and changes can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A Martian surface rock classification system based on meta-attention mechanism, characterized by: The system comprises: An image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; The image acquisition module is used to acquire an image of Mars, wherein the image of Mars includes a rocky area; The neural network model is used to process the Mars image to obtain a classification result corresponding to the rock area; wherein the neural network model includes an embedding unit, multiple transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units and the feedforward neural network are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain the adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

2. The system according to claim 1, wherein: The adjusted attention value corresponding to the i-th image block The formula for determining is: Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the i-th image block to the s-th image block.

3. The system according to claim 2, characterized in that The weight value w of each neuron in the feedforward neural network is i The formula for determining is: Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0; The formula for determining the weight matrix of the feedforward neural network is ω i,j for: W2=w i,j W1; Wherein, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

4. The system according to claim 3, characterized in that The image acquisition module is further configured to acquire a first number of training samples, each training sample including an image of Mars in a rocky area and a classification result corresponding to the rocky area; the image acquisition module is further configured to perform rotation processing, flip processing, and / or mirror processing on the first number of training samples to obtain a training sample set, the training sample set including a second number of training samples, wherein the first number is smaller than the second number, and the second number of training samples includes the same number of images of Mars in a rocky area for each classification result; The system also includes a model training module; The model training module is used to construct an objective function; based on the objective function, the neural network model is iteratively trained according to the training sample set to obtain a trained neural network model; Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of .

5. The system according to claim 4, characterized in that The cross entropy function is: L ce (ρ1,ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2); Among them, ρ1 is the true probability and ρ2 is the predicted probability; The discard control function is: Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

6. A method for classifying Martian surface rocks based on a meta-attention mechanism, characterized in that: Applied to a Martian surface rock classification system, the system includes an image acquisition module and a neural network model; the output end of the image acquisition module is connected to the input end of the neural network model; The method comprises: The image acquisition module acquires a Martian image, wherein the Martian image includes a rocky area; The neural network model processes the Mars image to obtain a classification result corresponding to the rock area; wherein the neural network model includes an embedding unit, multiple transformer units and a feedforward neural network connected in sequence, the embedding unit is used to determine the feature vector corresponding to each of the multiple image blocks included in the Mars image; the multiple transformer units and the feedforward neural network are used to obtain the classification result corresponding to the rock area according to the feature vector corresponding to each image block; the neural network model is configured with a meta-attention mechanism, for each transformer unit, the meta-attention mechanism is used to assign an attention mask to each image block, and to adjust the self-attention value corresponding to each image block according to the attention mask to obtain the adjusted attention value corresponding to each image block; the meta-attention mechanism is also used to determine the weight value of each neuron in the feedforward neural network, and to adjust the weight matrix of the feedforward neural network based on the weight value of each neuron in the feedforward neural network.

7. The method according to claim 6, characterized in that The adjusted attention value corresponding to the i-th image block The formula for determining is: Among them, m j is the attention mask of the i-th image block to the j-th image block, is the self-attention value of the i-th image block to the j-th image block, m s is the attention mask of the i-th image block to the s-th image block, is the self-attention value of the i-th image block to the s-th image block.

8. The method according to claim 7, characterized in that The weight value w of each neuron in the feedforward neural network is i The formula for determining is: Among them, t i,l is the relationship value corresponding to the i-th image block in the l-th classification task, t i,k is the relationship value corresponding to the i-th image block in the k-th classification task, g i is a random sampling value in Gumbel distribution, τ is a preset parameter, and τ is greater than 0; The formula for determining the weight matrix of the feedforward neural network is ω i,j for: W2=w i,j W1; Wherein, W2 is the weight matrix of the feedforward neural network after adjustment, W1 is the weight matrix of the feedforward neural network before adjustment, and w i,j is the weight matrix determined according to the weight value of each neuron.

9. The method according to claim 8, characterized in that The system further includes a model training module, and the method further includes: The image acquisition module acquires a first number of training samples, each training sample including an image of Mars in a rocky area and a classification result corresponding to the rocky area; The image acquisition module performs rotation processing, flip processing, and / or mirror processing on the first number of training samples to obtain a training sample set, where the training sample set includes a second number of training samples, wherein the first number is smaller than the second number, and the second number of training samples includes the same number of Mars images of rock regions for each classification result; The model training module constructs an objective function; The model training module iteratively trains the neural network model based on the objective function and the training sample set to obtain a trained neural network model; Among them, the objective function is the cross entropy function L ce and the discard control function L dc The sum of .

10. The method according to claim 9, characterized in that The cross entropy function is: L ce (ρ1,ρ2)=-(ρ1log(ρ2))+(1-ρ1)log(1-ρ2); Among them, ρ1 is the true probability and ρ2 is the predicted probability; The discard control function is: Wherein, L is the second number, n is the number of image blocks included in each training sample, is the binary mask of the i-th image block of the l-th training sample; λ is the preset coefficient.

Images