Remote sensing image recognition method, model and device

Abstract

The embodiment of the invention relates to the technical field of small sample learning and cross-domain remote sensing image recognition, and provides a remote sensing image recognition method, model and device, and the method comprises the steps: obtaining a to-be-recognized target remote sensing image, carrying out the image block division of the target remote sensing image, and obtaining a plurality of equal-size image blocks; performing feature embedding on each image block to generate a feature vector of each image block; performing feature interaction on the feature vector of each image block and features of an existing model training set through a self-attention mechanism, and optimizing the feature vector of each image block; and identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain an identification result corresponding to the target remote sensing image. Therefore, the adaptability, the learning performance, the classification precision, the practical application flexibility, the application efficiency and the practicability of the model to different domain features can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of small sample learning and cross-domain remote sensing image recognition, and in particular to a remote sensing image recognition method, model and device. Background Art

[0002] In the field of small sample learning and cross-domain remote sensing image recognition, common methods include prototype network (ProtoNet), network combining twin network and prototype network (Spatial Pooling Net, SPNet), layer-wise relevance propagation (Layer-wise Relevance Propagation, LRP), cross-domain small sample classification based on feature transformation (Feature-wiseTransformation, FWT), deep nearest neighbor neural network (Deep Neural Networks, DN4) and dual adaptive representation alignment (Dara), etc. The prototype network performs classification by calculating the average feature vector of each category, and has certain performance in small sample scenarios; SPNet combines the advantages of twin networks and prototype networks, and uses the similarity between the support set and the query set for small sample remote sensing classification; the LRP method enhances the interpretability of the model by tracing back the importance of features layer by layer; FWT reduces the difference between the source domain and the target domain through feature transformation, and improves the performance of the model in the target domain; DN4 uses deep feature neighborhoods for classification, which is effective in small sample tasks; Dara enhances the generalization ability of cross-domain small sample classification through dual alignment of features and classifiers. These methods have made significant progress in the field of small sample learning and cross-domain remote sensing image recognition, and provide a variety of technical paths to solve problems such as insufficient data and distribution differences between domains.

[0003] Although the above methods have made some progress in cross-domain small sample remote sensing image recognition, they still have significant shortcomings. First, the lack of effective cross-domain feature adaptation is one of the main problems. For example, the prototype network (ProtoNet) and the deep nearest neighbor neural network (DN4) cannot fully cope with the significant distribution differences between the source domain and the target domain, resulting in poor generalization performance. In cross-domain scenarios, the feature distributions of different data sets often vary greatly, which makes it impossible for the model to maintain a high level of accuracy in the target domain.

[0004] Secondly, the reliance on target domain data limits the practical application of the technology. For example, feature transformation-based FWT and dual adaptive representation alignment (Dara) require target domain data to participate in training or unsupervised alignment to improve the cross-domain generalization ability of the model. However, in the field of remote sensing, due to the difficulty in obtaining target domain data and data privacy and security issues, it is usually impractical to obtain target domain data, which limits the practical application scenarios of these methods.

[0005] In addition, the limitation of feature extraction is also an important disadvantage. Most methods use global feature extraction, which makes the model susceptible to interference from image background noise, especially in remote sensing images, where background information is often very complex and has a lot of interference. For scenes that need to focus on local features, this global feature extraction cannot effectively highlight key targets, affecting the classification accuracy of the model.

[0006] Finally, the complexity of training and application is also a major limitation of existing technologies. Some cross-domain small sample learning methods require multi-stage training or fine-tuning to cope with inter-domain differences, which not only increases the complexity of model development, but also significantly increases the cost of deployment and application. Therefore, existing methods face the problems of low efficiency and high cost in the application process, and it is difficult to meet the rapid response requirements in actual needs. Summary of the invention

[0007] The present invention provides a remote sensing image recognition method, model and device to solve the defects of the prior art models, such as poor generalization performance, limited practical application scenarios, low classification accuracy, low efficiency and high cost, and to improve the model's adaptability to different domain features, learning performance, classification accuracy, flexibility of practical application, and improvement of application efficiency and practicability.

[0008] The present invention provides a remote sensing image recognition method, comprising: Acquire a remote sensing image of a target to be identified and divide the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; Perform feature embedding on each image block to generate a feature vector for each image block; Performing feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to optimize the feature vector of each image block; The target remote sensing image to be identified is identified based on the optimized feature vector of each image block to obtain an identification result corresponding to the target remote sensing image.

[0009] In a possible implementation, the method further includes: Extract the key features of each image block through convolutional neural network; The key features are subjected to feature embedding processing to generate a feature vector for each image block.

[0010] In a possible implementation, the method further includes: Calculating the cosine similarity between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism; Based on the cosine similarity, the feature vector of each image block is interactively adapted to the features of the existing model training set to optimize the feature vector of each image block.

[0011] In a possible implementation, the method further includes: Classifying each image block based on the optimized feature vector of each image block; Aggregate image blocks of the same category to obtain multiple image block sets, each of which corresponds to a target category; The recognition result corresponding to the target remote sensing image is determined based on the target classification corresponding to each image block set.

[0012] In a possible implementation, the method further includes: The target classification corresponding to the image block set with the largest number of image blocks is taken as the recognition result corresponding to the target remote sensing image.

[0013] The present invention also provides a remote sensing image recognition model, which is applied to a remote sensing image recognition method, comprising: Image block division module and cross-domain adaptation module; The image block division module is used to divide the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; The cross-domain adaptation module includes a cross adaptation module, which is used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block; The cross-domain adaptive module recognizes the target remote sensing image by optimizing the prototype triplet loss function.

[0014] The present invention also provides a remote sensing image recognition device, comprising the following modules: An image acquisition and segmentation module is used to acquire a remote sensing image of a target to be identified and to segment the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; A feature embedding module, used to embed features for each image block and generate a feature vector for each image block; A feature interaction module, used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism, so as to optimize the feature vector of each image block; The image recognition module is used to recognize the target remote sensing image to be recognized based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image recognition method as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the remote sensing image recognition method as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the remote sensing image recognition method as described above is implemented.

[0018] The remote sensing image recognition method, model and device provided by the present invention obtain a target remote sensing image to be identified and divide the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; embed features for each image block to generate a feature vector for each image block; perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block; identify the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain the recognition result corresponding to the target remote sensing image. Compared with the existing technology, some networks lack effective cross-domain feature adaptation capabilities, the dependence on target domain data limits the practical application of the technology, the limitations of feature extraction methods and the complexity of training and application lead to poor generalization performance of the model, limited practical application scenarios, low classification accuracy, low efficiency and high cost. This solution improves the model's adaptability to features of different domains through image block-based feature extraction and cross-domain adaptation mechanism, so that the model trained in the source domain can be better generalized to the target domain; by combining the prototype triplet loss function, the model's intra-class clustering and inter-class distinction capabilities under the condition of a few samples are enhanced, thereby significantly improving the performance of cross-domain small sample learning; local feature extraction based on image blocks is adopted, combined with the self-attention mechanism, to effectively focus on key areas in the image, reduce the interference of background noise on the model, and improve classification accuracy; the cross-domain adaptation method that does not require target domain data enables the model to effectively perform cross-domain classification when only using source domain data, greatly reducing the dependence on target domain data and increasing the flexibility of practical applications; through end-to-end automated design, the steps of image block division, feature extraction, cross-domain adaptation, etc. are seamlessly integrated, which simplifies the model training process and improves the efficiency and practicality of the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is one of the flow charts of the remote sensing image recognition method provided by the present invention.

[0021] Figure 2 This is the second flow chart of the remote sensing image recognition method provided by the present invention.

[0022] Figure 3 It is a network structure diagram of the remote sensing image recognition method provided by the present invention.

[0023] Figure 4 It is a structural schematic diagram of the remote sensing image recognition device provided by the present invention.

[0024] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0027] Figure 1 This is one of the flow charts of the remote sensing image recognition method provided by the present invention, such as Figure 1 As shown, the method includes the following: S11, obtaining a remote sensing image of a target to be identified and dividing the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size.

[0028] In the embodiment of the present invention, first, a remote sensing image recognition model needs to be pre-trained. The specific training process is not described in detail here. Remote sensing image recognition is performed through the remote sensing image recognition model. Then, the target remote sensing image to be identified in the input model is divided into image blocks, and the entire image is divided into multiple image blocks of equal size. Among them, the target remote sensing image to be identified can be high-resolution remote sensing image data obtained from platforms such as satellites and drones.

[0029] Optionally, the target remote sensing image can also be enhanced and normalized. Image enhancement improves image quality through contrast adjustment, denoising and other methods to better extract features; normalization is to standardize pixel values ​​to the range of (0, 1) or (−1, 1) for easier model processing.

[0030] S12: Perform feature embedding on each image block to generate a feature vector for each image block.

[0031] The feature embedding of each image block can be extracted through a convolutional neural network to generate a feature vector. Feature embedding refers to converting the original pixel information or other features of an image block into a more compact and abstract representation. This representation is usually a high-dimensional vector that can capture the key features of the image block (such as texture, color, shape, etc.) while removing redundant information. The purpose of feature embedding is to enable computers to process and understand image content more efficiently.

[0032] S13. Perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block.

[0033] The self-attention mechanism is used to calculate the similarity between the feature vector of each image block and the features of the existing model training set; based on the similarity, the feature vector of each image block and the features of the existing model training set are interactively adapted to optimize the feature vector of each image block. The existing model training set is the training set used in pre-training the remote sensing image recognition model.

[0034] S14, identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain an identification result corresponding to the target remote sensing image.

[0035] Based on the optimized feature vector of each image block, each image block is classified; the image blocks of the same category are aggregated to obtain multiple image block sets, each image block set corresponds to a target classification; based on the target classification corresponding to each image block set, the recognition result corresponding to the target remote sensing image is determined. For example, the target classification corresponding to the image block set with the largest number of image blocks is used as the recognition result corresponding to the target remote sensing image.

[0036] The remote sensing image recognition method provided by the present invention obtains a target remote sensing image to be recognized and divides the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; embeds features for each image block to generate a feature vector for each image block; performs feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to perform cross-domain feature adaptation, and optimizes the feature vector of each image block; recognizes the target remote sensing image to be recognized based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image. Compared with the prior art, some networks lack effective cross-domain feature adaptation capabilities, their dependence on target domain data limits the practical application of the technology, the limitations of feature extraction methods, and the complexity of training and application lead to poor generalization performance of the model, limited practical application scenarios, low classification accuracy, low efficiency and high cost. This method improves the model's adaptability to features of different domains through image block-based feature extraction and cross-domain adaptation mechanism, so that the model trained in the source domain can be better generalized to the target domain; by combining the prototype triplet loss function, the model's intra-class clustering and inter-class distinction capabilities under the condition of a few samples are enhanced, thereby significantly improving the performance of cross-domain small sample learning; local feature extraction based on image blocks is adopted, combined with the self-attention mechanism, to effectively focus on the key areas in the image, reduce the interference of background noise on the model, and improve classification accuracy; the cross-domain adaptation method without the need for target domain data enables the model to effectively perform cross-domain classification when only using source domain data, greatly reducing the dependence on target domain data and increasing the flexibility of practical applications; through end-to-end automated design, the steps of image block division, feature extraction, cross-domain adaptation and other steps are seamlessly integrated, which simplifies the model training process and improves the efficiency and practicality of the application.

[0037] Figure 2 This is the second flow chart of the remote sensing image recognition method provided by the present invention, such as Figure 2 As shown, the method includes the following: S21, obtaining a target remote sensing image to be identified and dividing the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size.

[0038] In the embodiment of the present invention, high-resolution remote sensing image data of the target to be identified is first obtained from a platform such as a satellite or an unmanned aerial vehicle. The target remote sensing image is divided into multiple patches (Patches) of equal size by a patch-based approach for subsequent processing. Patch division can improve computational efficiency and enhance the extraction of local features.

[0039] Specifically, the input remote sensing image Divide into Image blocks of equal size, each image block represents the local information of the image. This method can highlight local features and effectively reduce the interference of background noise on the classification task. The formula is as follows: (1) in, Represents the image image blocks, is the image partitioning function.

[0040] S22. Extract the key features of each image block through convolutional neural network.

[0041] S23, performing feature embedding processing on the key features to generate a feature vector for each image block.

[0042] The above-mentioned multiple image blocks are then extracted through Convolutional Neural Networks (CNN) to extract the features of each image block, select the most valuable features for target recognition, such as texture, shape or spectral features, and embed the extracted key features to generate a high-dimensional feature vector , the feature vector of each image block is its representation in the feature space, which is used for subsequent feature interaction and classification: (2) The feature extraction of this part retains the local details of the image and helps to capture key areas in the classification task.

[0043] S24. Calculate the similarity between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism.

[0044] S25. Based on the similarity, interactively adapt the feature vector of each image block to the features of the existing model training set to optimize the feature vector of each image block.

[0045] In view of the distribution difference between the source domain (the data set used for model training) and the target domain (the data set that the model needs to process in practical applications), the embodiment of the present invention uses a self-attention mechanism to perform feature interaction between the support set (in some meta-learning or few-sample learning scenarios, the support set is a set of labeled samples used for model learning and adjustment) and the query set (the query set is a set of unlabeled samples, and the model needs to predict the query set based on the learning results of the support set) to enhance the generalization ability of the model between different domains. Through the self-attention mechanism, the features of the support set interact with the features of the query set. This interaction can help the model better understand the feature relationship between different data sets, thereby improving the generalization ability of the model, which refers to the ability of the model to perform well on unseen data. In domain adaptation tasks, generalization ability is particularly important because there is a distribution difference between the target domain data and the source domain data. Through the self-attention mechanism for feature interaction, the model can better capture the common features between the source domain and the target domain, while adapting to the unique features of the target domain. This makes the performance of the model on the target domain more robust.

[0046] Specifically, the application of the self-attention mechanism: The cross-adaptation module uses the self-attention mechanism to weight and rearrange the support set and query set features to generate new feature representations. Compute the similarity between the support set and query set features, and weight the features: (3) (4) (5) in, Represent the query vector ,key ,value , The introduction of the self-attention mechanism enables the model to dynamically adjust features according to the relevance of the task and enhance the adaptability between different domains.

[0047] The interactive adaptation between the support set and the query set is shown in Table 1: Adaptation from support set to query set: support set as key and value, query set as query, and adjust query set characteristics.

[0048] Adaptation of query set to support set: The query set is used as key and value, and the support set is used as query to enhance the adaptation of the support set to the query set.

[0049] Table 1 Detailed steps of interactive adaptation between support set and query set

[0050] Query Generation method: Query vector It is achieved by taking the input image block features It is obtained by linear transformation through a weight matrix. The formula is: (6) in, is the query weight matrix, is the feature vector of the image patch.

[0051] key Generation method: Key vector is the feature of the input image block , Z (usually the same image patch features or other related image patch features) through a weight matrix It is obtained by linear transformation. The formula is: (7) in, is the key weight matrix, is the feature vector of another image patch.

[0052] value Generate method: Value vector is the feature of the input image block (generating the same features as the key vector) through a weight matrix It is obtained by linear transformation. The formula is: (8) in, is the weight matrix of values, is the feature vector of the image patch.

[0053] These vectors ( , , ) is extracted from the input features through a linear transformation and is used in the self-attention mechanism to calculate the similarity between different image patches in order to generate new representations.

[0054] After the above interaction, the features of the support set and the query set are optimized, which enhances their similarity and solves the problem of feature differences in cross-domain classification.

[0055] S26. Classify each image block based on the optimized feature vector of each image block.

[0056] In order to improve the generalization ability of the model in small sample learning scenarios, the present invention combines the prototype network loss (Prototype Loss) and the triplet loss (Triplet Loss) and proposes a prototype triplet loss function (PTLoss).

[0057] Prototype network loss: used to calculate the distance between the query sample and the prototype of the category to which it belongs. The formula is as follows: (9) in, represents the feature embedding of the query sample, Representation Class Prototype.

[0058] Triplet loss: Maximize the distance between prototypes of different categories and push away the features of samples of different categories. The formula is: (10) Combining the two loss functions, the final prototype triplet loss is: (11) By optimizing this loss function, the model has stronger intra-class clustering ability and inter-class differentiation ability under small sample conditions.

[0059] Use the trained remote sensing image recognition model to classify each optimized image patch. Output category: Assign a category label to each image patch, such as building, vegetation, or water.

[0060] S27, aggregating image blocks of the same category to obtain multiple image block sets, each image block set corresponding to an object category.

[0061] The image blocks of the same category are aggregated to form multiple image block sets, each set corresponding to a target category.

[0062] S28. Determine a recognition result corresponding to the target remote sensing image based on the target classification corresponding to each image block set.

[0063] The classification results of all image blocks are integrated into the original image to form a complete recognition result.

[0064] Optionally, the target classification corresponding to the image block set with the largest number of image blocks may be used as the recognition result corresponding to the target remote sensing image.

[0065] For example, if the target remote sensing image is a playground, and there are students and sports equipment on the playground, and the number of image blocks classified as playground is the largest in the recognition results, the target remote sensing image can be directly identified as playground. The categories of various objects in the image can also be identified, such as people, lawns, sports equipment, etc.

[0066] Figure 3 is a network structure diagram of the remote sensing image recognition method provided by the present invention, such as Figure 3 As shown, the remote sensing image recognition model of the embodiment of the present invention is applied to the above remote sensing image recognition method, which includes: an image block division module and a cross-domain adaptation module. The image block division module is used to divide the target remote sensing image into image blocks to obtain multiple image blocks of equal size; the cross-domain adaptation module includes a cross adaptation module, which is used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block; the cross-domain adaptation module recognizes the target remote sensing image by optimizing the prototype triplet loss function.

[0067] The embodiment of the present invention realizes the automation of the whole process from image input to final classification through an end-to-end training mode. By seamlessly integrating image block division, feature extraction, adaptive enhancement and prototype triplet loss, the training and reasoning process is simplified and the practical application efficiency of the model is improved.

[0068] According to the experimental results, the embodiments of the present invention perform well in cross-domain small sample remote sensing classification tasks. In particular, in the 5-way 1-shot and 5-shot settings, the present invention has achieved significant improvements in classification accuracy on the RSSCN7→NWPU and RSSCN7→AID datasets. For example, under the 1-shot and 5-shot settings, the classification accuracy of the present invention on the NWPU dataset is 2% and 3% higher than the best existing method, respectively. Compared with the existing methods, the cross-domain adaptation module is combined with the self-attention mechanism, so that the association between the support set and the query set features can be dynamically enhanced, which significantly improves the adaptability and generalization ability of the model in different domains.

[0069] Compared with the prior art which requires multi-stage feature extraction and fine-tuning, the embodiment of the present invention provides an end-to-end training framework that integrates image segmentation, feature extraction, cross-domain adaptation and classification steps. Experimental results show that the method of the present invention outperforms existing methods that use transfer learning and fine-tuning, such as Fixed and Ft All methods, on multiple data sets. This end-to-end training not only simplifies the model development process, but also improves the application efficiency of the model in different scenarios. As shown in Table 2: Table 2 Comparison of classification accuracy with existing algorithms

[0070] The prototype triplet loss function (PTLoss) proposed in the embodiment of the present invention combines the advantages of prototype network loss and triplet loss to improve the effect of intra-class feature aggregation and inter-class feature separation. By adjusting the parameter λ of the loss function in the experiment, it is found that the classification accuracy of the model is optimal when λ=0.45, especially in the 5-way 1-shot setting, it shows a high accuracy of 83.65%, which is about 24 percentage points higher than the accuracy of the traditional ProtoNet method (59.26%). This shows that PTLoss has significant advantages in enhancing the cross-domain classification ability of the model. As shown in Table 3: Table 3 Contribution of parameter λ in RSSCN7→AID method

[0071] The beneficial effects of the embodiments of the present invention include as follows: (1) Cross-domain adaptive mechanism based on image blocks: Different from traditional global feature extraction, this paper proposes to divide the input image into image blocks of equal size and deal with cross-domain problems through feature extraction and adaptation at the image block level. This fine-grained feature extraction method at the image block level enables the model to focus on highly recognizable areas in the image, thereby improving the robustness and classification accuracy of the model in small sample and cross-domain scenarios.

[0072] (2) Combination of cross-domain feature embedding and self-attention mechanism: The present invention adopts the self-attention mechanism to perform cross-domain feature interaction and enhancement on the image block features of the support set and the query set. Through the shared cross-domain attention module, the model can establish connections between features in different domains, improve the model's understanding of the similarity of features between the support set and the query set, and enhance the classification effect.

[0073] (3) Introduction of prototype triplet loss function: This paper combines ProtoNet Loss with Triplet Loss and proposes Prototype Triplet Loss (PTLoss) to better distinguish feature embeddings of different categories. By shortening the distance between samples of the same category and the prototype and pushing the distance between prototypes of different categories, the generalization ability of the model in cross-domain small sample classification tasks is significantly improved.

[0074] (4) End-to-end cross-domain small sample classification method: The method proposed in the present invention can directly perform cross-domain small sample classification on a single dataset through end-to-end training. Compared with traditional methods that require multiple training domains or staged fine-tuning, this method is more efficient and flexible in dealing with cross-domain problems and does not require additional target domain data to participate in training.

[0075] The remote sensing image recognition device provided by the present invention is described below. The remote sensing image recognition device described below and the remote sensing image recognition method described above can be referenced to each other.

[0076] Figure 4 It is a schematic diagram of the structure of the remote sensing image recognition device provided by the present invention, which specifically includes: The image acquisition and segmentation module 401 is used to acquire the target remote sensing image to be identified and segment the target remote sensing image into image blocks to obtain multiple image blocks of equal size. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0077] The feature embedding module 402 is used to embed features for each image block and generate a feature vector for each image block. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0078] The feature interaction module 403 is used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0079] The image recognition module 404 is used to recognize the target remote sensing image to be recognized based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0080] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the remote sensing image recognition method, which includes: obtaining a target remote sensing image to be identified and dividing the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; embedding features for each image block to generate a feature vector for each image block; performing feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block; identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image.

[0081] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the remote sensing image recognition method provided by the above-mentioned methods, which includes: obtaining a target remote sensing image to be identified and dividing the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; embedding features for each image block to generate a feature vector for each image block; performing feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to optimize the feature vector of each image block; and identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the remote sensing image recognition method provided by the above-mentioned methods, the method comprising: obtaining a target remote sensing image to be identified and dividing the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; embedding features for each image block to generate a feature vector for each image block; performing feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to optimize the feature vector of each image block; identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image recognition method, characterized in that: include: Acquire a remote sensing image of a target to be identified and divide the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; Perform feature embedding on each image block to generate a feature vector for each image block; Performing feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to optimize the feature vector of each image block; The target remote sensing image to be identified is identified based on the optimized feature vector of each image block to obtain an identification result corresponding to the target remote sensing image.

2. The method according to claim 1, characterized in that The step of embedding features for each image block to generate a feature vector for each image block includes: Extract the key features of each image block through convolutional neural network; The key features are subjected to feature embedding processing to generate a feature vector for each image block.

3. The method according to claim 1, characterized in that The step of performing feature interaction between the feature vector of each image block and the features of an existing model training set through a self-attention mechanism to optimize the feature vector of each image block includes: Calculating the cosine similarity between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism; Based on the cosine similarity, the feature vector of each image block is interactively adapted to the features of the existing model training set to optimize the feature vector of each image block.

4. The method according to claim 3, characterized in that The step of identifying the target remote sensing image to be identified based on the optimized feature vector of each image block to obtain an identification result corresponding to the target remote sensing image includes: Classifying each image block based on the optimized feature vector of each image block; Aggregate image blocks of the same category to obtain multiple image block sets, each of which corresponds to a target category; The recognition result corresponding to the target remote sensing image is determined based on the target classification corresponding to each image block set.

5. The method according to claim 4, characterized in that The determining the recognition result corresponding to the target remote sensing image based on the target classification corresponding to each image block set includes: The target classification corresponding to the image block set with the largest number of image blocks is taken as the recognition result corresponding to the target remote sensing image.

6. A remote sensing image recognition model, applied to the remote sensing image recognition method according to any one of claims 1 to 5, characterized in that: include: Image block division module and cross-domain adaptation module; The image block division module is used to divide the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; The cross-domain adaptation module includes a cross adaptation module, which is used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism to optimize the feature vector of each image block; The cross-domain adaptive module recognizes the target remote sensing image by optimizing the prototype triplet loss function.

7. A remote sensing image recognition device, characterized in that: include: An image acquisition and segmentation module is used to acquire a remote sensing image of a target to be identified and to segment the target remote sensing image into image blocks to obtain a plurality of image blocks of equal size; A feature embedding module, used to embed features for each image block and generate a feature vector for each image block; A feature interaction module, used to perform feature interaction between the feature vector of each image block and the features of the existing model training set through a self-attention mechanism, so as to optimize the feature vector of each image block; The image recognition module is used to recognize the target remote sensing image to be recognized based on the optimized feature vector of each image block to obtain a recognition result corresponding to the target remote sensing image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the remote sensing image recognition method according to any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image recognition method according to any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the remote sensing image recognition method according to any one of claims 1 to 5 is implemented.

Images