Method, device, equipment, medium and product for generating remote sensing image recognition model

Abstract

The present application provides a remote sensing image recognition model generation method, device, equipment, medium, and product, relating to the fields of artificial intelligence technology, financial technology, or other related fields. The method comprises: obtaining image description text and sample images; inputting the image description text into a semantic understanding model to generate corresponding identification element information and background description information; inputting the identification element information and the sample image into a remote sensing image segmentation model to generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; inputting the background description information into a background generation model to generate a remote sensing background map; fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background map to generate a remote sensing training sample; and training a preset remote sensing image recognition model based on the remote sensing training sample to generate a corresponding target remote sensing image recognition model. The method of the present application can improve the training and generation effect of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, financial technology or other related fields, and in particular to a remote sensing image recognition model generation method, device, equipment, medium and product. Background Art

[0002] In the financial sector, satellite remote sensing imagery is often used for post-loan monitoring of projects. Therefore, the accuracy and efficiency of remote sensing image recognition models are particularly important. Rapid iterative optimization of remote sensing image recognition models requires a large number of remote sensing training samples.

[0003] At present, it is difficult to produce high-precision remote sensing images, and businesses generally choose to acquire remote sensing images on a quarterly or even annual basis. It is difficult for the model to learn enough target samples for training.

[0004] Therefore, the current number of remote sensing training samples is relatively small, resulting in poor training and generation effects of remote sensing image recognition models and slow iteration speed. Summary of the Invention

[0005] The present application provides a remote sensing image recognition model generation method, device, equipment, medium and product to solve the problem that the number of current remote sensing training samples is small, resulting in poor remote sensing image recognition model training results and slow iteration speed.

[0006] The first aspect of the present application provides a remote sensing image recognition model generation method, comprising:

[0007] Obtain image description text and sample images corresponding to the remote sensing training samples to be generated;

[0008] Inputting the image description text into a semantic understanding model trained to convergence, generating corresponding recognition element information and background description information;

[0009] Inputting the identification element information and the example image into a large remote sensing image segmentation model trained to convergence, generating a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; the example image at least includes the identification element corresponding to the identification element information;

[0010] Inputting the background description information into a background generation model trained to convergence to generate a remote sensing background map;

[0011] fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample;

[0012] A preset remote sensing image recognition model is trained based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

[0013] Furthermore, in the above method, the remote sensing image segmentation large model includes: an encoder and a decoder;

[0014] The step of inputting the identification element information and the sample image into a large remote sensing image segmentation model trained to convergence, and generating a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image, comprises:

[0015] Using an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image;

[0016] A decoder is used to perform attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image.

[0017] Furthermore, in the method described above, the encoder includes a swin-transformer network structure;

[0018] The step of using an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image includes:

[0019] Using an encoder to perform feature extraction and image block processing on the sample image to generate image feature vectors corresponding to the image blocks and position feature vectors corresponding to relative positions between the image blocks;

[0020] Generating a position representation vector corresponding to the example image according to the image feature vector corresponding to the image block and the relative position vector between the image blocks;

[0021] The position representation vector is input into the swin-transformer network structure to generate the encoding information matrix corresponding to all image blocks.

[0022] Furthermore, in the method described above, the decoder includes an attention layer and a swin-transformer network structure;

[0023] The decoder performs attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image, including:

[0024] Generate a corresponding original attention map according to the position representation vector, the identification element information and the attention layer;

[0025] Generate a corresponding encoding attention map according to the encoding information matrix, the identification element information and the attention layer;

[0026] Fitting the encoded attention map based on the original attention map, and inputting the fitted encoded attention map into the swin-transformer network structure to generate a segmented image of the same size as the image feature vector;

[0027] The segmented image is processed for classification prediction to generate the identification element category corresponding to each pixel in the segmented image.

[0028] Furthermore, in the above method, the step of fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample includes:

[0029] Binarize the identification element category corresponding to each pixel in the segmented image to generate a mask label corresponding to each pixel;

[0030] fusing the segmented image and the remote sensing background image to generate a corresponding fused image;

[0031] The mask annotation and the fused image are fused to generate a remote sensing training sample.

[0032] Furthermore, in the above method, fusing the segmented image and the remote sensing background image to generate a corresponding fused image includes:

[0033] Performing size assimilation processing on the segmented image and the remote sensing background image so that the sizes of the processed segmented image and the remote sensing background image are the same;

[0034] The processed segmented image is convolved and resampled based on the preset spectral response function to generate simulated grayscale values of the full-color band;

[0035] Performing Schmidt orthogonal transformation on the processed remote sensing background image to generate the corresponding Schmidt orthogonal transformation result;

[0036] Adjust the simulated grayscale value accordingly based on a preset mean-variance adjustment algorithm to generate an adjusted simulated grayscale value;

[0037] The adjusted analog grayscale value is used to replace the first component in the Schmidt orthogonalization transformation result, and the Schmidt orthogonalization inverse transformation is performed on the Schmidt orthogonalization transformation result to generate an inverse transformation result;

[0038] The first band in the inverse transformation result is removed to generate a fused image corresponding to the fusion of the segmented image and the remote sensing background image.

[0039] Furthermore, in the method described above, the background generation model is an artificial intelligence image generation model;

[0040] The step of inputting the background description information into a background generation model trained to convergence to generate a remote sensing background map includes:

[0041] Determining image keywords from the background description information using an artificial intelligence image generation model;

[0042] An artificial intelligence image generation model is used to generate a corresponding remote sensing background image based on the image keywords.

[0043] Furthermore, the method as described above, before inputting the identification element information and the example image into a large remote sensing image segmentation model trained to convergence to generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image, further includes:

[0044] Acquire a training sample, wherein the training sample includes: a remote sensing training image and annotation information corresponding to the remote sensing training image;

[0045] Inputting the training samples into a preset remote sensing image segmentation large model, and training the preset remote sensing image segmentation large model based on a preset random masking module; the preset random masking module is used to randomly mask pixels of image blocks when the preset remote sensing image segmentation large model performs image block processing;

[0046] Determining whether the preset remote sensing image segmentation model satisfies a preset convergence condition based on a segmented image output by the preset remote sensing image segmentation model and a recognition element category corresponding to each pixel in the segmented image;

[0047] If the preset remote sensing image segmentation large model meets the convergence condition, the preset remote sensing image segmentation large model that meets the convergence condition is determined as the remote sensing image segmentation large model trained to convergence.

[0048] Furthermore, in the above method, the training of a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model includes:

[0049] Dividing the remote sensing training samples into a training set and a validation set according to a preset ratio;

[0050] Iteratively train the preset remote sensing image recognition model using the training set to generate the preset remote sensing image recognition model corresponding to each iterative version;

[0051] Based on the validation set, the model accuracy of the preset remote sensing image recognition model corresponding to each iterative version is verified, and the average accuracy value corresponding to the preset remote sensing image recognition model of each iterative version is generated;

[0052] The preset remote sensing image recognition model corresponding to the largest average precision mean is determined as the target remote sensing image recognition model.

[0053] A second aspect of the present application provides a remote sensing image recognition model generation device, comprising:

[0054] An acquisition module is used to obtain image description text and sample images corresponding to the remote sensing training samples to be generated;

[0055] A first generation module is used to train the image description text input to a convergent semantic understanding model to generate corresponding recognition element information and background description information;

[0056] a second generation module, configured to input the identification element information and the example image into a large remote sensing image segmentation model trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; the example image at least including the identification element corresponding to the identification element information;

[0057] A third generation module is configured to input the background description information into a background generation model trained to convergence to generate a remote sensing background map;

[0058] A fusion module is used to fuse the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample;

[0059] The fourth generation module is used to train a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

[0060] Furthermore, in the above-mentioned device, the remote sensing image segmentation large model includes: an encoder and a decoder;

[0061] The second generating module is specifically used for:

[0062] An encoder is used to perform feature extraction and image segmentation processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image; a decoder is used to perform attention mechanism processing and classification prediction processing on the position representation vector and the coding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image.

[0063] Furthermore, in the device as described above, the encoder includes a swin-transformer network structure;

[0064] When the second generation module uses an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image, it is specifically used to:

[0065] An encoder is used to perform feature extraction and image block processing on the example image to generate image feature vectors corresponding to the image blocks and position feature vectors corresponding to the relative positions between the image blocks; a position representation vector corresponding to the example image is generated based on the image feature vectors corresponding to the image blocks and the relative position vectors between the image blocks; the position representation vector is input into the swin-transformer network structure to generate a coding information matrix corresponding to all image blocks.

[0066] Furthermore, in the apparatus as described above, the decoder includes an attention layer and a swin-transformer network structure;

[0067] The second generation module, when using a decoder to perform attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image, is specifically used to:

[0068] According to the position representation vector, the identification element information and the attention layer, a corresponding original attention map is generated; according to the encoding information matrix, the identification element information and the attention layer, a corresponding encoding attention map is generated; the encoding attention map is fitted based on the original attention map, and the fitted encoding attention map is input into the swin-transformer network structure to generate a segmented image of the same size as the image feature vector; the segmented image is classified and predicted to generate the identification element category corresponding to each pixel in the segmented image.

[0069] Furthermore, in the above device, the fusion module is specifically used to:

[0070] The identification element category corresponding to each pixel in the segmented image is binarized to generate a mask annotation corresponding to each pixel; the segmented image and the remote sensing background image are fused to generate a corresponding fused image; and the mask annotation and the fused image are fused to generate a remote sensing training sample.

[0071] Furthermore, in the above-mentioned device, when the fusion module fuses the segmented image and the remote sensing background image to generate a corresponding fused image, it is specifically used to:

[0072] The segmented image and the remote sensing background image are size assimilated so that the sizes of the processed segmented image and the remote sensing background image are the same; the processed segmented image is convolutionally resampled based on a preset spectral response function to generate a simulated grayscale value of the full-color band; the processed remote sensing background image is Schmidt orthogonalized to generate a corresponding Schmidt orthogonalized transformation result; the simulated grayscale value is adjusted accordingly based on a preset mean variance adjustment algorithm to generate an adjusted simulated grayscale value; the adjusted simulated grayscale value replaces the first component in the Schmidt orthogonalized transformation result, and the Schmidt orthogonalized inverse transformation is performed on the Schmidt orthogonalized transformation result to generate an inverse transformation result; the first band in the inverse transformation result is removed to generate a fused image corresponding to the fusion of the segmented image and the remote sensing background image.

[0073] Furthermore, in the above-mentioned device, the background generation model is an artificial intelligence image generation model;

[0074] The third generation module is specifically used for:

[0075] An artificial intelligence image generation model is used to determine image keywords from the background description information; and an artificial intelligence image generation model is used to generate a corresponding remote sensing background image based on the image keywords.

[0076] Furthermore, the device as described above further comprises:

[0077] A training module is used to obtain training samples, wherein the training samples include: remote sensing training images and annotation information corresponding to the remote sensing training images; inputting the training samples into a preset remote sensing image segmentation large model, and training the preset remote sensing image segmentation large model based on a preset random masking module; the preset random masking module is used to randomly mask the pixels of the image blocks when the preset remote sensing image segmentation large model performs image segmentation processing; determining whether the preset remote sensing image segmentation large model meets the preset convergence condition based on the segmented image output by the preset remote sensing image segmentation large model and the identification element category corresponding to each pixel in the segmented image; if the preset remote sensing image segmentation large model meets the convergence condition, then determining the preset remote sensing image segmentation large model that meets the convergence condition as the remote sensing image segmentation large model trained to convergence.

[0078] Furthermore, in the above device, the fourth generating module is specifically configured to:

[0079] The remote sensing training samples are divided into a training set and a validation set according to a preset ratio; the preset remote sensing image recognition model is iteratively trained using the training set to generate a preset remote sensing image recognition model corresponding to each iterative version; the model accuracy of the preset remote sensing image recognition model corresponding to each iterative version is verified based on the validation set to generate an average accuracy mean value corresponding to each iterative version of the preset remote sensing image recognition model; the preset remote sensing image recognition model corresponding to the largest average accuracy mean value is determined as the target remote sensing image recognition model.

[0080] A third aspect of the present application provides an electronic device, comprising: a memory and a processor;

[0081] The memory stores computer-executable instructions;

[0082] The processor executes the computer-executable instructions stored in the memory to implement the remote sensing image recognition model generation method as described in any one of the first aspects.

[0083] The fourth aspect of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the remote sensing image recognition model generation method described in any one of the first aspects.

[0084] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the remote sensing image recognition model generation method described in any one of the first aspects.

[0085] The present application provides a remote sensing image recognition model generation method, apparatus, device, medium, and product, the method comprising: obtaining image description text and an example image corresponding to a remote sensing training sample to be generated; inputting the image description text into a semantic understanding model trained to convergence to generate corresponding identification element information and background description information; inputting the identification element information and the example image into a remote sensing image segmentation model trained to convergence to generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; the example image at least includes an identification element corresponding to the identification element information; inputting the background description information into a background generation model trained to convergence to generate a remote sensing background image; fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample; and training a preset remote sensing image recognition model based on the remote sensing training sample to generate a corresponding target remote sensing image recognition model. The remote sensing image recognition model generation method of the present application inputs the identification element information and the example image into a remote sensing image segmentation model trained to convergence to generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image. The background description information is input into a background generation model trained to convergence to generate a remote sensing background image. Based on the segmented image, the identification element categories corresponding to each pixel in the segmented image, and the remote sensing background image, a remote sensing training sample containing the remote sensing background and specific identification elements is generated. This reduces the difficulty of generating remote sensing training samples, making them more convenient and efficient. Furthermore, training a preset remote sensing image recognition model based on the remote sensing training samples improves the training and generation performance of the remote sensing image recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0087] Figure 1 A scene graph that can implement the remote sensing image recognition model generation method of the embodiment of the present application;

[0088] Figure 2 Schematic diagram of the process of generating the remote sensing image recognition model provided in this application Figure 1 ;

[0089] Figure 3 Schematic diagram of the process of generating the remote sensing image recognition model provided in this application Figure 2 ;

[0090] Figure 4 A schematic diagram of the structure of the remote sensing image recognition model generation device provided in this application;

[0091] Figure 5This is a schematic diagram of the structure of the electronic device provided in this application.

[0092] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0094] In the technical solutions of the embodiments of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0096] It should be noted that the disclosed remote sensing image recognition model generation method, apparatus, device, medium, and product can be used in the fields of artificial intelligence technology, financial technology, or other related fields. They can also be used in any field other than artificial intelligence technology, financial technology, or other related fields. The application fields of the disclosed remote sensing image recognition model generation method, apparatus, device, medium, and product are not limited.

[0097] The technical solution of the present application is described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0098] In order to clearly understand the technical solution of this application, we first introduce the existing technology in detail. The post-loan monitoring task of engineering projects based on remote sensing target detection technology has made great progress in recent years. However, the demand for rapid iterative optimization of remote sensing models in complex business scenarios still faces great challenges. The main problems are as follows:

[0099] Acquiring high-precision remote sensing imagery is expensive. To save costs, businesses often choose to acquire remote sensing imagery on a quarterly or even annual basis. This makes it difficult for models to learn enough target samples for training. Remote sensing images have complex backgrounds, and background features vary significantly between seasons in some regions, significantly impacting the accuracy of target detection tasks within the image. Consequently, it's difficult for models to learn enough target samples for training, resulting in poor training results and slow iteration speeds for remote sensing image recognition models.

[0100] To address the problem of a limited number of remote sensing training samples in existing technologies, which leads to poor training and generation results for remote sensing image recognition models and slow iteration speeds, the inventors discovered that, to address this issue, a large remote sensing image segmentation model can be used to segment and extract sample images containing recognition elements, generating segmented images. A background generation model can then be used to generate a remote sensing background image. Simultaneously, the segmented image and the remote sensing background image are fused to obtain a remote sensing image suitable for training. This solves the problem of insufficient remote sensing training samples.

[0101] Specifically, obtain the image description text and sample image corresponding to the remote sensing training sample to be generated. Input the image description text into the semantic understanding model that has been trained to convergence, and generate corresponding identification element information and background description information. Input the identification element information and sample image into the remote sensing image segmentation model that has been trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image. The sample image at least includes the identification element corresponding to the identification element information. Input the background description information into the background generation model that has been trained to convergence, and generate a remote sensing background image. The segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image are fused to generate a remote sensing training sample. Based on the remote sensing training sample, the preset remote sensing image recognition model is trained to generate a corresponding target remote sensing image recognition model.

[0102] The remote sensing image recognition model generation method of the present application inputs recognition element information and sample images to a converged remote sensing image segmentation model, generates a segmented image corresponding to the recognition element information and a recognition element category corresponding to each pixel in the segmented image. Inputs background description information to a background generation model trained to convergence to generate a remote sensing background map. Thus, based on the segmented image, the recognition element category corresponding to each pixel in the segmented image, and the remote sensing background map, remote sensing training samples with remote sensing background and specific recognition elements are fused and generated. This reduces the difficulty of generating remote sensing training samples, and remote sensing training samples can be generated more conveniently and quickly. At the same time, training a preset remote sensing image recognition model based on the remote sensing training samples improves the training generation effect of the remote sensing image recognition model.

[0103] Based on the above creative findings, the inventor proposed the technical solution of this application.

[0104] The following describes the application scenarios of the remote sensing image recognition model generation method provided in the embodiment of the present application. Figure 1 As shown, 1 is a first electronic device and 2 is a second electronic device. The network architecture for the application scenario corresponding to the remote sensing image recognition model generation method provided in the embodiments of the present application includes: a first electronic device 1 and a second electronic device 2. The second electronic device 2 may be a database, for example. The second electronic device 2 stores image description text and example images corresponding to the remote sensing training samples to be generated. The image description text and example images can be provided by the user and input into the second electronic device 2.

[0105] For example, when a remote sensing image recognition model needs to be generated, the following process is performed:

[0106] ① The second electronic device 2 sends the image description text and the sample image to the first electronic device 1.

[0107] ② The first electronic device 1 trains the image description text input into a convergent semantic understanding model to generate corresponding recognition element information and background description information.

[0108] ③ The first electronic device 1 inputs the identification element information and the sample image into a large remote sensing image segmentation model that has been trained to convergence, and generates a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image. The sample image includes at least the identification element corresponding to the identification element information.

[0109] ④ The first electronic device 1 inputs the background description information into the background generation model trained to convergence to generate a remote sensing background image.

[0110] ⑤ The first electronic device 1 fuses the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample. After generating the remote sensing training sample, the remote sensing training sample can be sent to an electronic device for training a remote sensing image recognition model, thereby improving the training effect of the remote sensing image recognition model.

[0111] The embodiments of the present application are introduced below with reference to the accompanying drawings.

[0112] Figure 2 Schematic diagram of the process of generating the remote sensing image recognition model provided in this application Figure 1 ,like Figure 2 As shown, in this embodiment, the execution subject of the embodiment of the present application is a remote sensing image recognition model generation device, which can be integrated into an electronic device. The remote sensing image recognition model generation method provided in this embodiment includes the following steps:

[0113] Step S101: Obtain image description text and sample images corresponding to the remote sensing training sample to be generated.

[0114] In this embodiment, the acquisition method may be a method in which the user inputs the image description text and the sample image, or may be acquired from a device storing the image description text and the sample image, which is not limited in this embodiment.

[0115] Step S102: Input the image description text into a semantic understanding model that has been trained to convergence, and generate corresponding recognition element information and background description information.

[0116] In this embodiment, the identification element information includes the identification element category.

[0117] The semantic understanding model primarily performs entity recognition (NAME Entity Recognition) tasks, identifying key entities within user-entered text descriptions. Entity recognition addresses the sequence labeling problem in natural language processing: given an input sentence, each part of the sentence must be labeled with an entity (e.g., a place name, a date, etc.). In this embodiment, the semantic understanding module primarily focuses on two key entity types: image context descriptions and specific identification element categories.

[0118] The semantic understanding model can be trained in advance. During the training phase, this embodiment can use the pre-training model ERNIE (ERNIE is a semantic understanding pre-training framework based on continuous learning) to collect text descriptions of a large amount of project monitoring information provided by the business, and mark the image background descriptions (such as jungles, cities, snow-covered mountains, etc.) and specific identification element categories (such as windmills, ships, oil barrels, etc.) in different projects as training samples. Use the training samples to fine-tune the ERNIE model, set the warmup dynamic learning rate, and finally save the model with the best effect on the validation set. Warmup is a commonly used technology that can effectively alleviate the training instability of deep neural networks in the initial stage, accelerate the convergence speed of the model, and improve the generalization ability of the model.

[0119] Step S103: Input the identification element information and the sample image into a large remote sensing image segmentation model that has been trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image. The sample image at least includes the identification element corresponding to the identification element information.

[0120] In this embodiment, the remote sensing image segmentation model is used to segment corresponding portions of the example image based on the identification element information, thereby generating a segmented image corresponding to the identification element information and the identification element category corresponding to each pixel in the segmented image. For example, if the example image includes identification elements such as a building, a windmill, and a car, and the identification element information is a windmill, the remote sensing image segmentation model can be used to segment the example image to obtain the image content corresponding to the windmill and the identification element category corresponding to each pixel in the image content, such as the identification element category being background or the identification element category being windmill.

[0121] Step S104: input the background description information into the background generation model trained to convergence to generate a remote sensing background map.

[0122] In this embodiment, the background generation model can adopt an artificial intelligence image generation model, so as to directly generate a corresponding remote sensing background image based on the background description information.

[0123] Step S105 , fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample.

[0124] By fusing the segmented image with identification elements, the identification element categories corresponding to each pixel in the segmented image, and the remote sensing background image with background, remote sensing training samples that contain both identification elements and background and have corresponding annotations of identification elements can be generated.

[0125] Step S106: training a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

[0126] Generally speaking, remote sensing training samples can be divided into training sets and validation sets, so that the preset remote sensing image recognition model can be trained based on the training set, and the accuracy of the preset remote sensing image recognition model can be verified based on the validation set to generate the corresponding target remote sensing image recognition model.

[0127] The embodiment of the present application provides a remote sensing image recognition model generation method, which obtains image description text and example images corresponding to the remote sensing training samples to be generated. The image description text is input into a semantic understanding model that is trained to convergence, and corresponding identification element information and background description information are generated. The identification element information and the example image are input into a remote sensing image segmentation model that is trained to convergence, and a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image are generated. The example image at least includes an identification element corresponding to the identification element information. The background description information is input into a background generation model that is trained to convergence, and a remote sensing background map is generated. The segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background map are fused to generate a remote sensing training sample. The preset remote sensing image recognition model is trained based on the remote sensing training sample to generate a corresponding target remote sensing image recognition model.

[0128] The remote sensing image recognition model generation method of the present application inputs recognition element information and sample images to a converged remote sensing image segmentation model, generates a segmented image corresponding to the recognition element information and a recognition element category corresponding to each pixel in the segmented image. Inputs background description information to a background generation model trained to convergence to generate a remote sensing background map. Thus, based on the segmented image, the recognition element category corresponding to each pixel in the segmented image, and the remote sensing background map, remote sensing training samples with remote sensing background and specific recognition elements are fused and generated. This reduces the difficulty of generating remote sensing training samples, and remote sensing training samples can be generated more conveniently and quickly. At the same time, training a preset remote sensing image recognition model based on the remote sensing training samples improves the training generation effect of the remote sensing image recognition model.

[0129] Figure 3 Schematic diagram of the process of generating the remote sensing image recognition model provided in this application Figure 2 ,like Figure 3 As shown, the remote sensing image recognition model generation method provided in this embodiment is based on the remote sensing image recognition model generation method provided in the previous embodiment of this application, and further refines the process of generating training samples. The remote sensing image recognition model generation method provided in this embodiment includes the following steps.

[0130] Step S201: Obtain image description text and sample images corresponding to the remote sensing training sample to be generated.

[0131] In this embodiment, the implementation of step 201 is similar to the implementation of step 101 in the previous embodiment, and will not be repeated here.

[0132] Step S202: Input the image description text into a trained convergent semantic understanding model to generate corresponding recognition element information and background description information.

[0133] In this embodiment, the implementation of step 202 is similar to the implementation of step 102 in the previous embodiment, and will not be repeated here.

[0134] It should be noted that, in this embodiment, the remote sensing image segmentation model includes: an encoder and a decoder.

[0135] Optionally, in this embodiment, before performing remote sensing image segmentation, a large remote sensing image segmentation model may be trained as follows:

[0136] The training samples are obtained, where the training samples include: remote sensing training images and annotation information corresponding to the remote sensing training images.

[0137] The training samples are input into a preset remote sensing image segmentation model, and the preset remote sensing image segmentation model is trained based on a preset random masking module. The preset random masking module is used to randomly mask the pixels of the image blocks when the preset remote sensing image segmentation model performs image segmentation processing.

[0138] Whether the preset remote sensing image segmentation model meets the preset convergence condition is determined based on the segmented image output by the preset remote sensing image segmentation model and the identification element category corresponding to each pixel in the segmented image.

[0139] If the preset remote sensing image segmentation large model meets the convergence condition, the preset remote sensing image segmentation large model that meets the convergence condition is determined as the remote sensing image segmentation large model trained to convergence.

[0140] In this embodiment, a variety of image annotation datasets are introduced for the self-supervised learning task of a large remote sensing image segmentation model, forming a combination of training datasets corresponding to remote sensing training images. The training datasets cover a large number of general segmentation datasets and remote sensing-specific segmentation datasets. The general datasets increase data diversity and are used to enhance the general segmentation capabilities of the pre-trained model. The remote sensing-specific public datasets are used to enhance the professional segmentation capabilities of the pre-trained model in remote sensing scenarios. The three main representative datasets are described below. Other datasets can also be used, and this embodiment does not limit them:

[0141] Segment Anything 1-Billion (SA-1B): This dataset consists of over 11 million diverse, high-resolution images and their corresponding high-quality segmentation mask annotations, sourced from multiple contributors across regions and income levels. The images primarily feature locations, objects, and scenes, with varying subject matter. The masks range from large objects like buildings to fine-grained details like small animals. This dataset is primarily used to train the model's segmentation capabilities for general objects.

[0142] Remote Sensing Public Dataset: This dataset contains public datasets for various remote sensing land segmentation competitions. It consists of more than 1 million remote sensing images of different resolutions, scenes, and objects. It is mainly used to train the model's generalization ability in remote sensing scenarios.

[0143] Satellite-acquired datasets: Remote sensing image data is further captured from satellite and aerial platforms, covering different objects and scenes on six continents. They are mainly used to fine-tune the model for final downstream task training and improve the model's adaptability to different scenarios.

[0144] Since current conventional supervised learning requires a large amount of labeled data to train the basic model, and since remote sensing images have multiple resolutions and varying quality, obtaining labeled data requires a large number of remote sensing experts and a lot of labeling time. This embodiment designs a generative self-supervised learning training method based on multi-scale random masks, aiming to reconstruct masked pixels in remote sensing images of different resolutions and learn general feature representations in remote sensing image distributions.

[0145] The idea behind this method is to randomly mask pixel blocks in certain areas of an image and require the network to reconstruct the original image. Convergence is achieved when the accuracy of the reconstructed original image exceeds a preset threshold. Unlike conventional random masking methods, which generate image blocks of fixed size and then randomly mask them completely, this embodiment employs a multi-scale random masking method to address the multi-resolution nature of remote sensing images and avoid losing feature information about small objects in the image.

[0146] First, a variety of image block sizes are designed in the image block generation stage. Larger block sizes (64*64, 128*128, etc.) are designed for images with higher resolutions (such as 0.5m, 0.75m resolution), and smaller block sizes (14*14, 28*28, etc.) are designed for images with lower resolutions (such as 10m resolution), thereby ensuring that the pixel proportion of small objects in image blocks of different resolutions is high. Second, the strategy of random complete masking within the image block is replaced by masking with small windows of 1*1, 3*3, and 5*5, thereby effectively retaining the pixel feature information of some small targets in each image block. This embodiment randomly masks the pixels of the image block when performing image segmentation processing using a large remote sensing image segmentation model, and requires the network to be able to reconstruct the original image, thereby improving the training effect of the large remote sensing image segmentation model.

[0147] Step S203: Using an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image.

[0148] The position representation vector is related to the position information of the example image, and the encoding information matrix is related to the image information of the example image.

[0149] Optionally, in this embodiment, the encoder includes a swin-transformer network structure.

[0150] S203 may specifically be:

[0151] The encoder is used to perform feature extraction and image block processing on the sample image to generate image feature vectors corresponding to the image blocks and position feature vectors corresponding to the relative positions between the image blocks.

[0152] A position representation vector corresponding to the example image is generated according to the image feature vector corresponding to the image block and the relative position vector between the image blocks.

[0153] The position representation vector is input into the swin-transformer network structure to generate the encoding information matrix corresponding to all image blocks.

[0154] In this embodiment, the model structure introduces swin-transformer as the encoder backbone network. After receiving the sample image, the encoder extracts features from the sample image to generate the original image matrix vector X. The encoder divides the input image into blocks, and adds the image feature vector of the image block and the position feature vector of the relative position of the image block on the original image to obtain the position representation vector X' corresponding to the original image matrix. The position representation vector X' is passed into the swin-transformer backbone network, and after N consecutive swin-transformer blocks, the encoding information matrix C of all image blocks of the original image is obtained. The dimension of the encoding information matrix is n*d, where n represents the number of image blocks and d represents the dimension of each image block.

[0155] In step S204, a decoder is used to perform attention mechanism processing and classification prediction processing on the position representation vector and the coding information matrix based on the recognition element information to generate a segmented image corresponding to the image recognition element and a recognition element category corresponding to each pixel in the segmented image.

[0156] Optionally, in this embodiment, the decoder includes an attention layer and a swin-transformer network structure.

[0157] S204 may specifically include:

[0158] According to the position representation vector, recognition element information and attention layer, the corresponding original attention map is generated.

[0159] According to the encoding information matrix, recognition factor information and attention layer, the corresponding encoding attention map is generated.

[0160] The encoded attention map is fitted based on the original attention map, and the fitted encoded attention map is input into the swin-transformer network structure to generate a segmented image of the same size as the image feature vector.

[0161] The segmented image is processed for classification prediction to generate the identification element category corresponding to each pixel in the segmented image.

[0162] To address the difficulty in identifying key areas of shared features between classes and across samples when segmenting fine-grained visual objects in remote sensing imagery, this embodiment designs a self-enhanced attention mechanism by adding an attention layer to the decoder. First, the decoder inputs the encoded information matrix C into an ESE (Effective Squeeze and Extraction) attention layer to generate an encoded attention map Ce. This attention layer processes the encoded information matrix C based on the identification factor information. Simultaneously, the position representation vector X' is input into an ESE attention layer, which processes the position representation vector X' based on the identification factor information to generate the original attention map X'e.

[0163] The encoded attention map is fitted and optimized using the position information in the original attention map X'e, optimizing the KL divergence (Kullback-Leibler divergence), also known as relative entropy, between Ce and X'e, thereby optimizing the correspondence between positions and features in the encoded attention map. The fitted encoded attention map is then fed into N consecutive swin-transformer blocks to obtain a segmented image of the same size as the input original image matrix vector X. Finally, the image segmentation task head is attached to the pre-trained backbone. The task head mainly consists of a linear layer and a softmax layer, which is used to predict the segmentation result category of each pixel, ultimately generating the identification feature category corresponding to each pixel in the segmented image.

[0164] Step S205 : inputting the background description information into the background generation model trained to convergence to generate a remote sensing background map.

[0165] Optionally, in this embodiment, the background generation model is an artificial intelligence image generation model. S205 may specifically be:

[0166] An artificial intelligence image generation model is used to determine image keywords from background description information.

[0167] An artificial intelligence image generation model is used to generate corresponding remote sensing background images based on image keywords.

[0168] In this embodiment, the artificial intelligence image generation model can adopt a stable diffusion (SD) model, and the corresponding remote sensing background image can be generated through image keywords.

[0169] To achieve the task of generating complex remote sensing backgrounds, the present invention constructs a multi-resolution, multi-scene remote sensing-specific training dataset and introduces the dreambooth training method to fine-tune the SD model.

[0170] In this example, a variety of image annotation datasets are introduced for the background generation model fine-tuning task, forming a training dataset combination. The following is an introduction to the main datasets:

[0171] Million-Aid dataset: Contains millions of remote sensing scene instances and 51 scene categories, covering different scenes such as agricultural land, industrial land, residential land, and water areas. It has the advantages of high spatial resolution, large scale, and global distribution.

[0172] SIRI-WHU dataset: Contains 12 types of images obtained from satellite images, including scenes such as ports, parks, residences, and rivers.

[0173] NWPU-RESISC45 dataset: Contains more than 30,000 high-resolution remote sensing images, covering 45 scene categories such as airports, snow-capped mountains, jungles, deserts, and commercial areas.

[0174] Other data sets may also be used, and this embodiment does not limit this.

[0175] To address the limited output domain and expressiveness of the original stable diffusion (SD) model when generating complex remote sensing background images and to avoid language drift and overfitting, this example introduces the Dreambooth algorithm to fine-tune the generative model, thereby achieving the function of authentically restoring keyword entities in images.

[0176] The principle of the dreambooth algorithm is to input training pictures and keywords, and then use the method of adding "special identifier" weights to entity descriptions. Overall, it is based on the scene category features that artificial intelligence has learned, and then uses the special features learned by special identifiers to modify the scene, making the background image more complex and realistic.

[0177] When training the original SD model using the aforementioned dataset, the Dreambooth algorithm was introduced. Training images with key entities (such as vehicles) and text with a special identifier [V] for these key entities were prepared. These were then fed into the SD model to fine-tune the entire model network, enhancing the complexity and realism of the generated image details. For example, when generating a scene like "a parking lot full of cars," the model would add a special identifier [V] to the vehicle description. By learning the personalized details of various vehicles, it can generate more reasonable and complex parking scene images.

[0178] Step S206 : Binarize the identification element category corresponding to each pixel in the segmented image to generate a mask label corresponding to each pixel.

[0179] After the identification feature category is binarized, if 1 represents the background and 0 represents the identification feature, the value corresponding to each pixel can be output and used as a mask annotation for subsequent model training.

[0180] Step S207: Fusing the segmented image and the remote sensing background image to generate a corresponding fused image.

[0181] Optionally, in this embodiment, image fusion is mainly performed by fusing a multispectral low-resolution segmented image with a high-resolution panchromatic band background generated image, thereby obtaining a result sample image with richer information.

[0182] Optionally, in this embodiment, S207 may specifically be:

[0183] The segmented image and the remote sensing background image are size assimilated so that the sizes of the processed segmented image and the remote sensing background image are the same.

[0184] The processed segmented image is convolved and resampled based on the preset spectral response function to generate simulated grayscale values of the full-color band.

[0185] Perform Schmidt orthogonal transformation on the processed remote sensing background image to generate the corresponding Schmidt orthogonal transformation result.

[0186] The simulated grayscale value is adjusted accordingly based on a preset mean-variance adjustment algorithm to generate an adjusted simulated grayscale value.

[0187] The adjusted analog grayscale value is used to replace the first component in the Schmidt orthogonalization transformation result, and the Schmidt orthogonalization inverse transformation is performed on the Schmidt orthogonalization transformation result to generate an inverse transformation result.

[0188] The first band in the inverse transformation result is removed to generate the corresponding fused image after the segmented image and the remote sensing background image are fused.

[0189] The segmented image and the remote sensing background image are size assimilated, and the size of the multispectral low-resolution segmented image can be assimilated according to the size of the high-resolution panchromatic band remote sensing background image, that is, the size of the segmented image becomes the same as the size of the remote sensing background image.

[0190] Assume that the multispectral low-resolution segmented image is X. According to the preset spectral response function, X is convolved with a certain weight matrix W, and the simulated grayscale value P of the full-color band is obtained by resampling:

[0191]

[0192] The Schmidt orthogonalization transformation function H is introduced to perform Schmidt orthogonalization transformation on the processed remote sensing background image, as follows:

[0193]

[0194] Among them, T represents the band number that needs to be transformed, Z represents the band of the remote sensing background image, μ T represents the mean value of band T, φ(H l ,Z T ) represents the quotient of the covariance of the band Z of the remote sensing background image and the band H of the Schmidt orthogonal transformation result and the variance of the band Z of the remote sensing background image and the band H of the Schmidt orthogonal transformation result.

[0195] The simulated grayscale value is adjusted accordingly based on the preset mean variance adjustment algorithm, and the adjusted simulated grayscale value is generated as follows:

[0196]

[0197] Among them, P is the grayscale value of the low-resolution panchromatic band image, k1 is the gain, and k2 is the offset. The specific calculation is as follows:

[0198]

[0199] k2=μ I -(k1×μ P )

[0200] Among them, I represents the brightness component of the segmented image, v represents the variance of the gray value P of the panchromatic band image, μ I and μ P are the mean values of the brightness component I and the grayscale value P of the panchromatic band image, respectively.

[0201] Use the adjusted low-resolution panchromatic band Replace the first component H1 after Schmidt orthogonalization, keep the other components after Schmidt orthogonalization of the high-resolution panchromatic band, and perform inverse transformation:

[0202]

[0203] Finally, the first band of the inverse transformation result z is removed to output the fused image Z'.

[0204] Step S208: Fusing the mask annotation and the fused image to generate remote sensing training samples.

[0205] In this embodiment, in the scenario of post-loan monitoring of engineering projects based on satellite remote sensing images, the problems of poor model training effect and slow iteration speed in remote sensing image recognition modeling are actually addressed. By training a large remote sensing image segmentation model and accessing a background generation model based on artificial intelligence technology, users can easily and quickly generate training image samples containing different complex backgrounds and different recognition elements, thereby greatly improving the modeling efficiency of the remote sensing image recognition training model and the accuracy of the prediction model.

[0206] Optionally, in this embodiment, the specific steps of training a preset remote sensing image recognition model based on remote sensing training samples to generate a corresponding target remote sensing image recognition model may be as follows:

[0207] The remote sensing training samples are divided into a training set and a validation set according to a preset ratio;

[0208] Iteratively train the preset remote sensing image recognition model using the training set to generate the preset remote sensing image recognition model corresponding to each iterative version;

[0209] Based on the validation set, the model accuracy of the preset remote sensing image recognition model corresponding to each iterative version is verified, and the average accuracy value corresponding to the preset remote sensing image recognition model of each iterative version is generated;

[0210] The preset remote sensing image recognition model corresponding to the largest average precision mean is determined as the target remote sensing image recognition model.

[0211] In this embodiment, the remote sensing training samples can be divided into a training set and a validation set at a preset ratio of 7:3 for the training of the preset remote sensing image recognition model. The preset remote sensing image recognition model can essentially be described using the framework of statistical inference, focusing on the probability of making type I errors and type II errors, which are usually described by accuracy and recall. Accuracy describes how accurate the model is, that is, how many of the results predicted as positive examples are true positive examples; recall describes how complete the model is, that is, how many of the true samples are predicted as positive examples by our model. Different tasks have different preferences for the two types of errors, and often strive to reduce one type of error when the error of the other type does not exceed a certain threshold. In detection, mAP (meanAverage Precision) is used as a unified indicator to take both types of errors into consideration.

[0212] Specifically, for each picture in the validation set, the preset remote sensing image recognition model outputs multiple prediction boxes (often far exceeding the number of real boxes). In this embodiment, IoU (Intersection Over Union) is used to mark whether the prediction box is correct. After marking, the recall rate will always increase with the increase of prediction boxes. The accuracy rate is averaged at different recall levels to obtain AP, and then all categories are averaged according to their proportion to obtain mAP. Through iterative training, mAP is evaluated on the validation set after each iterative version is set. After the training is completed, the preset remote sensing image recognition model corresponding to the iterative version with the largest mAP is exported as the final output model of the training.

[0213] Figure 4 This is a schematic diagram of the structure of the remote sensing image recognition model generation device provided in this application, such as Figure 4 As shown, in this embodiment, the remote sensing image recognition model generation device 300 can be set in an electronic device, such as a terminal device, and the remote sensing image recognition model generation device 300 includes:

[0214] The acquisition module 301 is used to acquire image description text and sample images corresponding to the remote sensing training samples to be generated.

[0215] The first generation module 302 is used to train the image description text input to a convergent semantic understanding model to generate corresponding recognition element information and background description information.

[0216] The second generation module 303 is configured to input the identification element information and the example image into a large remote sensing image segmentation model that has been trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image. The example image includes at least the identification element corresponding to the identification element information.

[0217] The third generation module 304 is configured to input the background description information into a background generation model trained to convergence, and generate a remote sensing background map.

[0218] The fusion module 305 is used to fuse the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample.

[0219] The fourth generating module 306 is configured to train a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

[0220] The remote sensing image recognition model generation device provided in this embodiment can be executed Figure 2 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 2 The method embodiments shown are similar and will not be described in detail here.

[0221] The remote sensing image recognition model generation device provided in this application is based on the remote sensing image recognition model generation device provided in the previous embodiment, and further refines the remote sensing image recognition model generation device. The remote sensing image recognition model generation device 300 includes:

[0222] Optionally, in this embodiment, the remote sensing image segmentation model includes: an encoder and a decoder.

[0223] The second generating module 303 is specifically used for:

[0224] The encoder extracts features and segments the sample image into blocks to generate the corresponding position representation vector and encoding information matrix. The decoder performs attention mechanism processing and classification prediction on the position representation vector and encoding information matrix based on the recognition element information, generating a segmented image corresponding to the image recognition element and the recognition element category corresponding to each pixel in the segmented image.

[0225] Optionally, in this embodiment, the encoder includes a swin-transformer network structure.

[0226] When the second generating module 303 uses the encoder to perform feature extraction and image block processing on the example image to generate the position representation vector and the encoding information matrix corresponding to the example image, it is specifically used to:

[0227] The encoder extracts features from the sample image and processes the image into blocks, generating image feature vectors corresponding to the image blocks and positional feature vectors corresponding to the relative positions between the blocks. Based on these image feature vectors and the relative positional vectors, a positional representation vector corresponding to the sample image is generated. This positional representation vector is then fed into the Swin-Transformer network to generate a coded information matrix corresponding to all image blocks.

[0228] Optionally, in this embodiment, the decoder includes an attention layer and a swin-transformer network structure.

[0229] The second generation module 303, when using the decoder to perform attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the recognition element information to generate a segmented image corresponding to the image recognition element and a recognition element category corresponding to each pixel in the segmented image, is specifically used to:

[0230] Based on the position representation vector, identification element information, and attention layer, the corresponding original attention map is generated. Based on the encoding information matrix, identification element information, and attention layer, the corresponding encoded attention map is generated. The encoded attention map is fitted based on the original attention map and input into the swin-transformer network structure to generate a segmented image of the same size as the image feature vector. The segmented image is classified and predicted to generate the identification element category corresponding to each pixel in the segmented image.

[0231] Optionally, in this embodiment, the fusion module 305 is specifically configured to:

[0232] Binarize the identified feature categories corresponding to each pixel in the segmented image to generate a mask annotation corresponding to each pixel. Fuse the segmented image with the remote sensing background image to generate the corresponding fused image. Fuse the mask annotation and fused image to generate remote sensing training samples.

[0233] Optionally, in this embodiment, when the fusion module 305 fuses the segmented image and the remote sensing background image to generate a corresponding fused image, it is specifically used to:

[0234] The segmented image and the remote sensing background image are size-assimilated so that the processed segmented image and the remote sensing background image have the same size. The processed segmented image is convolutionally resampled based on a preset spectral response function to generate simulated grayscale values for the panchromatic band. The processed remote sensing background image is Schmidt orthogonalized to generate the corresponding Schmidt orthogonalized transformation result. The simulated grayscale values are adjusted accordingly based on a preset mean-variance adjustment algorithm to generate adjusted simulated grayscale values. The adjusted simulated grayscale value replaces the first component of the Schmidt orthogonalized transformation result, and the Schmidt orthogonalized transformation result is inversely transformed to generate an inverse transformation result. The first band in the inverse transformation result is removed to generate the corresponding fused image after the segmented image and the remote sensing background image are fused.

[0235] Optionally, in this embodiment, the background generation model is an artificial intelligence image generation model.

[0236] The third generating module 304 is specifically used for:

[0237] An artificial intelligence image generation model is used to determine image keywords from background description information. An artificial intelligence image generation model is used to generate corresponding remote sensing background images based on image keywords.

[0238] Optionally, in this embodiment, the remote sensing image recognition model generation device 300 further includes:

[0239] A training module is used to obtain training samples, which include: remote sensing training images and annotation information corresponding to the remote sensing training images. The training samples are input into a preset remote sensing image segmentation large model, and the preset remote sensing image segmentation large model is trained based on a preset random masking module. The preset random masking module is used to randomly mask the pixels of the image blocks when the preset remote sensing image segmentation large model performs image segmentation processing. According to the segmented image output by the preset remote sensing image segmentation large model and the identification element category corresponding to each pixel in the segmented image, it is determined whether the preset remote sensing image segmentation large model meets the preset convergence condition. If the preset remote sensing image segmentation large model meets the convergence condition, the preset remote sensing image segmentation large model that meets the convergence condition is determined as the remote sensing image segmentation large model trained to convergence.

[0240] Optionally, in this embodiment, the fourth generating module 306 is specifically configured to:

[0241] Remote sensing training samples are divided into a training set and a validation set according to a preset ratio. The preset remote sensing image recognition model is iteratively trained using the training set to generate the preset remote sensing image recognition model corresponding to each iteration. The model accuracy of each iteration of the preset remote sensing image recognition model is verified using the validation set to generate the mean average precision for each iteration of the preset remote sensing image recognition model. The preset remote sensing image recognition model with the largest mean average precision is determined as the target remote sensing image recognition model.

[0242] The remote sensing image recognition model generation device provided in this embodiment can be executed Figure 2-Figure 3 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 2-Figure 3 The method embodiments shown are similar and will not be described in detail here.

[0243] According to an embodiment of the present application, the present application also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0244] like Figure 5 As shown, Figure 5 is a schematic diagram of the structure of the electronic device provided by the present application. The electronic device is intended to be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application as described and / or claimed herein.

[0245] like Figure 5As shown, the electronic device includes: a processor 401 and a memory 402. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device.

[0246] Memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to perform the remote sensing image recognition model generation method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the remote sensing image recognition model generation method provided in this application.

[0247] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the remote sensing image recognition model generation method in the embodiment of the present application (for example, the attached Figure 4 The processor 401 executes the non-transient software programs, instructions, and modules stored in the memory 402 to execute various functional applications and data processing of the electronic device, thereby implementing the remote sensing image recognition model generation method in the above method embodiment.

[0248] At the same time, this embodiment also provides a computer product. When the instructions in the computer product are executed by the processor of the electronic device, the electronic device can execute the remote sensing image recognition model generation method of the above embodiment.

[0249] Those skilled in the art will readily conceive of other implementations of the embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the embodiments of the present application, which follow the general principles of the embodiments of the present application and include common knowledge or customary technical means in the art not disclosed in the embodiments of the present application.

[0250] It should be understood that the embodiments of the present application are not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present application is limited only by the appended claims.

Claims

1. A remote sensing image recognition model generation method, characterized in that: include: Obtain image description text and sample images corresponding to the remote sensing training samples to be generated; Inputting the image description text into a semantic understanding model trained to convergence, generating corresponding recognition element information and background description information; Input the identification element information and the sample image into a large remote sensing image segmentation model trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; The example image at least includes an identification element corresponding to the identification element information; Inputting the background description information into a background generation model trained to convergence to generate a remote sensing background map; fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample; A preset remote sensing image recognition model is trained based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

2. The method according to claim 1, characterized in that The remote sensing image segmentation large model includes: an encoder and a decoder; The step of inputting the identification element information and the sample image into a large remote sensing image segmentation model trained to convergence, and generating a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image, comprises: Using an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image; A decoder is used to perform attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image.

3. The method according to claim 2, characterized in that The encoder includes a swin-transformer network structure; The step of using an encoder to perform feature extraction and image block processing on the example image to generate a position representation vector and a coding information matrix corresponding to the example image includes: Using an encoder to perform feature extraction and image block processing on the sample image to generate image feature vectors corresponding to the image blocks and position feature vectors corresponding to relative positions between the image blocks; Generating a position representation vector corresponding to the example image according to the image feature vector corresponding to the image block and the relative position vector between the image blocks; The position representation vector is input into the swin-transformer network structure to generate the encoding information matrix corresponding to all image blocks.

4. The method according to claim 3, characterized in that The decoder includes an attention layer and a swin-transformer network structure; The decoder performs attention mechanism processing and classification prediction processing on the position representation vector and the encoding information matrix based on the identification element information to generate a segmented image corresponding to the image identification element and an identification element category corresponding to each pixel in the segmented image, including: Generate a corresponding original attention map according to the position representation vector, the identification element information and the attention layer; Generate a corresponding encoding attention map according to the encoding information matrix, the identification element information and the attention layer; Fitting the encoded attention map based on the original attention map, and inputting the fitted encoded attention map into the swin-transformer network structure to generate a segmented image of the same size as the image feature vector; The segmented image is processed for classification prediction to generate the identification element category corresponding to each pixel in the segmented image.

5. The method according to claim 4, characterized in that The step of fusing the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample includes: Binarize the identification element category corresponding to each pixel in the segmented image to generate a mask label corresponding to each pixel; fusing the segmented image and the remote sensing background image to generate a corresponding fused image; The mask annotation and the fused image are fused to generate a remote sensing training sample.

6. The method according to claim 5, characterized in that The fusing of the segmented image and the remote sensing background image to generate a corresponding fused image includes: Performing size assimilation processing on the segmented image and the remote sensing background image so that the sizes of the processed segmented image and the remote sensing background image are the same; The processed segmented image is convolved and resampled based on the preset spectral response function to generate simulated grayscale values of the full-color band; Performing Schmidt orthogonal transformation on the processed remote sensing background image to generate the corresponding Schmidt orthogonal transformation result; Adjust the simulated grayscale value accordingly based on a preset mean-variance adjustment algorithm to generate an adjusted simulated grayscale value; The adjusted analog grayscale value is used to replace the first component in the Schmidt orthogonalization transformation result, and the Schmidt orthogonalization inverse transformation is performed on the Schmidt orthogonalization transformation result to generate an inverse transformation result; The first band in the inverse transformation result is removed to generate a fused image corresponding to the fusion of the segmented image and the remote sensing background image.

7. The method according to claim 6, characterized in that The background generation model is an artificial intelligence image generation model; The step of inputting the background description information into a background generation model trained to convergence to generate a remote sensing background map includes: Determining image keywords from the background description information using an artificial intelligence image generation model; An artificial intelligence image generation model is used to generate a corresponding remote sensing background image based on the image keywords.

8. The method according to any one of claims 1 to 7, characterized in that Before inputting the identification element information and the sample image into a large remote sensing image segmentation model trained to convergence and generating a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image, the method further includes: Acquire a training sample, wherein the training sample includes: a remote sensing training image and annotation information corresponding to the remote sensing training image; Inputting the training samples into a preset remote sensing image segmentation large model, and training the preset remote sensing image segmentation large model based on a preset random masking module; the preset random masking module is used to randomly mask pixels of image blocks when the preset remote sensing image segmentation large model performs image segmentation processing; Determining whether the preset remote sensing image segmentation model satisfies a preset convergence condition based on a segmented image output by the preset remote sensing image segmentation model and a recognition element category corresponding to each pixel in the segmented image; If the preset remote sensing image segmentation large model meets the convergence condition, the preset remote sensing image segmentation large model that meets the convergence condition is determined as the remote sensing image segmentation large model trained to convergence.

9. The method according to claim 6, characterized in that The step of training a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model includes: Dividing the remote sensing training samples into a training set and a validation set according to a preset ratio; Iteratively train the preset remote sensing image recognition model using the training set to generate the preset remote sensing image recognition model corresponding to each iterative version; Based on the validation set, the model accuracy of the preset remote sensing image recognition model corresponding to each iterative version is verified, and the average accuracy value corresponding to the preset remote sensing image recognition model of each iterative version is generated; The preset remote sensing image recognition model corresponding to the largest average precision mean is determined as the target remote sensing image recognition model.

10. A remote sensing image recognition model generation device, characterized in that: include: An acquisition module is used to obtain image description text and sample images corresponding to the remote sensing training samples to be generated; A first generation module is used to train the image description text input to a convergent semantic understanding model to generate corresponding recognition element information and background description information; A second generation module is configured to input the identification element information and the sample image into a large remote sensing image segmentation model that has been trained to convergence, and generate a segmented image corresponding to the identification element information and an identification element category corresponding to each pixel in the segmented image; The example image at least includes an identification element corresponding to the identification element information; A third generation module is configured to input the background description information into a background generation model trained to convergence to generate a remote sensing background map; A fusion module is used to fuse the segmented image, the identification element category corresponding to each pixel in the segmented image, and the remote sensing background image to generate a remote sensing training sample; The fourth generation module is used to train a preset remote sensing image recognition model based on the remote sensing training samples to generate a corresponding target remote sensing image recognition model.

11. An electronic device, characterized in that: include: memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the remote sensing image recognition model generation method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the remote sensing image recognition model generation method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the remote sensing image recognition model generation method according to any one of claims 1 to 9 is implemented.

Images