Model training method and device, and remote sensing image intelligent segmentation method and device based on large model

Abstract

The application provides a model training method, a large model-based remote sensing image intelligent segmentation method and device, comprising: inputting a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold; based on the first segmentation mask image, determining a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image; based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region, determining a first model loss value of the first image segmentation model; based on the first model loss value, adjusting a first model parameter of the first image segmentation model to obtain a target image segmentation model; when the target image segmentation model is used for remote sensing image segmentation, the segmentation accuracy of the remote sensing image can be improved to a certain extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and in particular to a model training method, a remote sensing image intelligent segmentation method and device based on a large model. BACKGROUND

[0002] In satellite remote sensing images, different objects on the ground surface can be included, such as buildings, roads, vegetation, water bodies, etc. Remote sensing images cover a wide range and can provide ground surface conditions in a large area, which is crucial for environmental monitoring, urban planning, disaster response, etc. With the development of remote sensing technology, the amount of data obtained from remote sensing images is also increasing, and manual interpretation becomes impractical. Therefore, it is particularly important to develop effective automatic interpretation algorithms.

[0003] In related technologies, some deep learning networks can be used to automatically perform image segmentation tasks, such as Unet, SegNet, PSPNet, Deeplab series, etc. The accuracy of these networks in natural scene image segmentation tasks is getting higher and higher.

[0004] However, since remote sensing images have many image features, the above methods do not have high segmentation accuracy for remote sensing images when processing the segmentation task of remote sensing images. SUMMARY

[0005] In view of the above problems, the embodiments of the present application provide a model training method, a remote sensing image intelligent segmentation method and device based on a large model, an electronic device and a readable storage medium, in order to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, the embodiments of the present application provide a model training method, which comprises:

[0007] inputting a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold value;

[0008] determining the segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image based on the first segmentation mask image;

[0009] determining a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region;

[0010] adjusting the first model parameter of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to a target remote sensing image.

[0011] Optionally, the first image segmentation model comprises a first image feature extraction sub-model, and the method further comprises:

[0012] performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image;

[0013] inputting the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model;

[0014] determining a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0015] adjusting second model parameters of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model.

[0016] Optionally, the determining of the second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image comprises:

[0017] determining a mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0018] determining a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0019] determining the second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value.

[0020] Optionally, the determining of the mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image comprises:

[0021] inputting the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively;

[0022] determining a first relative position of a mask region in the first mask feature map and a second relative position same as the first relative position in the sample feature map;

[0023] determining a first pixel value of the mask region and a number of pixels contained in the mask region, and determining a second pixel value of a region in the sample feature map at the second relative position;

[0024] Determine a mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value, and the pixel quantity.

[0025] Optionally, determining the first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region comprises:

[0026] Determining a mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region.

[0027] Determining a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region.

[0028] Determining the first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0029] Optionally, determining the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region comprises:

[0030] Determining a binary cross-entropy loss value and a Dice loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region, respectively.

[0031] Determining the mask prediction loss value of the first image segmentation model based on the binary cross-entropy loss value and the Dice loss value.

[0032] In a second aspect, an embodiment of the present application provides a method for intelligent segmentation of remote sensing images based on a large model, which comprises:

[0033] Obtaining a target remote sensing image.

[0034] Inputting the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model, wherein the target image segmentation model is obtained based on the model training method according to any one of claims 1 to 6.

[0035] Optionally, the method further comprises:

[0036] Inputting the target remote sensing image into a target uncertainty estimation model to obtain a confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model.

[0037] generate, based on the confidence of each of the divided regions, an uncertainty estimation heat map corresponding to the first remote sensing divided image.

[0038] In a third aspect, an embodiment of the present application provides a model training device, the device comprising:

[0039] a first input and output module configured to input a sample remote sensing image in a sample remote sensing image dataset into a first image segmentation model to obtain a first divided mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold value;

[0040] a first determination module configured to determine, based on the first divided mask image, a divided prediction type corresponding to each of the preset divided regions in the sample remote sensing image respectively;

[0041] a second determination module configured to determine, based on the divided prediction type and a sample classification label corresponding to each of the preset divided regions respectively, a first model loss value of the first image segmentation model;

[0042] a first adjustment module configured to adjust, based on the first model loss value, a first model parameter of the first image segmentation model to obtain a target image segmentation model; wherein the target image segmentation model is configured to generate a target divided mask image corresponding to a target remote sensing image.

[0043] Optionally, the first image segmentation model comprises a first image feature extraction sub-model, and the device further comprises:

[0044] a mask module configured to perform a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image;

[0045] a second input and output module configured to input the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model;

[0046] a third determination module configured to determine, based on the first remote sensing mask image and the sample remote sensing image, a second model loss value of the second image feature extraction sub-model;

[0047] a second adjustment module configured to adjust, based on the second model loss value, a second model parameter of the second image feature extraction sub-model to obtain the first image feature extraction sub-model.

[0048] Optionally, the third determination module comprises:

[0049] a first determination sub-module configured to determine, based on the first remote sensing mask image and the sample remote sensing image, a mask image reconstruction loss value of the second image feature extraction sub-model.

[0050] a second determining sub-module, configured to determine a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0051] a third determining sub-module, configured to determine a second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value.

[0052] Optionally, the first determining sub-module comprises:

[0053] an input and output unit, configured to input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively;

[0054] a first determining unit, configured to determine a first relative position of a mask region in the first mask feature map and determine a second relative position same as the first relative position in the sample feature map;

[0055] a second determining unit, configured to determine a first pixel value of the mask region and a number of pixels contained in the mask region, and determine a second pixel value of a region where the second relative position is located in the sample feature map;

[0056] a third determining unit, configured to determine a mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

[0057] Optionally, the second determining module comprises:

[0058] a fourth determining sub-module, configured to determine a mask prediction loss value of the first image segmentation model based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region respectively;

[0059] a fifth determining sub-module, configured to determine a classification loss value of the first image segmentation model based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region respectively;

[0060] a sixth determining sub-module, configured to determine a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0061] Optionally, the fourth determining sub-module comprises:

[0062] The fourth determining unit is configured to determine a binary cross entropy loss value and a Dice loss value of the first image segmentation model respectively based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region.

[0063] The fifth determining unit is configured to determine a mask prediction loss value of the first image segmentation model based on the binary cross entropy loss value and the Dice loss value.

[0064] In a fourth aspect, an embodiment of the present application provides a device for intelligent segmentation of a remote sensing image based on a large model, and the device comprises:

[0065] The acquisition module is configured to acquire a target remote sensing image.

[0066] The first input-output module is configured to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model, wherein the target image segmentation model is obtained based on the model training method according to any one of the preceding embodiments.

[0067] Optionally, the device further comprises:

[0068] The second input-output module is configured to input the target remote sensing image into a target uncertainty estimation model to obtain a confidence level of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model.

[0069] The generation module is configured to generate an uncertainty estimation heat map corresponding to the first remote sensing segmentation image based on the confidence level of each segmentation region.

[0070] In a fifth aspect, an embodiment of the present application provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the model training method according to any one of the preceding embodiments or the method for intelligent segmentation of a remote sensing image based on a large model.

[0071] In a sixth aspect, an embodiment of the present application provides a readable storage medium, which stores a program or instructions, and the program or instructions are executed by a processor to implement the model segmentation method according to any one of the preceding embodiments or the method for intelligent segmentation of a remote sensing image based on a large model.

[0072] Specific advantages are as follows:

[0073] The embodiment of the application inputs a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold, based on the first segmentation mask image, a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image is determined, based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region, a first model loss value of the first image segmentation model is determined, based on the first model loss value, a first model parameter of the first image segmentation model is adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to a target remote sensing image, the first image segmentation model can be trained by using the sample remote sensing image data set with the average information entropy greater than or equal to the first threshold to obtain the target image segmentation model, the model can learn the visual diversity of the remote sensing image, and the segmentation accuracy of the remote sensing image can be improved to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed to be used in the description of the embodiments of the application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0075] Figure 1 is a flowchart of a model training method provided by the embodiments of the application;

[0076] Figure 2 is a flowchart of another model training method provided by the embodiments of the application;

[0077] Figure 3 is a flowchart of a remote sensing image intelligent segmentation method based on a large model provided by the embodiments of the application;

[0078] Figure 4 is a flowchart of a heat map generation method provided by the embodiments of the application;

[0079] Figure 5 is a logic block diagram of a model training device provided by the embodiments of the application;

[0080] Figure 6 is a logic block diagram of a remote sensing image intelligent segmentation device based on a large model provided by the embodiments of the application;

[0081] Figure 7 is a schematic diagram of an electronic device provided by the embodiments of the application. DETAILED DESCRIPTION

[0082] The exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings so as to be more thoroughly understood. Although exemplary embodiments of the present application are shown in the drawings, it is to be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application will be thoroughly and completely understood, and will fully convey the scope of the present application to those skilled in the art.

[0083] Referring to Figure 1 , Figure 1 A flowchart of a model training method provided by an embodiment of the present application is shown in FIG. 1. The method comprises the following steps.

[0084] In step 101, a sample remote sensing image in a sample remote sensing image dataset is input into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold.

[0085] In an embodiment of the present application, the data source of the sample remote sensing image dataset can be some commonly used datasets, such as Million-AID, NAIP, RSD46-WHU, MLRSNet, RESISC45, PatternNet, etc. The information entropy of each remote sensing image data contained in these datasets can be calculated, and remote sensing image data is selected from the above-mentioned datasets according to the information entropy to constitute a new dataset, and the average information entropy of the sample remote sensing image contained in the new dataset is greater than or equal to the first threshold, i.e. the sample remote sensing image dataset. After obtaining the sample remote sensing image dataset, the sample remote sensing image in the dataset can be input into the first image segmentation model, so that the first segmentation mask image output by the first image segmentation model can be obtained. The first image segmentation model can be a commonly used image segmentation model such as U-net or Mask2Former; the first segmentation mask image can be a binary image, and in the first segmentation mask image, a black line of a preset width can be used to represent the segmentation edge, and the image area between the lines can be white.

[0086] In step 102, based on the first segmentation mask image, the respective segmentation prediction types of each preset segmentation region in the sample remote sensing image are determined.

[0087] In an embodiment of the present application, the preset segmentation region can be determined according to the annotation information corresponding to the sample remote sensing image, i.e. the correct segmentation result corresponding to the sample remote sensing image. The respective segmentation prediction types of each preset segmentation region in the sample remote sensing image can be determined according to the first segmentation mask image. The segmentation prediction types can be pre-set, such as "bird", "vehicle", "building", "background", etc.

[0088] In step 103, a first model loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region.

[0089] In an embodiment of the present application, the segmentation region of the first segmentation mask image can not completely coincide with the preset segmentation region in the sample remote sensing image. In this way, in the non-coincidence region, the segmentation prediction type can deviate from the sample classification label corresponding to each preset segmentation region. The first model loss value of the first image segmentation model can be determined according to the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The first model loss value can be a cross-entropy loss value or a contrast loss value.

[0090] In step 104, the first model parameter of the first image segmentation model is adjusted based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to a target remote sensing image.

[0091] In an embodiment of the present application, the first model parameter of the first image segmentation model can be adjusted according to the first model loss value, so that the target segmentation model can be obtained. The adjustment of the first model parameter can be repeatedly performed multiple times, and after each adjustment, the process of steps 101 to 104 can be repeatedly performed to adjust the first model parameter again. In the case that the number of training times of the first image segmentation model reaches a preset number of times, or the first model loss value meets a convergence condition, the adjustment of the first model parameter of the first image segmentation model can be stopped, and the first image segmentation model obtained after the last adjustment is determined as the target image segmentation model.

[0092] In an embodiment of the present application, by inputting the sample remote sensing image in the sample remote sensing image data set into the first image segmentation model, the first segmentation mask image output by the first image segmentation model is obtained; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold value, based on the first segmentation mask image, the segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image is determined, based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region, the first model loss value of the first image segmentation model is determined, based on the first model loss value, the first model parameter of the first image segmentation model is adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to a target remote sensing image, the first image segmentation model can be trained by the sample remote sensing image data set with an average information entropy greater than or equal to the first threshold value to obtain the target image segmentation model, which can enable the model to learn the visual diversity of remote sensing images and improve the segmentation accuracy of remote sensing images to a certain extent.

[0093] Referring toFigure 2 , Figure 2 Another model training method provided by the embodiment of the present application is shown in the flowchart, which can include the following steps:

[0094] In step 201, a random mask operation is performed on the sample remote sensing image to obtain a sample remote sensing mask image.

[0095] In the embodiment of the present application, a random mask operation can be performed on the sample remote sensing image, that is, a random mask is added to the sample remote sensing image, so that a sample remote sensing mask image can be obtained. Compared with the sample remote sensing image, the sample remote sensing mask image can miss part of the image features, and the missing area can constitute a mask area.

[0096] In step 202, the sample remote sensing mask image is input into the second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model.

[0097] In the embodiment of the present application, the second image feature extraction sub-model can be a SwinV2-G network model. Before training, the model can randomly generate the values of the model parameters and the weights of the model contained in the model, so as to initialize the SwinV2-G model. The sample remote sensing mask image can be input into the second image feature extraction sub-model, so that a first remote sensing mask image output by the second image feature extraction sub-model can be obtained. The first remote sensing mask image is the image backbone of the sample remote sensing image, and can include one or more image features of the sample remote sensing image.

[0098] In step 203, a second model loss value of the second image feature extraction sub-model is determined based on the first remote sensing mask image and the sample remote sensing image.

[0099] In the embodiment of the present application, the second model loss value of the second image feature extraction sub-model can be determined according to the first remote sensing mask image and the sample remote sensing image. The second model loss value can be a binary cross-entropy loss value, a contrast loss value, or a combination of the above two loss values.

[0100] Optionally, in step 203, the following sub-steps can be included:

[0101] In sub-step 2031, a mask image reconstruction loss value of the second image feature extraction sub-model is determined based on the first remote sensing mask image and the sample remote sensing image.

[0102] In the embodiments of the present application, the mask image reconstruction loss value is an index used to measure the ability of a model to reconstruct the masked part of an image in a self-supervised learning model such as a mask autoencoder (MAE). The size of the mask image reconstruction loss value can reflect the accuracy of the model in reconstructing the masked part of the image. A lower loss value generally means that the model has better image reconstruction capability. The location of the mask reconstruction region in the first remote sensing mask image can be determined, and the location corresponding to the location of the mask region in the sample remote sensing image can be determined. Then, according to the reconstructed pixel value and the original pixel value of the above-mentioned location, the mask image reconstruction loss value of the second image feature extraction sub-model can be calculated.

[0103] Optionally, in substep 2031, the following substeps can be included:

[0104] Substep A1, input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively.

[0105] In the embodiments of the present application, the feature pyramid network (Feature Pyramid Networks, FPN) is a network structure widely used in deep learning target detection, mainly used to build multi-scale feature representation to effectively detect targets of different sizes. FPN can simultaneously retain rich semantic information and detailed information on feature maps of different scales. These fused feature maps can be directly used for target detection tasks such as bounding box regression and classification, thereby improving the detection performance of the model on multi-scale targets. The first remote sensing mask image and the sample remote sensing image can be input into the feature pyramid network respectively, so that the first mask feature map and the sample feature map output by the feature pyramid network respectively can be obtained. The first mask feature map can correspond to the first remote sensing mask image, and the sample feature map can correspond to the sample remote sensing image.

[0106] Substep A2, determine a first relative position of the mask region in the first mask feature map, and determine a second relative position in the sample feature map that is the same as the first relative position.

[0107] In the embodiments of the present application, after obtaining the first mask feature map and the sample feature map, the first relative position of the mask region in the first mask feature map and the second relative position in the sample feature map that is the same as the first relative position can be further determined.

[0108] Substep A3, determine the first pixel value of the mask region and the number of pixels contained in the mask region, and determine the second pixel value of the region where the second relative position is located in the sample feature map.

[0109] In the embodiments of this application, the first pixel value and the number of pixels of the mask region can be determined, and the second pixel value of the region where the second relative position is located in the sample feature map can be determined.

[0110] Sub-step A4: Based on the first pixel value, the second pixel value, and the number of pixels, determine the mask image reconstruction loss value of the second image feature extraction sub-model.

[0111] In the embodiments of this application, the mask image reconstruction loss value of the second image feature extraction sub-model can be calculated based on the first pixel value, the second pixel value, and the number of pixels. The calculation method is shown in Equation 1 below:

[0112] (Equation 1)

[0113] In Equation 1, This represents the loss value for masked image reconstruction. This indicates the number of the multi-scale feature map after fusion through FPN. Indicates the area that is covered The original pixel value, i.e., the second pixel value. This refers to the reconstructed pixel values ​​generated in these areas, i.e., the first pixel values. This represents the total number of pixels that are masked, i.e., the number of pixels in the masked area.

[0114] In the embodiments of this application, by inputting the first remote sensing mask image and the sample remote sensing image into the feature pyramid network respectively, the first mask feature map and the sample feature map output by the feature pyramid network are obtained respectively. The first relative position of the mask region in the first mask feature map is determined, and the second relative position in the sample feature map that is the same as the first relative position is determined. The first pixel value of the mask region and the number of pixels contained in the mask region are determined, and the second pixel value of the region where the second relative position is located in the sample feature map is determined. Based on the first pixel value, the second pixel value, and the number of pixels, the mask image reconstruction loss value of the second image feature extraction sub-model is determined. The feature pyramid network can be used to participate in the calculation of the mask image reconstruction loss value of the second image feature extraction sub-model, which can improve the accuracy of the mask image reconstruction loss value to a certain extent and improve the prediction accuracy of the final obtained first image feature extraction sub-model.

[0115] Sub-step 2032: Based on the first remote sensing mask image and the sample remote sensing image, determine the feature matching loss value of the second image feature extraction sub-model.

[0116] In the embodiments of the present application, the feature matching loss (FML) is a loss function used to measure the accuracy of feature restoration, mainly used in the single-image reflection separation task. The feature matching loss evaluates the performance of the model by comparing the accuracy of the features of the information of the restored non-reflection image. The feature matching loss value of the second image feature extraction sub-model can be calculated according to the first remote sensing mask image and the sample remote sensing image, and the specific calculation method is shown in the following formula 2:

[0117] (Formula 2)

[0118] In formula 2, represents the feature matching loss value, represents the image features of the first remote sensing mask image, represents the image features of the sample remote sensing image, the symbol “ ” represents the modulo operation, and the symbol “ ” represents the L2 norm, represents a preset linear mapping function.

[0119] In step 2033, the second model loss value of the second image feature extraction sub-model is determined based on the mask image reconstruction loss value and the feature matching loss value.

[0120] In the embodiments of the present application, the second model loss value of the second image feature extraction sub-model can be calculated according to the mask image reconstruction loss value and the feature matching loss value. The mask image reconstruction loss value and the feature matching loss value can be directly added to obtain the second model loss value. A balance coefficient can also be used to balance the proportion of the mask image reconstruction loss value and the feature matching loss value in the second model loss value, as shown in the following formula 3:

[0121] (Formula 3)

[0122] In formula 3, represents the second model loss value, represents the mask image reconstruction loss value, represents the feature matching loss value, represents a balance coefficient.

[0123] In the embodiments of the present application, by determining the mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image, determining the feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image, and determining the second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value, the mask image reconstruction loss value and the feature matching loss value can be calculated, and the second model loss value can be calculated based on the mask image reconstruction loss value and the feature matching loss value, which can improve the accuracy of the second model loss value to a certain extent and improve the prediction accuracy of the finally obtained first image feature extraction sub-model.

[0124] In step 204, the second model parameters of the second image feature extraction sub-model are adjusted based on the second model loss value to obtain the first image feature extraction sub-model.

[0125] In the embodiments of the present application, the second model parameters of the second image feature extraction sub-model can be adjusted according to the second model loss value, so that the first image feature extraction sub-model can be obtained. The adjustment of the second model parameters can be repeatedly performed multiple times, and after each adjustment, the process of steps 201 to 204 can be repeatedly performed to adjust the second model parameters again. In the case that the number of training times of the second image feature extraction sub-model reaches a preset number of times, or the second model loss value meets the convergence condition, the adjustment of the second model parameters of the second image feature extraction sub-model can be stopped, and the second image feature extraction sub-model obtained after the last adjustment is determined as the first image feature extraction sub-model.

[0126] In the embodiments of the present application, by performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image, inputting the sample remote sensing mask image into the second image feature extraction sub-model to obtain the first remote sensing mask image output by the second image feature extraction sub-model, determining the second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image, and adjusting the second model parameters of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model, noise can be added to the sample remote sensing image in a random mask manner, and the second image feature extraction sub-model can be trained based on the sample remote sensing mask image with added noise to obtain the first image feature extraction sub-model, which can improve the robustness and prediction accuracy of the first image feature extraction sub-model to a certain extent.

[0127] In step 205, a sample remote sensing image in a sample remote sensing image dataset is input into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; the first image segmentation model comprises a first image feature extraction sub-model; and an average information entropy of the sample remote sensing image is greater than or equal to a first threshold.

[0128] In an embodiment of the present application, the first image segmentation model can comprise a first image feature extraction sub-model, which can be a backbone network in the first image segmentation model. The first image feature extraction sub-model can be used to perform first-stage processing on the sample remote sensing image to extract image backbone features in the sample remote sensing image. The first image segmentation model can further comprise a pixel decoder and a Transformer decoder, wherein the pixel decoder can be used to up-sample low-resolution features from the image backbone features to generate high-resolution pixel-by-pixel feature representations, and the Transformer decoder can process the pixel-by-pixel embedding representations to generate the first segmentation mask image. Other implementation contents of this step can refer to the embodiment contents of step 101, which will not be repeated here.

[0129] In step 206, based on the first segmentation mask image, a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image is determined.

[0130] In an embodiment of the present application, the implementation contents of this step can refer to the embodiment contents of step 102, which will not be repeated here.

[0131] In step 207, based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region, a first model loss value of the first image segmentation model is determined.

[0132] In an embodiment of the present application, the implementation contents of this step can refer to the embodiment contents of step 103, which will not be repeated here.

[0133] Optionally, in step 207, the following sub-step can be included:

[0134] In sub-step 2071, based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region, a mask prediction loss value of the first image segmentation model is determined.

[0135] In an embodiment of the present step, the mask prediction loss value can be used to measure the similarity between the mask generated by the model and the target mask. The mask is the edge texture information in the image. The mask prediction loss value of the first image segmentation model can be determined according to the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The value range of the mask prediction loss value can be a value less than or equal to 0. The higher the matching degree of the segmentation prediction type and the sample classification label, the larger the mask prediction loss value. When the mask prediction loss value is 0, it means that the segmentation prediction type and the sample classification label are completely matched, that is, the segmentation prediction types of each preset segmentation region are completely the same as the sample classification label.

[0136] Optionally, in sub-step 2071, the following sub-steps can be included:

[0137] Sub-step B1, based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region, respectively determining the binary cross-entropy loss value and the Dice loss value of the first image segmentation model.

[0138] In an embodiment of the present application, the mask prediction loss value can include a binary cross-entropy loss value and a Dice loss value. The binary cross-entropy loss value can be used to measure the difference between the predicted probability of each pixel and the true value, and the Dice loss value can be used to measure the similarity of two masks. The binary cross-entropy loss value and the Dice loss value of the first image segmentation model can be calculated according to the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The calculation method of the binary cross-entropy loss value is shown in the following formula 4:

[0139] (Formula 4)

[0140] In formula 4, represents the binary cross-entropy loss value, represents the number of pixels in the mask region, represents the pixel number, represents the probability that the segmentation prediction type of pixel x is the i-th class, which can be obtained by calculating the similarity between the segmentation prediction type and the sample classification label, represents that the segmentation prediction type is not the i-th class (i.e., the segmentation prediction type does not correspond to the i-th sample classification label), 1 represents that the segmentation prediction type is the i-th class (i.e., the segmentation prediction type corresponds to the i-th sample classification label). log is the logarithm operator symbol.

[0141] The calculation method of the Dice loss value is shown in the following formula 5:

[0142] (Formula 5)

[0143] In formula 5, denotes the Dice loss value, denotes the number of pixels in the mask area, denotes the pixel number, denotes the probability that the segmentation prediction type of pixel x is the i-th class, which can be obtained by calculating the similarity between the segmentation prediction type and the sample classification label, denotes that the segmentation prediction type is not the i-th class (i.e., the segmentation prediction type does not correspond to the i-th sample classification label), 1 indicates that the segmentation prediction type is the i-th class (i.e., the segmentation prediction type corresponds to the i-th sample classification label).

[0144] Sub-step B2, determining a mask prediction loss value of the first image segmentation model based on the binary cross-entropy loss value and the Dice loss value.

[0145] In the embodiments of the present application, the mask prediction loss value of the first image segmentation model can be calculated according to the binary cross-entropy loss value and the Dice loss value. The binary cross-entropy loss value and the Dice loss value can be directly added to obtain the mask prediction loss value. A balance coefficient can also be added to balance the proportion of the binary cross-entropy loss value and the Dice loss value in the mask prediction loss value, as shown in the following formula 6:

[0146] (Formula 6)

[0147] In formula 6, denotes the mask prediction loss value, denotes the Dice loss value, denotes the binary cross-entropy loss value, is a balance coefficient, which can be set according to requirements.

[0148] In the embodiments of the present application, the binary cross-entropy loss value and the Dice loss value of the first image segmentation model are determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation area, respectively, the mask prediction loss value of the first image segmentation model is determined based on the binary cross-entropy loss value and the Dice loss value, the mask prediction loss value can be composed of the binary cross-entropy loss value and the Dice loss value, which can improve the accuracy of the mask prediction loss value to a certain extent.

[0149] Sub-step 2072, determining a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation area.

[0150] In the embodiments of the present application, the classification loss value can be used to measure the similarity between the predicted class label of each generated mask and the sample class label. For each first segmentation mask image, the first image segmentation model can further output a predicted probability distribution As a segmentation prediction type, where C represents the number of categories. represents the predicted probability of the i-th first segmentation mask image including category j. The type of the classification loss value is cross-entropy loss, and its calculation method is shown in the following formula 7:

[0151] (Formula 7)

[0152] In formula 7, represents the classification loss value, represents the c-th value in the classification one-hot vector (i.e. the sample class label) of the first segmentation mask image, and In the classification one-hot vector where the c-th value is located, only one value is 1 and the rest are 0. represents the predicted probability of the i-th first segmentation mask image including category c.

[0153] Sub-step 2073, determining a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0154] In the embodiments of the present application, the first model loss value of the first image segmentation model can be calculated according to the mask prediction loss value and the classification loss value. The mask prediction loss value and the classification loss value can be directly added to obtain the first model loss value. A balance coefficient can also be added to adjust the proportion of the mask prediction loss value and the classification loss value in the first model loss value, as shown in the following formula 8:

[0155] (Formula 8)

[0156] In the above formula 8, is the first model loss value, is the mask prediction loss value, is the classification loss value, is the balance coefficient.

[0157] In the embodiments of the present application, the mask prediction loss value of the first image segmentation model is determined based on the sample classification label corresponding to each preset segmentation area of the segmentation prediction type, the classification loss value of the first image segmentation model is determined based on the sample classification label corresponding to each preset segmentation area of the segmentation prediction type, and the first model loss value of the first image segmentation model is determined based on the mask prediction loss value and the classification loss value. The first model loss value can be composed of the mask prediction loss value and the classification loss value, which can improve the richness, accuracy and availability of the first model loss value to some extent, and can improve the prediction segmentation performance of the finally obtained target image segmentation model.

[0158] Step 208, adjusting the first model parameter of the first image segmentation model based on the first model loss value to obtain a target image segmentation model.

[0159] In the embodiments of the present application, when the first model parameter is adjusted, the second model parameter of the first image feature extraction sub-model can be kept unchanged, that is, only the model parameters of other parts of the first image segmentation model except the first image feature extraction sub-model can be adjusted. Other implementation contents of this step can refer to the embodiment contents of step 104, which will not be described here.

[0160] Reference Figure 3 , Figure 3 A flowchart of a remote sensing image intelligent segmentation method based on a large model provided by the embodiments of the present application, which can include:

[0161] Step 301, obtaining a target remote sensing image.

[0162] In the embodiments of the present application, the target remote sensing image can be obtained. The initial data source of the target remote sensing image can be a remote sensing satellite.

[0163] Step 302, inputting the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on any of the model training methods described above.

[0164] In the embodiments of the present application, the target remote sensing image can be input into the target image segmentation model, so that the first remote sensing segmentation image output by the target image segmentation model can be obtained. The target image segmentation model can be obtained by any of the above model training methods. The first remote sensing segmentation image is the segmentation result of the target remote sensing image.

[0165] In the embodiments of the present application, by acquiring a target remote sensing image, the target remote sensing image is input into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on the model training method described in any of the above, and the target image segmentation model can be used to segment the target remote sensing image, which can improve the accuracy of remote sensing image segmentation to a certain extent.

[0166] Based on the above-described embodiments, with reference to Figure 4 , Figure 4 A flowchart of a heat map generation method provided in the embodiments of the present application is shown in FIG. 3. After step 302, the method can further include:

[0167] In step 303, the target remote sensing image is input into a target uncertainty estimation model to obtain the confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model.

[0168] In the embodiments of the present application, the target uncertainty estimation model can be composed of a backbone network, a Resnet50 network model and a multilayer perception machine, which are used to analyze and predict the uncertainty part, so as to find the high-uncertainty region for labeling and complete subsequent manual correction. The backbone network can be the same as the first image feature extraction submodel in the target image segmentation model. The output of the backbone network can be connected to the input of the Resnet50, and the output of the Resnet50 can be connected to the input of the multilayer perception machine. The target remote sensing image can be input into the target uncertainty estimation model for step-by-step processing by the backbone network, the Resnet50 network model and the multilayer perception machine, and finally the confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model can be obtained. The training process of the target uncertainty estimation model can be as follows: for an input image , the feature is obtained after the backbone network, the feature can obtain a prediction result after the other parts of the target image segmentation model except the backbone network, and the feature can obtain an uncertainty estimation result after the other parts of the uncertainty estimation network except the backbone network. The difference between the prediction result and the manual annotation can obtain the area where the model prediction is wrong , and when the difference between and satisfies the preset convergence condition, the target uncertainty estimation model is obtained. U E ​​

[0169] In step 304, an uncertainty estimation heat map corresponding to the first remote sensing segmentation image is generated based on the confidence of each segmentation region.

[0170] In the embodiments of the present application, the uncertainty estimation heat map corresponding to the first remote sensing segmentation image can be generated according to the confidence of each segmentation region. In the heat map, the lower the confidence, the higher the proportion of the blue part in the heat color; the higher the confidence, the higher the proportion of the red part in the heat color.

[0171] In the embodiments of the present application, by inputting the target remote sensing image into the target uncertainty estimation model, the confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model is obtained, and the uncertainty estimation heat map corresponding to the first remote sensing segmentation image is generated based on the confidence of each segmentation region. The segmentation performance of the target image segmentation model can be evaluated using the uncertainty estimation model, which improves the visibility of the segmentation result of the target image segmentation model and helps to further improve the segmentation efficiency of the target image segmentation model through other methods.

[0172] Reference Figure 5 , Figure 5 A logic block diagram of a model training device provided in the embodiments of the present application is provided, and the device 500 can include:

[0173] The first input and output module 501 is configured to input a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold value.

[0174] The first determination module 502 is configured to determine a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image based on the first segmentation mask image.

[0175] The second determination module 503 is configured to determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region.

[0176] The first adjustment module 504 is configured to adjust a first model parameter of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is configured to generate a target segmentation mask image corresponding to a target remote sensing image.

[0177] Optionally, the first image segmentation model includes a first image feature extraction sub-model, and the device 500 further includes:

[0178] a mask module, configured to perform a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image;

[0179] a second input and output module, configured to input the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model;

[0180] a third determination module, configured to determine a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0181] a second adjustment module, configured to adjust a second model parameter of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model.

[0182] Optionally, the third determination module comprises:

[0183] a first determination sub-module, configured to determine a mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0184] a second determination sub-module, configured to determine a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image;

[0185] a third determination sub-module, configured to determine the second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value.

[0186] Optionally, the first determination sub-module comprises:

[0187] an input and output unit, configured to input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively;

[0188] a first determination unit, configured to determine a first relative position of a mask region in the first mask feature map and determine a second relative position same as the first relative position in the sample feature map;

[0189] a second determination unit, configured to determine a first pixel value of the mask region and a number of pixels contained in the mask region, and determine a second pixel value of a region where the second relative position is located in the sample feature map;

[0190] a third determination unit, configured to determine the mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

[0191] Optionally, the second determining module 503 comprises:

[0192] A fourth determining sub-module, configured to determine a mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region respectively;

[0193] A fifth determining sub-module, configured to determine a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region respectively;

[0194] A sixth determining sub-module, configured to determine a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0195] Optionally, the fourth determining sub-module comprises:

[0196] A fourth determining unit, configured to determine a binary cross entropy loss value and a Dice loss value of the first image segmentation model based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region respectively;

[0197] A fifth determining unit, configured to determine a mask prediction loss value of the first image segmentation model based on the binary cross entropy loss value and the Dice loss value.

[0198] The model training apparatus in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. For example, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application are not limited in this regard.

[0199] The model training apparatus in the embodiments of the present application can be an apparatus with an operating system. The operating system can be an Android operating system, can be a Linux, Windows operating system, etc., can also be other possible operating systems, and the embodiments of the present application do not make specific limitations.

[0200] The model training apparatus provided in the embodiments of the present application can implement Figures 1-2 The method embodiments implement various processes, and to avoid repetition, details are not described herein.

[0201] Referring to Figure 6 , Figure 6 is a logic block diagram of a remote sensing image intelligent segmentation apparatus based on a large model provided by the embodiments of the present application. The apparatus 600 can include:

[0202] The acquisition module 601 is configured to acquire a target remote sensing image.

[0203] The first input and output module 602 is configured to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model. The target image segmentation model is obtained based on the model training method according to any of the above.

[0204] Optionally, the apparatus 600 further includes:

[0205] The second input and output module is configured to input the target remote sensing image into a target uncertainty estimation model to obtain a confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model.

[0206] The generation module is configured to generate an uncertainty estimation heat map corresponding to the first remote sensing segmentation image based on the confidence of each segmentation region.

[0207] The device for intelligent segmentation of remote sensing images based on a large model in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or another device other than a terminal. For example, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like. The embodiments of the present application are not limited in this regard.

[0208] The device for intelligent segmentation of remote sensing images based on a large model in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, a Linux operating system, a Windows operating system, or another possible operating system, and the embodiments of the present application are not limited in this regard.

[0209] The device for intelligent segmentation of remote sensing images based on a large model provided in the embodiments of the present application can implement the method embodiments Figures 3-4 The processes implemented by the method embodiments are not repeated here to avoid repetition.

[0210] The embodiments of the present application provide an electronic device, which is shown in Figure 7 The electronic device 70 includes a processor 701, a memory 702, and a computer program 7021 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the program, the model training method or the method for intelligent segmentation of remote sensing images based on a large model of the foregoing embodiments is implemented.

[0211] The embodiments of the present application also provide a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the model training method or the method for intelligent segmentation of remote sensing images based on a large model disclosed in the embodiments of the present application are implemented.

[0212] The embodiments of the present application further provide a computer program product, which, when running on an electronic device, causes a processor to implement the steps in the model training method or the remote sensing image intelligent segmentation method based on a large model.

[0213] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0214] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the methods, devices, electronic devices and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal devices to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal devices produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0215] These computer program instructions can also be stored in a computer readable memory that can guide the computer or other programmable data processing terminal devices to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0216] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0217] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0218] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0219] The above provides a resource decoupling system, execution method and device for deep learning application. The principle and implementation of the present application are described by specific examples. The above example is only used to help understand the method and core idea of the present application. For those skilled in the art, the specific implementation and application range can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A model training method, characterized in that, The method comprises: inputting a sample remote sensing image in a sample remote sensing image dataset into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein an average information entropy of the sample remote sensing image is greater than or equal to a first threshold value; determining, based on the first segmentation mask image, a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image respectively; determining, based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region respectively, a first model loss value of the first image segmentation model; adjusting, based on the first model loss value, a first model parameter of the first image segmentation model to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to a target remote sensing image; the first image segmentation model comprises a first image feature extraction sub-model, and the method further comprises: performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image; inputting the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model; determining, based on the first remote sensing mask image and the sample remote sensing image, a second model loss value of the second image feature extraction sub-model; adjusting, based on the second model loss value, a second model parameter of the second image feature extraction sub-model to obtain the first image feature extraction sub-model; the determining, based on the first remote sensing mask image and the sample remote sensing image, the second model loss value of the second image feature extraction sub-model comprises: determining, based on the first remote sensing mask image and the sample remote sensing image, a mask image reconstruction loss value of the second image feature extraction sub-model; determining, based on the first remote sensing mask image and the sample remote sensing image, a feature matching loss value of the second image feature extraction sub-model; determining, based on the mask image reconstruction loss value and the feature matching loss value, the second model loss value of the second image feature extraction sub-model; the determining, based on the first remote sensing mask image and the sample remote sensing image, the mask image reconstruction loss value of the second image feature extraction sub-model comprises: inputting the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively; determining a first relative position of a mask region in the first mask feature map and a second relative position same as the first relative position in the sample feature map; determining a first pixel value of the mask region and a number of pixels contained in the mask region, and determining a second pixel value of a region at the second relative position in the sample feature map; determining, based on the first pixel value, the second pixel value and the number of pixels, the mask image reconstruction loss value of the second image feature extraction sub-model.

2. The method of claim 1, wherein, The first model loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The mask prediction loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The classification loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The first model loss value of the first image segmentation model is determined based on the mask prediction loss value and the classification loss value.

3. The method of claim 2, wherein, The first model loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The binary cross-entropy loss value and the Dice loss value of the first image segmentation model are respectively determined based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation region. The mask prediction loss value of the first image segmentation model is determined based on the binary cross-entropy loss value and the Dice loss value.

4. A large model-based remote sensing image intelligent segmentation method, characterized in that, The method comprises: Obtaining a target remote sensing image; Inputting the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on the model training method of any one of claims 1 to 3.

5. The method of claim 4, wherein, The method further comprises: Inputting the target remote sensing image into a target uncertainty estimation model to obtain a confidence of each segmentation region in the first remote sensing segmentation image output by the target uncertainty estimation model; Generating an uncertainty estimation heat map corresponding to the first remote sensing segmentation image based on the confidence of each segmentation region.

6. A model training apparatus characterized by comprising: The device comprises: A first input and output module configured to input a sample remote sensing image in a sample remote sensing image dataset into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold value; A first determination module configured to determine a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image based on the first segmentation mask image; A second determination module configured to determine a first model loss value of the first image segmentation model based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region; A first adjustment module configured to adjust a first model parameter of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is configured to generate a target segmentation mask image corresponding to a target remote sensing image; The first image segmentation model comprises a first image feature extraction sub-model, and the model training device further comprises: A mask module configured to perform a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image; The second input and output module is configured to input the sample remote sensing mask image into the second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model. The third determination module is configured to determine a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image. The second adjustment module is configured to adjust a second model parameter of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model. The third determination module includes: The first determination submodule is configured to determine a mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image. The second determination submodule is configured to determine a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image. The third determination submodule is configured to determine a second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value. The first determination submodule includes: The input and output unit is configured to input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively to obtain a first mask feature map and a sample feature map output by the feature pyramid network respectively. The first determination unit is configured to determine a first relative position of a mask region in the first mask feature map and determine a second relative position same as the first relative position in the sample feature map. The second determination unit is configured to determine a first pixel value of the mask region and a number of pixels contained in the mask region, and determine a second pixel value of a region in the sample feature map at the second relative position. The third determination unit is configured to determine a mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

7. A large model-based remote sensing image intelligent segmentation device, characterized in that, The device includes: The acquisition module is configured to acquire a target remote sensing image. The first input and output module is configured to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model, wherein the target image segmentation model is obtained based on the model training method in any one of claims 1 to 3.

Images