Model training method, and remote sensing image intelligent segmentation method and device based on large model

Abstract

The invention provides a model training method and a remote sensing image intelligent segmentation method and device based on a large model, and the method comprises the steps: inputting a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model, and obtaining a first segmentation mask image outputted by the first image segmentation model; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold value, based on the first segmentation mask image, determining a segmentation prediction type corresponding to each preset segmentation region in the sample remote sensing image, and based on the segmentation prediction type and a sample classification label corresponding to each preset segmentation region, performing classification on the sample remote sensing image. According to the method, the first model loss value of the first image segmentation model is determined, the first model parameter of the first image segmentation model is adjusted based on the first model loss value, the target image segmentation model is obtained, and when the target image segmentation model is used for remote sensing image segmentation, the segmentation accuracy of the remote sensing image can be improved to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a model training method, and a remote sensing image intelligent segmentation method and device based on a large model. Background Art

[0002] Satellite remote sensing images can include different objects on the surface, such as buildings, roads, vegetation, water bodies, etc. Remote sensing images have a wide coverage and can provide surface conditions over a large area, which is crucial for environmental monitoring, urban planning, disaster response and other fields. With the development of remote sensing technology, the amount of data obtained from remote sensing images is increasing, and manual interpretation has become impractical. Therefore, it is particularly important to develop effective automatic interpretation algorithms.

[0003] In related technologies, some deep learning networks can be used to automatically perform image segmentation tasks, such as Unet, SegNet, PSPNet, Deeplab series, etc. These networks have increasingly higher accuracy in natural scene image segmentation tasks.

[0004] However, since remote sensing images have more image features, the above method has low segmentation accuracy for remote sensing images when processing the segmentation task of remote sensing images. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a model training method, a remote sensing image intelligent segmentation method based on a large model, an apparatus, an electronic device and a readable storage medium to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect of an embodiment of the present application, a model training method is provided, the method comprising: Inputting a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold; Based on the first segmentation mask image, determining the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image; Determining a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each of the preset segmentation areas; Based on the first model loss value, the first model parameters of the first image segmentation model are adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

[0007] Optionally, the first image segmentation model includes a first image feature extraction sub-model, and the method further includes: Performing a random masking operation on the sample remote sensing image to obtain a sample remote sensing mask image; Inputting the sample remote sensing mask image into a second image feature extraction submodel to obtain a first remote sensing mask image output by the second image feature extraction submodel; Determining a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Based on the second model loss value, the second model parameters of the second image feature extraction sub-model are adjusted to obtain the first image feature extraction sub-model.

[0008] Optionally, determining a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image includes: Determining a mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Determining a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Based on the mask image reconstruction loss value and the feature matching loss value, a second model loss value of the second image feature extraction sub-model is determined.

[0009] Optionally, determining the mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image includes: Inputting the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively, to obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network; Determine a first relative position of the mask area in the first mask feature map, and determine a second relative position in the sample feature map that is the same as the first relative position; Determine a first pixel value of the mask area and the number of pixels included in the mask area, and determine a second pixel value of the area where the second relative position in the sample feature map is located; Based on the first pixel value, the second pixel value and the number of pixels, a mask image reconstruction loss value of the second image feature extraction sub-model is determined.

[0010] Optionally, determining the first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each preset segmentation area includes: Determining a mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; Determining a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; Based on the mask prediction loss value and the classification loss value, a first model loss value of the first image segmentation model is determined.

[0011] Optionally, determining the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each preset segmentation area includes: Based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas, respectively determining a binary cross entropy loss value and a Dice loss value of the first image segmentation model; Based on the binary cross entropy loss value and the Dice loss value, a mask prediction loss value of the first image segmentation model is determined.

[0012] In a second aspect, an embodiment of the present application provides a remote sensing image intelligent segmentation method based on a large model, the method comprising: Acquire target remote sensing images; The target remote sensing image is input into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on the model training method described in any one of claims 1 to 6.

[0013] Optionally, the method further comprises: Inputting the target remote sensing image into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model; Based on the confidence levels of the respective segmented regions, an uncertainty estimation heat map corresponding to the first remote sensing segmented image is generated.

[0014] In a third aspect, an embodiment of the present application provides a model training device, the device comprising: A first input-output module is used to input a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold; A first determination module, configured to determine, based on the first segmentation mask image, the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image; A second determination module, used to determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each of the preset segmentation areas; The first adjustment module is used to adjust the first model parameters of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

[0015] Optionally, the first image segmentation model includes a first image feature extraction sub-model, and the device further includes: A mask module, used for performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image; A second input-output module, used for inputting the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model; A third determination module, configured to determine a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; The second adjustment module is used to adjust the second model parameters of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model.

[0016] Optionally, the third determining module includes: A first determination submodule, configured to determine a mask image reconstruction loss value of the second image feature extraction submodel based on the first remote sensing mask image and the sample remote sensing image; A second determination submodule, configured to determine a feature matching loss value of the second image feature extraction submodel based on the first remote sensing mask image and the sample remote sensing image; The third determination submodule is used to determine the second model loss value of the second image feature extraction submodel based on the mask image reconstruction loss value and the feature matching loss value.

[0017] Optionally, the first determining submodule includes: An input-output unit, used to input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively, to obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network; A first determining unit, configured to determine a first relative position of the mask area in the first mask feature map, and determine a second relative position in the sample feature map that is the same as the first relative position; A second determining unit, used to determine a first pixel value of the mask area and the number of pixels included in the mask area, and determine a second pixel value of an area where a second relative position in the sample feature map is located; The third determining unit is used to determine the mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

[0018] Optionally, the second determining module includes: A fourth determination submodule, used to determine the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; A fifth determination submodule, used to determine a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; The sixth determination submodule is used to determine a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0019] Optionally, the fourth determining submodule includes: a fourth determining unit, configured to determine a binary cross entropy loss value and a Dice loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation regions; A fifth determining unit is used to determine a mask prediction loss value of the first image segmentation model based on the binary cross entropy loss value and the Dice loss value.

[0020] In a fourth aspect, an embodiment of the present application provides a remote sensing image intelligent segmentation device based on a large model, the device comprising: An acquisition module is used to acquire a target remote sensing image; The first input-output module is used to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on any of the model training methods described above.

[0021] Optionally, the device further comprises: A second input-output module, used for inputting the target remote sensing image into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model; A generation module is used to generate an uncertainty estimation heat map corresponding to the first remote sensing segmented image based on the confidence of each segmented area.

[0022] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement a model training method as described in any one of the above, or a remote sensing image intelligent segmentation method based on a large model.

[0023] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the model segmentation method as described in any one of the above items is implemented, or a remote sensing image intelligent segmentation method based on a large model.

[0024] The specific beneficial effects are: In an embodiment of the present application, a sample remote sensing image in a sample remote sensing image data set is input into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold value, and based on the first segmentation mask image, the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image is determined, and based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation area, a first model loss value of the first image segmentation model is determined, and based on the first model loss value, a first model parameter of the first image segmentation model is adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image, and the first image segmentation model can be trained using a sample remote sensing image data set whose average information entropy is greater than or equal to the first threshold value to obtain a target image segmentation model, so that the model can learn the visual diversity of remote sensing images and improve the segmentation accuracy of remote sensing images to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 It is a flowchart of a model training method provided in an embodiment of the present application; Figure 2 It is a flowchart of another model training method provided in an embodiment of the present application; Figure 3 It is a flowchart of a remote sensing image intelligent segmentation method based on a large model provided in an embodiment of the present application; Figure 4It is a flow chart of a method for generating a heat map provided in an embodiment of the present application; Figure 5 It is a logic block diagram of a model training device provided in an embodiment of the present application; Figure 6 It is a logic block diagram of a remote sensing image intelligent segmentation device based on a large model provided in an embodiment of the present application; Figure 7 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0028] Reference Figure 1 , Figure 1 A flow chart of a model training method provided in an embodiment of the present application, the method comprising: Step 101: input a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold.

[0029] In an embodiment of the present application, the data source of the sample remote sensing image data set may be some commonly used data sets, such as Million-AID, NAIP, RSD46-WHU, MLRSNet, RESISC45, PatternNet, etc. The information entropy of each remote sensing image data contained in these data sets can be calculated, and the remote sensing image data can be screened from the above data sets according to the information entropy to form a new data set, and the average information entropy of the sample remote sensing images contained in the new data set is greater than or equal to the first threshold, that is, the sample remote sensing image data set. After obtaining the sample remote sensing image data set, the sample remote sensing image in the data set can be input into the first image segmentation model, so that the first segmentation mask image output by the first image segmentation model can be obtained. Among them, the first image segmentation model can be a commonly used image segmentation model such as U-net, Mask2Former, etc.; the first segmentation mask image can be a binary image, in which a black line of a preset width can be used to represent the segmentation edge, and the image area between the lines can be white.

[0030] Step 102: Determine the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image based on the first segmentation mask image.

[0031] In an embodiment of the present application, the preset segmentation area can be determined according to the annotation information corresponding to the sample remote sensing image, and the annotation information is the correct segmentation result corresponding to the sample remote sensing image. The segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image can be determined according to the first segmentation mask image. The segmentation prediction type can be preset, such as "bird", "vehicle", "building", "background", etc.

[0032] Step 103: Determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas.

[0033] In an embodiment of the present application, the segmentation area of ​​the first segmentation mask image may not completely overlap with the preset segmentation area in the sample remote sensing image. In this way, in the non-overlapping area, the segmentation prediction type may deviate from the sample classification labels corresponding to each preset segmentation area. The first model loss value of the first image segmentation model may be determined based on the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area. The first model loss value may be a cross entropy loss value or a contrast loss value.

[0034] Step 104: Based on the first model loss value, adjust the first model parameters of the first image segmentation model to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

[0035] In an embodiment of the present application, the first model parameters of the first image segmentation model can be adjusted according to the first model loss value, so that the target segmentation model can be obtained. The adjustment of the first model parameters can be repeated multiple times, and after each adjustment, the process of steps 101 to 104 can be repeated to adjust the first model parameters again. When the number of training times of the first image segmentation model reaches a preset number of times, or the first model loss value meets the convergence condition, the adjustment of the first model parameters of the first image segmentation model can be stopped, and the first image segmentation model obtained after the last adjustment is determined as the target image segmentation model.

[0036] In an embodiment of the present application, a sample remote sensing image in a sample remote sensing image data set is input into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image data set is greater than or equal to a first threshold value, and based on the first segmentation mask image, the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image is determined, and based on the segmentation prediction type and the sample classification label corresponding to each preset segmentation area, the first model loss value of the first image segmentation model is determined, and based on the first model loss value, the first model parameter of the first image segmentation model is adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image, and the first image segmentation model can be trained by a sample remote sensing image data set whose average information entropy is greater than or equal to the first threshold value to obtain a target image segmentation model, which can enable the model to learn the visual diversity of remote sensing images and improve the segmentation accuracy of remote sensing images to a certain extent.

[0037] Reference Figure 2 , Figure 2 A flowchart of another model training method provided in an embodiment of the present application, the method may include: Step 201, performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image; In an embodiment of the present application, a random mask operation may be performed on the sample remote sensing image, that is, a random mask is added to the sample remote sensing image, so as to obtain a sample remote sensing mask image. Compared with the sample remote sensing image, the sample remote sensing mask image may lack some image features, and the missing area may constitute a mask area.

[0038] Step 202: input the sample remote sensing mask image into the second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model.

[0039] In an embodiment of the present application, the second image feature extraction submodel may be a SwinV2-G network model, which may randomly generate various model parameter values ​​and model weight values ​​contained in the model before training, thereby initializing the SwinV2-G model. The sample remote sensing mask image may be input into the second image feature extraction submodel, thereby obtaining a first remote sensing mask image output by the second image feature extraction submodel. The first remote sensing mask image is the image backbone of the sample remote sensing image, and may include one or more image features of the sample remote sensing image.

[0040] Step 203: Determine a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image.

[0041] In an embodiment of the present application, a second model loss value of the second image feature extraction submodel can be determined based on the first remote sensing mask image and the sample remote sensing image. The second model loss value can be a binary cross entropy loss value, a contrast loss value, or a combination of the above two loss values.

[0042] Optionally, step 203 may include the following sub-steps: Sub-step 2031, based on the first remote sensing mask image and the sample remote sensing image, determining the mask image reconstruction loss value of the second image feature extraction sub-model.

[0043] In an embodiment of the present application, the mask image reconstruction loss value is an indicator used to measure the ability of the model to reconstruct the masked portion of the image in a self-supervised learning model such as a masked autoencoder (MAE). The size of the mask image reconstruction loss value can reflect the accuracy of the model in reconstructing the masked portion of the image. A lower loss value generally means that the model has better image reconstruction capabilities. The location of the masked reconstructed area in the first remote sensing mask image can be determined, and the location corresponding to the location of the masked area in the sample remote sensing image can be determined, and then the mask image reconstruction loss value of the second image feature extraction submodel can be calculated based on the reconstructed pixel value and the original pixel value of the above position.

[0044] Optionally, sub-step 2031 may include the following sub-steps: Sub-step A1: input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively, and obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network.

[0045] In an embodiment of the present application, Feature Pyramid Networks (FPN) is a network structure widely used in deep learning target detection, mainly used to construct multi-scale feature representations to effectively detect targets of different sizes. FPN can simultaneously retain rich semantic information and detail information on feature maps of different scales. These fused feature maps can be directly used for target detection tasks, such as bounding box regression and classification, thereby improving the model's detection performance for multi-scale targets. The first remote sensing mask image and the sample remote sensing image can be respectively input into the feature pyramid network, so that the first mask feature map and the sample feature map output by the feature pyramid network can be obtained. Among them, the first mask feature map can correspond to the first remote sensing mask image, and the sample feature map can correspond to the sample remote sensing image.

[0046] Sub-step A2: determining a first relative position of the mask area in the first mask feature map, and determining a second relative position in the sample feature map that is the same as the first relative position.

[0047] In an embodiment of the present application, after obtaining the first mask feature map and the sample feature map, the first relative position of the mask area in the first mask feature map and the second relative position in the sample feature map that is the same as the above first relative position can be further determined.

[0048] Sub-step A3, determining the first pixel value of the mask area and the number of pixels contained in the mask area, and determining the second pixel value of the area where the second relative position in the sample feature map is located.

[0049] In an embodiment of the present application, a first pixel value and the number of pixels in the mask area can be determined, and a second pixel value in the area where the second relative position in the sample feature map is located can be determined.

[0050] Sub-step A4, determining a mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

[0051] In an embodiment of the present application, the mask image reconstruction loss value of the second image feature extraction sub-model can be calculated according to the first pixel value, the second pixel value and the number of pixels, and the calculation method thereof is shown in the following formula 1: (Formula 1) In formula 1, represents the mask image reconstruction loss value, Indicates the number of the fused multi-scale feature map formed by FPN. Indicates the masked area The original pixel value, that is, the second pixel value, Represents the reconstructed pixel values ​​generated in these areas, that is, the first pixel values, Represents the total number of masked pixels, that is, the number of pixels in the mask area.

[0052] In an embodiment of the present application, a first remote sensing mask image and a sample remote sensing image are respectively input into a feature pyramid network to obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network, a first relative position of the mask area in the first mask feature map is determined, and a second relative position that is the same as the first relative position in the sample feature map is determined, a first pixel value of the mask area and the number of pixels contained in the mask area are determined, and a second pixel value of the area where the second relative position is located in the sample feature map is determined. Based on the first pixel value, the second pixel value and the number of pixels, a mask image reconstruction loss value of the second image feature extraction submodel is determined. The feature pyramid network can be used to participate in the calculation of the mask image reconstruction loss value of the second image feature extraction submodel, which can improve the accuracy of the mask image reconstruction loss value to a certain extent and improve the prediction accuracy of the first image feature extraction submodel finally obtained.

[0053] Sub-step 2032, determining a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image.

[0054] In the embodiments of the present application, Feature Matching Loss (FML) is a loss function used to measure the accuracy of feature restoration, and is mainly used in single image reflection separation tasks. Feature matching loss evaluates the performance of the model by comparing the feature accuracy of the information of the restored non-reflection image. The feature matching loss value of the second image feature extraction sub-model can be calculated based on the first remote sensing mask image and the sample remote sensing image, and the specific calculation method is shown in Formula 2 below: (Formula 2) In formula 2, represents the feature matching loss value, represents the image features of the first remote sensing mask image, Represents the image features of the sample remote sensing image, symbol " " represents modulo, symbol " " indicates L2 paradigm, Represents a preset linear mapping function.

[0055] Sub-step 2033, determining a second model loss value of the second image feature extraction sub-model based on the mask image reconstruction loss value and the feature matching loss value.

[0056] In an embodiment of the present application, the second model loss value of the second image feature extraction submodel can be calculated based on the mask image reconstruction loss value and the feature matching loss value. The mask image reconstruction loss value and the feature matching loss value can be directly added to obtain the second model loss value. A balance coefficient can also be used to balance the proportion of the mask image reconstruction loss value and the feature matching loss value in the second model loss value, as shown in Formula 3 below: (Formula 3) In formula 3, represents the loss value of the second model, represents the mask image reconstruction loss value, represents the feature matching loss value, Represents the balance coefficient.

[0057] In an embodiment of the present application, the mask image reconstruction loss value of the second image feature extraction submodel is determined based on the first remote sensing mask image and the sample remote sensing image, the feature matching loss value of the second image feature extraction submodel is determined based on the first remote sensing mask image and the sample remote sensing image, and the second model loss value of the second image feature extraction submodel is determined based on the mask image reconstruction loss value and the feature matching loss value. The mask image reconstruction loss value and the feature matching loss value can be calculated, and the second model loss value can be calculated by the mask image reconstruction loss value and the feature matching loss value. The accuracy of the second model loss value can be improved to a certain extent, and the prediction accuracy of the first image feature extraction submodel finally obtained can be improved.

[0058] Step 204: Based on the second model loss value, adjust the second model parameters of the second image feature extraction sub-model to obtain the first image feature extraction sub-model.

[0059] In an embodiment of the present application, the second model parameters of the second image feature extraction submodel can be adjusted according to the second model loss value, so that the first image feature extraction submodel can be obtained. The adjustment of the second model parameters can be repeated multiple times, and after each adjustment, the process of steps 201 to 204 can be repeated to adjust the second model parameters again. When the training times of the second image feature extraction submodel reaches a preset number of times, or the second model loss value meets the convergence condition, the adjustment of the second model parameters of the second image feature extraction submodel can be stopped, and the second image feature extraction submodel obtained after the last adjustment is determined as the first image feature extraction submodel.

[0060] In an embodiment of the present application, a sample remote sensing mask image is obtained by performing a random mask operation on a sample remote sensing image, and the sample remote sensing mask image is input into a second image feature extraction submodel to obtain a first remote sensing mask image output by the second image feature extraction submodel. Based on the first remote sensing mask image and the sample remote sensing image, a second model loss value of the second image feature extraction submodel is determined. Based on the second model loss value, the second model parameters of the second image feature extraction submodel are adjusted to obtain a first image feature extraction submodel. Noise can be added to the sample remote sensing image by random masking, and the second image feature extraction submodel can be trained by using the sample remote sensing mask image after adding noise to obtain a first image feature extraction submodel. This can improve the robustness and prediction accuracy of the first image feature extraction submodel to a certain extent.

[0061] Step 205: input the sample remote sensing image in the sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the first image segmentation model includes a first image feature extraction sub-model; and the average information entropy of the sample remote sensing image is greater than or equal to a first threshold.

[0062] In an embodiment of the present application, the first image segmentation model may include a first image feature extraction submodel, and the first image feature extraction submodel may be a backbone network in the first image segmentation model. The first image feature extraction submodel may be used to perform a first stage of processing on a sample remote sensing image to extract image backbone features in the sample remote sensing image. The first image segmentation model may also include a pixel decoder and a Transformer decoder, wherein the pixel decoder may be used to upsample low-resolution features from image backbone features to generate a high-resolution pixel-by-pixel feature representation, and the Transformer decoder may process the pixel-by-pixel embedded representation to generate a first segmentation mask image. For other implementation contents of this step, please refer to the embodiment contents of step 101, which will not be repeated here.

[0063] Step 206: Determine the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image based on the first segmentation mask image.

[0064] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 102, which will not be repeated here.

[0065] Step 207: Determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas.

[0066] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 103, which will not be repeated here.

[0067] Optionally, step 207 may include the following sub-steps: Sub-step 2071, determining the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas.

[0068] In an embodiment of this step, the mask prediction loss value can be used to measure the similarity between the mask generated by the model and the target mask. The mask is the edge texture information in the image. The mask prediction loss value of the first image segmentation model can be determined according to the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area. The value range of the mask prediction loss value can be a value less than or equal to 0. Among them, the higher the degree of match between the segmentation prediction type and the sample classification label, the greater the mask prediction loss value. When the mask prediction loss value is 0, it means that the segmentation prediction type and the sample classification label are completely matched, that is, the segmentation prediction type of each preset segmentation area is exactly the same as the sample classification label.

[0069] Optionally, sub-step 2071 may include the following sub-steps: Sub-step B1, based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas, respectively determine the binary cross entropy loss value and the Dice loss value of the first image segmentation model.

[0070] In an embodiment of the present application, the mask prediction loss value may include a binary cross entropy loss value and a Dice loss value. Among them, the binary cross entropy loss value can be used to measure the gap between the predicted probability and the true value of each pixel, and the Dice loss value can be used to measure the similarity between two masks. The binary cross entropy loss value and the Dice loss value of the first image segmentation model can be calculated according to the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area. The calculation method of the binary cross entropy loss value is shown in Formula 4 below: (Formula 4) In formula 4, represents the binary cross entropy loss value, represents the number of pixels in the mask area, Indicates the pixel number, It represents the probability that the segmentation prediction type of pixel x is the i-th category, which can be obtained by calculating the similarity between the segmentation prediction type and the sample classification label. Indicates that the segmentation prediction type is not the i-th category (that is, the segmentation prediction type does not correspond to the i-th sample classification label). 1 means the segmentation prediction type is the i-th category (i.e. the segmentation prediction type corresponds to the i-th sample classification label). log is the logarithmic operator.

[0071] The calculation method of Dice loss value is as shown in the following formula 5: (Formula 5) In formula 5, represents the Dice loss value, represents the number of pixels in the mask area, Indicates the pixel number, It represents the probability that the segmentation prediction type of pixel x is the i-th category, which can be obtained by calculating the similarity between the segmentation prediction type and the sample classification label. Indicates that the segmentation prediction type is not the i-th category (that is, the segmentation prediction type does not correspond to the i-th sample classification label). 1 means that the segmentation prediction type is the i-th category (that is, the segmentation prediction type corresponds to the i-th sample classification label).

[0072] Sub-step B2: determining a mask prediction loss value of the first image segmentation model based on the binary cross entropy loss value and the Dice loss value.

[0073] In an embodiment of the present application, the mask prediction loss value of the first image segmentation model can be calculated based on the binary cross entropy loss value and the Dice loss value. The mask prediction loss value can be obtained by directly adding the binary cross entropy loss value and the Dice loss value. A balance coefficient can also be added to balance the proportion of the binary cross entropy loss value and the Dice loss value in the mask prediction loss value, as shown in the following formula 6: (Formula 6) In formula 6, represents the mask prediction loss value, represents the Dice loss value, represents the binary cross entropy loss value, It is the balance coefficient and can be set according to requirements.

[0074] In an embodiment of the present application, the binary cross entropy loss value and the Dice loss value of the first image segmentation model are determined based on the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area, and the mask prediction loss value of the first image segmentation model is determined based on the binary cross entropy loss value and the Dice loss value. The mask prediction loss value can be formed by the binary cross entropy loss value and the Dice loss value, which can improve the accuracy of the mask prediction loss value to a certain extent.

[0075] Sub-step 2072, determining the classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas.

[0076] In an embodiment of the present application, the classification loss value can be used to measure the similarity between the predicted category label of each generated mask and the sample classification label. For each first segmentation mask image, the first image segmentation model can also output a predicted probability distribution As the segmentation prediction type, where C represents the number of categories. It indicates the predicted probability that the i-th first segmentation mask image includes category j. The type of classification loss value is cross entropy loss, and its calculation method is shown in the following formula 7: (Formula 7) In formula 7, represents the classification loss value, Represents the cth value in the classification one-hot vector (i.e., sample classification label) of the first segmentation mask image. In the classification one-hot vector, only one value is 1 and the rest are 0. represents the predicted probability that the i-th first segmentation mask image includes category c.

[0077] Sub-step 2073, determining a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0078] In an embodiment of the present application, the first model loss value of the first image segmentation model can be calculated based on the mask prediction loss value and the classification loss value. The mask prediction loss value and the classification loss value can be directly added to obtain the first model loss value. A balance coefficient can also be added to adjust the proportion of the mask prediction loss value and the classification loss value in the first model loss value, as shown in the following formula 8: (Formula 8) In the above formula 8, is the loss value of the first model, is the mask prediction loss value, is the classification loss value, is the balance coefficient.

[0079] In an embodiment of the present application, the mask prediction loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area, and the classification loss value of the first image segmentation model is determined based on the segmentation prediction type and the sample classification labels corresponding to each preset segmentation area. The first model loss value of the first image segmentation model is determined based on the mask prediction loss value and the classification loss value. The first model loss value can be formed by the mask prediction loss value and the classification loss value, which improves the richness, accuracy and availability of the first model loss value to a certain extent, and can improve the prediction segmentation performance of the target image segmentation model finally obtained.

[0080] Step 208: Based on the first model loss value, adjust the first model parameters of the first image segmentation model to obtain a target image segmentation model.

[0081] In the embodiment of the present application, when adjusting the first model parameters, the second model parameters of the first image feature extraction submodel can be kept unchanged, that is, only the model parameters of other parts of the first image segmentation model other than the first image feature extraction submodel can be adjusted. For other implementation contents of this step, please refer to the embodiment contents of step 104, which will not be repeated here.

[0082] Reference Figure 3 , Figure 3 A schematic diagram of a process of a remote sensing image intelligent segmentation method based on a large model provided in an embodiment of the present application, the method may include: Step 301, acquiring a target remote sensing image.

[0083] In an embodiment of the present application, a target remote sensing image may be acquired, and the initial data source of the target remote sensing image may be a remote sensing satellite.

[0084] Step 302: input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmented image output by the target image segmentation model; wherein the target image segmentation model is obtained based on any of the above-mentioned model training methods.

[0085] In an embodiment of the present application, the target remote sensing image can be input into the target image segmentation model, so as to obtain the first remote sensing segmentation image output by the target image segmentation model. The target image segmentation model can be obtained by any of the above model training methods. The first remote sensing segmentation image is the segmentation result of the target remote sensing image.

[0086] In an embodiment of the present application, a target remote sensing image is acquired and input into a target image segmentation model to obtain a first remote sensing segmented image output by the target image segmentation model; wherein the target image segmentation model is obtained based on any of the model training methods described above, and the target image segmentation model can be used to segment the target remote sensing image, which can improve the accuracy of remote sensing image segmentation to a certain extent.

[0087] Based on the above implementation, Figure 4 , Figure 4 A schematic diagram of a heat map generation method provided in an embodiment of the present application. After step 302, the method may further include: Step 303: input the target remote sensing image into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model.

[0088] In an embodiment of the present application, a target uncertainty estimation model may be composed of a backbone network, a Resnet50 network model, and a multi-layer perceptron, and is used to analyze the uncertain part of the prediction, so as to find out the high uncertainty areas for marking and complete the subsequent manual error correction. Among them, the backbone network may be the same as the first image feature extraction sub-model in the target image segmentation model. The output of the backbone network may be connected to the input of Resnet50, and the output of Resnet50 may be connected to the input of the multi-layer perceptron. The target remote sensing image may be input into the target uncertainty estimation model, and the backbone network, Resnet50 network model, and multi-layer perceptron may perform step-by-step processing, and finally the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model may be obtained. The training process of the target uncertainty estimation model may be as follows: for the input image , after passing through the backbone network, the features are obtained ,feature The prediction results can be obtained by the other parts of the target image segmentation model except the backbone network. ,feature After the uncertainty estimation network except the backbone network, the uncertainty estimation result can be obtained. , let the prediction result With manual annotation By subtracting, we can get the area where the model predicts errors. , when training the uncertainty estimation network, let return .when U and E When the difference between them meets the preset convergence condition, the target uncertainty estimation model can be obtained.

[0089] Step 304: Generate an uncertainty estimation heat map corresponding to the first remote sensing segmented image based on the confidence of each segmented area.

[0090] In an embodiment of the present application, an uncertainty estimation heat map corresponding to the first remote sensing segmented image can be generated according to the confidence of each segmented area. In the heat map, the lower the confidence, the higher the proportion of the blue part in the heat color; the higher the confidence, the higher the proportion of the red part in the heat color.

[0091] In an embodiment of the present application, the target remote sensing image is input into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model, and based on the confidence of each segmented area, an uncertainty estimation heat map corresponding to the first remote sensing segmented image is generated. The uncertainty estimation model can be used to evaluate the segmentation performance of the target image segmentation model, thereby improving the visibility of the segmentation results of the target image segmentation model, and helping to further improve the segmentation efficiency of the target image segmentation model through other methods.

[0092] Reference Figure 5 , Figure 5 A logical block diagram of a model training device provided in an embodiment of the present application, the device 500 may include: The first input-output module 501 is used to input the sample remote sensing image in the sample remote sensing image data set into the first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold; A first determination module 502 is used to determine the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image based on the first segmentation mask image; A second determination module 503 is used to determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; The first adjustment module 504 is used to adjust the first model parameters of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

[0093] Optionally, the first image segmentation model includes a first image feature extraction sub-model, and the apparatus 500 further includes: A mask module, used for performing a random mask operation on the sample remote sensing image to obtain a sample remote sensing mask image; A second input-output module, used for inputting the sample remote sensing mask image into a second image feature extraction sub-model to obtain a first remote sensing mask image output by the second image feature extraction sub-model; A third determination module, configured to determine a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; The second adjustment module is used to adjust the second model parameters of the second image feature extraction sub-model based on the second model loss value to obtain the first image feature extraction sub-model.

[0094] Optionally, the third determining module includes: A first determination submodule, configured to determine a mask image reconstruction loss value of the second image feature extraction submodel based on the first remote sensing mask image and the sample remote sensing image; A second determination submodule, configured to determine a feature matching loss value of the second image feature extraction submodel based on the first remote sensing mask image and the sample remote sensing image; The third determination submodule is used to determine the second model loss value of the second image feature extraction submodel based on the mask image reconstruction loss value and the feature matching loss value.

[0095] Optionally, the first determining submodule includes: An input-output unit, used to input the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively, to obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network; A first determining unit, configured to determine a first relative position of the mask area in the first mask feature map, and determine a second relative position in the sample feature map that is the same as the first relative position; A second determining unit, used to determine a first pixel value of the mask area and the number of pixels included in the mask area, and determine a second pixel value of an area where a second relative position in the sample feature map is located; The third determining unit is used to determine the mask image reconstruction loss value of the second image feature extraction sub-model based on the first pixel value, the second pixel value and the number of pixels.

[0096] Optionally, the second determining module 503 includes: A fourth determination submodule, used to determine the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; A fifth determination submodule, used to determine a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; The sixth determination submodule is used to determine a first model loss value of the first image segmentation model based on the mask prediction loss value and the classification loss value.

[0097] Optionally, the fourth determining submodule includes: a fourth determining unit, configured to determine a binary cross entropy loss value and a Dice loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation regions; A fifth determining unit is used to determine a mask prediction loss value of the first image segmentation model based on the binary cross entropy loss value and the Dice loss value.

[0098] The model training device in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobilepersonal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0099] The model training device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, a Linux operating system, a Windows operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0100] The model training device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0101] Reference Figure 6 , Figure 6 A logic block diagram of a remote sensing image intelligent segmentation device based on a large model provided in an embodiment of the present application, the device 600 may include: An acquisition module 601 is used to acquire a target remote sensing image; The first input-output module 602 is used to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmented image output by the target image segmentation model; wherein the target image segmentation model is obtained based on any of the model training methods described above.

[0102] Optionally, the device 600 further includes: A second input-output module, used for inputting the target remote sensing image into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model; A generation module is used to generate an uncertainty estimation heat map corresponding to the first remote sensing segmented image based on the confidence of each segmented area.

[0103] The remote sensing image intelligent segmentation device based on the large model in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmentedreality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personaldigital assistant, PDA), etc., and can also be a server, a network attached storage (Network AttachedStorage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0104] The large model-based remote sensing image intelligent segmentation device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, a Linux operating system, a Windows operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0105] The remote sensing image intelligent segmentation device based on a large model provided in the embodiment of the present application can achieve Figure 3 to Figure 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0106] The present application embodiment provides an electronic device, see Figure 7 The electronic device 70 includes: a processor 701, a memory 702, and a computer program 7021 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the program, the model training method of the aforementioned embodiment is implemented, or the remote sensing image intelligent segmentation method based on a large model is implemented.

[0107] An embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the model training method disclosed in the embodiment of the present application, or the steps in the remote sensing image intelligent segmentation method based on a large model is implemented.

[0108] The embodiment of the present application also provides a computer program product, which, when running on an electronic device, enables a processor to implement the model training method disclosed in the embodiment of the present application, or the steps in the remote sensing image intelligent segmentation method based on a large model.

[0109] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0110] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0114] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0115] The above is a detailed introduction to a resource decoupling system, execution method and device for deep learning applications provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model training method, characterized in that: The method comprises: Inputting a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold; Based on the first segmentation mask image, determining the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image; Determining a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each of the preset segmentation areas; Based on the first model loss value, the first model parameters of the first image segmentation model are adjusted to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

2. The method according to claim 1, characterized in that The first image segmentation model includes a first image feature extraction sub-model, and the method further includes: Performing a random masking operation on the sample remote sensing image to obtain a sample remote sensing mask image; Inputting the sample remote sensing mask image into a second image feature extraction submodel to obtain a first remote sensing mask image output by the second image feature extraction submodel; Determining a second model loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Based on the second model loss value, the second model parameters of the second image feature extraction sub-model are adjusted to obtain the first image feature extraction sub-model.

3. The method according to claim 2, characterized in that The determining, based on the first remote sensing mask image and the sample remote sensing image, a second model loss value of the second image feature extraction sub-model comprises: Determining a mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Determining a feature matching loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image; Based on the mask image reconstruction loss value and the feature matching loss value, a second model loss value of the second image feature extraction sub-model is determined.

4. The method according to claim 3, characterized in that The step of determining the mask image reconstruction loss value of the second image feature extraction sub-model based on the first remote sensing mask image and the sample remote sensing image includes: Inputting the first remote sensing mask image and the sample remote sensing image into a feature pyramid network respectively, to obtain a first mask feature map and a sample feature map respectively output by the feature pyramid network; Determine a first relative position of the mask area in the first mask feature map, and determine a second relative position in the sample feature map that is the same as the first relative position; Determine a first pixel value of the mask area and the number of pixels included in the mask area, and determine a second pixel value of the area where the second relative position in the sample feature map is located; Based on the first pixel value, the second pixel value and the number of pixels, a mask image reconstruction loss value of the second image feature extraction sub-model is determined.

5. The method according to claim 1, characterized in that The determining, based on the segmentation prediction type and the sample classification labels respectively corresponding to each of the preset segmentation regions, a first model loss value of the first image segmentation model comprises: Determining a mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; Determining a classification loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas; Based on the mask prediction loss value and the classification loss value, a first model loss value of the first image segmentation model is determined.

6. The method according to claim 5, characterized in that The determining the mask prediction loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each preset segmentation area includes: Based on the segmentation prediction type and the sample classification labels corresponding to each of the preset segmentation areas, respectively determining a binary cross entropy loss value and a Dice loss value of the first image segmentation model; Based on the binary cross entropy loss value and the Dice loss value, a mask prediction loss value of the first image segmentation model is determined.

7. A remote sensing image intelligent segmentation method based on a large model, characterized in that: The method comprises: Acquire target remote sensing images; The target remote sensing image is input into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on the model training method described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The method further comprises: Inputting the target remote sensing image into a target uncertainty estimation model to obtain the confidence of each segmented area in the first remote sensing segmented image output by the target uncertainty estimation model; Based on the confidence levels of the respective segmented regions, an uncertainty estimation heat map corresponding to the first remote sensing segmented image is generated.

9. A model training device, characterized in that: The device comprises: A first input-output module is used to input a sample remote sensing image in a sample remote sensing image data set into a first image segmentation model to obtain a first segmentation mask image output by the first image segmentation model; wherein the average information entropy of the sample remote sensing image is greater than or equal to a first threshold; A first determination module, configured to determine, based on the first segmentation mask image, the segmentation prediction type corresponding to each preset segmentation area in the sample remote sensing image; A second determination module, used to determine a first model loss value of the first image segmentation model based on the segmentation prediction type and the sample classification labels respectively corresponding to each of the preset segmentation areas; The first adjustment module is used to adjust the first model parameters of the first image segmentation model based on the first model loss value to obtain a target image segmentation model; wherein the target image segmentation model is used to generate a target segmentation mask image corresponding to the target remote sensing image.

10. A remote sensing image intelligent segmentation device based on a large model, characterized in that: The device comprises: An acquisition module is used to acquire a target remote sensing image; The first input-output module is used to input the target remote sensing image into a target image segmentation model to obtain a first remote sensing segmentation image output by the target image segmentation model; wherein the target image segmentation model is obtained based on the model training method described in any one of claims 1 to 6.

Images