Mask-guided remote sensing image precise boundary segmentation model evolution method

Abstract

The application belongs to the technical field of pattern recognition, and specifically discloses a mask-guided remote sensing image accurate boundary segmentation model evolution method, which comprises the following steps: obtaining an original remote sensing image, and constructing a mask image according to the labeling information of the original remote sensing image; constructing a latent diffusion model module of the remote sensing image accurate boundary segmentation model; based on a deterministic diffusion process, using the mask image to guide the latent diffusion model module to learn a denoising process, and outputting mask features; constructing a Transformer prediction head module of the remote sensing image accurate boundary segmentation model; constructing a loss function to optimize and train the remote sensing image accurate boundary segmentation model, and completing the evolution of the remote sensing image accurate boundary segmentation model. The application solves the problem that the randomness of Gaussian noise in the traditional random diffusion process brings unnecessary disturbance to U-Net features, and causes the U-Net features to be not applicable to a deterministic segmentation task.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pattern recognition, and particularly relates to a mask-guided remote sensing image accurate boundary segmentation model evolution method. BACKGROUND

[0002] Remote sensing instance segmentation is a basic task in the field of remote sensing, and has a wide range of applications in building extraction and land measurement. This task requires accurate identification of instances, including their corresponding categories and masks. One of the key challenges of remote sensing image segmentation is to accurately depict object boundaries. However, the high-altitude perspective of remote sensing images often makes it difficult to extract fine-grained boundary contours.

[0003] Existing methods can be roughly divided into two main strategies. The first strategy focuses on designing specialized modules to enhance feature extraction, thereby generating smoother boundaries and reducing jagged artifacts. The second strategy uses edge-aware supervision to promote clearer and more accurate boundary delineation while mitigating blurring effects. However, these methods mainly rely on traditional convolutional neural networks (CNNs) trained from scratch on specific datasets, and these neural networks do not fully utilize the prior knowledge contained in the base model. Compared to a single convolutional neural network trained on a specific dataset, a base model utilizes a large and powerful visual transformer trained on a massive dataset, which can capture rich visual knowledge.

[0004] The latent diffusion model (LDM) has shown great potential in generation tasks. It is trained based on the larger LAION-5B dataset, which enables the LDM to incorporate more extensive prior knowledge. In addition, the LDM is specifically designed for generation tasks, allowing it to capture more accurate context and detailed information, thereby enhancing its ability to generate reliable high-quality images. SUMMARY

[0005] The purpose of the present application is to solve the problem that the randomness of Gaussian noise in the traditional random diffusion process brings unnecessary disturbance to the U-Net features, making it unsuitable for deterministic segmentation tasks, and to propose a mask-guided remote sensing image accurate boundary segmentation model evolution method.

[0006] The technical solution of the present application is: a mask-guided remote sensing image accurate boundary segmentation model evolution method, comprising the following steps:

[0007] Obtain the original remote sensing image, and construct a mask image according to the annotation information of the original remote sensing image;

[0008] Construct a latent diffusion model module of the remote sensing image accurate boundary segmentation model;

[0009] Based on the deterministic diffusion process, a latent diffusion model module is used to learn a denoising process guided by the mask image, and a mask feature is output;

[0010] A Transformer prediction head module of the remote sensing image accurate boundary segmentation model is constructed.

[0011] A loss function is constructed to optimize and train the remote sensing image accurate boundary segmentation model containing the latent diffusion model module and the Transformer prediction head module, and the evolution of the remote sensing image accurate boundary segmentation model is completed.

[0012] As a preferred, the mask image is constructed according to the label information of the original remote sensing image, specifically:

[0013] The original remote sensing image is divided into multiple blocks, each instance is distinguished by a unique RGB color, and each block is assigned a fixed color based on the absolute position of the instance mask center in the original remote sensing image, and an RGB mask image is obtained.

[0014] As a preferred, the latent diffusion model module is composed of a variational autoencoder and a U-Net;

[0015] The variational autoencoder is used to receive the original remote sensing image and the mask image, and respectively extracts features from the original remote sensing image and the mask image to obtain the latent representation of the original remote sensing image and the latent representation of the mask image.

[0016] The U-Net is used to receive the latent representation of the original remote sensing image and the latent representation of the mask image, remove the noise of the latent representation of the original remote sensing image and the noise of the latent representation of the mask image, and obtain multi-level features of the intermediate layer in the L={3,6,9} layer of the U-Net.

[0017] As a preferred, the deterministic diffusion process reduces the noise caused by random Gaussian noise by directly mapping between the original remote sensing image and the mask image; the deterministic diffusion process is specifically:

[0018]

[0019]

[0020] Wherein, denotes the objective function of the deterministic diffusion process, denotes the mathematical expectation under the joint probability distribution, denotes the latent representation of the mask image, denotes the U-Net parameters to be learned, denotes the latent representation containing noise at time step ​denotes the L2 norm for computing the mean square error, denotes the hyperparameters of the noise schedule, denotes the latent representation of the original remote sensing image.

[0021] As a preference, the Transformer prediction head module comprises a classification branch and a mask branch;

[0022] The classification branch is used to predict a score vector , and the score vector is used to predict a class;

[0023] The mask branch is used to fuse the latent representation obtained from the U-Net of the latent diffusion model module and predict a mask .

[0024] As a preference, the Transformer prediction head module is composed of L stacked layers, each of which contains a mask attention unit, a self-attention unit and a feedforward network connected in turn;

[0025] The mask attention unit is used to limit attention to instance regions;

[0026] The self-attention unit is used to realize the interaction between instances;

[0027] The feedforward network is used to introduce nonlinear operations to abstract and strengthen the features extracted by the attention layer.

[0028] As a preference, the Transformer prediction head module uses a learnable query to distinguish different objects, and a learnable position embedding to indicate spatial positions; wherein, denotes the real number field, denotes the number of predicted instances, denotes the dimension of the feature;

[0029] In the first stacked layer, the mask attention unit takes as the query, as the key, and as the value, wherein, , and denote the vectors for transforming the query, key and value, denotes the query of the stacked layer mask attention unit, denotes the multi-level features of the intermediate layer obtained from the U-Net;

[0030] The self-attention unit takes the output of the mask attention unit as the query, key and value, thereby realizing the interaction between different objects.

[0031] As preferred, the loss function comprises a segmentation loss and a diffusion loss, and the specific formula is:

[0032]

[0033] wherein, represents the loss function, represents the first segmentation loss, represents the diffusion loss, represents the number of segmentation losses.

[0034] As preferred, the segmentation loss comprises a classification loss and a mask loss, the classification loss is a cross-entropy loss, and the mask loss is composed of a binary cross-entropy loss and a Dice loss; and the specific formula of the segmentation loss is:

[0035]

[0036] wherein, represents the classification loss, represents the weight factor of the classification loss, represents a score vector predicted by a Transformer prediction head module, represents a real class label of a target in a given image, represents the binary cross-entropy loss, represents the weight factor of the binary cross-entropy loss, represents a mask predicted by the Transformer prediction head module, represents a mask of the target in the given image, represents the Dice loss, represents the weight factor of the Dice loss.

[0037] As preferred, the diffusion loss is an MSE loss, and the specific formula is:

[0038]

[0039] wherein, represents a latent representation of a mask image, represents a U-Net parameter to be learned, represents a latent representation containing noise at a time step .

[0040] The beneficial effects of the present application are:

[0041] 1.The present application is directed to precise instance segmentation of remote sensing images, and proposes a mask-guided diffusion model segmentation architecture. By introducing an RGB mask image to guide the denoising process of the latent diffusion model, it can learn more fine and accurate key content and layout features, thereby improving the accuracy of boundary contour prediction.

[0042] 2.The present application adopts a deterministic diffusion process when integrating the mask image. This method reduces the inherent randomness of traditional diffusion models, making the feature representation in U-Net more accurate and better suited for aerial segmentation tasks. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of a mask-guided precise boundary segmentation model evolution method for remote sensing images is shown.

[0044] Figure 2 A flowchart of a mask-guided precise boundary segmentation model evolution method for remote sensing images is shown. DETAILED DESCRIPTION

[0045] Exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the present application, and are not intended to limit the scope of the present application.

[0046] Example 1:

[0047] The present application uses a mask image to guide the latent diffusion model (LDM) to obtain finer features for the segmentation task. Traditional random diffusion processes condition on the original image by modeling the distribution of the mask image with Gaussian noise and using a U-Net to learn the denoising process. However, the randomness of the Gaussian noise can cause unnecessary disturbance to the U-Net features, making it less suitable for deterministic segmentation tasks. To alleviate this problem and better adapt to remote sensing image segmentation, the present application uses a deterministic diffusion process for mask image integration. The deterministic diffusion process directly maps the original image to the mask image, effectively reducing randomness. In addition, the number of time steps in the deterministic diffusion process is reduced to 1, further reducing randomness. As shown in Figure 1 and Figure 2 A mask-guided precise boundary segmentation model evolution method for remote sensing images, comprising the following steps:

[0048] S1. Obtain the original remote sensing image and construct the mask image according to the annotation information of the original remote sensing image;

[0049] S2. Construct the latent diffusion model module of the remote sensing image precise boundary segmentation model;

[0050] The potential diffusion model module is composed of a variational autoencoder and a U-Net;

[0051] The variational autoencoder is configured to receive the original remote sensing image and the mask image, and perform feature extraction on the original remote sensing image and the mask image respectively to obtain a potential representation of the original remote sensing image and a potential representation of the mask image.

[0052] The U-Net is configured to receive the potential representation of the original remote sensing image and the potential representation of the mask image, remove noise of the potential representation of the original remote sensing image and noise of the potential representation of the mask image, and obtain multi-level features of intermediate layers in L={3, 6, 9} layers of the U-Net.

[0053] S3. Based on the deterministic diffusion process, the mask image is used to guide the potential diffusion model module to learn a denoising process, and mask features are outputted;

[0054] S4. A Transformer prediction head module of the remote sensing image accurate boundary segmentation model is constructed.

[0055] S5. A loss function is constructed to optimize and train the remote sensing image accurate boundary segmentation model comprising the potential diffusion model module and the Transformer prediction head module, and evolution of the remote sensing image accurate boundary segmentation model is completed.

[0056] In the embodiment, the mask image is an RGB mask label image, which is generated from the annotation of the corresponding original image. Referring to Painter, the mask image distinguishes each instance by using different RGB colors, and the color of each instance is assigned based on the absolute position of the center of the instance in the image. The mask image is constructed as follows: given an original remote sensing image , the mask image is obtained from the annotation information corresponding to the original remote sensing image . Wherein, represents a real number field, represents the height of the original remote sensing image, represents the width of the original remote sensing image; the present application assigns a color to each instance mask center based on the absolute position of the center in the image. The present application conceptually divides the original remote sensing image into 16x20x20 blocks, corresponding to the RGB three channels respectively. Then a fixed color is assigned to each block, and if the center of the mask is located in the block, the corresponding coloring is performed and the RGB mask image is obtained.

[0057] In this embodiment, the latent diffusion model LDM is inspired by non-equilibrium thermodynamics and can effectively learn the data distribution of a given dataset. Specifically, the input data is first extracted by the VAE encoder to obtain a latent representation, and then gradually perturbed by noise added in a fixed Markov chain at fixed time steps until the data distribution approaches the standard normal distribution. Subsequently, the diffusion model is trained by the reverse process to learn the denoising process that reconstructs the original data distribution in the latent space from the standard normal distribution. The construction of the latent diffusion model module is as follows: the latent diffusion model in the present application mainly consists of a variational autoencoder (VAE) encoder and a U-Net. The input image is processed by the VAE encoder to extract a latent representation, which is then subjected to step noise perturbation and transmitted to the U-Net. The U-Net is trained to learn to remove noise from the latent features. During training, the parameters of the VAE model remain unchanged, and the parameters of the U-Net are optimized.

[0058] In this embodiment, remote sensing image segmentation is a deterministic task, but the randomness of the Gaussian noise in the diffusion model introduces uncertainty, resulting in excessive noise in the U-Net features. Therefore, the present application uses a deterministic diffusion process to combine the mask image. The deterministic diffusion process directly maps between the original image and the mask image, effectively reducing the apparent noise caused by random Gaussian noise. In addition, the present application reduces the time step to 1 to further reduce randomness. When the mask image is introduced, the deterministic diffusion process helps to obtain more stable and consistent U-Net features. Subsequently, the feature is transmitted to the Transformer prediction head module for class and mask prediction of individual objects.

[0059] Based on the deterministic diffusion process, the mask image is used to guide the latent diffusion model module to learn the denoising process as follows: in order to introduce the mask image to the latent diffusion model and better adapt to deterministic aerial segmentation, the present application uses a deterministic diffusion process to combine the mask image. The deterministic diffusion process directly maps between the original remote sensing image and the mask image, thereby avoiding the multi-step noise addition steps usually involved in the diffusion process.

[0060] The original remote sensing image and its corresponding mask image are paired as input. The original remote sensing image and the mask image are both processed by a variational autoencoder (VAE) encoder to obtain latent representations and respectively, where represents the number of channels of the latent representation, represents the height of the latent representation, represents the width of the latent representation. The deterministic diffusion process is defined as:

[0061]

[0062]

[0063] wherein, represents an objective function of a deterministic diffusion process, represents a mathematical expectation under a joint probability distribution, represents a latent representation of a mask image, represents a U-Net parameter to be learned, represents a latent representation containing noise at time step t, represents a time step, represents an L2 norm, and the purpose is to calculate the mean square error (MSE), represents a hyperparameter of noise scheduling, represents a latent representation of an original remote sensing image.

[0064] In this embodiment, in order to predict the segmentation of each instance, the application proposes a Transformer prediction head module. The purpose of the prediction head is to determine the class and bounding box of each object. The Transformer prediction head takes a learnable query as input to distinguish different objects. The construction of the Transformer prediction head module is as follows: the application first introduces a learnable query to distinguish different objects, and uses a learnable position embedding to indicate the spatial position, wherein represents the number of predicted instances. The Transformer prediction head module is composed of stacked layers, each layer containing a mask attention module, a self-attention module and a feed-forward network (FFN). In the first l layer, the mask attention module takes as the query, and and as the key and value, respectively. Among them, , and represent the vectors for the transformation of the query, key and value, represents the query of the mask attention unit of the th stacked layer, represents the multi-level features of the intermediate layer obtained from the U-Net. In order to focus attention on the object area rather than the entire feature map, the application also adds an attention mask . The self-attention module uses the output of the mask attention as the query, key and value, so as to realize the interaction between different objects. The fully connected network (FFN) is composed of two fully connected layers with a dimension of 2048.

[0065] Subsequently, the invention constructs a classification branch and a mask branch, wherein the classification branch predicts a score vector , which is used to predict the category; the mask branch fuses the mask features obtained from the U-Net to predict the mask .

[0066] In the present embodiment, the loss function is constructed as follows: the training loss function of the present invention consists of two parts: segmentation loss and diffusion loss. Therefore, the specific formula of the loss function is:

[0067]

[0068] wherein, denotes the loss function, denotes the first segmentation loss, denotes the diffusion loss, denotes the number of segmentation losses.

[0069] The segmentation loss includes a classification loss and a mask loss, the classification loss is a cross-entropy loss, and the mask loss consists of a binary cross-entropy loss and a Dice loss; the specific formula of the segmentation loss is:

[0070]

[0071] wherein, denotes the classification loss, denotes the weight factor of the classification loss, denotes the score vector predicted by the Transformer prediction head module, denotes the true category label of the target in the given image, denotes the binary cross-entropy loss, denotes the weight factor of the binary cross-entropy loss, denotes the mask predicted by the Transformer prediction head module, denotes the mask of the target in the given image, denotes the Dice loss, denotes the weight factor of the Dice loss.

[0072] The diffusion loss is an MSE loss, and the specific formula is:

[0073]

[0074] wherein, denotes the latent representation of the mask image, denotes the U-Net parameters to be learned, denotes the latent representation containing noise at time step t.

[0075] For the problem of precise segmentation of remote sensing images, the application proposes a mask-guided diffusion model segmentation architecture. The architecture captures more accurate structural information by guiding the LDM with a mask image, which is beneficial for boundary prediction. The mask image is an RGB image generated from the corresponding label of the input image, in which each instance is assigned a different color to ensure clear separation. Compared with text guidance, which mainly points to semantic features, mask guidance provides more fine and accurate key content and layout features, making it more effective in precise contour prediction.

[0076] Embodiment 2:

[0077] Based on embodiment 1, the application embodiment conducts experiments on a widely used remote sensing instance segmentation dataset, namely NWPU, to illustrate the effect of the method proposed by the application.

[0078] The NWPU dataset is a widely used remote sensing instance segmentation dataset, containing ten categories: aircraft, ship, storage tank, baseball field, tennis court, basketball court, track and field, port, bridge and vehicle. The dataset contains high spatial resolution images collected from Google Earth, with a resolution range of 0.5 meters to 2 meters. The experimental results are shown in Table 1. It can be obtained that the method proposed by the application achieves 71.5 and 70.4 , surpassing the performance of current advanced remote sensing image instance segmentation methods.

[0079] Table 1 Comparison results of the application and the latest method on the remote sensing instance segmentation task

[0080]

[0081] Embodiment 3:

[0082] Based on embodiment 1, the application embodiment provides a mask-guided remote sensing image precise boundary segmentation model evolution system, which can be used to implement the mask-guided remote sensing image precise boundary segmentation model evolution method as described in the preceding embodiments. The system comprises:

[0083] A mask image construction module is used to obtain an original remote sensing image and construct a mask image according to the label information of the original remote sensing image.

[0084] A latent diffusion model construction module is used to construct a latent diffusion model module of the remote sensing image precise boundary segmentation model.

[0085] A denoising learning module is used to guide the latent diffusion model module to learn a denoising process based on a deterministic diffusion process using the mask image, and output mask features.

[0086] a prediction head construction module, configured to construct a Transformer prediction head module of the remote sensing image accurate boundary segmentation model;

[0087] a loss function construction module, configured to construct a loss function to optimize and train the remote sensing image accurate boundary segmentation model including the latent diffusion model module and the Transformer prediction head module, and complete evolution of the remote sensing image accurate boundary segmentation model.

[0088] According to the embodiments of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0089] In an example embodiment, the electronic device comprises at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the mask-guided remote sensing image accurate boundary segmentation model evolution method as described in Embodiment 1 above.

[0090] In an example embodiment, the readable storage medium can be a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the mask-guided remote sensing image accurate boundary segmentation model evolution method according to Embodiment 1 above.

[0091] In an example embodiment, the computer program product comprises a computer program which, when executed by a processor, implements the mask-guided remote sensing image accurate boundary segmentation model evolution method according to Embodiment 1 above.

[0092] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, so that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program codes can be executed entirely on a machine, partially on a machine, partially on a machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage medium would include one or more lines of electrical wire, portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0095] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0096] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0097] Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and understanding of the principles of the application and should not be construed as limiting the scope of the application to such specifically enumerated embodiments. Various other specific adaptations and combinations of features can be made in accordance with the teachings of the application without departing from the spirit thereof and these adaptations and combinations are intended to fall within the scope of the application.

Claims

1. A mask-guided remote sensing image accurate boundary segmentation model evolution method, characterized in that, The method comprises the following steps: obtaining an original remote sensing image, and constructing a mask image according to the annotation information of the original remote sensing image; constructing a latent diffusion model module of a remote sensing image accurate boundary segmentation model; based on a deterministic diffusion process, using the mask image to guide the latent diffusion model module to learn a denoising process, and outputting mask features; constructing a Transformer prediction head module of the remote sensing image accurate boundary segmentation model; constructing a loss function to optimize and train the remote sensing image accurate boundary segmentation model comprising the latent diffusion model module and the Transformer prediction head module, and completing the evolution of the remote sensing image accurate boundary segmentation model; The mask image is constructed according to the annotation information of the original remote sensing image, specifically: dividing the original remote sensing image into multiple blocks, each instance being distinguished by a unique RGB color, assigning a fixed color to each block based on the absolute position of the instance mask center in the original remote sensing image, and obtaining an RGB mask image; The latent diffusion model module is composed of a variational autoencoder and a U-Net; The variational autoencoder is used to receive the original remote sensing image and the mask image, and respectively extract features from the original remote sensing image and the mask image to obtain latent representations of the original remote sensing image and the mask image; The U-Net is used to receive the latent representations of the original remote sensing image and the mask image, remove the noise of the latent representations of the original remote sensing image and the noise of the latent representations of the mask image, and obtain multi-level features of intermediate layers in the L={3, 6, 9} layers of the U-Net.

2. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that, The deterministic diffusion process reduces the noise caused by random Gaussian noise by directly mapping between the original remote sensing image and the mask image; the deterministic diffusion process is specifically: in, The objective function represents the deterministic diffusion process. Denotes the mathematical expectation under the joint probability distribution. This represents the latent representation of the masked image. This indicates the U-Net parameters that need to be learned. Indicates at time step Potential characterization of noise at that time This represents the L2 norm used to calculate the mean squared error. The hyperparameters representing noise scheduling, This represents the potential representation of the original remote sensing image.

3. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that, The Transformer prediction head module includes a classification branch and a mask branch; The classification branch is used to predict a score vector The score vector is used to predict a class; The mask branch is used to fuse the latent representation obtained from the U-Net of the latent diffusion model module and to predict the resulting mask .

4. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 3, characterized in that, The Transformer prediction head module is composed of L stacked layers, each stacked layer containing a mask attention unit, a self-attention unit and a feedforward network connected in turn; The mask attention unit is used to limit attention to the instance region; The self-attention unit is used to realize the interaction between instances; The feedforward network is used to introduce a nonlinear operation to abstract and strengthen the features extracted by the attention layer.

5. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 4, characterized in that, Transformer prediction head module with learnable queries Differentiating between objects, with learnable positional embeddings indicating spatial positions; wherein, denotes the real number domain, denotes the number of predicted instances, denotes the dimension of the features; In the jth stacked layer, the mask attention unit takes as the query, takes as the key, and takes as the value, where , and denote vectors for the transformation of the query, the key, and the value, denotes the query of the jth stacked layer mask attention unit, denotes the multi-level feature of the intermediate layer obtained from the U-Net; The self-attention unit takes the output of the mask attention unit as the query, key and value, thereby realizing the interaction between different objects.

6. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that, The loss function includes a segmentation loss and a diffusion loss, and the specific formula is: wherein, represents a loss function, represents a first segmentation loss, represents a diffusion loss, represents a number of segmentation losses.

7. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 6, characterized in that, The segmentation loss includes a classification loss and a mask loss, the classification loss is a cross-entropy loss, and the mask loss is composed of a binary cross-entropy loss and a Dice loss; the specific formula of the segmentation loss is: wherein, denotes a classification loss, denotes a weight factor for the classification loss, denotes a score vector predicted by the Transformer prediction head module, denotes a true class label of the object in the given image, denotes a binary cross-entropy loss, denotes a weight factor for the binary cross-entropy loss, denotes a mask predicted by the Transformer prediction head module, denotes a mask of the object in the given image, denotes a Dice loss, denotes a weight factor for the Dice loss.

8. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 7, characterized in that, The diffusion loss is an MSE loss, and the specific formula is: wherein, represents a latent representation of a mask image, represents the U-Net parameters to be learned, represents the latent representation containing noise at time step t.

Images