Mask-guided remote sensing image precise boundary segmentation model evolution method

Abstract

The invention belongs to the technical field of mode recognition, and particularly discloses a mask-guided remote sensing image accurate boundary segmentation model evolution method, which comprises the following steps of: acquiring an original remote sensing image, and constructing a mask image according to annotation information of the original remote sensing image; constructing a potential diffusion model module of a remote sensing image accurate boundary segmentation model; based on a deterministic diffusion process, using the mask image to guide the potential diffusion model module to learn a denoising process, and outputting mask features; constructing a Transform prediction head module of the accurate boundary segmentation model of the remote sensing image; and constructing a loss function to carry out optimization training on the remote sensing image precise boundary segmentation model, and completing evolution of the remote sensing image precise boundary segmentation model. According to the method, the problem that the randomness of Gaussian noise in the traditional random diffusion process brings unnecessary disturbance to the U-Net feature, so that the U-Net feature is not suitable for a deterministic segmentation task is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of pattern recognition, and in particular relates to a mask-guided remote sensing image precise boundary segmentation model evolution method. Background Art

[0002] Remote sensing instance segmentation is a fundamental task in remote sensing, with widespread applications in areas such as building extraction and land surveying. This task requires the precise identification of instances, including their corresponding categories and masks. A key challenge in remote sensing image segmentation is accurately delineating object boundaries. However, the aerial perspective of remote sensing images often complicates the extraction of fine-grained boundary contours.

[0003] Existing methods can be roughly divided into two main strategies. The first strategy focuses on designing specialized modules to enhance feature extraction, thereby generating smoother boundaries and reducing jagged artifacts. The second strategy uses edge-aware supervision to promote clearer and more precise boundary delineation while mitigating blurring effects. However, these methods mainly rely on traditional convolutional neural networks (CNNs) trained from scratch on specific datasets, which do not fully utilize the prior knowledge embedded in the base model. Compared with a single convolutional neural network trained on a specific dataset, the base model utilizes a large and powerful visual transformer trained on massive datasets, which can capture rich visual knowledge.

[0004] The Latent Diffusion Model (LDM) has shown great promise in image generation tasks. Training on the larger LAION-5B dataset enables LDM to incorporate a wider range of prior knowledge. Furthermore, LDM is specifically designed for image generation, enabling it to capture more precise context and detailed information, enhancing its ability to generate reliable, high-quality images. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that the randomness of Gaussian noise in the traditional random diffusion process brings unnecessary disturbances to the U-Net features, making it unsuitable for deterministic segmentation tasks. A mask-guided evolutionary method for precise boundary segmentation models of remote sensing images is proposed.

[0006] The technical solution of the present invention is: a mask-guided remote sensing image precise boundary segmentation model evolution method, comprising the following steps: Obtain the original remote sensing image and construct a mask image based on the annotation information of the original remote sensing image; A latent diffusion model module for building accurate boundary segmentation models for remote sensing images; Based on the deterministic diffusion process, the mask image is used to guide the latent diffusion model module to learn the denoising process and output the mask features; Build a Transformer prediction head module for accurate boundary segmentation models of remote sensing images; A loss function is constructed to optimize the training of the remote sensing image precise boundary segmentation model including the latent diffusion model module and the Transformer prediction head module, completing the evolution of the remote sensing image precise boundary segmentation model.

[0007] Preferably, the mask image is constructed according to the annotation information of the original remote sensing image, specifically: The original remote sensing image is divided into multiple blocks, each instance is distinguished by a unique RGB color, and a fixed color is assigned to each block based on the absolute position of the instance mask center in the remote sensing image and the original remote sensing image to obtain an RGB mask image.

[0008] Preferably, the potential diffusion model module is composed of a variational autoencoder and a U-Net; The variational autoencoder is used to receive the original remote sensing image and the mask image, and perform feature extraction on the original remote sensing image and the mask image respectively to obtain a potential representation of the original remote sensing image and a potential representation of the mask image; The U-Net is used to receive the latent representation of the original remote sensing image and the latent representation of the mask image, remove noise from the latent representation of the original remote sensing image and the latent representation of the mask image, and obtain multi-level features of the intermediate layer in the L={3,6,9} layer of the U-Net.

[0009] Preferably, the deterministic diffusion process reduces the noise caused by random Gaussian noise by directly mapping between the original remote sensing image and the mask image; the deterministic diffusion process is specifically as follows:

[0010]

[0011] in, represents the objective function of the deterministic diffusion process, represents the mathematical expectation under the joint probability distribution, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, Indicates that at time step The potential representation of noise when represents the L2 norm used to calculate the mean square error, represents the hyperparameters of the noise schedule, Represents the latent representation of the original remote sensing image.

[0012] Preferably, the Transformer prediction head module includes a classification branch and a mask branch; The classification branch is used to predict the score vector , the score vector is used to predict the category; The mask branch is used to fuse the latent representations obtained from the U-Net of the latent diffusion model module and predict the mask .

[0013] Preferably, the Transformer prediction head module consists of L stacked layers, each stacked layer comprising a mask attention unit, a self-attention unit, and a feedforward network connected in sequence; The mask attention unit is used to limit attention to the instance area; The self-attention unit is used to realize interaction between instances; The feedforward network is used to introduce nonlinear operations to abstract and enhance the features extracted by the attention layer.

[0014] As an option, the Transformer prediction head module is based on a learnable query Distinguish different objects and indicate spatial locations with learnable position embeddings; where represents the field of real numbers, represents the number of prediction instances, Represents the dimension of the feature; In the stacked layers, the masked attention unit is As a query, As a key, As values, where 、 and vectors representing transformations for queries, keys, and values, Indicates the A query of a stacked layer of masked attention units, Represents the multi-level features of the intermediate layer obtained from U-Net; The self-attention unit uses the output of the masked attention unit as query, key, and value, thereby enabling interactions between different objects.

[0015] Preferably, the loss function includes segmentation loss and diffusion loss, and the specific formula is:

[0016] in, represents the loss function, Indicates the segmentation loss, represents the diffusion loss, Represents the amount of segmentation loss.

[0017] Preferably, the segmentation loss includes classification loss and mask loss, the classification loss is cross entropy loss, and the mask loss is composed of binary cross entropy loss and Dice loss; the specific formula of the segmentation loss is:

[0018] in, represents the classification loss, represents the weight factor of classification loss, represents the score vector predicted by the Transformer prediction head module, represents the true category label of the target in a given image, represents the binary cross entropy loss, represents the weight factor of binary cross entropy loss, represents the mask predicted by the Transformer prediction head module, represents the mask of the object in a given image, represents the Dice loss, Represents the weight factor of Dice loss.

[0019] Preferably, the diffusion loss is MSE loss, and the specific formula is:

[0020] in, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, Indicates that at time step The potential representation of noise when .

[0021] The beneficial effects of the present invention are: 1. This paper proposes a mask-guided diffusion model segmentation architecture for accurate instance segmentation in remote sensing images. By introducing an RGB mask image to guide the denoising process of the latent diffusion model, it can learn more detailed and precise key content and layout features, thereby improving the accuracy of boundary contour prediction.

[0022] 2. This paper adopts a deterministic diffusion process when integrating the mask image. This approach reduces the inherent randomness of traditional diffusion models, making the feature representation in U-Net more precise and better suited for aerial segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Shown is a flowchart of a mask-guided evolutionary method for accurate boundary segmentation models of remote sensing images.

[0024] Figure 2Shown is a flowchart of a mask-guided evolutionary method for accurate boundary segmentation models of remote sensing images. DETAILED DESCRIPTION

[0025] The exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the accompanying drawings are merely exemplary and are intended to illustrate the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0026] Example 1: The present invention utilizes the mask image to guide the latent diffusion model (LDM) to obtain more refined features for segmentation tasks. The traditional random diffusion process takes the original image as a condition, introduces Gaussian noise to model the distribution of the mask image, and uses U-Net to learn the denoising process. However, the randomness of Gaussian noise will bring unnecessary disturbances to the U-Net features, making it less suitable for deterministic segmentation tasks. In order to alleviate this problem and better adapt to remote sensing image segmentation, the present invention adopts a deterministic diffusion process for mask image integration. The deterministic diffusion process directly maps the original image to the mask image, thereby effectively reducing the randomness. In addition, the number of time steps in the deterministic diffusion process is reduced to 1, which further reduces the randomness. As Figure 1 and Figure 2 As shown, a mask-guided remote sensing image accurate boundary segmentation model evolution method includes the following steps: S1. Obtain an original remote sensing image and construct a mask image based on the annotation information of the original remote sensing image; S2. Build a latent diffusion model module for accurate boundary segmentation of remote sensing images; The potential diffusion model module is composed of a variational autoencoder and a U-Net; The variational autoencoder is used to receive the original remote sensing image and the mask image, and perform feature extraction on the original remote sensing image and the mask image respectively to obtain a potential representation of the original remote sensing image and a potential representation of the mask image; The U-Net is used to receive the latent representation of the original remote sensing image and the latent representation of the mask image, remove noise from the latent representation of the original remote sensing image and the latent representation of the mask image, and obtain multi-level features of the intermediate layer in the L={3,6,9} layer of the U-Net.

[0027] S3. Based on the deterministic diffusion process, the mask image is used to guide the latent diffusion model module to learn the denoising process and output the mask features. S4. Build a Transformer prediction head module for accurate boundary segmentation of remote sensing images. S5. Construct a loss function to optimize the training of the remote sensing image precise boundary segmentation model including the latent diffusion model module and the Transformer prediction head module to complete the evolution of the remote sensing image precise boundary segmentation model.

[0028] In this embodiment, the mask image is an RGB mask label image generated by the annotation of its corresponding original image. Referring to Painter, the mask image uses different RGB colors to distinguish each instance, and the color of each instance is assigned based on the absolute position of its center in the image. Constructing the mask image is specifically as follows: Given the original remote sensing image The present invention obtains the mask image through the annotation information corresponding to the original remote sensing image .in, represents the field of real numbers, Indicates the height of the original remote sensing image, Represents the width of the original remote sensing image. The present invention assigns a color to each instance mask based on its absolute position within the image. Conceptually, the original remote sensing image is divided into 16×20×20 blocks, corresponding to the three RGB channels. Each block is then assigned a fixed color. If the center of the mask falls within the block, it is colored accordingly, yielding an RGB mask image.

[0029] In this embodiment, the latent diffusion model LDM is inspired by non-equilibrium thermodynamics and can effectively learn the data distribution of a given data set. Specifically, the input data is first subjected to a VAE encoder to extract a latent representation, and then is gradually perturbed, where noise is gradually added in a fixed Markov chain within a fixed time step until the data distribution approaches a standard normal distribution. Subsequently, the diffusion model is trained through an inverse process to learn a denoising process that reconstructs the distribution of the original data in the latent space from the standard normal distribution. The latent diffusion model module is specifically constructed as follows: the latent diffusion model in the present invention is mainly composed of a variational autoencoder (VAE) encoder and a U-Net. The input image is subjected to a VAE encoder to extract a latent representation, and then the latent representation is subjected to a VAE encoder. The noise perturbation is then fed into the U-Net. The U-Net is trained to remove the noise from the latent features. During training, the parameters of the VAE model remain unchanged, while the parameters of the U-Net are optimized.

[0030] In this embodiment, remote sensing image segmentation is a deterministic task. However, the randomness of Gaussian noise in the diffusion model introduces uncertainty, resulting in excessive noise in the U-Net features. Therefore, the present invention utilizes a deterministic diffusion process to incorporate a mask image. This deterministic diffusion process directly maps between the original image and the mask image, effectively mitigating the noticeable noise caused by random Gaussian noise. Furthermore, the present invention reduces the time step to 1 to further reduce randomness. When the mask image is introduced, the deterministic diffusion process helps to obtain more stable and consistent U-Net features. This feature is then passed to the Transformer prediction head module for category and mask prediction for individual objects.

[0031] Based on a deterministic diffusion process, the mask image is used to guide the latent diffusion model module in learning the denoising process. Specifically, to introduce the mask image into the latent diffusion model and better adapt it to deterministic aerial segmentation, the present invention utilizes a deterministic diffusion process to incorporate the mask image. The deterministic diffusion process directly maps between the original remote sensing image and the mask image, thus avoiding the multiple noise addition steps typically involved in the diffusion process.

[0032] Original remote sensing image and its corresponding mask image are paired as input. Both the original remote sensing image and the mask image are processed by the variational autoencoder (VAE) encoder to obtain the potential representation and ,in, represents the number of channels of the latent representation, represents the height of the potential representation, represents the width of the potential representation. The deterministic diffusion process is defined as:

[0033]

[0034] in, represents the objective function of the deterministic diffusion process, represents the mathematical expectation under the joint probability distribution, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, represents the potential representation containing noise at time step t, represents the time step, Represents the L2 norm, the purpose is to calculate the mean square error (MSE), represents the hyperparameters of the noise schedule, Represents the latent representation of the original remote sensing image.

[0035] In this embodiment, in order to predict the segmentation of each instance, the present invention proposes a Transformer prediction head module. The goal of the prediction head is to determine the category and bounding box of each object. The Transformer prediction head takes a learnable query as input to distinguish different objects. The specific steps of constructing the Transformer prediction head module are as follows: the present invention first introduces a learnable query to distinguish different objects, while using learnable position embedding to indicate the spatial location, where Represents the number of predicted instances. The Transformer prediction head module consists of Each layer consists of a mask attention module, a self-attention module and a feed-forward network (FFN). l In the layer, the mask attention module is As a query, and As keys and values ​​respectively. 、 and vectors representing transformations for queries, keys, and values, Indicates the A query of a stacked layer of masked attention units, In order to focus on the object area rather than the entire feature map, the present invention also adds an attention mask The self-attention module uses the output of masked attention as query, key, and value, thus enabling interactions between different objects. The fully connected network (FFN) consists of two fully connected layers with a dimension of 2048.

[0036] Subsequently, the invention constructs a classification branch and a mask branch, wherein the classification branch predicts the score vector , which is used to predict the category; the mask branch fuses the mask features obtained from U-Net to predict the mask .

[0037] In this embodiment, the loss function is constructed as follows: The training loss function of the present invention consists of two parts: segmentation loss and diffusion loss. Therefore, the specific formula of the loss function is:

[0038] in, represents the loss function, Indicates the segmentation loss, represents the diffusion loss, Represents the amount of segmentation loss.

[0039] The segmentation loss includes classification loss and mask loss. The classification loss is cross entropy loss, and the mask loss is composed of binary cross entropy loss and Dice loss. The specific formula of segmentation loss is:

[0040] in, represents the classification loss, represents the weight factor of classification loss, represents the score vector predicted by the Transformer prediction head module, represents the true category label of the target in a given image, represents the binary cross entropy loss, represents the weight factor of binary cross entropy loss, represents the mask predicted by the Transformer prediction head module, represents the mask of the object in a given image, represents the Dice loss, Represents the weight factor of Dice loss.

[0041] The diffusion loss is the MSE loss, and the specific formula is:

[0042] in, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, represents the latent representation containing noise at time step t.

[0043] To address the problem of accurate segmentation of remote sensing images, this paper proposes a mask-guided diffusion model segmentation architecture. This architecture uses a mask image to guide the LDM, capturing more precise structural information and facilitating boundary prediction. The mask image is an RGB image generated from the corresponding annotations of the input image, where each instance is assigned a different color to ensure clear separation. Compared to text guidance, which primarily focuses on semantic features, mask guidance provides more refined and accurate key content and layout features, making it more effective in accurately predicting contours.

[0044] Example 2: Based on Example 1, the present invention conducts experiments on a widely used remote sensing instance segmentation dataset, namely, NWPU, to illustrate the effect of the method proposed in the present invention.

[0045] The NWPU dataset is a widely used remote sensing instance segmentation dataset that contains ten categories: aircraft, ships, storage tanks, baseball fields, tennis courts, basketball courts, track and field fields, ports, bridges, and vehicles. The dataset contains high-spatial resolution images collected from Google Earth with a resolution range of 0.5 meters to 2 meters. The experimental results are shown in Table 1. It can be seen that the proposed method achieves 71.5% accuracy in the NWPU dataset. and 70.4 , surpassing the performance of current state-of-the-art remote sensing image instance segmentation methods.

[0046] Table 1 Comparison results of the present invention and the latest methods on remote sensing instance segmentation tasks

[0047] Example 3: Based on Example 1, this embodiment of the present invention provides a mask-guided remote sensing image precise boundary segmentation model evolution system, which can be used to implement the mask-guided remote sensing image precise boundary segmentation model evolution method as described in the aforementioned embodiment. The system includes: A mask image construction module is used to obtain the original remote sensing image and construct a mask image based on the annotation information of the original remote sensing image; Latent diffusion model construction module, which is used to construct a latent diffusion model module for accurate boundary segmentation model of remote sensing images; The denoising learning module is used to guide the latent diffusion model module to learn the denoising process based on the deterministic diffusion process using the mask image and output the mask features; Prediction head building module, which is used to build the Transformer prediction head module of the remote sensing image precise boundary segmentation model; The loss function construction module is used to construct a loss function to optimize the training of the remote sensing image precise boundary segmentation model including the latent diffusion model module and the Transformer prediction head module, and complete the evolution of the remote sensing image precise boundary segmentation model.

[0048] According to an embodiment of the present invention, the present invention further provides an electronic device, a readable storage medium, and a computer program product.

[0049] In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the mask-guided remote sensing image precise boundary segmentation model evolution method as described in Example 1 above.

[0050] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the mask-guided remote sensing image precise boundary segmentation model evolution method according to the above embodiment 1.

[0051] In an exemplary embodiment, a computer program product includes a computer program, which, when executed by a processor, implements the mask-guided remote sensing image precise boundary segmentation model evolution method according to the above embodiment 1.

[0052] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0053] In the context of the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0054] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0055] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0056] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0057] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A mask-guided remote sensing image accurate boundary segmentation model evolution method, characterized by: The following steps are involved: Obtain the original remote sensing image and construct a mask image based on the annotation information of the original remote sensing image; A latent diffusion model module for building accurate boundary segmentation models for remote sensing images; Based on the deterministic diffusion process, the mask image is used to guide the latent diffusion model module to learn the denoising process and output the mask features; Build a Transformer prediction head module for accurate boundary segmentation models of remote sensing images; A loss function is constructed to optimize the training of the remote sensing image precise boundary segmentation model including the latent diffusion model module and the Transformer prediction head module, completing the evolution of the remote sensing image precise boundary segmentation model.

2. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that: The mask image is constructed according to the annotation information of the original remote sensing image, specifically: The original remote sensing image is divided into multiple blocks, each instance is distinguished by a unique RGB color, and a fixed color is assigned to each block based on the absolute position of the instance mask center in the remote sensing image and the original remote sensing image to obtain an RGB mask image.

3. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that: The potential diffusion model module is composed of a variational autoencoder and a U-Net; The variational autoencoder is used to receive the original remote sensing image and the mask image, and perform feature extraction on the original remote sensing image and the mask image respectively to obtain a potential representation of the original remote sensing image and a potential representation of the mask image; The U-Net is used to receive the latent representation of the original remote sensing image and the latent representation of the mask image, remove noise from the latent representation of the original remote sensing image and the latent representation of the mask image, and obtain multi-level features of the intermediate layer in the L={3,6,9} layer of the U-Net.

4. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 3, characterized in that: The deterministic diffusion process reduces the noise caused by random Gaussian noise by directly mapping between the original remote sensing image and the mask image. The deterministic diffusion process is specifically as follows: in, represents the objective function of the deterministic diffusion process, represents the mathematical expectation under the joint probability distribution, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, Indicates that at time step The potential representation of noise when represents the L2 norm used to calculate the mean square error, represents the hyperparameters of the noise schedule, Represents the latent representation of the original remote sensing image.

5. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 3, characterized in that: The Transformer prediction head module includes a classification branch and a mask branch; The classification branch is used to predict the score vector , the score vector is used to predict the category; The mask branch is used to fuse the latent representations obtained from the U-Net of the latent diffusion model module and predict the mask .

6. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 5, characterized in that: The Transformer prediction head module consists of L stacked layers, each of which contains a mask attention unit, a self-attention unit, and a feedforward network connected in sequence; The mask attention unit is used to limit attention to the instance area; The self-attention unit is used to realize interaction between instances; The feedforward network is used to introduce nonlinear operations to abstract and enhance the features extracted by the attention layer.

7. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 6, characterized in that: Transformer prediction head module with learnable query Distinguish different objects and indicate spatial locations with learnable position embeddings; where represents the field of real numbers, represents the number of prediction instances, Represents the dimension of the feature; In the stacked layers, the masked attention unit is As a query, As a key, As values, where 、 and vectors representing transformations for queries, keys, and values, Indicates the A query of a stacked layer of masked attention units, Represents the multi-level features of the intermediate layer obtained from U-Net; The self-attention unit uses the output of the masked attention unit as query, key, and value, thereby enabling interactions between different objects.

8. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 1, characterized in that: The loss function includes segmentation loss and diffusion loss, and the specific formula is: in, represents the loss function, Indicates the segmentation loss, represents the diffusion loss, Represents the amount of segmentation loss.

9. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 8, characterized in that: The segmentation loss includes classification loss and mask loss. The classification loss is cross entropy loss, and the mask loss is composed of binary cross entropy loss and Dice loss. The specific formula of segmentation loss is: in, represents the classification loss, represents the weight factor of classification loss, represents the score vector predicted by the Transformer prediction head module, represents the true category label of the target in a given image, represents the binary cross entropy loss, represents the weight factor of binary cross entropy loss, represents the mask predicted by the Transformer prediction head module, represents the mask of the object in a given image, represents the Dice loss, Represents the weight factor of Dice loss.

10. The mask-guided remote sensing image precise boundary segmentation model evolution method according to claim 8, characterized in that: The diffusion loss is the MSE loss, and the specific formula is: in, represents the latent representation of the mask image, Indicates the U-Net parameters that need to be learned, Indicates that at time step The potential representation of noise when .

Images