Thangka image segmentation method based on edge feature guidance and detail feature denoising

Abstract

The invention provides a Thangka image segmentation method based on edge feature guidance and detail feature denoising, relates to the technical field of computer vision and image processing, and introduces a detail feature attention enhancement module based on a U-Net network. Training a detail feature attention enhancement module by adding random noise and performing reverse denoising; secondly, the trained module is applied to a feature map of the Thangka image, Gaussian noise is added for iterative denoising, and a real sample is generated; thirdly, in combination with multi-head self-attention of different window sizes and global average pooling, constructing a window multi-head self-attention pooling pyramid to capture multi-scale feature information in a global context; and finally, through introduction of fusion interaction of edge features and semantic information, feature layer information output of different scales is optimized by using DW convolution, and an edge feature extraction effect is improved through an edge loss function. According to the method, the de-noising and segmentation effects on the Thangka image are effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and image processing, and in particular to a thangka image segmentation method based on edge feature guidance and detail feature denoising. Background Art

[0002] Thangka image segmentation technology, as an important innovation in the field of cultural protection, records the structural layout and complex patterns of Thangka works through meticulous image segmentation processing. Image noise removal and detail feature enhancement are key technologies to improve image quality and resolution. In recent years, research on Thangka images has mainly focused on damaged area detection, and only a few scholars have focused on semantic segmentation of Thangka elements. There are also various problems with semantic segmentation, including high image blur in the dataset, which leads to unclear segmentation of the headdress part of the Thangka image, better extraction of the Buddha part in the Thangka image, but the number of categories in the segmentation result is small, resulting in failure to capture other fine-grained elements in the Thangka image, and the use of multi-scale fusion and attention mechanism to achieve accurate extraction of Thangka elements, but the division of each element is not accurate enough.

[0003] To solve the above problems, a thangka image segmentation method based on edge feature guidance and detail feature denoising is proposed. This method is different from the traditional single processing mode that only relies on segmentation labels. By introducing a carefully crafted edge feature map as a benchmark for edge guidance, it can accurately capture and segment thangka image information.

[0004] In the prior art, publication number CN116229104A discloses a salient target detection method guided by edge features, which extracts edge data of salient objects from a true value map of salient objects, and uses the edge data as an additional preliminary training set; uses a basic network to extract image features, and based on the image features, obtains edge features and multi-scale position features of salient objects respectively; based on a multi-layer perceptron, dynamically learns the relationship between edge features and multi-scale position features to obtain edge enhancement fusion features; sends the edge enhancement fusion features to a convolutional network for refinement processing, and obtains the final segmentation result based on the refined edge enhancement fusion features.

[0005] The main problems with the above methods are: the dynamic learning of the relationship between edge features and multi-scale position features by the multi-layer perceptron may introduce too much model complexity and easily lead to overfitting problems on small sample data sets; the above methods mainly focus on the extraction of edge and position features, ignoring the contextual information in the image, which may lead to information loss or feature redundancy, and lack the ability to understand complex backgrounds and object interactions, affecting the final segmentation results.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a thangka image segmentation method based on edge feature guidance and detail feature denoising to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions: A thangka image segmentation method based on edge feature guidance and detail feature denoising, the specific steps include: Step 1: Collect thangka images according to the set time step, and perform detail feature enhancement and denoising of thangka images based on the U-Net network. Introduce a detail feature enhancement attention module into the U-Net network structure, use the collected thangka images and encoder as the input of the detail feature enhancement attention module, add random noise, and train the detail feature enhancement attention module; Step 2: Input the feature map of the thangka image into the trained detail feature enhancement attention module. In the forward process, add Gaussian noise to the feature map to obtain the noisy feature map. In the reverse process, iterative denoising is performed on the noisy feature map and the target result to generate a real sample. Step 3: Input the processed feature map, combine the window multi-head self-attention of different window sizes with the global average pooling, and generate a window multi-head self-attention pooling pyramid to capture feature information of different scales in the global context information; Step 4: Fuse the edge features and semantic information of features of different scales, introduce feature layer information of different scales, perform average pooling and maximum pooling on the feature layer information respectively, and splice them to generate the output of feature layer information of different scales. Fuse the output of feature layer information through DW convolution to generate edge features, and optimize the extraction of edge features through edge loss function.

[0009] Furthermore, the principle for training the detail feature enhanced attention module is: ; ; ; ; in, ,express is a feature map with C channels, H height, and W width, and , is The feature map obtained by 3×3 full-dimensional dynamic convolution operation, Represents the feature map To adjust the size, It means to rearrange the dimensions of the resized feature map, including exchanging the order of the number of channels, height and width of the image. represents matrix multiplication, represents the feature map of enhanced detail features obtained by matrix multiplication, , represents a feature map with 2C channels, H height, and W width. Indicates that and After concat, the feature map is obtained by 3×3 full-dimensional dynamic convolution operation. represents the addition of elements, represents a feature map integrating different feature information obtained by element addition, represents the feature map combining detail features and semantic information obtained by matrix multiplication, Indicates normalizing the value of the feature map. Indicates the convolution operation on the normalized feature map. Represents the feature map of the final output after detail feature enhancement and convolution processing.

[0010] Furthermore, the principle of generating real samples by enhancing the attention module through detail features is based on: The forward process can be expressed as: ; in, represents the conditional probability distribution of the forward diffusion process, represents the feature map after noise addition at time step t, represents the feature map at time step t-1, represents the index of the time step, and , represents the maximum value of the time step, represents the variance of the noise at time step t, that is, the noise intensity added at time step t, represents a Gaussian distribution, and the mean of the Gaussian distribution is , the variance is , represents the covariance matrix of the noise, represents the result of the forward process; ; ; ; in, represents the diffusion probability corresponding to the step length t, represents the cumulative product of all diffusion probabilities from time step 1-t, represents the original thangka image without noise, represents Gaussian noise; According to the feature map after adding noise and the target result, iterative denoising is performed to generate real samples. As The approximation of denoising can be expressed as: ; in, represents the conditional probability distribution of the reverse denoising process, represents the learning parameters of the model, represents the back diffusion variable, represents the noise variance of the model parameterization; The output after production detail feature enhancement is based on the formula: ; in, Represents the predicted value of the original thangka image; The loss function is used to calculate the module loss based on the following principle: ; in, represents the mean square error loss function, Represents the joint expectation of three variables: time step, original thangka image and random noise.

[0011] Furthermore, the principle for generating the window multi-head self-attention pooling pyramid is: The feature map is divided into multiple windows, and the feature representation is performed on each window. The formula is: ; in, Indicates The feature representation of the window is represents a window of size M×M and number of channels C; The self-attention of the generated window is based on the formula: ; ; ; ; ; in, represents the query vector, represents the query weight matrix, represents the key vector, represents the key weight matrix, represents a value vector, represents the value weight matrix, represents the attention weight, represents the transpose of the key vector, represents the dimension of the key, and , Indicates The self-attention output of the window; The calculation of each attention head is: ; The output of multi-head self-attention is: ; in, Indicates The output of an attention head is Respectively represent The query weight matrix, key weight matrix and value weight matrix corresponding to the attention heads are Display Window The multi-head self-attention output, The learned weight matrix representing the output; For the multi-head self-attention output features of each window, global average pooling is performed based on the formula: ; in, Indicates The feature map output by multi-head self-attention of the window, Indicates The result of global average pooling of the feature maps output by multi-head self-attention of windows. Indicates After the self-attention process, the window The vector of position, Represents the row and column indices respectively; The input feature map is processed at multiple scales by different scaling factors, and the above is repeated for each scale. and Calculate to generate feature vectors of each scale, concatenate feature vectors of multiple scales to obtain the final multi-scale feature representation, based on the following formula: ; in, represents multi-scale feature representation, Indicates from the scale The generated feature vector, Represents the scaling factor.

[0012] Furthermore, the principle of generating edge features and optimizing the extraction of edge features through edge loss function is as follows: Assume that the input feature layer information of different scales are , the formula for splicing and generating the output of feature layer information of different scales is based on: ; ; in, Indicates that the original feature layer information and The output after multiple maximum pooling, represents the first maximum pooling operation, It means that the second maximum pooling is performed on the basis of the first maximum pooling. It means that the third maximum pooling is performed on the basis of the first two maximum poolings. Indicates that the original feature layer information The output after multiple average pooling, represents the first average pooling operation, represents the second average pooling operation, represents the third average pooling operation; The Dice loss function is selected as the edge loss function, and the formula is: ; in, Represents the input image The marginal loss, Represents the edge feature map of the model output, Indicates upsampling the edge feature map output by the model. Represents the edge features of the real image.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention introduces a detail feature enhancement attention module to more effectively focus on and strengthen the key detail features in the thangka image. By adding Gaussian noise in the forward process and performing iterative denoising in the reverse process, more refined noise suppression is achieved, so that the model can maintain good performance when facing images of different noise levels and types; based on the introduction of detail feature enhancement and denoising modules, the model effectively reduces the noise in the feature map while retaining the image detail information as much as possible, which helps to more accurately identify the structural boundaries of the thangka, thereby improving the segmentation accuracy of the thangka image. In addition, the detail feature enhancement module and the denoising module have adaptive adjustment capabilities, so that the model can flexibly respond to diverse and complex scenes, and can adapt to thangka images of different clarity during segmentation, further enhancing the robustness and accuracy of segmentation.

[0014] The present invention also combines the window multi-head self-attention mechanism with the global average pooling to generate a window multi-head self-attention pooling pyramid, captures different scale features in the global context information, and flexibly captures feature information of different scales in the thangka image by selecting a variety of window sizes. This self-attention mechanism enables the model to automatically focus on key areas in the thangka image, such as important religious symbols and characters, thereby optimizing the expression of features and improving the accuracy of segmentation. The detail information provided by the small window is combined with the context information provided by the large window, so that the model can pay attention to details and grasp the overall situation, effectively extract multi-scale context features in the image, and retain spatial information, ensuring the accurate extraction of complex patterns and details in the thangka image, thereby further improving the segmentation performance of the algorithm.

[0015] The present invention also introduces feature layer information of different scales and fuses them, and performs average pooling and maximum pooling on the feature layer information respectively, so as to effectively ensure the uniformity and importance of features, improve the expressiveness of features, and form more comprehensive and comprehensive features and edge features based on DW convolution, so that when processing thangka images, the segmentation process is faster and more efficient, while maintaining high-quality segmentation effects, further enhancing the effectiveness of feature representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of a method flow chart of an embodiment of the present invention; Figure 2 This is a flow chart of a module for training detail features to enhance attention according to an embodiment of the present invention; Figure 3 A schematic diagram of a detail feature enhancement attention module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a window multi-head self-attention pooling pyramid according to an embodiment of the present invention; Figure 5 Schematic diagram of a multi-scale edge feature guidance module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0018] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] Example: See also Figures 1 to 5 , the present invention provides a technical solution: A thangka image segmentation method based on edge feature guidance and detail feature denoising, the specific steps include: Step 1: Collect thangka images according to the set time step, and perform detail feature enhancement and denoising of thangka images based on the U-Net network. Introduce a detail feature enhancement attention module into the U-Net network structure, use the collected thangka images and encoder as the input of the detail feature enhancement attention module, add random noise, and train the detail feature enhancement attention module; In this embodiment, high-definition thangka images are collected, and the thangka images are annotated at the pixel level using Labelme software, and the images are scaled to a size of 512×512 to establish a high-definition thangka dataset; In this embodiment, the principle of training the detail feature enhanced attention module is: ; ; ; ; in, ,express is a feature map with C channels, H height, and W width, and , is The feature map obtained by 3×3 full-dimensional dynamic convolution operation, Represents the feature map To adjust the size, It means to rearrange the dimensions of the resized feature map, including exchanging the order of the number of channels, height and width of the image. represents matrix multiplication, represents the feature map of enhanced detail features obtained by matrix multiplication, , represents a feature map with 2C channels, H height, and W width. Indicates that and After concat, the feature map is obtained by 3×3 full-dimensional dynamic convolution operation. represents the addition of elements, represents a feature map integrating different feature information obtained by element addition, Represents the feature map obtained by matrix multiplication that combines detail features and semantic information. The collected image and the result after the image is processed by the encoder are used as input. Indicates normalizing the value of the feature map. Indicates the convolution operation on the normalized feature map. Represents the feature map of the final output after detail feature enhancement and convolution processing.

[0020] generate and The formula is: ; ; in, Express Apply a 3×3 convolution operation, Indicates the spliced Apply a 3×3 convolution operation; right Resize to ensure consistent size when concatenating feature maps, and rearrange The operation is to change the dimension order of the feature map, changing the dimension from Transformed to To facilitate subsequent operations.

[0021] Step 2: Input the feature map of the thangka image into the trained detail feature enhancement attention module. In the forward process, add Gaussian noise to the feature map to obtain the noisy feature map. In the reverse process, iterative denoising is performed on the noisy feature map and the target result to generate a real sample. In this embodiment, the principle for generating real samples by enhancing the attention module through detail features is: The forward process can be expressed as: ; in, represents the conditional probability distribution of the forward diffusion process, represents the feature map after noise addition at time step t, represents the feature map at time step t-1, represents the index of the time step, and , represents the maximum value of the time step, when When set to the maximum number, the reverse process can better approximate the Gaussian distribution. represents the variance of the noise at time step t, that is, the noise intensity added at time step t, represents a Gaussian distribution, and the mean of the Gaussian distribution is , the variance is , represents the covariance matrix of the noise, represents the result of the forward process; According to the input noise-free original thangka image and the known diffusion probability, the above forward process can be expressed as: ; ; ; ; in, represents the diffusion probability corresponding to the step length t, represents the cumulative product of all diffusion probabilities from time step 1-t, represents the original thangka image without noise, represents Gaussian noise; According to the feature map after adding noise and the target result, iterative denoising is performed to generate real samples. As The approximation of denoising can be expressed as: ; in, represents the conditional probability distribution of the reverse denoising process, represents the learning parameters of the model, represents the back diffusion variable, represents the noise variance of the model parameterization; During each training session, Parameterize the denoising model to obtain the predicted ; The output after production detail feature enhancement is based on the formula: ; in, Represents the predicted value of the original thangka image; The loss function is used to calculate the module loss based on the following principle: ; in, represents the mean square error loss function, Represents the joint expectation of three variables: time step, original thangka image and random noise.

[0022] The diffusion model is a generative model based on the Markov chain, which gradually transforms the Gaussian noise distribution into a more complex distribution, making it suitable for fields such as semantic segmentation and text-to-image generation. The diffusion model includes a forward process and a reverse process, namely the learning denoising stage and the implementation denoising stage. In the forward process, Gaussian noise is added to the feature map of the model to obtain a noisy feature map, and denoising is performed in the reverse process.

[0023] Step 3: Input the processed feature map, combine the window multi-head self-attention of different window sizes with the global average pooling, and generate a window multi-head self-attention pooling pyramid to capture feature information of different scales in the global context information; In this embodiment, the multi-head self-attention mechanism calculates multiple self-attention mechanisms in parallel, so that the model can capture the relationship between elements in the input sequence in different subspaces. CNN is used as the backbone network, and the window self-attention mechanisms of different window sizes are combined to generate a window multi-head self-attention pooling pyramid to achieve the capture of global context information of feature information of different scales. In this embodiment, the principle for generating the window multi-head self-attention pooling pyramid is: The feature map is divided into multiple windows, and the feature representation is performed on each window. The formula is: ; in, Indicates The feature representation of the window is represents a window of size M×M and number of channels C; The self-attention of the generated window is based on the formula: ; ; ; ; ; in, represents the query vector, represents the query weight matrix, represents the key vector, represents the key weight matrix, represents a value vector, represents the value weight matrix, represents the attention weight, represents the transpose of the key vector, represents the dimension of the key, Indicates The self-attention output of the window; Weight Matrix The dimensions of , the weight matrix is ​​updated in each training cycle through the ADAM optimization algorithm; The calculation of each attention head is: ; The output of multi-head self-attention is: ; in, Indicates The output of an attention head is Respectively represent The query weight matrix, key weight matrix and value weight matrix corresponding to the attention heads are Display Window The multi-head self-attention output, The learned weight matrix representing the output; For the multi-head self-attention output features of each window, global average pooling is performed based on the formula: ; in, Indicates The feature map output by multi-head self-attention of the window, Indicates The result of global average pooling of the feature maps output by multi-head self-attention of windows. Indicates After the self-attention process, the window The vector of position, Represents the row and column indices respectively; The input feature map is processed at multiple scales by different scaling factors, and the above is repeated for each scale. and Calculate to generate feature vectors of each scale, concatenate feature vectors of multiple scales to obtain the final multi-scale feature representation, based on the following formula: ; in, represents multi-scale feature representation, Indicates from the scale The generated feature vector, Represents the scaling factor.

[0024] After the feature layer, the window self-attention mechanism (WMSA) with window sizes of 2, 4, 6, and 8 and global average pooling are used. WMSA with different window sizes can focus on local and global features at the same time. Small windows capture fine-grained local spatial information, and large windows effectively capture global contextual information. The information interaction between WMSA with different window sizes can achieve a complementary effect, combining the detail information provided by the small window with the contextual information provided by the large window, so that the model can focus on both details and the overall situation. The output of WMSA with different window sizes is spliced ​​and fused with the global average pooling result, and the fused feature layer is reduced to the target number of channels in combination with a 1×1 convolutional layer, so that the WMSA with different window sizes forms a complementary effect, so that the model can focus on both details and the overall situation.

[0025] Step 4: Fuse the edge features and semantic information of features of different scales, introduce feature layer information of different scales, perform average pooling and maximum pooling on the feature layer information respectively, and splice them to generate the output of feature layer information of different scales. Fuse the output of feature layer information through DW convolution to generate edge features, and optimize the extraction of edge features through edge loss function.

[0026] In this embodiment, the principle of generating edge features and optimizing the extraction of edge features through the edge loss function is: Assume that the input feature layer information of different scales are , feature layer , indicating the number of channels is , height is , width is The feature map contains semantic information and is extracted by applying average pooling of different scales of pooling kernels in parallel. The diversified feature representation in the feature layer effectively reduces the information loss that may occur during the pooling process, ensuring that important features are fully preserved. On this basis, the most significant features in the region are retained through maximum pooling. Indicates the number of channels , height is , width is The feature map contains spatial information. By serially cascading and applying pooling kernels of different sizes for maximum pooling, the feature layer The edge features are deeply extracted. The serial maximum pooling uses pooling kernels of different sizes in each layer and extracts features in a layer-by-layer progressive manner, thereby effectively capturing edge information from subtle to global ranges. The formula for splicing and generating the output of feature layer information of different scales is based on: ; ; in, Indicates that the original feature layer information The output after multiple maximum pooling, represents the first maximum pooling operation, It means that the second maximum pooling is performed on the basis of the first maximum pooling. It means that the third maximum pooling is performed on the basis of the first two maximum poolings. Indicates that the original feature layer information The output after multiple average pooling, represents the first average pooling operation, represents the second average pooling operation, represents the third average pooling operation; Through DW convolution and after downsampling Perform feature fusion to generate a more comprehensive and comprehensive feature representation and edge features , Integrate The spatial information in the image is extracted, and the fine edge features and high-level semantic information in the image are extracted, and the edge features are predicted and generated. As edge guidance information, the effectiveness of features is further enhanced, and the edge loss mechanism is introduced. Guided by , the extraction of edge features is directly optimized.

[0027] In each channel, a separate convolution kernel is used for convolution. The size of each convolution kernel is K×K. For each channel of the input feature map, For unbalanced data sets, the Dice loss function is selected as the marginal loss function, and the formula is: ; in, Represents the input image The marginal loss, Represents the edge feature map of the model output, Indicates upsampling the edge feature map output by the model. Represents the edge features of the real image.

[0028] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0029] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0030] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0031] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A thangka image segmentation method based on edge feature guidance and detail feature denoising, characterized in that: The specific steps include: Step 1: Collect thangka images according to the set time step, and perform detail feature enhancement and denoising of thangka images based on the U-Net network. Introduce a detail feature enhancement attention module into the U-Net network structure, use the collected thangka images and encoder as the input of the detail feature enhancement attention module, add random noise, and train the detail feature enhancement attention module; Step 2: Input the feature map of the thangka image into the trained detail feature enhancement attention module. In the forward process, add Gaussian noise to the feature map to obtain the noisy feature map. In the reverse process, iterative denoising is performed on the noisy feature map and the target result to generate a real sample. Step 3: Input the processed feature map, combine the window multi-head self-attention of different window sizes with the global average pooling, and generate a window multi-head self-attention pooling pyramid to capture feature information of different scales in the global context information; Step 4: Fuse the edge features and semantic information of features of different scales, introduce feature layer information of different scales, perform average pooling and maximum pooling on the feature layer information respectively, and splice them to generate the output of feature layer information of different scales. Fuse the output of feature layer information through DW convolution to generate edge features, and optimize the extraction of edge features through edge loss function.

2. The method for thangka image segmentation based on edge feature guidance and detail feature denoising according to claim 1, characterized in that: The principle of training the detail feature enhanced attention module in step 1 is: ； ； ； ； in, ,express is a feature map with C channels, H height, and W width, and , is The feature map obtained by 3×3 full-dimensional dynamic convolution operation, Represents the feature map To adjust the size, It means to rearrange the dimensions of the resized feature map, including exchanging the order of the number of channels, height and width of the image. represents matrix multiplication, represents the feature map of enhanced detail features obtained by matrix multiplication, , represents a feature map with 2C channels, H height, and W width. Indicates that and After concat, the feature map is obtained by 3×3 full-dimensional dynamic convolution operation. represents the addition of elements, represents a feature map integrating different feature information obtained by element addition, represents the feature map combining detail features and semantic information obtained by matrix multiplication, Indicates normalizing the value of the feature map. Indicates the convolution operation on the normalized feature map. Represents the feature map of the final output after detail feature enhancement and convolution processing.

3. The method for thangka image segmentation based on edge feature guidance and detail feature denoising according to claim 1, characterized in that: The principle of generating real samples by enhancing the attention module through detail features in step 2 is: The forward process can be expressed as: ； in, represents the conditional probability distribution of the forward diffusion process, represents the feature map after noise addition at time step t, represents the feature map at time step t-1, represents the index of the time step, and , represents the maximum value of the time step, represents the variance of the noise at time step t, that is, the noise intensity added at time step t, represents a Gaussian distribution, and the mean of the Gaussian distribution is , the variance is , represents the covariance matrix of the noise, represents the result of the forward process; ； ； ； in, represents the diffusion probability corresponding to the step length t, represents the cumulative product of all diffusion probabilities from time step 1-t, represents the original thangka image without noise, represents Gaussian noise; According to the feature map after adding noise and the target result, iterative denoising is performed to generate real samples. As The approximation of denoising can be expressed as: ； in, represents the conditional probability distribution of the reverse denoising process, represents the learning parameters of the model, represents the back diffusion variable, represents the noise variance of the model parameterization; The output after production detail feature enhancement is based on the formula: ； in, Represents the predicted value of the original thangka image; The loss function is used to calculate the module loss based on the following principle: ； in, represents the mean square error loss function, Represents the joint expectation of three variables: time step, original thangka image and random noise.

4. The method for thangka image segmentation based on edge feature guidance and detail feature denoising according to claim 1, characterized in that: The principle for generating the window multi-head self-attention pooling pyramid in step 3 is: The feature map is divided into multiple windows, and the feature representation is performed on each window. The formula is: ； in, Indicates The feature representation of the window is represents a window of size M×M and number of channels C; The self-attention of the generated window is based on the formula: ； ； ； ； ； in, represents the query vector, represents the query weight matrix, represents the key vector, represents the key weight matrix, represents a value vector, represents the value weight matrix, represents the attention weight, represents the transpose of the key vector, represents the dimension of the key, and , Indicates The self-attention output of the window; The calculation of each attention head is: ； The output of multi-head self-attention is: ； in, Indicates The output of an attention head is Respectively represent The query weight matrix, key weight matrix and value weight matrix corresponding to the attention heads are Display Window The multi-head self-attention output of The learned weight matrix representing the output; For the multi-head self-attention output features of each window, global average pooling is performed based on the formula: ； in, Indicates The feature map output by multi-head self-attention of the window, Expressing the The result of global average pooling of the feature maps output by multi-head self-attention of windows. Indicates After the self-attention process, the window The vector of position, Represents the row and column indices respectively; The input feature map is processed at multiple scales by different scaling factors, and the above is repeated for each scale. and Calculate to generate feature vectors of each scale, concatenate feature vectors of multiple scales to obtain the final multi-scale feature representation, based on the following formula: ； in, represents multi-scale feature representation, Indicates from the scale The generated feature vector, Represents the scaling factor.

5. The method for thangka image segmentation based on edge feature guidance and detail feature denoising according to claim 1, characterized in that: The principle of generating edge features in step 4 and optimizing the extraction of edge features through edge loss function is: Assume that the input feature layer information of different scales are , the formula for splicing and generating the output of feature layer information of different scales is based on: ； ； in, Indicates that the original feature layer information The output after multiple maximum pooling, represents the first maximum pooling operation, It means that the second maximum pooling is performed on the basis of the first maximum pooling. It means that the third maximum pooling is performed on the basis of the first two maximum poolings. Indicates that the original feature layer information The output after multiple average pooling, represents the first average pooling operation, represents the second average pooling operation, represents the third average pooling operation; The Dice loss function is selected as the edge loss function, and the formula is: ； in, Represents the input image The marginal loss, Represents the edge feature map of the model output, Indicates upsampling the edge feature map output by the model. Represents the edge features of the real image.

Images