Morphological gradual region replacement method based on SAM semantic segmentation and user guidance

Abstract

The application discloses a morphological gradual change region replacement method based on SAM semantic segmentation and user guidance, relates to the technical field of computer vision and image processing, and comprises the following steps: performing semantic segmentation on a to-be-processed image through a SAM model, extracting multi-level semantic features and performing standardization and dimension reduction, extracting causal factors based on independent component analysis, generating a directed causal factor correlation graph through Granger causality test, and generating a causal attribution probability graph through reverse mapping; constructing a structured causal graph, generating a causal mask through a graph convolution network; encoding original interaction signals into a guidance heat map; constructing a diffusion equation, forming a gradual change control equation by dynamically fusing the intensity distribution of the guidance heat map and an image semantic diffusion term, and iteratively solving the gradual change control equation; generating an anisotropic morphological operation kernel according to the geometric curvature characteristics of each region in the replacement mask; and fusing the optimized replacement mask and target content based on a gradient domain optimization algorithm to generate a gradual change replacement image.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a morphological gradient region replacement method based on SAM semantic segmentation and user guidance. Background Art

[0002] In recent years, deep learning-based image segmentation and editing technologies have made significant progress, especially in the field of semantic segmentation. The Segment Anything Model (SAM), through its large-scale pre-training architecture, can achieve high-precision object segmentation with zero-shot generalization. The SAM model, combined with the Vision Transformer (ViT) encoder and a lightweight decoder, can generate multi-level semantic feature maps, providing a robust feature representation foundation for object recognition in complex scenes. At the same time, user interaction guidance technology has gradually become a research hotspot in the field of image editing. By generating heat maps through interactive signals such as clicks and graffiti, user intent can be converted into spatial weight distributions, enhancing the controllability of the editing process. In addition, the combination of morphological operations and gradient domain optimization algorithms provides pixel-level accuracy for local area replacement.

[0003] However, existing technologies still have significant limitations in the deep fusion of semantic segmentation and user guidance, interference area suppression, and sub-pixel gradient control. First, traditional semantic segmentation methods rely on single-level feature extraction, which makes it difficult to distinguish areas with high overlap or similar textures, resulting in blurred mask boundaries and mis-segmentation; second, the mathematical coupling between user interaction signals and semantic features is insufficient. Existing methods mostly use linear superposition or threshold truncation to fuse heat maps, which cannot achieve intent-driven dynamic diffusion direction adjustment. In addition, the traditional diffusion equation lacks a causal isolation mechanism for non-correlated interference areas, and misdiffusion across semantic boundaries can cause content tampering and distortion. At the mask optimization level, fixed-structure morphological operation kernels are difficult to adapt to complex curvature changes, resulting in jagged edges or excessive smoothing. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a morphological gradient region replacement method based on SAM semantic segmentation and user guidance to solve the problems of cross-semantic boundary interference diffusion, insufficient dynamic coupling between user intention and semantic features, and difficulty in adaptive generation of anisotropic morphological kernels in existing semantic segmentation methods.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides a morphological gradient region replacement method based on SAM semantic segmentation and user guidance, which includes: performing semantic segmentation on the input image to be processed through the SAM model to generate an initial binary mask, and extracting a multi-scale feature map from the multi-level block features of the Vision Transformer encoder, and forming a multi-level semantic feature through spatial alignment and channel splicing; standardizing and reducing the multi-level semantic features, extracting causal factors based on independent component analysis, generating a directed causal factor association graph through the Granger causality test, and reverse mapping to the pixel space to generate a causal attribution probability graph; constructing a structured causal graph using a simple linear iterative clustering algorithm, propagating semantic information through a graph convolutional network and segmenting to generate a causal mask; receiving the original interaction signal input by the user, and encoding the original interaction signal into a causal mask that is consistent with the original graph. A guided heat map with image resolution matching is constructed; based on the causal mask and the guided heat map, a diffusion equation that integrates user intention and semantic features is constructed, and a gradient control equation is formed by dynamically integrating the intensity distribution of the guided heat map and the image semantic diffusion term; a numerical solution method is used to iteratively solve the gradient control equation to obtain a replacement mask with sub-pixel gradient transition; an adaptive anisotropic morphological operation kernel is generated according to the geometric curvature characteristics of each region in the replacement mask, and dynamic erosion and dilation are performed to generate an optimized replacement mask; the optimized replacement mask is fused with the target content based on the gradient domain optimization algorithm to generate a gradient replacement image.

[0008] As a preferred solution of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance described in the present invention, wherein: the input image to be processed is semantically segmented by the SAM model to generate an initial binary mask, and a multi-scale feature map is extracted from the multi-level block features of the Vision Transformer encoder, and multi-level semantic features are formed through spatial alignment and channel splicing, including the following steps:

[0009] Perform resolution adjustment and boundary filling processing on the image to be processed to generate a pre-processed image;

[0010] The preprocessed image is input into the Vision Transformer encoder of the SAM model, and multi-layer block features are generated through multi-layer Transformer encoding;

[0011] The multi-level block features are input into the SAM model decoder, and a segmentation probability map is generated based on the output of the SAM model decoder. After binarization, an initial binary mask aligned with the original image is obtained.

[0012] Multi-scale feature maps are extracted from different levels of the Vision Transformer encoder, and multi-level semantic features are formed through spatial alignment and channel splicing.

[0013] As a preferred scheme of the morphological gradual change region replacement method based on SAM semantic segmentation and user guidance according to the application, wherein: the structured causal graph is constructed by using a simple linear iterative clustering algorithm, semantic information is propagated through a graph convolution network, and a causal mask is segmented, including the following steps,

[0014] According to the pixel-level causal attribution probability map, the superpixel nodes are clustered based on a simple linear iterative clustering algorithm, the weighted directed edges are established between the superpixel nodes according to the direction and intensity of the edges in the directed causal factor association graph, and a structured causal graph is generated.

[0015] The semantic information is propagated along the direction of the causal edge by using a graph convolution network, the node features are updated, and the updated node features are threshold segmented to generate a causal mask.

[0016] As a preferred scheme of the morphological gradual change region replacement method based on SAM semantic segmentation and user guidance according to the application, wherein: the original interaction signal is encoded into a guide heat map matching the resolution of the original image, including the following steps,

[0017] A two-dimensional Gaussian distribution heat field is generated with each click coordinate as the center, the Gaussian kernel standard deviation is adaptively adjusted according to the image resolution, and a Gaussian heat distribution map is generated.

[0018] The continuous coordinate sequence of the scribble trajectory is converted into a binary mask, the trajectory width is expanded to a preset pixel value by morphological dilation, and a continuous heat distribution map is generated by applying Gaussian blur.

[0019] The maximum value operation is performed on the Gaussian heat distribution map and the continuous heat distribution map pixel by pixel, and the initial heat map is merged.

[0020] The initial heat map is enhanced from the interaction device input resolution to the original image resolution using a bilinear interpolation algorithm, and is standardized to generate a guide heat map.

[0021] As a preferred scheme of the morphological gradual change region replacement method based on SAM semantic segmentation and user guidance according to the application, wherein: the diffusion equation fusing user intention and semantic features is constructed based on the causal mask and the guide heat map, the gradual change control equation is formed by dynamically fusing the intensity distribution of the guide heat map and the image semantic diffusion term, including the following steps,

[0022] The basic diffusion equation is constructed based on the anisotropic heat conduction equation, the intensity distribution of the guide heat map is taken as the external source term of the basic diffusion equation, the local guidance intensity of the user intention to the diffusion direction is controlled through the weight coefficient, and the intermediate diffusion equation containing the user guidance term is generated.

[0023] Extract semantic boundary gradients based on causal masks, calculate the diffusion coefficient matrix, suppress diffusion across semantic boundaries and enhance smooth diffusion within semantic regions, and output the semantic diffusion coefficient matrix;

[0024] The semantic diffusion coefficient matrix and the user-guided term are fused into the basic diffusion equation to generate the gradual control equation.

[0025] As a preferred solution of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance of the present invention, wherein: the iterative solution of the gradient control equation using a numerical solution method includes the following steps:

[0026] Use the enhanced semantic feature field as the initial input feature field, set the initial number of iterations and the convergence threshold;

[0027] The gradient control equation is discretized in time and space to generate a discrete iterative formula, and the incremental update value of the enhanced semantic feature field is calculated pixel by pixel. The enhanced semantic feature field is updated and the number of iterations is accumulated until the preset maximum number of iterations or the convergence condition is reached;

[0028] Calculate the mean square error between the current feature field and the previous iteration result. If the error is less than the convergence threshold or the maximum number of iterations is reached, terminate the iteration and output the steady-state feature field.

[0029] As an optimal solution of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance described in the present invention, wherein: the generating of an adaptive anisotropic morphological operation kernel according to the geometric curvature characteristics of each region in the replacement mask refers to applying the Hessian matrix to the replacement mask to calculate the local curvature, extracting the principal curvature direction and curvature intensity of each pixel coordinate, and generating an anisotropic morphological kernel set based on the principal curvature direction and curvature intensity.

[0030] As a preferred solution of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance described in the present invention, wherein: the fusion of the optimized replacement mask and the target content based on the gradient domain optimization algorithm to generate the gradient replacement image refers to extracting the gradient information of the target region based on the optimized replacement mask and generating an intermediate fused image through the gradient domain optimization algorithm;

[0031] Perform Laplace pyramid decomposition on the intermediate fused image and the original image, and reconstruct the gradient replacement image after fusion;

[0032] Perform gamma correction and color balance on the multi-scale fused image and output the final gradient replacement image.

[0033] In a second aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement according to the first aspect of the present application.

[0034] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement according to the first aspect of the present application.

[0035] The present application has the following beneficial effects: by constructing a structured causal graph and generating a causal mask, non-causal noise regions are identified and suppressed, and the semantic association strength is modeled using superpixel nodes and weighted directed edges, thereby avoiding the over-segmentation problem of traditional threshold segmentation for complex topological structures; by embedding the guide heat map as an external source term into the anisotropic heat conduction equation, combining the diffusion coefficient matrix generated by the causal mask to dynamically balance the user intention and semantic constraints, and dynamically adjusting the local diffusion rate to suppress diffusion across the semantic boundary, the present application breaks through the limitation of traditional diffusion models which only rely on image gradients, and realizes the mathematical coupling of user-guided intensity and semantic logic. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The overall flowchart of the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement.

[0038] Figure 2 The flowchart of generating a causal mask in the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement.

[0039] Figure 3 The flowchart of generating a guide heat map in the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement.

[0040] Figure 4 The flowchart of dynamic diffusion and morphological optimization in the method for semantic segmentation based on SAM and user-guided morphological gradual region replacement. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0044] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a morphological gradient region replacement method based on SAM semantic segmentation and user guidance, comprising the following steps:

[0045] S1. The input image to be processed is semantically segmented through the SAM model to generate an initial binary mask. Multi-scale feature maps are extracted from the multi-level block features of the Vision Transformer encoder, and multi-level semantic features are formed through spatial alignment and channel splicing.

[0046] S1.1. Perform resolution adjustment and boundary filling processing on the image to be processed to generate a preprocessed image that meets the SAM model input specifications.

[0047] Specifically, the target size is determined according to the input requirements of the Segment Anything Model (SAM). The long side of the image is scaled to the target size using the bicubic interpolation algorithm while maintaining the original aspect ratio. The short side is adaptively adjusted proportionally, and the insufficient area on the short side is mirror-symmetrically filled to make up the target size, generating a size-normalized intermediate image.

[0048] Content-aware hybrid filling is applied to the edge area of ​​the intermediate image. By identifying the texture features of the main area of ​​the image, synthetic content that is continuous with the main texture is generated in the filled area, eliminating the boundary artifacts caused by filling.

[0049] The padded intermediate image is channel-separated and normalized, and the pixel values ​​are linearly mapped to the numerical range defined during the SAM model pre-training, and the preprocessed image that meets the SAM model input specifications is output.

[0050] Furthermore, the pre-training of the SAM model specifically includes: the input image is uniformly scaled to a fixed resolution (e.g., 1024×1024 pixels), the short side is bicubic interpolated to maintain the ratio, and the long side is symmetrically padded to the target size. The pixel values ​​are normalized to the range [0, 1.0] to match the mean and standard deviation of the ImageNet dataset. The Vision Transformer (ViT-H / 16) is used as the image encoder and trained on a dataset containing 11 million images. The training objective is to generate high-quality segmentation masks based on randomly sampled points, boxes, or scribble prompts. The mask IoU loss and focal loss are jointly optimized to balance the learning weights of foreground and background regions.

[0051] S1.2. Input the preprocessed image into the Vision Transformer encoder of the SAM model and generate multi-layer block features through multi-layer Transformer encoding.

[0052] Specifically, the preprocessed image is divided into a non-overlapping block sequence of fixed size, and each image block is mapped to an embedding vector through a linear projection layer, and then a learnable position code is added to form an initial block feature sequence containing spatial position information; the initial block feature sequence is input into the multi-layer Transformer module of the Vision Transformer encoder of the SAM model, and each layer performs self-attention calculation and feedforward neural network processing in turn, updates the block feature representation and passes it to the next layer; the block feature matrix of the corresponding depth is extracted at the output positions of the 4th, 8th, 12th, 16th, 20th, and 24th layers of the Vision Transformer encoder, and the block features of each layer are restored to the original image spatial resolution through bilinear interpolation, and then spliced ​​along the channel dimension to generate a multi-level block feature set.

[0053] S1.3. Generate a segmentation probability map based on the output of the SAM model decoder, and obtain an initial binary mask aligned with the original image after binarization.

[0054] Multi-level block features are fed into the SAM model decoder, where semantic information from different levels is integrated through a cross-scale feature fusion module (e.g., Feature Pyramid Network (FPN)). Specifically, high-level features (e.g., the 24th layer) are upsampled using bilinear interpolation to match the resolution of low-level features. The upsampled high-level features are then added channel-by-channel to the low-level features to enhance detail preservation and generate a preliminary fused feature matrix. The Squeeze-and-Excitation module (SE Block) adjusts the channel weights of the preliminary fused feature matrix to suppress redundant channels and enhance the response of key channels, generating an optimized fused feature matrix. A 1×1 convolution operation is performed on the fused feature matrix to compress the number of channels to 1, generating a low-resolution segmentation logits matrix, where each element represents the unnormalized confidence score that the corresponding image block belongs to the target region.

[0055] It should be noted that the Squeeze-and-Excitation module (SE Block) is an attention mechanism used to enhance the channel feature representation capabilities of convolutional neural networks (CNNs) or visual Transformer models. Its core idea is to dynamically learn the importance weights of each feature channel, suppress redundant channels, and enhance the feature responses of key channels, thereby improving the model's ability to focus on the target area.

[0056] Furthermore, the low-resolution segmentation logits matrix output by the SAM model decoder is upsampled to the original image resolution through bilinear interpolation to generate a segmentation probability map that is spatially aligned with the original image. The Sigmoid function is applied pixel by pixel to the segmentation probability map to convert the probability value, and the logits value is mapped to the interval [0,1] to form a normalized probability distribution. The segmentation probability threshold is set according to the numerical distribution of the segmentation probability map, and the normalized probability distribution is binarized. Pixels with probability values ​​greater than or equal to the segmentation probability threshold are marked as target areas (value 1), and the remaining pixels are marked as background areas (value 0), generating an initial binary mask that is completely consistent with the original image resolution. Finally, a connected domain analysis is performed on the initial binary mask to eliminate isolated noise areas with an area smaller than the preset area threshold, and output the optimized initial binary mask. The preset area threshold is obtained by statistically analyzing the area distribution of noise areas in historical data, and taking the 95% quantile as the preset area threshold.

[0057] S1.4. Extract multi-scale feature maps from different levels of the Vision Transformer encoder, and form multi-level semantic features through spatial alignment and channel concatenation.

[0058] The block feature matrices of the corresponding levels are extracted from the output positions of the 4th, 8th, 12th, 16th, 20th, and 24th layers of the Vision Transformer encoder, and the block feature matrices of each level are adjusted to the same spatial resolution as the original image through a bilinear interpolation algorithm. The adjusted feature matrices of each level are spliced ​​along the channel dimension in order from shallow to deep levels to form a multi-scale fusion feature matrix with increasing number of channels, and multi-level semantic features are output.

[0059] S2. Standardize and reduce the dimensionality of multi-level semantic features, extract causal factors based on independent component analysis, generate a directed causal factor association graph through Granger causality test, and reversely map it to pixel space to generate a causal attribution probability graph; use a simple linear iterative clustering algorithm to construct a structured causal graph, propagate semantic information through a graph convolutional network, and segment and generate a causal mask.

[0060] S2.1. Normalize and reduce the dimensionality of multi-level semantic features, output the reduced-dimensional semantic feature vector, extract causal factors based on independent component analysis, and use the initial binary mask to filter background area noise.

[0061] The multi-level semantic features are subjected to channel-by-channel zero mean and unit variance normalization. The mean and standard deviation of each channel are calculated and the eigenvalues ​​are mapped to a standard normal distribution with a mean of 0 and a variance of 1. The standardized multi-level semantic features are input into the principal component analysis algorithm. Based on the eigenvalue decomposition results of the covariance matrix, the top N principal component directions with a cumulative variance contribution rate exceeding 95% are selected as projection basis vectors. The standardized feature matrix is ​​linearly projected along the principal component directions to generate a low-dimensional semantic feature vector of dimension w, where the value of w is automatically determined by the principal component analysis algorithm according to the cumulative variance contribution rate threshold. Finally, the low-dimensional semantic feature vector after dimensionality reduction is output.

[0062] Furthermore, the initial binary mask is pixel-aligned with the multi-level semantic features to ensure consistent spatial resolution. A masking operation is applied to the multi-level semantic features to force the feature values ​​of background areas to zero and eliminate noise interference. Independent component analysis is performed on the low-dimensional semantic feature vector to extract statistically independent causal factors (such as edge factors, texture factors, and illumination factors). Each causal factor corresponds to a local or global feature with clear semantic meaning in the processed image.

[0063] S2.2. Based on the Granger causality test between causal factors, the temporal prediction relationship between causal factors is analyzed. If the temporal change of factor A can predict the change of factor B, a directed edge is established between A and B to generate a directed causal factor association graph.

[0064] Specifically, the causal factors are arranged in spatial scanning order within the image space. The response values ​​of each causal factor at different spatial locations are converted into pseudo-time series data to generate the time series response sequence of each factor. A Granger causality test is performed on the pseudo-time series of each pair of causal factors (A, B). The prediction error is compared with the prediction error when the lagged term of factor A is included using a vector autoregression (VAR) model. If the inclusion of the lagged term of factor A significantly reduces the prediction error of factor B (F test p < 0.05), then A is considered to have a Granger causal effect on B. For causal factor pairs that pass the test (A→B), the standardized causal effect strength is calculated. The mean of the absolute value of the lagged term coefficient of factor A in the VAR model is used as the edge weight to generate a weighted directed edge set. A topological sorting algorithm is used to de-cyclicalize the directed edge set, removing edges that form loops (preferentially retaining edges with higher weights). This generates the final directed causal factor association graph, where nodes represent causal factors and edges represent Granger causal strengths.

[0065] The causal factors are mapped back to the pixel space through linear inverse transformation to generate the attribution probability matrix of each pixel to each causal factor. The attribution probability matrix is ​​normalized to ensure that the sum of the causal factor attribution probabilities of each pixel is 1, and a pixel-level causal attribution probability map is generated (the size is consistent with the original image, and the number of channels is the number of causal factors).

[0066] S2.3. Based on the pixel-level causal attribution probability graph and the simple linear iterative clustering (SLIC) algorithm, pixels belonging to the same causal factor and spatially adjacent are clustered into superpixel nodes. According to the edge direction and strength in the directed causal factor association graph, weighted directed edges are established between superpixel nodes to generate a structured causal graph.

[0067] A graph convolutional network (GCN) is applied on the structured causal graph to propagate semantic information along the causal edge direction, update the node features to enhance the consistency of causally related regions, suppress non-causal correlation interference, and obtain updated node features.

[0068] S2.4. Perform threshold segmentation on the updated node features, retain the areas where the causal weight is higher than the preset global threshold, generate a binary causal mask, fill the holes through morphological closing operations, and output enhanced semantic features that isolate irrelevant interference.

[0069] Specifically, based on the updated node features, the mean causal weights of all pixels within each node's corresponding region are calculated to generate a node-level causal weight matrix, which quantifies the strength of the causal association between each region and the target semantics. A global threshold (e.g., 0.6) is set for the node-level causal weight matrix. Nodes with weights greater than or equal to the global threshold are marked as target regions (value 1), and all others are marked as background (value 0), generating an initial binary causal mask.

[0070] A morphological closing operation is performed on the initial binary causal mask, using a circular kernel (e.g., with a radius of 3 pixels) to dilate and then erode it, filling small holes in the mask and smoothing its edges. Connected domain analysis is performed on the morphologically closed mask, removing isolated regions smaller than a preset threshold (e.g., 50 pixels) to generate an optimized causal mask. The causal mask is then multiplied pixel by pixel by the original semantic features, retaining the feature values ​​within the mask and setting the features outside the mask to zero. This output is an enhanced semantic feature that isolates irrelevant interference. The enhanced semantic features of all pixels in the image are spatially arranged to form a two-dimensional matrix of enhanced semantic feature fields, representing the collection and distribution of semantic features across the entire image space.

[0071] S3. Receive the original interaction signal input by the user, and encode the original interaction signal into a guidance heat map that matches the resolution of the original image.

[0072] S3.1. Receive the original interaction signal of the click coordinates or graffiti trajectory generated by the user through the interactive device, record it as a discrete coordinate point sequence or a continuous trajectory path, extract its spatial position information and filter out invalid input.

[0073] Specifically, the original coordinate data generated by the user's click or graffiti operation, that is, the original interaction signal, including the screen coordinates of the click event or the continuous coordinate sequence of the touch trajectory, is monitored and captured in real time through the API interface of the input device; the original coordinate data is mapped to the actual pixel coordinate system of the original image according to the scaling ratio and offset of the image display area, and a normalized coordinate point set matching the image resolution is generated; a validity check is performed on the normalized coordinate point set, coordinate points exceeding the image boundary are removed, and repeated clicks or trajectory overlapping areas are deduplicated to generate a valid coordinate point set; the valid coordinate point set is stored in timestamp order as a discrete click coordinate point sequence or a continuous trajectory path.

[0074] S3.2. Generate a two-dimensional Gaussian distribution thermal field centered at each click coordinate. The standard deviation of the Gaussian kernel is adaptively adjusted according to the image resolution to ensure that the thermal coverage range matches the physical size of the target semantic area, and generate an initial Gaussian thermal distribution map.

[0075] Furthermore, the Gaussian kernel standard deviation is calculated proportionally based on the original image resolution and the average physical size of the target semantic area to ensure that the thermal field coverage is linearly related to the size of the target area. Based on the adaptive standard deviation, a two-dimensional Gaussian kernel matrix is ​​generated. The kernel size is automatically expanded to cover the 3σ range based on the standard deviation, generating a standardized Gaussian kernel weight distribution. At the image position corresponding to each click coordinate, a local thermal field is generated with the two-dimensional Gaussian kernel matrix as the center, and the kernel weight value is filled into the image area centered on the click coordinate. All local thermal fields are superimposed on the same thermal distribution map, and the intensity value of the overlapping area is maximized to generate the initial Gaussian thermal distribution map.

[0076] The continuous coordinate sequence of the graffiti trajectory is converted into a binary mask, the trajectory width is expanded to a preset pixel value through morphological dilation, and Gaussian blur is applied to the dilated area to generate a continuous heat distribution map with a smooth transition.

[0077] Specifically, the continuous coordinate sequence of the graffiti trajectory is converted into an all-zero matrix consistent with the original image resolution. The pixel coordinates corresponding to the trajectory path are assigned a value of 1 to generate an initial binary mask. A morphological dilation operation is performed on the initial binary mask using a circular structure kernel, expanding the trajectory path width to a preset pixel value, generating a dilated binary mask with uniform width. A Gaussian blur algorithm is applied to the dilated binary mask, smoothing edge transitions according to a preset blur radius to generate a continuously gradient low-resolution thermal distribution map. A bilinear interpolation algorithm is used to upsample the low-resolution thermal distribution map to the original image resolution, generating an initial continuous thermal distribution map aligned with the image space.

[0078] Linearly map the pixel values ​​of the initial Gaussian heat map and the initial continuous heat map to the interval [0, 1] to generate a standardized Gaussian heat map and a continuous heat map. Maximize the Gaussian heat map and the continuous heat map pixel by pixel and merge them into the initial heat map, preserving the peak intensity of the user's guidance intention.

[0079] S3.3. Use a bilinear interpolation algorithm to scale the initial heatmap from the interactive device input resolution to the original image resolution, generating an intermediate heatmap that is spatially aligned with the original image.

[0080] Get the input resolution of the initial heat map (such as 256×256) and the target resolution of the original image (such as 4096×4096), and calculate the scaling ratio of the two in the width and height directions. Determine the sampling interval and weight matrix of the interpolation kernel based on the scaling ratio, and generate the pixel coordinate mapping relationship and weight coefficient required for bilinear interpolation. For each pixel coordinate of the initial heat map, perform a weighted sum of the intensity values ​​of the four nearest neighboring pixels according to the interpolation kernel parameters, and calculate the interpolation result of the corresponding position at the target resolution. Fill the interpolation calculation result into the blank matrix of the target resolution to generate an intermediate heat map that is strictly aligned with the spatial coordinates of the original image, ensuring that each pixel coordinate corresponds one-to-one to the original image area. Perform a mean filtering operation on the edge area of ​​the intermediate heat map to eliminate boundary jagged artifacts that may be introduced during the interpolation process.

[0081] The pixel values ​​of the intermediate heat map are linearly mapped to the interval [0, 1], with the highest intensity area corresponding to 1 and the background area corresponding to 0, to generate a standardized guided heat map.

[0082] S4. Based on the causal mask and the guided heat map, a diffusion equation is constructed that integrates user intention and semantic features. The intensity distribution of the guided heat map and the image semantic diffusion term are dynamically integrated to form a gradient control equation.

[0083] S4.1. Construct a basic diffusion equation based on the anisotropic heat conduction equation, express the evolution of the enhanced semantic feature field as a gradient-driven diffusion process, and dynamically adjust the diffusion coefficient according to the enhanced semantic features.

[0084] Specifically, the basic diffusion equation is defined based on the Perona-Malik anisotropic diffusion equation, and the temporal evolution of the enhanced semantic feature field is expressed as a gradient-driven diffusion form. The enhanced semantic feature field is used as the feature field to be diffused, and the central difference method is used to calculate the spatial gradient of each pixel to drive the diffusion direction and generate a gradient amplitude map; based on the enhanced semantic feature gradient extracted by the causal mask, the initial diffusion coefficient matrix is ​​generated through an exponential decay function. , suppressing the diffusion across semantic boundaries (high gradient areas ), preserve smoothness within the semantic region (low gradient region );

[0085] During the diffusion process, the diffusion coefficient matrix is ​​updated according to the real-time feature gradient to ensure that the diffusion rate is dynamically adapted to the current semantic features, and the updated diffusion coefficient is output.

[0086] S4.2. Use the intensity distribution of the guided heat map as the external source term of the basic diffusion equation, control the local guidance intensity of the user intention on the diffusion direction through the weight coefficient, and generate an intermediate diffusion equation including the user guidance term.

[0087] Furthermore, the intensity distribution of the guidance heatmap is added to the basic diffusion equation as an external source term, and the difference between the heatmap intensity and the current feature field intensity is used as the driving term to generate a preliminary correction equation that incorporates user intent. The weight coefficient is dynamically adjusted based on the local intensity value of the guidance heatmap, with regions with higher heatmap intensity receiving greater weights, thereby strengthening the guidance effect of user-specified areas and generating a weighted correction equation. The weighted user guidance term is added to the diffusion term in the basic diffusion equation to form an intermediate diffusion equation that incorporates both semantic diffusion and user guidance, preserving the integrity of the equation's mathematical form.

[0088] An explicit Euler discretization test is performed on the intermediate diffusion equation to verify the matching of the time step and the weight coefficient, and to ensure that the user-guided term does not destroy the numerical stability during the iteration process.

[0089] S4.3. Extract semantic boundary gradients based on the causal mask, calculate the diffusion coefficient matrix, suppress diffusion across semantic boundaries and enhance smooth diffusion within the semantic region, and output the semantic-driven semantic diffusion coefficient matrix.

[0090] Furthermore, the Sobel operator is applied to the causal mask to calculate horizontal and vertical gradients, generating a semantic boundary gradient magnitude map to quantify the edge strength of the semantic region. Based on this semantic boundary gradient magnitude map, the gradient magnitudes are mapped to diffusion coefficients using an exponential decay function, with the coefficients approaching 0 for high-gradient regions (semantic boundaries) and 1 for low-gradient regions (semantic interiors). This generates an initial diffusion coefficient matrix. A mean filter is performed on the semantic interior regions (low-gradient regions) in the initial diffusion coefficient matrix to improve the consistency of the diffusion coefficients and ensure a uniform diffusion rate within the semantic region. The diffusion coefficients of semantic boundary regions (high-gradient regions) are reset to zero to prevent cross-boundary feature diffusion, generating the final semantic diffusion coefficient matrix.

[0091] S4.4. The semantic diffusion coefficient matrix and the user guidance term are integrated into the basic diffusion equation to generate a gradient control equation that dynamically balances user intent and semantic constraints.

[0092] Specifically, the semantic diffusion coefficient matrix and the user-guided term in the intermediate diffusion equation are received, the default diffusion coefficient in the basic diffusion equation is replaced with the semantic diffusion coefficient matrix, and the enhanced semantically constrained diffusion term is generated.

[0093] The user guidance term and the semantic constraint diffusion term are superimposed to form a gradual control equation that includes both user intention and semantic constraints. The expression is:

[0094] ;

[0095] in, is the feature field to be optimized (such as image intensity, temperature field, etc.), with the space and time change, is the semantic diffusion coefficient, Provide weights for users and control the impact of user goals. A user-specified target field (such as a heat map or target image).

[0096] S5. A numerical solution method is used to iteratively solve the gradient control equation to obtain a replacement mask containing a sub-pixel gradient transition.

[0097] S5.1. Use the enhanced semantic feature field as the initial input feature field and set the initial number of iterations and the convergence threshold.

[0098] Furthermore, the initial number of iterations is set based on the original image resolution (e.g., 500 times for a 4096×4096 image), and the convergence threshold is set to one ten-thousandth (0.0001) according to the numerical dynamic range of the enhanced semantic feature field (e.g., maximum intensity 1.0).

[0099] The gradient control equation is discretized in time and space to generate a discrete iterative formula, and the incremental update value of the enhanced semantic feature field is calculated pixel by pixel. The enhanced semantic feature field is updated and the number of iterations is accumulated until the preset maximum number of iterations or the convergence condition is reached;

[0100] Specifically, the spatial derivative terms in the gradually varying governing equations are discretized using a central difference scheme to generate an approximate spatial discrete expression for the spatially discretized partial differential equation. The time derivative terms are discretized using an explicit Euler method, converting the continuous time evolution into a discrete time-step iterative relationship. By substituting the spatial discrete terms into the time-step discretized equations and solving for the characteristic field at the next time step, the spatial and temporal discretization results are combined to generate a discrete iterative formula.

[0101] Based on a discrete iterative formula, the current enhanced semantic feature field is input and the gradient difference between it and neighboring pixels is calculated pixel by pixel to generate an incremental update value. This incremental update value is scaled by the time step and added to the current feature field, updating the entire field data and incrementing the iteration counter. The mean square error between the current feature field and the previous iteration result is calculated. If the error is less than the convergence threshold or the maximum number of iterations is reached, the iteration is terminated and the steady-state feature field is output. Otherwise, the updated feature field is used as input to enter the next iteration. The steady-state feature field is bilinearly interpolated and upsampled to the original image resolution. The target area is extracted through global threshold segmentation, and a replacement mask containing sub-pixel gradient transitions is generated.

[0102] S6. Generate an adaptive anisotropic morphological operation kernel according to the geometric curvature characteristics of each region in the replacement mask, perform dynamic erosion and dilation, and generate an optimized replacement mask.

[0103] S6.1. Apply Hessian matrix to calculate local curvature on the replacement mask, extract the principal curvature direction and curvature intensity of each pixel coordinate, and generate a geometric curvature feature map.

[0104] Specifically, after performing Gaussian filtering with a standard deviation σ = 1.5 on the replacement mask to eliminate noise, the second-order spatial derivatives of each pixel point are calculated by a Sobel operator to construct a Hessian matrix. A Scharr filter is used to optimize the direction sensitivity, and the maximum eigenvalue and corresponding eigenvector of the Hessian matrix are solved. The eigenvector direction is taken as the principal curvature direction and encoded using the radian value, and the absolute value of the eigenvalue is taken as the curvature intensity quantization value. The principal curvature direction is mapped to a 0-180° polar coordinate color ring to generate a direction encoding map, and the curvature intensity is normalized by Min-Max to generate an intensity mapping map in the interval 0-1. The direction encoding map and the intensity mapping map are spliced along the channel dimension to output a two-channel geometric curvature feature map containing geometric curvature direction and intensity information.

[0105] Based on the principal curvature direction of the geometric curvature feature map, the rotation angle of the morphological kernel is adjusted, and the kernel size is dynamically scaled according to the curvature intensity to generate a set of anisotropic morphological kernels that adapt to the local geometry.

[0106] Further, the principal curvature direction channel data is extracted from the geometric curvature feature map, and the direction angle is converted to a rotation angle matrix of the morphological kernel by polar coordinate mapping. According to the normalized value of the curvature intensity channel, the intensity is mapped to a preset kernel size range (such as 1-5 pixels) to generate a dynamic scaling coefficient matrix by linear interpolation. Based on the preset elliptical kernel basic template, the kernel is subjected to affine rotation transformation according to the rotation angle matrix, and the major and minor axis sizes of the kernel are adjusted according to the dynamic scaling coefficient matrix. For each pixel coordinate, a corresponding rotated and scaled morphological kernel is generated, which is stored as a kernel set according to the spatial coordinate index, and an anisotropic morphological kernel set that strictly matches the local geometric structure is output.

[0107] It should also be noted that the preset elliptical kernel basic template is set according to the geometric characteristics of the target object, the application scenario requirements and experimental verification, and by adjusting parameters such as the major and minor axis ratio and size, it is ensured that the local structure can be effectively matched after rotation and scaling.

[0108] For each pixel coordinate of the replacement mask, a local erosion operation is performed according to the corresponding anisotropic morphological kernel to suppress small noise in high curvature areas and preserve edge details.

[0109] Furthermore, each pixel coordinate of the replacement mask is traversed, and the rotated and scaled kernel template corresponding to the pixel coordinate is obtained by indexing from the anisotropic morphological kernel set. The local area of ​​the mask is intercepted with the pixel as the center and a logical "AND" operation is performed with the rotated and scaled kernel template. The minimum grayscale value of all pixels in the kernel coverage area in the operation result is assigned to the central pixel to complete local corrosion; for high curvature areas (pixels with curvature intensity ≥ 0.8), an additional secondary corrosion operation is applied to strengthen the suppression of isolated noise points by reducing the kernel size to 50% of the original size and increasing the grayscale judgment threshold to 0.9; edge contrast detection is performed on the eroded replacement mask, and when the pixel gradient value is higher than the edge protection gradient threshold, its original grayscale value is restored to retain edge sharpness, generating a denoised optimized replacement mask.

[0110] It should be noted that the Sobel operator gradient calculation is performed on the eroded mask image, the cumulative distribution function (CDF) of the gradient amplitude of the entire image is statistically calculated, and the gradient amplitude corresponding to 95% of the CDF value is selected as the edge protection gradient threshold.

[0111] S7. Based on the gradient domain optimization algorithm, the optimized replacement mask is fused with the target content to generate a visually consistent gradient replacement image.

[0112] S7.1. Extract the gradient information of the target area based on the optimized replacement mask and perform weighted fusion with the gradient information of the enhanced semantic feature field to generate a fused gradient field that preserves the target edge and background texture.

[0113] Specifically, the Sobel operator is used to calculate the horizontal and vertical gradients of the optimized replacement mask to generate the gradient amplitude map and direction map of the target area; Gaussian derivative filtering with a standard deviation σ=1.2 is performed on the enhanced semantic feature field to extract the gradient amplitude map of the background texture; the target and background gradient amplitudes are linearly weighted pixel by pixel according to the confidence weight of the replacement mask (weight 0.8 for the target area, weight 0.3 for the transition area, weight 0.1 for the background area), and the initial fused gradient field is generated; the Poisson equation is used to solve the local compatible gradient direction for the gradient direction conflict area (pixels with direction difference ≥30°) to eliminate directional inconsistency; finally, the optimized gradient amplitude and direction map are merged into vector form, and the fused gradient field that retains the sharp edges of the target and the continuous texture of the background is output.

[0114] The difference between the fused gradient field and the original image gradient field is used as a constraint term to construct a least squares optimization equation. The objective function includes a gradient consistency term and a pixel smoothing term to generate a gradient domain optimization equation.

[0115] Specifically, a gradient consistency term is constructed based on the pixel-by-pixel gradient amplitude difference between the fused gradient field and the original image gradient field, and the square sum of the horizontal and vertical gradient differences of each pixel is calculated as the constraint matrix; the pixel smoothing term is defined as the sum of the squares of the second-order Laplace operator differences of the grayscale value differences of adjacent pixels, which is weighted by λ=0.3 (dynamically adjusted according to the image signal-to-noise ratio) and then superimposed with the gradient consistency term; the objective function is expressed as a linear combination of the gradient consistency term and the pixel smoothing term, which can be expanded as follows: A sparse matrix equation of the form is a pentadiagonal symmetric matrix containing gradient weights and smoothing coefficients, To fuse the difference vector between the gradient field and the original gradient; store the equations in compressed sparse column (CSC) format and verify the positive definiteness of the matrix, and output the gradient domain optimization equations that can be numerically solved.

[0116] The preconditioned conjugate gradient method is used to solve the gradient domain optimization equation, calculate the optimal intensity value of each pixel, and generate an intermediate fused image with consistent gradient.

[0117] Furthermore, the sparse matrix of the gradient domain optimization equation is (penta-diagonal symmetric structure) and the difference vector are loaded into memory and the solution vector is initialized Calculate the initial residual as the column vector form of the original image pixel intensity , and construct a sparse matrix based on The dynamic incomplete Cholesky preconditioning matrix is ​​used to reduce the condition number of the equation. Furthermore, the incomplete Cholesky decomposition is used to generate the lower triangular matrix and the diagonal matrix , such that:

[0118] ;

[0119] in, Keep only The lower triangular elements of the non-zero filling level ≤ 2;

[0120] Initialize the solution vector to zero and calculate the initial residual. Use forward and back substitution to solve the initial solution vector to zero and calculate the initial residual, generate a preprocessing residual vector with a low condition number, and update the search direction to the preprocessing residual vector.

[0121] It should be noted that the search direction is The direction used to update the solution vector in the iteration indicates in which direction the current solution needs to be adjusted to approach the optimal solution more quickly. For example, in image fusion, the search direction may point to the areas with the largest gradient differences, and these areas are corrected first. The initial direction is the preprocessing residual, and the subsequent directions are generated by combining the preprocessing residual with the historical direction to maintain orthogonality.

[0122] At each iteration Calculate the step size and update the solution vector and residual; set the residual threshold based on the relative error of the initial residual, for example, set the residual threshold to multiples of the initial residual norm, that is, , iterate round by round until the residual norm is lower than the set residual threshold or the maximum number of iterations is reached; otherwise, calculate the new preprocessing residual and update the search direction;

[0123] The final solution vector Rearrange the original image into a two-dimensional matrix according to its size, use adaptive Min-Max normalization to map the pixel intensity to the integer range of [0,255], and output an intermediate fused image with consistent gradient, where the pixel intensity satisfies the equation: ), is the Laplace smoothing operator, is the preset weight coefficient.

[0124] S7.2. Perform Laplacian pyramid decomposition on the intermediate fused image and the original image, align brightness and contrast at multiple scales, and reconstruct the resulting multi-scale fused image after fusion.

[0125] Perform five-layer Laplacian pyramid decomposition on the intermediate fusion image and the original image respectively. Specifically, the standard deviation = 1.6 Gaussian kernel is used to downsample the two images step by step to generate a Gaussian pyramid. After upsampling each layer of the Gaussian pyramid back to its original size, the residual is subtracted from the upper layer to extract the Laplacian pyramid layer containing details of different spatial frequencies. The root mean square error (RMSE) between the intermediate fused image and the original image is calculated in each layer of the pyramid. With the goal of minimizing the error, the affine transformation is used to adjust the brightness bias of each layer of the intermediate fused image. and contrast scaling factor , the expression is,

[0126] ;

[0127] ;

[0128] ;

[0129] in, After alignment layer images, After fusion layer images, For the The scaling factor of the layer is used to adjust the standard deviation of the fused image. For the The translation factor of the layer is used to adjust the mean of the fused image. For the fused image The standard deviation of the layer, The original image The standard deviation of the layer, The original image The mean of the layer, For the fused image The mean of the layer, and are the mean and standard deviation within the stratum, respectively;

[0130] The aligned intermediate fusion pyramid layer is mixed with the original image pyramid layer according to adaptive weights. The low-frequency layer (l ≥ 3) is mainly based on the weight of the original image (0.8), and the high-frequency layer (l ≤ 2) is mainly based on the weight of the fusion image (0.7). The mixed pyramid layers are upsampled step by step from the top to the bottom, and the clamping operation is performed to limit the pixel values ​​to the range of [0, 255] to reconstruct a multi-scale fused image with continuous brightness distribution and seamless texture.

[0131] Perform gamma correction and color balance on the multi-scale fusion image to adjust it to be consistent with human eye perception, and output the final gradient replacement image.

[0132] This embodiment also provides a computer device, which is suitable for the case of a morphological gradient region replacement method based on SAM semantic segmentation and user guidance, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the morphological gradient region replacement method based on SAM semantic segmentation and user guidance proposed in the above embodiment.

[0133] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0134] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the morphological gradient region replacement method based on SAM semantic segmentation and user guidance as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0135] In summary, the present invention constructs a structured causal graph and generates a causal mask to identify and suppress non-causally related noise areas, and uses superpixel nodes and weighted directed edges to model semantic association strength, thereby avoiding the over-segmentation problem of traditional threshold segmentation for complex topological structures; by embedding the guided heat map as an external source term into the anisotropic heat conduction equation, combined with the diffusion coefficient matrix generated by the causal mask to dynamically balance user intentions and semantic constraints, dynamically adjust the local diffusion rate, and suppress diffusion across semantic boundaries, breaking through the limitation of traditional diffusion models that only rely on image gradients, and realizing the mathematical coupling of user guidance strength and semantic logic.

[0136] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A morphological gradient region replacement method based on SAM semantic segmentation and user guidance, characterized by: include, The SAM model is used to perform semantic segmentation on the input image to generate an initial binary mask. Multi-scale feature maps are extracted from the multi-level block features of the Vision Transformer encoder, and multi-level semantic features are formed through spatial alignment and channel splicing. Multi-level semantic features are standardized and reduced in dimension. Causal factors are extracted based on independent component analysis and background noise is filtered using an initial binary mask. A directed causal factor association graph is generated through the Granger causality test and reversely mapped to the pixel space to generate a causal attribution probability graph. A structured causal graph is constructed using a simple linear iterative clustering algorithm, and semantic information is propagated and segmented using a graph convolutional network to generate a causal mask. Receive the original interaction signal input by the user, and encode the original interaction signal into a guidance heat map matching the resolution of the original image; Based on the causal mask and the guided heat map, a diffusion equation is constructed that integrates user intention and semantic features. The intensity distribution of the guided heat map and the image semantic diffusion term are dynamically integrated to form a gradient control equation. A numerical solution method is used to iteratively solve the gradient control equation to obtain a replacement mask containing sub-pixel gradient transition; Generate an adaptive anisotropic morphological operation kernel according to the geometric curvature characteristics of each region in the replacement mask, perform dynamic erosion and expansion, and generate an optimized replacement mask; The optimized replacement mask is fused with the target content based on the gradient domain optimization algorithm to generate a gradient replacement image.

2. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 1, characterized in that: The SAM model is used to perform semantic segmentation on the input image to be processed, generate an initial binary mask, extract a multi-scale feature map from the multi-level block features of the Vision Transformer encoder, and form a multi-level semantic feature through spatial alignment and channel splicing, including the following steps: Perform resolution adjustment and boundary filling processing on the image to be processed to generate a pre-processed image; The preprocessed image is input into the Vision Transformer encoder of the SAM model, and multi-layer block features are generated through multi-layer Transformer encoding; The multi-level block features are input into the SAM model decoder, and a segmentation probability map is generated based on the output of the SAM model decoder. After binarization, an initial binary mask aligned with the original image is obtained. Multi-scale feature maps are extracted from different levels of the Vision Transformer encoder, and multi-level semantic features are formed through spatial alignment and channel splicing.

3. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 1, characterized in that: The method uses a simple linear iterative clustering algorithm to construct a structured causal graph, propagates semantic information through a graph convolutional network, and segments and generates a causal mask, including the following steps: According to the pixel-level causal attribution probability graph, superpixel nodes are clustered based on a simple linear iterative clustering algorithm. According to the edge direction and strength in the directed causal factor association graph, weighted directed edges are established between superpixel nodes to generate a structured causal graph. The graph convolutional network is used to propagate semantic information along the causal edge direction, update the node features, and perform threshold segmentation on the updated node features to generate the causal mask.

4. The method for replacing morphological gradient regions based on SAM semantic segmentation and user guidance according to claim 3, characterized in that: The encoding of the original interaction signal into a guidance heat map matching the original image resolution comprises the following steps: A two-dimensional Gaussian distribution thermal field is generated with each click coordinate as the center. The Gaussian kernel standard deviation is adaptively adjusted according to the image resolution to generate a Gaussian thermal distribution map. The continuous coordinate sequence of the graffiti trajectory is converted into a binary mask, the trajectory width is expanded to a preset pixel value through morphological dilation, and Gaussian blur is applied to generate a continuous heat distribution map; The Gaussian thermal distribution map and the continuous thermal distribution map are combined into the initial thermal map by taking the maximum value of each pixel; The initial heat map is scaled from the interactive device input resolution to the original image resolution using a bilinear interpolation algorithm and then normalized to generate a guided heat map.

5. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 4, characterized in that: The method constructs a diffusion equation that integrates user intention and semantic features based on the causal mask and the guided heat map, and forms a gradient control equation by dynamically integrating the intensity distribution of the guided heat map and the image semantic diffusion term, including the following steps: A basic diffusion equation is constructed based on the anisotropic heat conduction equation. The intensity distribution of the guided heat map is used as an external source term in the basic diffusion equation. The local guidance intensity of the user intention on the diffusion direction is controlled by the weight coefficient, generating an intermediate diffusion equation that includes the user guidance term. Extract semantic boundary gradients based on causal masks, calculate diffusion coefficient matrices, suppress diffusion across semantic boundaries and enhance smooth diffusion within semantic regions, and output semantic diffusion coefficient matrices. The semantic diffusion coefficient matrix and the user-guided term are fused into the basic diffusion equation to generate the gradual control equation.

6. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 5, characterized in that: The method of iteratively solving the gradual control equation by using a numerical solution method includes the following steps: Use the enhanced semantic feature field as the initial input feature field, set the initial number of iterations and the convergence threshold; The gradient control equation is discretized in time and space to generate a discrete iterative formula, and the incremental update value of the enhanced semantic feature field is calculated pixel by pixel. The enhanced semantic feature field is updated and the number of iterations is accumulated until the preset maximum number of iterations or the convergence condition is reached; Calculate the mean square error between the current feature field and the previous iteration result. If the error is less than the convergence threshold or the maximum number of iterations is reached, terminate the iteration and output the steady-state feature field.

7. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 6, characterized in that: Generating an adaptive anisotropic morphological operation kernel according to the geometric curvature characteristics of each region in the replacement mask refers to applying the Hessian matrix to the replacement mask to calculate the local curvature, extracting the principal curvature direction and curvature intensity of each pixel coordinate, and generating an anisotropic morphological kernel set based on the principal curvature direction and curvature intensity.

8. The morphological gradient region replacement method based on SAM semantic segmentation and user guidance according to claim 7, characterized in that: The step of fusing the optimized replacement mask with the target content based on the gradient domain optimization algorithm to generate a gradient replacement image refers to extracting gradient information of the target area based on the optimized replacement mask and generating an intermediate fused image through the gradient domain optimization algorithm; Perform Laplacian pyramid decomposition on the intermediate fused image and the original image, and reconstruct the gradient replacement image after fusion; Perform gamma correction and color balance on the multi-scale fused image and output the final gradient replacement image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the morphological gradient region replacement method based on SAM semantic segmentation and user guidance are implemented as described in any one of claims 1 to 8.

Images