Image inpainting method and system based on multi-scale hybrid feature modeling

Abstract

The application belongs to the technical field of image processing, and particularly relates to an image missing area repairing method and system based on multi-scale mixed feature modeling. The original image to be repaired and a missing area mask are spliced in the channel dimension, multi-layer normalization and convolution operations are performed on initial features, feature extraction is performed on a content channel branch and a gate channel branch, multi-layer convolution and linearization operations are performed to obtain channel gate features of each layer; the channel gate features of the last layer are split into spatial detail component features and long-range correlation component features, spatial detail components and long-range spatial components are obtained through linear mapping, and fusion output features of each layer are obtained through processing; and the fusion output features of the first layer are used as a repaired complete image. The scheme can guarantee the structural continuity of a large-area missing area, and effectively reduce blurring, artifacts and boundary fracture phenomena.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for repairing missing regions in images based on multi-scale hybrid feature modeling. Background Technology

[0002] Image restoration refers to the technique of using information from known areas in an image to reasonably fill in missing areas caused by occlusion, damage, or manual removal. This technique has wide applications in fields such as old photo restoration, image watermark removal, object removal, and visual content editing.

[0003] Existing image inpainting methods mainly include those based on convolutional neural networks (CNNs) and those based on attention mechanisms. CNN-based methods typically have high computational efficiency, but due to the limited local receptive field of convolution operations, they struggle to effectively model the relationships between distant pixels when dealing with large missing areas, easily leading to structural discontinuities or blurred textures in the inpainting results.

[0004] Attention-based methods improve the ability to restore the overall structure of an image to some extent by introducing a global modeling strategy. However, these methods have high computational complexity in high-resolution image restoration tasks and suffer from problems such as large memory consumption and slow inference speed, which limits their promotion in practical applications.

[0005] In recent years, state-space models have attracted attention due to their good global modeling capabilities and computational efficiency in sequence modeling tasks. However, the ability of state-space models alone to characterize local fine textures and spatial geometric relationships in two-dimensional image restoration tasks remains limited. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for repairing missing regions in images based on multi-scale hybrid feature modeling.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: This invention provides a method for image missing region repair based on multi-scale hybrid feature modeling, comprising the following steps: S1: Input the original image to be repaired and the corresponding missing region mask, concatenate the two in the channel dimension to obtain the input features, and then expand the input features through convolution and channel dimension to obtain the initial features.

[0008] S2: Normalize the initial features layer by layer, and perform pointwise convolutions along the channel dimension to obtain high-dimensional features containing twice the number of intermediate channels. Use depthwise convolutions to model the high-dimensional features in the spatial domain on the content branch channels to obtain content branch features. Finally, model the features in the spatial domain on the gated branch channels to obtain gated branch features. The content branch features are non-linearly activated and multiplied element-wise with the gated branch features to obtain spatial gated features. Channel compression is then performed to restore the initial feature dimension. Global average pooling is then performed to map the channel weights. After non-linear activation, these weights are multiplied element-wise with the spatial gated features to obtain the first-layer channel gated features. The first-layer channel gated features are downsampled and used to replace the initial features in S2. S2 is then executed, and so on, to obtain the channel gated features of each layer. S3: Take the channel-gated features of the last layer in step S2, decompose them into spatial detail components and long-range correlation components, and then obtain the spatial detail components and long-range spatial components through linear mapping. The spatial detail component is subjected to depthwise separable convolution, then added element-wise to the spatial detail component and non-linearly activated to obtain the enhanced spatial detail component. Finally, it is multiplied element-wise with the long-range correlation component to obtain the fusion feature of the last layer. The fusion feature of the last layer is refined and then residually superimposed with the spatial detail component features to obtain the fusion output feature of the last layer. It is then upsampled by skip connection with the channel gated feature of the corresponding layer in S2 and replaced with the channel gated feature of the last layer in step S2 in S3. S3 is then executed, and so on, to obtain the fusion output feature of each layer.

[0009] S4: Use the fused output features of the first layer obtained in step S3 as the repaired complete image.

[0010] Furthermore, the reconstruction loss is calculated for the missing mask region in step S1, the gating weights are set for the gating branch in step S2, the gating stability loss and feature alignment loss are calculated, and the multi-scale constraints are calculated for the fused features in step S3. We perform a weighted combination of reconstruction loss, gating stability loss, feature alignment loss, structural consistency constraint, and multi-scale constraint, and iteratively update the results by minimizing them.

[0011] Furthermore, in S1, the image to be repaired is a three-channel RGB image, and the mask for the missing region is a single-channel binary image.

[0012] Furthermore, spatial gating features are obtained in S2. It is achieved through the following formula:

[0013] in, Indicates content branching characteristics, denoted by δ, which represents the gated branch feature; δ represents the nonlinear activation function; and ⊙ represents element-wise multiplication.

[0014] Furthermore, channel gating features are obtained in S2. It is achieved through the following formula:

[0015] in, Represents spatial gating features, and GAP represents global average pooling. Represents the channel mapping function. ⊙ represents the activation function, and ⊙ represents element-wise multiplication.

[0016] Furthermore, in S3, the spatial detail component features and long-range correlation component features obtained through channel splitting are achieved using the following formula:

[0017]

[0018] in, Represents spatial detail components, For the spatial detail components of the split, For long-range correlated components, The long-range correlation component features representing the split are... This represents a linear mapping.

[0019] Furthermore, the enhanced spatial detail components in S3 are obtained through an enhanced response function, the specific calculation formula of which is as follows:

[0020] in, Represents spatial detail components, This represents depthwise separable convolution. ⊕ represents a nonlinear activation function, ⊕ represents element-wise addition, ⊙ represents element-wise multiplication, and y represents an enhanced response function.

[0021] Furthermore, the calculation of fused features in S3 is achieved through the following formula:

[0022] in, Indicates long-range correlation components, R represents the spatial detail components. fusion y represents the fusion feature, y represents the response function, and ⊙ represents element-wise multiplication.

[0023] Furthermore, the fused output features in S3 The calculation is performed using the following formula:

[0024] Among them, R fusion Indicates fusion characteristics, For spatial detail component features, ⊕ represents a linear mapping, and ⊕ represents element-wise addition.

[0025] This invention also provides an image missing region repair system based on multi-scale hybrid feature modeling, comprising the following modules: Initial feature acquisition module: The original image to be repaired and the corresponding missing region mask are input, and the two are concatenated in the channel dimension to obtain the input features. The input features are then expanded by convolution and channel dimension to obtain the initial features.

[0026] The channel-gated feature acquisition module: The initial features are layer-normalized, and then convolved point-by-point along the channel dimension to obtain high-dimensional features containing twice the number of intermediate channels. Depthwise convolution is used to model the high-dimensional features in the spatial domain of the content branch channels to obtain content branch features. Finally, spatial domain modeling is performed on the gated branch channels to obtain gated branch features. The content branch features are non-linearly activated and then multiplied element-wise with the gated branch features to obtain the spatially gated features. Finally, channel compression is performed to restore the initial feature dimensions. The spatial gated features are globally averaged and mapped to channel weights. After nonlinear activation, they are multiplied element-wise with the spatial gated features to obtain the channel gated features of the first layer. The channel gated features of the first layer are downsampled and used to replace the initial features in S2. S2 is then executed, and so on, to obtain the channel gated features of each layer.

[0027] The fusion output feature acquisition module: It takes the channel-gated features from the last layer in step S2, decomposes them into spatial detail component features and long-range correlation component features, and then obtains the spatial detail component and long-range spatial component through linear mapping. The spatial detail component is subjected to depthwise separable convolution, then added element-wise to the spatial detail component and non-linearly activated to obtain the enhanced spatial detail component. Finally, it is multiplied element-wise with the long-range correlation component to obtain the fusion feature of the last layer. The fusion feature of the last layer is refined and then residually superimposed with the spatial detail component features to obtain the fusion output feature of the last layer. It is then upsampled by skip connection with the channel gated feature of the corresponding layer in S2 and replaced with the channel gated feature of the last layer in step S2 in S3. S3 is then executed, and so on, to obtain the fusion output feature of each layer.

[0028] Image repair output module: The fused output features of the first layer obtained in step S3 are used as the repaired complete image.

[0029] Compared with the prior art, the technical solution provided by this invention has the following advantages: This solution enhances the realism of the generated results through a collaborative mechanism of "global structural modeling + local texture reconstruction". Specifically, the encoding end utilizes long-range dependency modeling and multi-scale hierarchical structures to establish stable global semantic constraints, ensuring the structural continuity of large-area missing regions.

[0030] The decoding end enhances the expression of local details through residual gating reconstruction units, and combines spatial gating and channel attention to adaptively enhance key texture areas, thereby effectively reducing blurring, artifacts and boundary breakage.

[0031] In this solution, the overall structure uses lightweight convolution and gating mechanisms to replace some of the heavy attention operations, which significantly reduces the model's dependence on hardware resources. It can complete image restoration processing on local devices or edge terminals without frequently calling cloud computing resources, thereby effectively reducing bandwidth consumption and network latency caused by image data transmission, and improving system real-time performance and engineering feasibility.

[0032] Regarding data security and privacy protection, this solution supports independent image restoration tasks on the device side, avoiding the data exposure risk associated with uploading the original image in traditional cloud-based restoration processes. Because sensitive image data can be processed locally in a closed loop, the probability of leakage during transmission is effectively reduced, resulting in higher data security capabilities. Detailed Implementation

[0033] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments.

[0034] Example 1 This embodiment provides a method for image missing region repair based on multi-scale hybrid feature modeling, including the following steps: S1: Input the original image to be repaired and the corresponding missing region mask, concatenate the two in the channel dimension to obtain the input features, and then expand the input features through convolution and channel dimension to obtain the initial features.

[0035] The input consists of the original image to be repaired and the corresponding missing region mask. The original image is a three-channel RGB image, and the missing region mask is a single-channel binary image used to identify the location of the region to be repaired in the image. The input performs unified processing on the original image and the missing mask, concatenating them along the channel dimension to form input features containing complete contextual information and missing region indication information. This enables the network to clearly distinguish between known and missing regions during subsequent feature modeling, improving the targeting of the repair. The processed input features are then fed into feature embedding. Feature embedding uses convolution operations to extract spatial features and expand the channel dimension of the input features, enhancing feature expressive power while maintaining the original spatial structure information, thus providing a richer feature foundation for deep feature modeling in the subsequent encoding stage.

[0036] S2: Normalize the initial features layer by layer, and perform pointwise convolutions along the channel dimension to obtain high-dimensional features containing twice the number of intermediate channels. Use depthwise convolutions to model the high-dimensional features in the spatial domain on the content branch channels to obtain content branch features. Finally, model the features in the spatial domain on the gated branch channels to obtain gated branch features. The content branch features are non-linearly activated and multiplied element-wise with the gated branch features to obtain spatial gated features. Channel compression is then performed to restore the initial feature dimension. Global average pooling is then performed to map the channel weights. After non-linear activation, these weights are multiplied element-wise with the spatial gated features to obtain the first-layer channel gated features. The first-layer channel gated features are downsampled and used to replace the initial features in S2. S2 is then executed, and so on, to obtain the channel gated features of each layer. In this embodiment, preferably, the channel-gated features of the first layer are downsampled and used as the input features of the second layer to replace the initial features in S2. Then, the operation of step S2 is performed again to obtain the channel-gated features of the second layer. This process is repeated three times to perform downsampling operations to reduce the input high-dimensional feature map in spatial dimension and map it to the deep semantic space, ensuring that the network can capture the global structural information of the image at different resolution scales.

[0037] In this embodiment, preferably, this module employs a spatial channel-gated feedforward network to achieve joint adaptive modeling of spatial information and channel information during deep feature transformation.

[0038] This network uses an internally integrated global statistical unit to extract the distribution patterns of brightness, hue, and macroscopic structure across the entire image area using global average pooling technology, transforming them into a highly abstract global context bias signal that encodes the macroscopic semantic environment of the image.

[0039] This signal is re-extended to the spatial dimension through a broadcast mechanism and aligned element-wise with local features at the channel level, achieving full-scale feature normalization using residual modulation.

[0040] After receiving the initial feature map, the spatial channel-gated feedforward network first performs layer normalization to reduce feature distribution differences and improve the numerical stability of subsequent nonlinear mappings. Then, it expands the initial features in the channel dimension through pointwise convolution to obtain a high-dimensional feature representation containing twice the number of intermediate channels, providing sufficient feature redundancy for gating operations.

[0041] Based on this, spatial domain modeling is performed on the expanded features using depthwise convolution, enabling each channel to independently capture local structural information.

[0042] The convolution result is divided into two parts in the channel dimension. One part is used as the content branch and the other part is used as the gate branch. After the content branch is processed by a non-linear activation function, it is multiplied with the gate branch element by element to complete the spatial gating operation.

[0043] The dynamic modulation of response intensity to characteristics at different spatial locations can be expressed in the following computational form:

[0044] Where x1 is the content feature, x2 is the gating feature, δ represents the nonlinear activation function, ⊙ represents element-wise multiplication, and Fs is the spatial gating feature obtained after multiplying the content branch and the gating branch.

[0045] Subsequently, the feature dimensions are restored to match the input through channel compression mapping to ensure that the module can be seamlessly embedded into the network structure and support residual connections.

[0046] To enhance the discriminative ability at the channel level, global average pooling is applied to the spatially gating features to extract statistical information for each channel. A lightweight mapping network is then used to generate channel weight coefficients, and the features are adaptively modulated channel by channel. This strengthens the response to key channels of the current task and suppresses redundant information. The channel gating process can be represented as follows:

[0047] Wherein, GAP represents global average pooling. Let F represent the channel mapping function, σ represent the activation function, and Fs represent the spatial gating feature obtained after multiplying the content branch and the gating branch. out This is a channel gating feature.

[0048] By organically combining spatial gating and channel gating, the spatial-channel-gated feedforward network maintains the efficient computational characteristics of the feedforward network while achieving fine-grained control over the spatial distribution of features and the importance of channels. This enables the network to more accurately characterize structural and semantic information in visual tasks such as image restoration, thereby improving the robustness of feature representation and reconstruction quality.

[0049] S3: Take the channel-gated features of the last layer in step S2, decompose them into spatial detail components and long-range correlation components, and then obtain the spatial detail components and long-range spatial components through linear mapping. The spatial detail component is subjected to depthwise separable convolution, then added element-wise to the spatial detail component and non-linearly activated to obtain the enhanced spatial detail component. Finally, it is multiplied element-wise with the long-range correlation component to obtain the fusion feature of the last layer. The fusion feature of the last layer is refined and then residually superimposed with the spatial detail component features to obtain the fusion output feature of the last layer. It is then upsampled by skip connection with the channel gated feature of the corresponding layer in S2 and replaced with the channel gated feature of the last layer in step S2 in S3. S3 is then executed, and so on, to obtain the fusion output feature of each layer.

[0050] In this embodiment, preferably, this module uses an integrated residual gating block to construct an active feature generation and dynamic information flow reshaping mechanism, which exhibits extremely high pertinence and operational efficiency in detail restoration.

[0051] This module constructs parallel spatial detail branches and long-range correlation branches, and introduces a residual coupling strategy based on a gating mechanism, enabling information at different scales to interact and selectively enhance within the same layer, thereby significantly improving feature representation capabilities without introducing high computational overhead.

[0052] In the specific implementation process, the residual gating block decomposes the channel gating features of the last layer in step S2 into channels through channel splitting. Spatial detail component features and The long-range spatial component features are obtained by linear mapping to obtain the spatial detail component and the long-range spatial component, which are calculated using the following formula:

[0053]

[0054] in, Represents spatial detail components, For the spatial detail components of the split, For long-range correlated components, The long-range correlation component features representing the split are... This represents a linear mapping.

[0055] The spatial detail component mainly preserves the fine-grained texture and edge information of the image, while the long-range correlation component is used to model long-distance dependencies and the overall semantic structure. In the spatial detail component, depthwise separable convolution is introduced as a location encoding module. By performing spatial convolution operations on each channel independently, spatial location information is effectively encoded while keeping the number of parameters low. The residual concatenation structure ensures the stable propagation of gradients in the deep network, thereby avoiding the gradient vanishing problem common in traditional deep networks.

[0056] Along the spatial detail component path, the module modulates the convolution output using a nonlinear activation function and combines it with residual connections to form an enhanced local response function. The calculation process can be expressed as follows:

[0057] Where DWConv represents depthwise separable convolution, x represents spatial detail component, ⊕ represents element-wise addition, ⊙ represents element-wise multiplication, and δ represents non-linear activation function.

[0058] Through the above-mentioned gating modulation mechanism, the long-range correlation component can determine the "passage degree" of the content branch at each spatial location. While preserving the input feature information, it introduces spatial context, making the spatial detail branch more discriminative and robust, thereby achieving adaptive control of the feature response amplitude.

[0059] The enhanced spatial detail components and long-range correlation components are coupled through element-wise multiplication to obtain fused features, achieving information selection based on a gating mechanism. The fusion process can be represented as follows:

[0060] in, Represents spatial detail components, R represents the long-range spatial components. fusion ⊙ indicates fusion feature, and ⊙ indicates element-wise multiplication.

[0061] This multiplicative fusion method can dynamically modulate the global feature response according to the importance of local structures, enabling the network to automatically strengthen global information consistent with the current semantics and suppress irrelevant or redundant components, thereby improving the effectiveness of feature representation.

[0062] The final layer's fused features are refined through linear mapping and then residual-superimposed with the spatial detail components to obtain the final fused output features:

[0063] Where T represents a linear mapping, R fusion Indicates fusion characteristics, It represents the spatial detail component features.

[0064] The fused output feature of the last layer is upsampled by skipping the channel gating feature of the corresponding layer in S2, and used as the input of the next layer. This replaces the channel gating feature of the last layer in step S2 of this step. S3 is then executed to obtain the fused output feature of the next layer. This process is repeated for each layer, with S3 performed to obtain the fused output feature of each layer. In this embodiment, preferably, three upsampling operations are performed to obtain the fused output feature of the first layer.

[0065] This residual output format retains the basic information of the input features while introducing the fused contextual features, enabling the module to continuously enhance its feature representation capabilities without destroying the original representation.

[0066] This sophisticated design enables deep coupling repair of structure and texture, ensuring a natural transition of texture granularity when object edges cross missing boundaries. This significantly enhances the model's ability to depict complex geometric structures while greatly improving parameter utilization efficiency in the decoding stage, providing solid performance support for high-quality, high-resolution image restoration.

[0067] S4: Use the fused output features of the first layer obtained in step S3 as the repaired complete image.

[0068] In this embodiment, preferably, after completing multi-level decoding and reconstruction, the decoding and reconstruction module outputs a feature representation with the same resolution as the original image. The output module further maps this feature to generate the repaired complete image result. The output repaired image completes the missing areas reasonably and maintains good consistency with the non-missing areas in terms of structure and texture, thereby achieving high-quality repair of the input image.

[0069] Example 2 This embodiment involves model training and optimization prior to embodiment 1.

[0070] Calculate the reconstruction loss for the missing mask region in step S1, set the gating weights for the gated branch in step S2, calculate the gating stability loss and feature alignment loss, and calculate the multi-scale constraints for the fused features in step S3. We perform a weighted combination of reconstruction loss, gating stability loss, feature alignment loss, structural consistency constraint, and multi-scale constraint, and iteratively update the results by minimizing them.

[0071] In this embodiment, an adaptive optimization method for reconstructing missing regions is proposed. Through the collaborative design of mask-aware input, regional differential supervision, gating stability constraints, and feature consistency modulation, the network parameters are jointly optimized, thereby significantly improving the model's recovery capability in scenarios with large-area missing areas and complex structures.

[0072] First, we construct training samples, assuming the complete image is I∈R. H×W×3 The mask for the missing region is M∈{0,1} H×W Where M=0 represents the missing region and M=1 represents the valid region, the damaged image I is generated by element-wise multiplication. d =I⊙M, and concatenate the mask and the damaged image in the channel dimension to form the network input X=Concat(Id,M), so that the model can explicitly perceive the location and extent of the area to be repaired.

[0073] The input is processed by feature embedding, differential encoder, and decoder to obtain the repaired result. =D(E(Φ(X))), where Φ(), E(), and D() represent the feature embedding function, encoding function, and decoding function, respectively.

[0074] To enhance the reconstruction capability of missing regions, this invention employs a region-differentiated reconstruction strategy, applying different weights to the missing and valid regions respectively. The reconstruction loss is defined as follows:

[0075] in For the repaired image, I is the complete image, M is the mask of the missing region, ‖‖1 represents the L1 paradigm, and ⊙ represents element-wise multiplication. The sum of the absolute values ​​of all pixel differences is calculated. If α>β, the model will prioritize optimizing the content generation ability of the missing region.

[0076] Meanwhile, considering the numerous gating structures introduced by the spatial channel gating feedforward network and residual gating blocks in the network, to avoid gating response oversaturation or failure, the gating weight is set as g=σ(z), where σ() is the activation function. Information constraints are applied to it to maintain the effective response range, and its gating stability loss is:

[0077] This improves feature modulation capability and training stability.

[0078] To ensure structural continuity between the repaired region and its context, a structural consistency constraint based on spatial gradient is introduced:

[0079] in The image to be repaired is I, where I represents the complete image, and |||1 represents the L1 paradigm. This represents the gradient operator, used to characterize edge and structural information.

[0080] Furthermore, to further improve the semantic consistency of the generated content, the missing regions and valid regions are statistically aligned in the encoding feature space. The feature alignment loss is: L align =‖μ(F hole )-μ(F valid )‖2+‖σ(F hole )-σ(F valid )‖2 Where F hole F represents the feature representation of the masked portion of an image after it has been encoded by a neural network. validThe feature representations extracted from the original, known parts of the image that are not occluded are given in the same layer of the network. μ() and σ() represent the feature mean and standard deviation, respectively, and |||2 represents the L2 norm.

[0081] To accommodate different levels of missing data, a progressive difficulty scheduling strategy is employed during training, causing the proportion of missing regions to gradually increase with each iteration. The relationship between these changes is as follows:

[0082] Where t is the current iteration step and N is the total number of training steps. This indicates the maximum change. This represents the minimum value of change, allowing the model to gradually transition from learning local texture completion to global structure reconstruction; furthermore, supervision is simultaneously applied to the output results at multiple scales during the decoding stage, forming multi-scale constraints:

[0083] in To repair the resulting image, where I represents the complete image, |||1 represents the L1 paradigm, and S represents the total number of convolution kernel scales, summation is used to converge multi-scale features. This represents the weighting coefficient.

[0084] The reconstruction loss, gating stability loss, feature alignment loss, structural consistency constraint, and multi-scale constraint are weighted and combined as follows: L=L rec +γL str +ηL gate +ξL align +δL ms Where γ represents the structural consistency constraint weight value, η represents the gating stability loss weight value, ξ represents the feature alignment loss weight value, and δ represents the multi-scale constraint weight value.

[0085] The network parameters are iteratively updated by minimizing the objective function.

[0086] I out = ⊙(1-M)+I⊙M in The image represents the repaired image, where I is the complete image and M is the mask for the missing region. out For the final output restored image, ⊙ represents element-wise multiplication.

[0087] Through the above optimization strategies, the model can learn the global structural information and local texture features of the missing region while maintaining computational efficiency, effectively reducing structural breaks, blurring and artifacts, thereby significantly improving the realism, continuity and stability of the image restoration results, and improving the model's generalization ability in complex scenes.

Claims

1. An image missing region repair method based on multi-scale hybrid feature modeling, characterized in that: Includes the following steps: S1: Input the original image to be repaired and the corresponding missing region mask, concatenate the two in the channel dimension to obtain the input features, and expand the input features through convolution and channel dimension to obtain the initial features; S2: Normalize the initial features layer by layer, and perform pointwise convolutions along the channel dimension to obtain high-dimensional features containing twice the number of intermediate channels. Use depthwise convolutions to model the high-dimensional features in the spatial domain on the content branch channels to obtain content branch features. Finally, model the features in the spatial domain on the gated branch channels to obtain gated branch features. The content branch features are non-linearly activated and multiplied element-wise with the gated branch features to obtain spatial gated features. Channel compression is then performed to restore the initial feature dimension. Global average pooling is then performed to map the channel weights. After non-linear activation, these weights are multiplied element-wise with the spatial gated features to obtain the first-layer channel gated features. The first-layer channel gated features are downsampled and used to replace the initial features in S2. S2 is then executed, and so on, to obtain the channel gated features of each layer. S3: Take the channel-gated features of the last layer in step S2, decompose them into spatial detail components and long-range correlation components, and then obtain the spatial detail components and long-range spatial components through linear mapping. The spatial detail component is subjected to depthwise separable convolution, then added element-wise to the spatial detail component and non-linearly activated to obtain the enhanced spatial detail component. Finally, it is multiplied element-wise with the long-range correlation component to obtain the fusion feature of the last layer. The fusion feature of the last layer is refined and then residually superimposed with the spatial detail component features to obtain the fusion output feature of the last layer. It is then upsampled by skip connection with the channel gated feature of the corresponding layer in S2 and replaced with the channel gated feature of the last layer in step S2 in S3. S3 is then executed, and so on, to obtain the fusion output feature of each layer. S4: Use the fused output features of the first layer obtained in step S3 as the repaired complete image.

2. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: Calculate the reconstruction loss for the missing mask region in step S1, set the gating weights for the gated branch in step S2, calculate the gating stability loss and feature alignment loss, and calculate the multi-scale constraints for the fused features in step S3. We perform a weighted combination of reconstruction loss, gating stability loss, feature alignment loss, structural consistency constraint, and multi-scale constraint, and iteratively update the results by minimizing them.

3. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: In S1, the image to be repaired is a three-channel RGB image, and the mask for the missing region is a single-channel binary image.

4. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: Spatial gating features obtained in S2 It is achieved through the following formula: ， in, Indicates content branching characteristics, denoted by δ, which represents the gated branch feature; δ represents the nonlinear activation function; and ⊙ represents element-wise multiplication.

5. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: Channel gating features obtained in S2 It is achieved through the following formula: ， in, Represents spatial gating features, and GAP represents global average pooling. Represents the channel mapping function. ⊙ represents the activation function, and ⊙ represents element-wise multiplication.

6. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: In S3, the spatial detail component features and long-range correlation component features obtained through channel splitting are achieved using the following formula: ， ， in, Represents spatial detail components, For the spatial detail components of the split, For long-range correlated components, The long-range correlation component features representing the split are... This represents a linear mapping.

7. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: The enhanced spatial detail components in S3 are obtained through an enhanced response function, and the specific calculation formula is as follows: ， in, Represents spatial detail components, This represents depthwise separable convolution. ⊕ represents a nonlinear activation function, ⊕ represents element-wise addition, ⊙ represents element-wise multiplication, and y represents an enhanced response function.

8. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 7, characterized in that: The calculation of fused features in S3 is achieved through the following formula: ， in, Indicates long-range correlation components, Represents spatial detail components, y represents the fusion feature, y represents the response function, and ⊙ represents element-wise multiplication.

9. The image missing region repair method based on multi-scale hybrid feature modeling according to claim 1, characterized in that: S3 fused output features The calculation is performed using the following formula: ， in, Indicates fusion features, For spatial detail component features, ⊕ represents a linear mapping, and ⊕ represents element-wise addition.

10. An image missing region repair system based on multi-scale hybrid feature modeling, characterized in that: Includes the following modules: Initial feature acquisition module: Input the original image to be repaired and the corresponding missing region mask, concatenate the two in the channel dimension to obtain the input features, and expand the input features through convolution and channel dimension to obtain the initial features; The channel-gated feature acquisition module: The initial features are layer-normalized, and then convolved point-by-point along the channel dimension to obtain high-dimensional features containing twice the number of intermediate channels. Depthwise convolution is used to model the high-dimensional features in the spatial domain of the content branch channels to obtain content branch features. Finally, spatial domain modeling is performed on the gated branch channels to obtain gated branch features. The content branch features are non-linearly activated and then multiplied element-wise with the gated branch features to obtain the spatially gated features. Finally, channel compression is performed to restore the initial feature dimensions. The spatial gated features are globally averaged and mapped to channel weights. After nonlinear activation, they are multiplied element-wise with the spatial gated features to obtain the channel gated features of the first layer. The channel gated features of the first layer are downsampled and replaced with the initial features in S2. S2 is then executed, and so on, to obtain the channel gated features of each layer. The fusion output feature acquisition module: It takes the channel-gated features from the last layer in step S2, decomposes them into spatial detail component features and long-range correlation component features, and then obtains the spatial detail component and long-range spatial component through linear mapping. The spatial detail component is subjected to depthwise separable convolution, then added element-wise to the spatial detail component and non-linearly activated to obtain the enhanced spatial detail component. Finally, it is multiplied element-wise with the long-range correlation component to obtain the fusion feature of the last layer. The fusion feature of the last layer is refined and then residually superimposed with the spatial detail component features to obtain the fusion output feature of the last layer. It is then upsampled by skip connection with the channel gated feature of the corresponding layer in S2 and replaced with the channel gated feature of the last layer in step S2 in S3. S3 is then executed, and so on, to obtain the fusion output feature of each layer. Image repair output module: The fused output features of the first layer obtained in step S3 are used as the repaired complete image.

Images