Texture-structure guided feature fusion and context refinement image restoration method

Abstract

The invention relates to a texture-structure guided feature fusion and context refinement image restoration method, belongs to the technical field of computer vision and digital image processing, and aims to solve the problems of discontinuous structure, fuzzy texture and poor structure and texture feature fusion effect in the existing image restoration technology. The method comprises the following steps: constructing a repair model comprising a texture generation network TGN, a structure prediction network SPN and an image reconstruction network IRN; a complete grey-scale image and an edge image are generated through a TGN to provide texture guidance, an SPN outputs a globally consistent structure prior image, an initial rough feature map is generated through a double-branch adaptive fusion module, then optimization processing is carried out through a multi-scale context refining module of an IRN, and a final restored image is output. According to the invention, the structure coherence and texture fidelity of the repaired image are effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and digital image processing technology, specifically relating to a texture-structure guided feature fusion and context-refined image restoration method. Background Technology

[0002] Image restoration utilizes the contextual information of known areas in an image to reconstruct missing or occluded areas in a semantically coherent and visually reasonable manner. It is a core technology in the fields of digital image processing and computer vision, and is widely used in scenarios such as old photo restoration and target removal. It has particularly important practical value in the digital protection of cultural heritage such as murals.

[0003] In existing technologies, traditional image inpainting methods are divided into two categories: diffusion-based and block-matching-based. These methods perform well only in scenarios with small-scale rule gaps. When faced with large, irregularly missing areas, they are prone to texture misalignment and structural discontinuities due to a lack of global semantic understanding and structural modeling capabilities. Deep learning-based image inpainting techniques have become the mainstream. Single-stage end-to-end methods struggle to simultaneously maintain global structural consistency and local texture fidelity. While existing multi-stage methods introduce structural priors to guide inpainting through task decomposition, they fail to achieve effective synergistic fusion of structural and texture features and lack the ability to capture long-range contextual dependencies. Consequently, the inpainting results still generally suffer from structural distortion and texture blurring, failing to meet the application requirements of high-precision image inpainting. Summary of the Invention

[0004] The purpose of this invention is to provide a texture-structure guided feature fusion and contextual refinement image inpainting method to address the aforementioned problems. This method integrates structural priors and texture information through a feature fusion module and a contextual refinement module, achieving robust image inpainting while maintaining global structural coherence and local texture consistency. The dual-branch adaptive fusion module dynamically balances texture and structural features based on the SPADE spatial adaptive modulation mechanism and a channel-level gating strategy implemented through gated convolution. Multi-scale contextual refinement adaptively adjusts feature weights at different scales through parallel multi-branch dilated convolution and a large kernel attention mechanism, thereby ensuring that the inpainting result is semantically coherent and visually realistic.

[0005] The method includes the following steps:

[0006] S1. Obtain the damaged RGB image to be repaired and the corresponding binary mask, and generate a damaged grayscale image and a damaged edge image based on the damaged RGB image; S2. Construct a texture-structure guided feature fusion and context refinement model FFCRNet, which includes a texture generation network TGN, a structure prediction network SPN, and an image reconstruction network IRN. The image reconstruction network IRN includes a dual-branch adaptive fusion module and a multi-scale context refinement module. S3. Input the binary mask, damaged grayscale image, and damaged edge image into the pre-trained texture generation network TGN, and output the repaired complete grayscale image and complete edge image; S4. Input the damaged RGB image and the corresponding binary mask into the pre-trained structure prediction network SPN, and output a globally consistent structure prior map; S5. Input the complete grayscale image, complete edge image and structural prior image into the dual-branch adaptive fusion module to generate the initial coarse feature map after fusion; S6. Input the initial coarse feature map, the damaged RGB image, and the binary mask into the pre-trained image reconstruction network IRN. After processing by the multi-scale context refinement module, the final repaired image is output.

[0007] Optionally, in step S3, the damaged grayscale image and the damaged edge image are represented as follows: ; ; in, This is a true grayscale image. For true edge maps, It is a binary mask. For element-wise multiplication; The texture generation network TGN adopts a generator-discriminator architecture. The generator is a dual-branch decoder structure with a shared encoder. A bottleneck layer composed of residual blocks is set between the encoder and the decoder. The dilation rate of the residual blocks is 1, 2, 4, 8, and 16 respectively. The output expression of the texture generation network TGN is: ; in, For TGN generator, This is the complete grayscale image after repair. This is the complete edge map after repair.

[0008] Optionally, in step S3, the total loss function of the texture generation network (TGN) is obtained by weighting and summing the reconstruction loss, perceptual loss, feature matching loss, and adversarial loss using their respective weight parameters. The formula for the total loss function is:

[0009] in, 、 、 and These are the weight parameters for reconstruction loss, perception loss, feature matching loss, and adversarial loss, respectively. The formula for reconstruction loss is: ; To achieve perceptual loss, the semantic and perceptual similarity between generated and real images is enhanced by comparing the differences between feature activation maps extracted from the pre-trained VGG-16 network, as shown in the formula: ; in, Indicates VGG-16 i Activation map of each pooling layer; The feature matching loss, derived from pix2pixHD, is used to stabilize adversarial training and promotes detail generation by constraining the consistency between generated and real images across the discriminator's multi-layer features. The formula is: ; in ， Indicates the first i The number of elements activated in the layer. It is a discriminator No. i Layer activation, T The total number of discriminator layers; To counteract losses, the formula is: ; in, For splicing.

[0010] Optionally, in step S4, the structure prediction network SPN adopts a pre-trained masked autoencoder MAE, and the masking strategy of MAE adopts a deterministic mask specified by a binary mask M. MAE's encoder is used to process visible image patches, and the decoder combines the encoded visible token, the learnable mask token, and the full location embedding to reconstruct the structural prior map. The damaged RGB image is represented as follows: ; in, A true RGB image. M For a binary mask, ⊙ represents element-wise multiplication; The output expression of the structure prediction network SPN is: ; in, This is a globally consistent structural prior graph.

[0011] Optionally, the specific steps of step S5 are as follows: S51. Transfer the complete grayscale image With complete edge map Initial features are obtained by channel splicing. F 1. For F 1. After processing by SPADE branch and gated convolution branch respectively, texture features are obtained by adaptive weighted fusion; The calculation process for the texture features includes: Preliminary fused features are obtained through independent convolutions: ; ; The SPADE branch output is: ; The output of the gated convolution branch is: ; The expression for calculating the fusion weight graph is: ; in, , These correspond to the fusion weights of the SPADE branch and the gated convolution branch, respectively. The texture feature expression is: ; S52. Integrating texture features with structural prior maps An initial coarse feature map is generated by fusing and aligning the features using the same dual-branch structure. F 0.

[0012] Optionally, in step S6, the image reconstruction network IRN adopts a generator-discriminator architecture, the generator adopts an encoder-decoder structure, and the multi-scale context refinement module is set in the bottleneck layer between the generator encoder and the decoder. The output expression of the image reconstruction network IRN is:

[0013] in, For IRN generators, F 0 represents the initial coarse feature map. For damaged RGB images, M It is a binary mask. For the final image repair.

[0014] Optionally, the total loss function of the image reconstruction network (IRN) is obtained by weighting and summing the reconstruction loss, perceptual loss, style loss, and adversarial loss using corresponding scaling parameters. The expression for the total loss function is:

[0015] in, Scaling parameters for reconstruction loss 、 Scaling parameters for perceptual loss 、 The scaling parameter for style loss. Scaling parameters to counteract loss; The formula for reconstruction loss is: ; To express the perceived loss, the formula is: ; in, Indicates the VGG-16's i Activation map of each pooling layer; The formula for style loss is: ; in, , For activation graph The constructed Gram matrix; To counteract losses, the formula is: ; in, It is the discriminator for IRN.

[0016] Optionally, in step S6, the multi-scale context refinement module includes a multi-scale parallel dilated convolution module (MSPP). The MSPP has four parallel dilated convolution branches with dilation rates of 1, 2, 4, and 8, which extract multi-granularity local features and then fuse them. The enhanced feature representation is obtained through residual connections. The output expression of the MSPP is: ; in, The encoded features output by the IRN encoder of the image reconstruction network. To enhance feature representation.

[0017] Optionally, the multi-scale context refinement module further includes a large-kernel efficient attention mechanism (LKA), which sequentially passes through a large-kernel spatial attention module and an efficient channel attention module to enhance the feature representation. Weighted modulation of spatial and channel dimensions is performed to capture long-range dependencies; The output expression of the large-kernel spatial attention module is: ; in, For the generated spatial attention map, The sigmoid activation function is used, and ⊙ represents element-wise multiplication; The output expression of the high-efficiency channel attention module is: ; in, For the generated channel attention weights, This represents the final characteristic after LKA modulation.

[0018] As can be seen from the above technical solutions, the present invention has the following advantages: This invention solves the problem of existing methods failing to simultaneously ensure global structural coherence and local texture fidelity by decoupling the texture generation network and the structure prediction network, thereby obtaining complete texture detail cues and globally consistent structural priors. This avoids structural breaks and texture blurring in the restoration results. A dual-branch adaptive fusion module enables dynamic adaptive fusion of texture and structural features, avoiding unnatural boundary transitions and semantic drift caused by direct feature splicing. Combined with a multi-scale context refinement module, long-range dependencies are captured, ensuring consistency between the restored area and the global semantics. This invention maintains stable and excellent restoration performance across a wide occlusion range of 0%-60%, making it widely applicable to practical scenarios such as mural restoration and old photograph restoration. Attached Figure Description

[0019] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the texture-structure guided feature fusion and context-refined image restoration method in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall architecture of the texture-structure guided feature fusion and context refinement model FFCRNet in an embodiment of the present invention, wherein, Figure 2(a) in the diagram represents the texture generation network. Figure 2 (b) in the diagram represents the structure prediction network. Figure 2 (c) in the diagram represents the image reconstruction network; Figure 3 This is a schematic diagram of the dual-branch adaptive fusion module in an embodiment of the present invention, wherein, Figure 3 (a) in the diagram represents the edge and grayscale fusion submodule. Figure 3 (b) in the diagram represents the texture and structure fusion submodule; Figure 4 This is a schematic diagram of the multi-scale context refinement module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the large-kernel efficient attention mechanism LKA in an embodiment of the present invention, wherein, Figure 5 (a) in the diagram represents the large-kernel spatial attention module. Figure 5 (b) in the diagram represents the efficient channel attention module. Detailed Implementation

[0021] The various embodiments of the present invention will be described more comprehensively in the detailed steps of the texture-structure guided feature fusion and contextual refinement image inpainting method described below. To make the inventive objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solutions protected by the present invention. Obviously, the embodiments described below are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0024] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0025] Please see Figure 1 The diagram shows a flowchart of a texture-structure-guided feature fusion and context-refined image inpainting method, which includes: S1. Obtain the damaged RGB image to be repaired and the corresponding binary mask, and generate a damaged grayscale image and a damaged edge image based on the damaged RGB image; S2. Construct a texture-structure guided feature fusion and context refinement model FFCRNet, which includes a texture generation network TGN, a structure prediction network SPN, and an image reconstruction network IRN. The image reconstruction network IRN includes a dual-branch adaptive fusion module and a multi-scale context refinement module. S3. Input the binary mask, damaged grayscale image, and damaged edge image into the pre-trained texture generation network TGN, and output the repaired complete grayscale image and complete edge image; S4. Input the damaged RGB image and the corresponding binary mask into the pre-trained structure prediction network SPN, and output a globally consistent structure prior map; S5. Input the complete grayscale image, complete edge image and structural prior image into the dual-branch adaptive fusion module to generate the initial coarse feature map after fusion; S6. Input the initial coarse feature map, the damaged RGB image, and the binary mask into the pre-trained image reconstruction network IRN. After processing by the multi-scale context refinement module, the final repaired image is output.

[0026] As a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another texture-structure guided feature fusion and context-refined image inpainting method is provided, which includes the following steps: S1. Obtain the damaged RGB image to be repaired and the corresponding binary mask, and generate a damaged grayscale image and a damaged edge image based on the damaged RGB image; S2. Construct the texture-structure guided feature fusion and context refinement model FFCRNet, which includes a texture generation network TGN, a structure prediction network SPN, and an image reconstruction network IRN, such as Figure 2 As shown, Figure 2 (a) in the diagram represents the texture generation network. Figure 2 (b) in the diagram represents the structure prediction network. Figure 2 (c) in the diagram represents the image reconstruction network; S3. Input the binary mask, damaged grayscale image, and damaged edge image into the pre-trained texture generation network TGN, and output the repaired complete grayscale image and complete edge image; In this embodiment of the invention, in step S3, the damaged grayscale image and the damaged edge image are represented as follows: ; ; in, This is a true grayscale image. For true edge maps, It is a binary mask. For element-wise multiplication; The texture generation network (TGN) employs a generator-discriminator architecture. The generator is a dual-branch decoder structure with a shared encoder. A bottleneck layer composed of residual blocks is placed between the encoder and decoder, with the dilation rates of the residual blocks being 1, 2, 4, 8, and 16, respectively. The dual-branch decoder upsamples features on each branch and fuses them with features from the corresponding encoder stage through skip connections. Finally, each branch outputs the repaired edge map and grayscale image through convolutional layers, which are then processed by the discriminator. Constraints.

[0027] The output expression of the texture generation network TGN is: ; in, For TGN generator, This is the complete grayscale image after repair. This is the complete edge map after repair.

[0028] In step S3, the total loss function of the texture generation network (TGN) is obtained by weighting and summing the reconstruction loss, perceptual loss, feature matching loss, and adversarial loss using their respective weight parameters. The formula for the total loss function is:

[0029] in, 、 、 and These are the weight parameters for reconstruction loss, perception loss, feature matching loss, and adversarial loss, respectively. The formula for reconstruction loss is: ; To achieve perceptual loss, the semantic and perceptual similarity between generated and real images is enhanced by comparing the differences between feature activation maps extracted from the pre-trained VGG-16 network, as shown in the formula: ; in, Indicates VGG-16 i Activation map of each pooling layer; The feature matching loss, derived from pix2pixHD, is used to stabilize adversarial training and promotes detail generation by constraining the consistency between generated and real images across the discriminator's multi-layer features. The formula is: ; in ， Indicates the first i The number of elements activated in the layer. It is a discriminator No. i Layer activation, T The total number of discriminator layers; To counteract losses, the formula is: ; in, For splicing.

[0030] S4. Input the damaged RGB image and the corresponding binary mask into the pre-trained structure prediction network SPN, and output a globally consistent structure prior map; In this embodiment of the invention, in step S4, the structure prediction network SPN adopts a pre-trained masked autoencoder MAE, and the masking strategy of MAE adopts a deterministic mask specified by a binary mask M. MAE's encoder is used to process visible image patches, and the decoder combines the encoded visible token, the learnable mask token, and the full location embedding to reconstruct the structural prior map. The damaged RGB image is represented as follows: ; in, A true RGB image. M For a binary mask, ⊙ represents element-wise multiplication; The output expression of the structure prediction network SPN is: ; in, This is a globally consistent structural prior graph.

[0031] It should be noted that the encoder only processes visible image patches, while the decoder reconstructs the structural graph by combining the encoded visible tokens with learnable mask tokens and full location embeddings. In this network, the masking strategy of the pre-trained mask autoencoder is changed from random masking to a deterministic masking specified by the occlusion mask M, to ensure that the predicted structure is accurately aligned with the missing regions. It provides a globally consistent spatial layout for subsequent image reconstruction networks. Compared with existing methods, the MAE-based structure prediction network better maintains global structural rationality and semantic consistency.

[0032] S5. Input the complete grayscale image, complete edge image and structural prior image into the dual-branch adaptive fusion module to generate the initial coarse feature map after fusion; It should be noted that the Image Reconstruction Network (IRN) aims to integrate texture and structural information to generate the final restored image, including a bi-branch adaptive fusion module and a multi-scale contextual refinement module. Specifically, and Texture features are generated by fusing through a dual-branch adaptive fusion module, and then combined with... Based on the preliminary rough results of the reconstruction Finally, a generator equipped with a multi-scale contextual refinement module produces the final restored image. At the same time, a discriminator is used to ensure visual realism. For example... Figure 3 The diagram shown is a schematic of the dual-branch adaptive fusion module. Figure 3 (a) in the diagram represents the edge and grayscale fusion submodule. Figure 3 (b) represents the texture and structure fusion submodule; the SPADE branch modulates normalized features through edge-guided affine transformations, thereby providing spatially variable structure regularization. Simultaneously, as a learnable feature filter, the gated convolution branch suppresses semantic noise in damaged areas and selectively enhances texture details. By integrating the two branches, this module ensures boundary consistency and improves detail realism.

[0033] In this embodiment of the invention, step S5 specifically involves the following steps: S51. Transfer the complete grayscale image With complete edge map Initial features are obtained by channel splicing. F 1. For F 1. After processing by SPADE branch and gated convolution branch respectively, texture features are obtained by adaptive weighted fusion; The calculation process for the texture features includes: Preliminary fused features are obtained through independent convolutions: ; ; The SPADE branch output is: ; The output of the gated convolution branch is: ; The expression for calculating the fusion weight graph is: ; in, , These correspond to the fusion weights of the SPADE branch and the gated convolution branch, respectively. The texture feature expression is: ; S52. Integrating texture features with structural prior maps An initial coarse feature map is generated by fusing and aligning the features using the same dual-branch structure. F 0.

[0034] S6. Input the initial coarse feature map, the damaged RGB image, and the binary mask into the pre-trained image reconstruction network IRN. After processing by the multi-scale context refinement module, the final repaired image is output.

[0035] In this embodiment of the invention, in step S6, the image reconstruction network IRN adopts a generator-discriminator architecture, the generator adopts an encoder-decoder structure, and the multi-scale context refinement module is set in the bottleneck layer between the generator encoder and the decoder. The output expression of the image reconstruction network IRN is:

[0036] in, For IRN generators, F 0 represents the initial coarse feature map. For damaged RGB images, M It is a binary mask. For the final image repair.

[0037] The total loss function of the image reconstruction network IRN is obtained by weighted summation of reconstruction loss, perceptual loss, style loss, and adversarial loss using corresponding scaling parameters. The expression for the total loss function is:

[0038] in, Scaling parameters for reconstruction loss 、 Scaling parameters for perceptual loss 、 The scaling parameter for style loss. Scaling parameters to counteract loss; The formula for reconstruction loss is: ; To express the perceived loss, the formula is: ; in, Indicates the VGG-16's i Activation map of each pooling layer; The formula for style loss is: ; in, , For activation graph The constructed Gram matrix; To counteract losses, the formula is: ; in, It is the discriminator for IRN.

[0039] In step S6, the multi-scale context refinement module includes a multi-scale parallel dilated convolution module (MSPP), which has four parallel dilated convolution branches, such as... Figure 4 As shown, the porosity is 1, 2, 4, and 8, respectively. After extracting multi-granularity local features and fusing them, the enhanced feature representation is obtained through residual connections. The output expression of MSPP is: ; in, The encoded features output by the image reconstruction network IRN encoder. To enhance feature representation.

[0040] In this embodiment of the invention, the multi-scale context refinement module further includes a large-kernel efficient attention mechanism, such as LKA. Figure 5 As shown, the LKA sequentially passes through a large-kernel spatial attention module and a high-efficiency channel attention module. Figure 5 (a) in the diagram represents the large-kernel spatial attention module. Figure 5 (b) in the diagram represents the efficient channel attention module, which enhances feature representation. Weighted modulation of spatial and channel dimensions is performed to capture long-range dependencies; The output expression of the large-kernel spatial attention module is: ; in, For the generated spatial attention map, The sigmoid activation function is used, and ⊙ represents element-wise multiplication; The output expression of the high-efficiency channel attention module is: ; in, For the generated channel attention weights, This represents the final characteristic after LKA modulation.

[0041] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0042] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A texture-structure guided feature fusion and context-refined image inpainting method, characterized in that, The method includes the following steps: S1. Obtain the damaged RGB image to be repaired and the corresponding binary mask, and generate a damaged grayscale image and a damaged edge image based on the damaged RGB image; S2. Construct a texture-structure guided feature fusion and context refinement model FFCRNet, which includes a texture generation network TGN, a structure prediction network SPN, and an image reconstruction network IRN. The image reconstruction network IRN includes a dual-branch adaptive fusion module and a multi-scale context refinement module. S3. Input the binary mask, damaged grayscale image, and damaged edge image into the pre-trained texture generation network TGN, and output the repaired complete grayscale image and complete edge image; S4. Input the damaged RGB image and the corresponding binary mask into the pre-trained structure prediction network SPN, and output a globally consistent structure prior map; S5. Input the complete grayscale image, complete edge image and structural prior image into the dual-branch adaptive fusion module to generate the initial coarse feature map after fusion; S6. Input the initial coarse feature map, the damaged RGB image, and the binary mask into the pre-trained image reconstruction network IRN. After processing by the multi-scale context refinement module, the final repaired image is output.

2. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 1, characterized in that, In step S3, the damaged grayscale image and the damaged edge image are represented as follows: ； ； in, This is a true grayscale image. For true edge maps, It is a binary mask. For element-wise multiplication; The texture generation network TGN adopts a generator-discriminator architecture. The generator is a dual-branch decoder structure with a shared encoder. A bottleneck layer composed of residual blocks is set between the encoder and the decoder. The dilation rate of the residual blocks is 1, 2, 4, 8, and 16 respectively. The output expression of the texture generation network TGN is: ； in, For TGN generator, This is the complete grayscale image after repair. This is the complete edge map after repair.

3. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 2, characterized in that, In step S3, the total loss function of the texture generation network (TGN) is obtained by weighting and summing the reconstruction loss, perceptual loss, feature matching loss, and adversarial loss using their respective weight parameters. The formula for the total loss function is: in, 、 、 and These are the weight parameters for reconstruction loss, perception loss, feature matching loss, and adversarial loss, respectively. The formula for reconstruction loss is: ； To achieve perceptual loss, the semantic and perceptual similarity between generated and real images is enhanced by comparing the differences between feature activation maps extracted from the pre-trained VGG-16 network, as shown in the formula: ； in, Indicates VGG-16 i Activation map of each pooling layer; The feature matching loss, derived from pix2pixHD, is used to stabilize adversarial training and promotes detail generation by constraining the consistency between generated and real images across the discriminator's multi-layer features. The formula is: ； in ， Indicates the first i The number of elements activated in the layer. It is a discriminator No. i Layer activation, T The total number of discriminator layers; To counteract losses, the formula is: ； in, For splicing.

4. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 1, characterized in that, In step S4, the structure prediction network SPN uses a pre-trained masked autoencoder MAE, and the masking strategy of MAE uses a deterministic mask specified by a binary mask M. MAE's encoder is used to process visible image patches, and the decoder combines the encoded visible token, the learnable mask token, and the full location embedding to reconstruct the structural prior map. The damaged RGB image is represented as follows: ； in, A true RGB image. M For a binary mask, ⊙ represents element-wise multiplication; The output expression of the structure prediction network SPN is: ； in, This is a globally consistent structural prior graph.

5. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 1, characterized in that, The specific steps of step S5 are as follows: S51. Transfer the complete grayscale image With complete edge map Initial features are obtained by channel splicing. F 1. For F 1. After processing by SPADE branch and gated convolution branch respectively, texture features are obtained by adaptive weighted fusion; The calculation process for the texture features includes: Preliminary fusion features are obtained through independent convolutions: ； ； The SPADE branch output is: ； The output of the gated convolution branch is: ； The expression for calculating the fusion weight graph is: ； in, , These correspond to the fusion weights of the SPADE branch and the gated convolution branch, respectively. The texture feature expression is: ； S52. Integrating texture features with structural prior maps An initial coarse feature map is generated by fusing and aligning the features using the same dual-branch structure. F 0.

6. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 1, characterized in that, In step S6, the image reconstruction network IRN adopts a generator-discriminator architecture, the generator adopts an encoder-decoder structure, and the multi-scale context refinement module is set in the bottleneck layer between the generator encoder and the decoder. The output expression of the image reconstruction network IRN is: in, For IRN generators, F 0 represents the initial coarse feature map. For damaged RGB images, M It is a binary mask. For the final image repair.

7. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 6, characterized in that, The total loss function of the image reconstruction network IRN is obtained by weighted summation of reconstruction loss, perceptual loss, style loss, and adversarial loss using corresponding scaling parameters. The expression for the total loss function is: in, Scaling parameters for reconstruction loss 、 Scaling parameters for perceptual loss 、 The scaling parameter for style loss. Scaling parameters to counteract loss; The formula for reconstruction loss is: ； To express the perceived loss, the formula is: ； in, Indicates the VGG-16's i Activation map of each pooling layer; The formula for style loss is: ； in, , For activation graph The constructed Gram matrix; To counteract losses, the formula is: ； in, It is the discriminator for IRN.

8. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 1, characterized in that, In step S6, the multi-scale context refinement module includes a multi-scale parallel dilated convolution module (MSPP). The MSPP has four parallel dilated convolution branches with dilation rates of 1, 2, 4, and 8. After extracting multi-granularity local features, these are fused and then connected via residual connections to obtain the enhanced feature representation. The output expression of the MSPP is: ； in, The encoded features output by the image reconstruction network IRN encoder. To enhance feature representation.

9. The texture-structure guided feature fusion and context-refined image inpainting method according to claim 8, characterized in that, The multi-scale context refinement module also includes a large-kernel efficient attention mechanism (LKA), which sequentially passes through a large-kernel spatial attention module and an efficient channel attention module to enhance the feature representation. Weighted modulation of spatial and channel dimensions is performed to capture long-range dependencies; The output expression of the large-kernel spatial attention module is: ； in, For the generated spatial attention map, The sigmoid activation function is used, and ⊙ represents element-wise multiplication; The output expression of the high-efficiency channel attention module is: ； in, For the generated channel attention weights, This represents the final characteristic after LKA modulation.

Images