Cultural relic image restoration method based on text-to-image large model and reference image guidance

Abstract

The application claims a kind of cultural relic image restoration method based on text-to-image large model and reference image guidance, first, the cultural relic image to be repaired is preprocessed, the damaged area mask information is obtained, and the multi-modal repair condition is constructed in combination with the text description input by user;Subsequently, under the joint constraint of text condition and reference image feature, the text-to-image large model generates a candidate repair result consistent with the original cultural relic in content structure, texture details and artistic style.The application realizes the fine completion of the damaged area by cross-modal feature alignment and adaptive feature fusion mechanism, which cooperatively models the text semantic feature, reference image visual feature and context information of the image to be repaired, thereby realizing the fine completion of the damaged area.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology, particularly its application in the restoration of cultural relic images. Specifically, it relates to a method for restoring cultural relic images based on a large text-image model and reference image guidance. This method utilizes a large text-image model to fuse text descriptions and visual features of reference images, thereby achieving automatic restoration of damaged or missing parts in cultural relic images. It is widely used in digital cultural relic protection, cultural heritage restoration, and image generation tasks in computer vision. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence and computer-aided design, the field of cultural relic restoration has also begun to shift towards digital virtual restoration, gradually relying on intelligent algorithms and automated restoration techniques. The application of these emerging technologies can restore the details and artistic style of cultural relics without damaging the original artifacts, thus improving restoration efficiency. Digital cultural relic restoration provides entirely new ideas and methods for the protection and restoration of cultural relics, especially when facing complex damage or loss of detail that is difficult to repair, where digital restoration demonstrates great potential. At the same time, it has broad application prospects in many fields such as virtual museum construction, digital archiving of cultural heritage, and the visual reconstruction of historical sites.

[0003] Current image restoration research primarily focuses on natural images, with limited application to historical or cultural relic images. Cultural relic images and natural images differ significantly in visual characteristics, detail processing, and creative intent. Natural images typically depict objects and scenes from the real world, with relatively regular details and textures; restoration efforts emphasize restoring physical properties and a sense of naturalness. In contrast, cultural relic images, such as ancient murals and sculptures, often possess colors and textures honed by time, exhibiting unique historical features and cultural characteristics. Restoration requires not only restoring the image's structure and details but also preserving the historical value and artistic style of the relic. Furthermore, damage in cultural relic images often involves traces of time and cultural background; restoration must consider these historical and emotional factors, whereas natural image restoration focuses more on the reproduction of objective reality. Therefore, cultural relic image restoration demands, in addition to technical precision, a greater emphasis on restoring historical and artistic value.

[0004] Existing methods for artifact image restoration mainly focus on traditional image processing techniques and deep learning-based automatic restoration methods. Traditional methods often rely on manual restoration or rule-based algorithms, such as image inpainting, interpolation, and texture synthesis. While these methods can restore missing surface parts of artifact images to some extent, their effectiveness is limited when dealing with details, stylistic consistency, and complex backgrounds. Existing deep learning methods are mostly limited to single-modal restoration, lacking a comprehensive consideration of the inherent semantics and artistic style of artifact images. Furthermore, these methods typically require large amounts of paired data for training, but obtaining large-scale labeled data is often quite difficult in the field of artifact image restoration.

[0005] This invention studies a method for restoring cultural relics images based on a large text-based image model and a reference image. This method improves the detail recovery capability and artistic style consistency of the restored image by combining multimodal features and cross-modal attention mechanisms. For cultural relics images with large areas of missing data, this invention proposes an innovative restoration scheme that does not rely on paired training datasets. Specifically, the method collects relevant text descriptions of the image to be restored and a reference image of the same style, using this information to guide the restoration process, thereby achieving high-quality image restoration. Therefore, the core challenge of this research lies in how to utilize multimodal information for restoration under conditions of lacking paired data, ensuring the complete restoration of the image's style, texture, and structure, while simultaneously considering both generation efficiency and artistic accuracy during the restoration process.

[0006] Unlike the method in patent application publication number 2025101649012, which relies on large paired datasets and LoRA fine-tuning techniques, the inpainting method proposed in this invention, based on a large model of text-derived images and reference image guidance, does not require paired datasets. It solves the problem of difficult data collection by combining text descriptions and reference images for unsupervised inpainting. Furthermore, this invention utilizes a pre-trained large model and avoids the need for large-scale fine-tuning through cross-modal attention mechanisms and adaptive feature fusion, thus improving the stability and style consistency of the inpainting results. In addition, the consistency of the inpainting results in terms of detail and artistic style is better guaranteed, especially when dealing with complex damaged areas, where this invention can ensure accurate matching of semantics and visual effects. Summary of the Invention

[0007] This invention aims to solve the problems of the prior art. It proposes a method for cultural relic image restoration based on a text-based image model and a reference image. The resulting model is an end-to-end cultural relic image restoration model, which takes the image of the cultural relic to be restored, a text description, and a reference image of the cultural relic as input. Through a multimodal restoration model that requires no additional training, it fully combines the features of the text and the image to achieve reasonable filling of missing areas, ensuring the consistency of the restoration result in semantics and artistic style.

[0008] The technical solution of the present invention is as follows:

[0009] A method for restoring cultural relics images based on large-scale models of original images and reference images, comprising the following steps:

[0010] Step 1: Obtain an image of the cultural relic to be restored, and preprocess the image, including adjusting the image size, identifying and marking the damaged areas in the image, and generating corresponding damaged area mask information to provide accurate input for subsequent restoration steps;

[0011] Step 2: Based on the visual content and attribute information of the cultural relic image, construct text description information corresponding to the cultural relic image. The text description information is used to characterize the subject matter, patterns, period characteristics, or artistic style of the cultural relic.

[0012] Step 3: Obtain at least one reference image, which is similar to the image of the cultural relic to be restored in terms of content structure, local texture or overall artistic style;

[0013] Step 4: Extract features from the reference image to obtain a visual feature representation of the reference image used for guided repair;

[0014] Step 5: Extract features from the text description information to obtain a semantic feature representation of the text description used to guide the repair;

[0015] Step 6: Construct a multimodal conditional input, taking the image of the cultural relic to be restored, the mask of the damaged area, the semantic features of the text, and the visual features of the reference image as conditional inputs to the large model of the text-to-image;

[0016] Step 7: Based on the generation mechanism of the large model of the text image, under the constraints of multimodal conditions, the damaged area is gradually generated or completed to obtain the initial repair result;

[0017] Step 8: Adjust the fusion weight between the repaired area and the undamaged area based on the confidence level or feature consistency of the generated results;

[0018] Step 9: Output the final image restoration result of the cultural relic after adjustment.

[0019] Furthermore, step 1 specifically includes: the data used for the image of the cultural relic to be restored is a digital image of the cultural relic, which includes images of cultural relics with historical and artistic value, such as grotto murals and landscape paintings, and contains image degradation information caused by natural aging, environmental erosion or human factors. The degradation information is manifested as local missing parts, color decay, texture breakage or structural discontinuity. The mask of the damaged area is a binary mask, which is used to distinguish the damaged area from the non-damaged area.

[0020] Furthermore, in step 2, the text description information is obtained by user input, automatically generated from the cultural relics database, or through semantic analysis of the cultural relics images.

[0021] Furthermore, in step 3: the reference image is derived from historical images of the same cultural relic, images of cultural relics of the same category, or images of cultural relics from the same period or with the same artistic style.

[0022] Furthermore, in steps 4 and 5: the feature extractor for the reference image and text description is a multimodal feature extraction model CLIP trained using contrastive learning. The CLIP model includes a text encoder and an image encoder. The text encoder is used to encode the text description to obtain text semantic features. Specifically, the text encoder adopts the BERT (Bidirectional Encoder Representations from Transformers) encoding method based on the Transformer architecture to capture deep semantic information in the text. The image encoder is used to encode the reference image to obtain image visual features. Specifically, the image encoder adopts the ViT (Vision Transformer) encoding method, extracting local and global features of the image through convolutional layers or self-attention mechanisms to obtain a high-dimensional image representation.

[0023] Furthermore, step 6 specifically includes: the large-scale model for the text-based image is a stable diffusion model (Stable Diffusion) combined with a lightweight adapter (IP-Adapter). Stable Diffusion performs diffusion modeling and generation of the image in the latent space. The IP-Adapter is used to inject textual semantic features and reference image visual features as conditions into the latent space diffusion process without changing the main structure of Stable Diffusion, thereby achieving the generation or reconstruction of the cultural relic image under multimodal constraints. The pre-trained model used by Stable Diffusion is a restoration model (stable-diffusion-xl-1.0-inpainting-0.1). The training objective function is:

[0024] .

[0025] in, Images representing cultural relics awaiting restoration. Indicates noise. This indicates the text description conditions. Indicates the conditions of the reference image. Indicates the time step. Indicates model parameters, Indicates time step The output image below.

[0026] Furthermore, step 7 specifically includes: the multimodal conditional constraint is a decoupled cross-attention mechanism that injects text semantic features and reference image visual features into the IP-Adapter, compatible with both text and images. The decoupled cross-attention refers to adding only one additional cross-attention layer for image features in each cross-attention layer of the Stable Diffusion UNet. During the training phase, only the parameters of the new cross-attention layer are trained, while the original UNet model remains frozen. Thus, the lightweight adapter can simultaneously accept text and images as input, providing additional control for the large text-to-image model, thereby providing reference image-guided restoration of cultural relics. The new decoupled cross-attention expression is as follows:

[0027] = V +

[0028] in, Since the original UNet network is frozen, only [the following is possible] in decoupled cross-attention. and It is trainable; This represents the new feature vector output by the attention mechanism layer. This represents the query matrix, which is composed of the input... With weight matrix The product obtained by multiplication; and Represents the "index" or "feature description" of the retrieved object; V and Represents the "specific content" of the object being retrieved; This represents the scaling factor.

[0029] Furthermore, in step 8, the fusion weight is the cue word guidance coefficient in the adapter, used to control the strictness with which the text-based image model should follow the cue words during sampling. Specifically, the fusion weight is dynamically adjusted according to the complexity of the repaired region and the requirements of the generated effect. For simple or known repaired regions, the fusion weight is lower, allowing the model to rely more on the visual features of the reference image; while for complex repaired regions with high detail requirements, the fusion weight is higher, and the model will more strictly follow the semantic guidance of the cue words, thereby ensuring the consistency of the generated result in style and semantics. This adjustment mechanism improves the accuracy and artistry of the repair result by balancing the relationship between semantic guidance and visual features through real-time optimization of weights during the generation process.

[0030] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the artifact image restoration method based on a large model of the artifact image and a reference image as described in any one of the claims.

[0031] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the artifact image restoration method based on a large model of the original image and a reference image as described in any one of the claims.

[0032] The advantages and beneficial effects of this invention are as follows:

[0033] This invention combines text descriptions and reference images, employing a decoupled cross-attention mechanism. Based on a text-generated image model and guided by reference images, this method for restoring cultural relics eliminates the need for large paired datasets, reducing data preparation work. This method effectively restores images of cultural relics with large areas of loss, ensuring accurate restoration of details while preserving the artistic style and historical authenticity of the relics, avoiding the stylistic inconsistencies and content leakage problems inherent in traditional restoration methods. Furthermore, the automated restoration process improves efficiency, making it particularly suitable for image restoration with complex backgrounds and missing details. It has broad application prospects and can provide technical support in various fields such as virtual museum construction and cultural heritage protection.

[0034] The innovation of this invention lies in its restoration method combining multimodal features and cross-modal attention mechanisms, as well as its restoration strategy that does not rely on paired labeled data. These steps and techniques are not conventional, because cross-modal feature fusion requires the model to simultaneously process and coordinate different types of data (text and images) and dynamically adjust their relative weights during generation. This requires not only powerful model capabilities but also complex mechanisms to ensure style consistency and semantic accuracy. Furthermore, traditional image restoration methods rely on training with large amounts of paired data, while cultural relic image restoration often lacks large-scale labeled datasets. This invention effectively circumvents this problem by using a pre-trained large model and reference images as guidance, providing a restoration solution that does not require paired datasets. The introduction of an adaptive feature fusion mechanism further breaks through the fixed weight limitation of traditional image restoration methods, enabling flexible adjustment of the generation strategy according to the complexity of the image, avoiding abrupt boundary changes or style inconsistencies, and further improving the restoration effect. Attached Figure Description

[0035] Figure 1 This invention provides a preferred embodiment of a training-free end-to-end cultural relic image restoration model.

[0036] Figure 2 This is a rendering of the artifact image restoration effect of the present invention;

[0037] Figure 3 This is a flowchart of the algorithm of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0039] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0040] A method for restoring cultural relics images based on large-scale models of original images and reference images, comprising the following steps:

[0041] Step 1: Obtain an image of the cultural relic to be repaired, and preprocess the image to generate corresponding mask information for the damaged area;

[0042] Step 2: Based on the visual content and attribute information of the cultural relic image, construct text description information corresponding to the cultural relic image. The text description information is used to characterize the subject matter, patterns, period characteristics, or artistic style of the cultural relic.

[0043] Step 3: Obtain at least one reference image, which is similar to the image of the cultural relic to be restored in terms of content structure, local texture or overall artistic style;

[0044] Step 4: Extract features from the reference image to obtain a visual feature representation of the reference image used for guided repair;

[0045] Step 5: Extract features from the text description information to obtain a semantic feature representation of the text description used to guide the repair;

[0046] Step 6: Construct a multimodal conditional input, taking the image of the cultural relic to be restored, the mask of the damaged area, the semantic features of the text, and the visual features of the reference image as conditional inputs to the large model of the text-to-image;

[0047] Step 7: Based on the generation mechanism of the large model of the text image, under the constraints of multimodal conditions, the damaged area is gradually generated or completed to obtain the initial repair result;

[0048] Step 8: Adjust the fusion weight between the repaired area and the undamaged area based on the confidence level or feature consistency of the generated results;

[0049] Step 9: Output the final image restoration result of the cultural relic after adjustment.

[0050] Furthermore, in step 1, the data used is digital images of cultural relics, including images of cultural relics with historical and artistic value such as grotto murals and landscape paintings. These images contain image degradation information caused by natural aging, environmental erosion, or human factors, manifested as local missing parts, color attenuation, texture breaks, or structural discontinuities. The damaged area mask is a binary or multi-valued mask used to distinguish between damaged and undamaged areas.

[0051] Furthermore, in step 2: the text description information is obtained by user input, automatically generated from the cultural relics database, or through semantic analysis of the cultural relics images;

[0052] Furthermore, in step 3: the reference image is derived from historical images of the same cultural relic, images of cultural relics of the same category, or images of cultural relics from the same period or with the same artistic style;

[0053] Furthermore, in steps 4 and 5: the feature extractor for the reference image and text description is a multimodal feature extraction model CLIP trained using a contrastive learning method. The CLIP model includes a text encoder and an image encoder. The text encoder is used to encode the text description to obtain text semantic features, and the image encoder is used to encode the reference image to obtain image visual features.

[0054] Furthermore, in step 6: the large-scale model for the cultural relic image is a stable diffusion model (Stable Diffusion) combined with a lightweight adapter (IP-Adapter). Stable Diffusion performs diffusion modeling and generation of the image in the latent space, which reduces computational complexity and improves generation efficiency compared to diffusion calculations in the image space. The IP-Adapter is used to inject textual semantic features and reference image visual features as conditions into the latent space diffusion process without changing the main structure of Stable Diffusion, thereby achieving the generation or reconstruction of cultural relic images under multimodal constraints. The pre-trained model used by Stable Diffusion is a restoration model (stable-diffusion-xl-1.0-inpainting-0.1); the training objective function is:

[0055]

[0056] Furthermore, in step 7, the multimodal conditional constraint is a decoupled cross-attention mechanism that injects text semantic features and reference image visual features into the IP-Adapter, compatible with both text and images. This decoupled cross-attention refers to adding only one additional cross-attention layer for image features in each cross-attention layer of the Stable Diffusion UNet. During training, only the parameters of the new cross-attention layer are trained, while the original UNet model remains frozen. Thus, the adapter can simultaneously accept text and images as input, providing additional control for the text-to-image model and thereby providing reference image-guided artifact image restoration. The new decoupled cross-attention expression is as follows:

[0057] = V +

[0058] in, Since the original UNet network is frozen, only [the following is possible] in decoupled cross-attention. and It is trainable.

[0059] Furthermore, in step 8: the fusion weight is the cue word guidance coefficient in the adapter, which is used to control the strictness with which the text-based large model should follow the cue words during sampling.

[0060] It is worth noting that this application utilizes a pre-trained text-based image model, combined with guidance from reference images, to restore missing regions in cultural relic images, encompassing the restoration of elements such as structure, texture, and style. Unlike existing methods, this application not only achieves high-quality restoration even in the absence of large-scale paired datasets, but also fully leverages the multimodal information from text descriptions and reference images, ensuring consistency in artistic style and semantics in the restoration results. Furthermore, this invention improves the model's restoration accuracy when handling complex damaged areas through a cross-modal attention mechanism, overcoming the shortcomings of traditional methods in detail restoration and style consistency.

[0061] Effect test:

[0062] This invention can restore damaged images of cultural relics (landscape paintings, murals). Figure 2This paper demonstrates the application effects of this method in the restoration of images of different types of cultural relics. In the figures, the first column shows the reference image, the second column shows the image to be restored, and the subsequent columns show the restoration results. By combining the reference image and text description, the multimodal cultural relic image restoration method proposed in this invention can effectively restore missing parts of the image, maintain stylistic consistency, and achieve high-fidelity restoration in detail and structure. Whether it is a complex landscape painting or a historical mural, this invention can accurately fill in missing areas while preserving the original artistic style and historical value as much as possible.

[0063] It should be noted that the user information (including but not limited to user device information, personal user information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the laws, regulations and standards of relevant countries and regions.

[0064] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0065] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for restoring cultural relics images based on a large-scale model of the original image and reference images, characterized in that, Includes the following steps: Step 1: Obtain an image of the cultural relic to be restored and preprocess the image, including adjusting the image size, identifying and marking the damaged areas in the image, and generating corresponding damaged area mask information to provide accurate input for subsequent restoration steps; Step 2: Based on the visual content and attribute information of the cultural relic image, construct text description information corresponding to the cultural relic image. The text description information is used to characterize the subject matter, patterns, period characteristics, or artistic style of the cultural relic. Step 3: Obtain at least one reference image, which is similar to the image of the cultural relic to be restored in terms of content structure, local texture or overall artistic style; Step 4: Extract features from the reference image to obtain a visual feature representation of the reference image used for guided repair; Step 5: Extract features from the text description information to obtain a semantic feature representation of the text description used to guide the repair; Step 6: Construct a multimodal conditional input, taking the image of the cultural relic to be restored, the mask of the damaged area, the semantic features of the text, and the visual features of the reference image as conditional inputs to the large model of the text-to-image; Step 7: Based on the generation mechanism of the large model of the text image, under the constraints of multimodal conditions, the damaged area is gradually generated or completed to obtain the initial repair result; Step 8: Adjust the fusion weight between the repaired area and the undamaged area based on the confidence level or feature consistency of the generated results; Step 9: Output the final image restoration result of the cultural relic after adjustment.

2. The method for cultural relic image restoration based on a large-scale model of the original image and reference images as described in claim 1, characterized in that, Step 1 specifically includes: the data used for the image of the cultural relic to be restored is a digital image of the cultural relic, which includes images of cultural relics with historical and artistic value, such as grotto murals and landscape paintings, and contains image degradation information caused by natural aging, environmental erosion or human factors. The degradation information is manifested as local missing parts, color attenuation, texture breakage or structural discontinuity. The mask of the damaged area is a binary mask, which is used to distinguish the damaged area from the non-damaged area.

3. The method for cultural relic image restoration based on a large-scale model of the original image and reference images as described in claim 1, characterized in that, Step 2: The text description information is obtained by user input, automatically generated from the cultural relics database, or by semantic analysis of cultural relics images.

4. The method for cultural relic image restoration based on a large model of the original image and reference images as described in claim 1, characterized in that, Step 3: The reference image is derived from historical images of the same cultural relic, images of cultural relics of the same category, or images of cultural relics from the same period or with the same artistic style.

5. The method for cultural relic image restoration based on a large model of the original image and reference images as described in claim 1, characterized in that, In steps 4 and 5: the feature extractor for the reference image and text description is a multimodal feature extraction model CLIP trained using contrastive learning. The CLIP model includes a text encoder and an image encoder. The text encoder is used to encode the text description to obtain text semantic features. Specifically, the text encoder adopts the BERT (Bidirectional Encoder Representations from Transformers) encoding method based on the Transformer architecture to capture deep semantic information in the text. The image encoder is used to encode the reference image to obtain image visual features. Specifically, the image encoder adopts the ViT (Vision Transformer) encoding method, extracting local and global features of the image through convolutional layers or self-attention mechanisms to obtain a high-dimensional image representation.

6. The method for cultural relic image restoration based on a large-scale model of the original image and reference images as described in claim 1, characterized in that, Step 6 specifically includes: the large-scale model for the cultural relic image is a stable diffusion model (Stable Diffusion) combined with a lightweight adapter (IP-Adapter). Stable Diffusion performs diffusion modeling and generation of the image in the latent space. The IP-Adapter is used to inject textual semantic features and reference image visual features as conditions into the latent space diffusion process without changing the main structure of Stable Diffusion, thereby achieving the generation or reconstruction of cultural relic images under multimodal constraints. The pre-trained model used by Stable Diffusion is a restoration model (stable-diffusion-xl-1.0-inpainting-0.1). The training objective function is: ； in, Images representing cultural relics awaiting restoration. Indicates noise. This indicates the text description conditions. Indicates the conditions of the reference image. Indicates the time step. Indicates model parameters, Indicates time step The output image below.

7. The method for restoring cultural relics images based on a large model of the original image and reference images as described in claim 1, characterized in that, Step 7 specifically includes: the multimodal conditional constraint is a decoupled cross-attention mechanism that injects text semantic features and reference image visual features into the IP-Adapter, compatible with both text and images. This decoupled cross-attention refers to adding only an additional cross-attention layer for image features in each cross-attention layer of the Stable Diffusion UNet. During training, only the parameters of the new cross-attention layer are trained, while the original UNet model remains frozen. Thus, the lightweight adapter can simultaneously accept text and images as input, providing additional control for the large text-to-image model, thereby providing reference image-guided artifact image restoration. The new decoupled cross-attention expression is as follows: = V + ； in, Since the original UNet network is frozen, only [the following is possible] in decoupled cross-attention. and It is trainable; This represents the new feature vector output by the attention mechanism layer. The query matrix represents the input... With weight matrix The product obtained by multiplication; and Represents the "index" or "feature description" of the retrieved object; V and Represents the "specific content" of the object being retrieved; This represents the scaling factor.

8. The method for cultural relic image restoration based on a large-scale model of the original image and reference images as described in claim 1, characterized in that, Step 8: The fusion weights are cue word guidance coefficients in the adapter, used to control the strictness with which the large model of the text-based image should follow the cue words during sampling. Specifically, the fusion weights are dynamically adjusted according to the complexity of the repaired region and the requirements of the generated effect. For simple or known repaired regions, the fusion weights are lower, allowing the model to rely more on the visual features of the reference image; For complex areas requiring high detail, the fusion weights are higher, and the model more strictly follows the semantic guidance of the prompts, ensuring consistency in style and semantics in the generated results. This adjustment mechanism improves the accuracy and artistry of the restoration results by optimizing weights in real time during the generation process and balancing the relationship between semantic guidance and visual features.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the cultural relic image restoration method based on the large model of the original image and the reference image as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cultural relic image restoration method based on the large model of the text image and the reference image as described in any one of claims 1 to 8.

Images