A high-fidelity automatic repair method and system for commercial auction images

Abstract

The application discloses a kind of high-fidelity full-automatic repair methods and systems of commercial image, belong to image processing technical field.The application first input reference original drawing and to be repaired generation drawing into multimodal vision big model, and automatically generate pixel-level target mask in combination with zero sample segmentation model;Then the mask is used to extract local visual features from the reference original drawing and encode into high-dimensional visual feature vector;Next, in the cross attention layer of latent diffusion model, the visual feature vector and the text feature vector are weighted and fused, guiding the model to redraw with high fidelity in the mask area;Finally, the closed-loop verification is carried out by calculating the perceptual similarity score, and the parameters are automatically adjusted and iteratively redrawn when unqualified.The application realizes the accurate restoration of local features without human intervention, effectively eliminates AI hallucinations, while maintaining global light and structure consistency, significantly improving the commercial qualification rate and processing efficiency of commercial images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a high-fidelity fully automatic restoration method and system for commercial photographs. Background Technology

[0002] Currently, generative image technology based on diffusion models has made significant progress and is widely used in commercial scenarios such as e-commerce photography, virtual try-on, and advertising design. Among them, image local repainting technology can complete and repair content in specified areas of an image under the condition of a given mask. Existing diffusion model-based inpainting methods, such as Stable Diffusion Inpainting, can greatly improve the visual coherence between the repaired area and the surrounding environment by introducing masking mechanisms and cross-modal attention mechanisms. However, these methods rely heavily on pure invention prompts to guide the generation process. When the area to be repaired involves complex commercial elements that require absolute precision (such as specific font brand logos, special process buttons, copyrighted printed patterns, specific stitching arrangements, etc.), the invention encoder is limited by the expression granularity and representation ability, making it difficult to accurately convey the above visual prior information. This causes the diffusion model to produce "hallucination" during the inference process, generating textures that seem reasonable but deviate significantly from the actual physical properties of the product.

[0003] To alleviate the limitations of the representation in this invention, visual prompting techniques, such as IP-Adapter, have emerged in recent years. These methods globally or locally encode a reference image and project the extracted visual features into the feature space of the invention within the diffusion model, thereby achieving image-level guidance for the generation process. However, existing visual prompting techniques are mostly used for overall style or structural transfer, lacking refined feature extraction and pixel-level injection mechanisms for local defect areas. This makes them difficult to directly apply to commercial photography restoration scenarios that require maintaining global lighting, pose, and background while performing high-fidelity restoration of only specific local areas.

[0004] Meanwhile, the development of multimodal large language models (LVLMs) such as GPT-4V and Qwen-VL, as well as zero-shot image segmentation models (such as Segment Anything Model, SAM), has made it possible to achieve automated defect detection and mask generation. Current technologies have not yet organically combined the semantic contrast capabilities of LVLMs with the fine-grained segmentation capabilities of SAMs to form an end-to-end automated detection and repair pipeline for local defects in commercial photography images.

[0005] Therefore, how to provide a high-fidelity, fully automated restoration method and system for commercial photographs is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a high-fidelity fully automatic restoration method and system for commercial photography images, which solves the technical problem in the prior art of how to achieve high-fidelity, pixel-level restoration of local fine visual features in commercial photography images without relying on manual intervention, while maintaining the consistency of global features.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a high-fidelity fully automatic restoration method for commercial photographs, comprising: S1: Obtain a reference original image and a generated image to be repaired, wherein the reference original image contains the correct physical properties, and the generated image to be repaired is a generated image with local defects; S2: Input the original reference image and the image to be repaired into the multimodal visual large model, output a spatial bounding box describing the defect area through semantic comparison, and use the spatial bounding box as a prompt to input the zero-sample image segmentation model to generate a pixel-level target mask on the image to be repaired. S3: Use the target mask to crop out the corresponding local image patch from the original reference image, and convert the local image patch into a high-dimensional visual feature vector through a visual feature encoding network; S4: Input the image to be repaired, the target mask, and the high-dimensional visual feature vector into the latent diffusion model. In the cross-attention layer of the latent diffusion model, the high-dimensional visual feature vector and the feature vector of the present invention are weighted and fused to guide the latent diffusion model to redraw within the coverage area of ​​the target mask and generate the initial redraw image. S5: Calculate the perceptual similarity score between the initial redrawn image and the reference original image within the area corresponding to the target mask. If the score is lower than a preset threshold, automatically adjust the redrawing parameters and return to S4 for a second redraw. Otherwise, output the current redrawn image as the final high-fidelity repaired image.

[0008] Furthermore, the spatial bounding box is a coarse-grained bounding box.

[0009] Furthermore, the zero-sample image segmentation model is the Segment Anything Model.

[0010] Furthermore, the target mask is a binary pixel-level mask.

[0011] Furthermore, the weighted fusion is performed according to a dynamic fusion weight λ, which is adaptively adjusted based on the area and / or texture complexity of the target mask: the λ value is increased when the target mask covers high-density identifiers or complex textures, and the λ value is decreased when the target mask covers smooth areas.

[0012] Furthermore, the automatic adjustment of redrawing parameters includes: Increase the expansion radius of the target mask to capture more of the surrounding context texture; Increase the injection weight of the high-dimensional visual feature vector in the cross-attention layer.

[0013] Furthermore, the potential diffusion model also includes a dual fine-tuning module, which comprises: The main body consistency constraint submodule is used to forcibly lock the material and structural properties of the repair area during the noise reduction process; The inference acceleration submodule is used to compress the number of denoising sampling steps to a preset low step range.

[0014] On the other hand, the present invention also provides a high-fidelity fully automatic restoration system for commercial photographs, comprising: Input module: used to obtain the original reference image and the image to be repaired; Intelligent detection and mask generation module: used to input the original reference image and the image to be repaired into a multimodal visual large model to output a spatial bounding box, and input the spatial bounding box into a zero-sample image segmentation model to generate a pixel-level target mask; Visual feature extraction module: used to crop local image patches from the original reference image using the target mask and convert them into high-dimensional visual feature vectors; Multimodal joint redrawing module: used to input the image to be repaired, the target mask and the high-dimensional visual feature vector into the latent diffusion model, and perform weighted fusion in the cross attention layer to generate the initial redrawing image; Closed-loop verification and iteration module: used to calculate the perceptual similarity score between the initial redrawn image and the reference original image in the target area, and automatically adjust parameters and trigger the redrawing module to perform secondary repair when the score is lower than the threshold; otherwise, output the current image.

[0015] As can be seen from the above technical solution, compared with the prior art, the present invention provides a high-fidelity fully automatic restoration method and system for commercial photography images, which has the following beneficial effects: 1. A smart detection and mask generation mechanism based on multimodal visual large model (LVLM) is proposed: by comparing the reference image and the generated image using LVLM, the spatial coordinates of local differences are automatically output, and a high-precision target mask is generated by combining the image segmentation algorithm, thus realizing the localization of the repair area without human intervention.

[0016] 2. Constructing a multimodal repair pipeline based on visual feature injection: The feature encoding network is used to transform the local physical information in the reference image into a high-dimensional visual feature vector, which directly guides the redrawing process of the diffusion model in the masked area, breaking through the feature representation limitations brought about by the traditional pure invention-driven approach.

[0017] 3. Design of a multi-dimensional collaborative constraint and closed-loop iterative system: Introduce a dual fine-tuning module for subject consistency constraints and inference acceleration, and combine it with an edge smoothing strategy and an adaptive evaluation retry mechanism. While ensuring the objective physical consistency of the image, the system's inference efficiency and output robustness are significantly improved. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the overall architecture of the method provided by the present invention; Figure 2 This is a qualitative comparison of the restoration effects of the method of the present invention and existing mainstream image local repainting methods on the same commercially available image. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1: Embodiment 1 of this invention discloses a high-fidelity fully automatic restoration method for commercial photographs, comprising: S1: Obtain a reference original image and a generated image to be repaired, wherein the reference original image contains the correct physical properties, and the generated image to be repaired is a generated image with local defects; S2: Input the original reference image and the image to be repaired into the multimodal visual large model, output a spatial bounding box describing the defect area through semantic comparison, and use the spatial bounding box as a prompt to input the zero-sample image segmentation model to generate a pixel-level target mask on the image to be repaired. S3: Use the target mask to crop out the corresponding local image patch from the original reference image, and convert the local image patch into a high-dimensional visual feature vector through a visual feature encoding network; S4: Input the image to be repaired, the target mask, and the high-dimensional visual feature vector into the latent diffusion model. In the cross-attention layer of the latent diffusion model, the high-dimensional visual feature vector and the feature vector of the present invention are weighted and fused to guide the latent diffusion model to redraw within the coverage area of ​​the target mask and generate the initial redraw image. S5: Calculate the perceptual similarity score between the initial redrawn image and the reference original image within the area corresponding to the target mask. If the score is lower than a preset threshold, automatically adjust the redrawing parameters and return to S4 for a second redraw. Otherwise, output the current redrawn image as the final high-fidelity repaired image.

[0022] Specifically, after obtaining the original reference image and the image to be repaired, see [link to relevant documentation]. Figure 1 As shown, this invention can be divided into four main stages, corresponding to steps S2-S5, with reference to the original drawing input ( ) and the generated image to be repaired ( Two input sources; Stage 1 (Large Model Intelligent Detection and Mask Generation): Multimodal Visual Large Model (LVLM) reception and It outputs a coarse-grained spatial bounding box (BBox); this BBox serves as a prompt for the zero-shot image segmentation model (SAM) to generate a pixel-level target mask. ).

[0023] Phase Two (Visual Feature Extraction Module): Using a target mask, crop a local image patch from the original reference image. And it is converted into a high-dimensional visual feature vector through a visual feature encoding network. ).

[0024] Phase 3 (Dual-Constraint Multimodal Joint Redrawing): The Latent Diffusion Base Model (LDM) receives the map to be repaired, the target mask, and the visual feature vector. Under the collaboration of the Dual LoRA module (which includes the subject consistency constraint and inference acceleration sub-module), it performs redrawing to generate the initial version of the redrawing map.

[0025] Phase Four (Iterative Closed-Loop Verification System): The difference assessment module calculates the perceptual similarity score between the initial redrawn image and the original reference image. If the score is below the threshold, the adaptive parameter adjustment module dynamically expands the mask expansion radius or increases the weight of visual feature injection, and returns the updated parameters to stage three for secondary redrawing; if the score is qualified, it proceeds to the final output.

[0026] Final output module: Outputs high-fidelity commercial images ( ).

[0027] For details, please refer to the original image ( ): An original product image containing 100% accurate physical properties (such as standard brand logo, specific stitching, and correct print layout) serves as a "visual truth" reference for the entire restoration process.

[0028] Image to be repaired ( ): Flawed images that are physically distorted, omitted, or altered due to partial "illusions" generated by the generated model after the pre-processing of AI clothing change or commercial photography.

[0029] Specifically, Phase 1: Large-scale intelligent detection and mask generation (LVLM & Mask Generation): Multimodal Visual Large Model (LVLM): Reception and By comparing spatial perception and deep semantics, it automatically identifies the differences in physical features between two images in terms of business logic and outputs a coarse-grained spatial bounding box (BBox) containing the error region.

[0030] Zero-Shot Image Segmentation (SAM): Receives the bounding box output by the LVLM as a geometric cue, in Edge refinement attention calculation is performed to automatically generate a high-precision binary pixel-level target mask. This completely replaces manual image cutout.

[0031] Phase Two: Visual Feature Extraction Module Feature patch cropping: using a mask From the original reference image Precisely crop out local image patches containing only the correct physical details. ).

[0032] Visual feature encoding network (Encoder): This is a feature extractor (such as a deep convolutional network) that transforms cropped feature patches into high-dimensional visual feature vectors. This vector, or Visual Prompt, will replace the traditional pure invention prompt as a "hard reference" for the repair network to generate correct textures.

[0033] Phase 3: Dual-constrained Multimodal Inpainting Latent Diffusion Foundation Model (LDM): The core reconstruction engine of the system. It receives the graph to be repaired and the mask. and extracted visual features In the cross-attention layer, visual features are forcibly injected into the denoising process, guiding the model to "imitate" the correct physical properties within the mask area. At the same time, the attention mechanism is used to capture ambient light and shadow at the mask edges, achieving seamless "soft fusion".

[0034] Dual LoRA module: Mounted as a bypass plugin on the base network. It includes two sub-modules: "Subject Consistency Constraint" and "Inference Acceleration". The former ensures that the generated materials and structures do not change; the latter optimizes the sampling trajectory to compress the generation steps to 4-8 steps, achieving extremely fast map generation.

[0035] Phase Four: Iterative Closed-loop Validation System Difference Assessment Module (Eval): After repair, extract the perceptual features of the initial redrawn image and the reference original image in the target area, and calculate the comprehensive similarity score. Adaptive parameter adjustment (Params): When the score is lower than the commercially acceptable threshold, the system automatically intercepts the generation and triggers a retry mechanism. The algorithm dynamically expands the mask's expansion radius (to capture more surrounding light and shadow to resolve harsh edges) or increases the injection weight of visual features in the cross-attention layer (to suppress the illusion of this invention and resolve inaccurate details), and then sends the updated parameters back to stage three for a second fast redraw.

[0036] Final Output: High-fidelity commercial images ( The final image, as determined by the closed-loop verification system, achieves 100% pixel-level lossless restoration of complex local physical features without compromising global lighting and model pose, and can be directly used for commercial release.

[0037] Specifically, intelligent difference detection and mask generation based on large models:

[0038] To address the efficiency bottleneck of manual mask selection, this invention proposes a dual-track detection mechanism that combines LVLM (Multimodal Large Model) and SAM (Zero-Sample Segmentation).

[0039] Semantic localization: Utilize LVLM cross-modal comparison between the "reference original image" and the "generated image to be repaired" to accurately identify generation defects that violate business logic (such as logo distortion) and output coarse-grained bounding boxes (BBoxes).

[0040] Pixel segmentation: The bounding box (BBox) is injected as a spatial cue into the SAM network, leveraging its extremely high boundary calculation capabilities to generate a pixel-level mask that closely matches the defect edge. The combination of these two methods ensures that redrawing is performed only within the absolutely necessary minimum scope, preserving the global features and lighting of the original image without loss.

[0041] Visual feature extraction and multimodal local redrawing: To address the issue that the invention alone cannot accurately represent complex commercial identifiers, this framework introduces a joint guidance strategy combining the invention with visual features.

[0042] Visual feature injection: Correct details of the original image are extracted using a mask and encoded as a high-dimensional visual feature tensor. In the cross-attention layer of the diffusion model, the attention of this invention and the visual feature attention are weighted and fused according to weight (λ) to directly intervene in the denoising process and avoid AI illusions.

[0043] Adaptive weight (λ) allocation: Dynamically adjust parameters based on mask area and texture complexity. When encountering high-density logos or complex wiring, λ is significantly increased to force the model to "imitate" physical details; when encountering smooth lighting and shadows, λ is decreased, relying on the invention and context to achieve a smooth reconstruction and a balance between detail and naturalness.

[0044] Cooperative control mechanism of the dual fine-tuning model: Two major modules are cascaded on top of the base model to meet stringent commercial standards: 1. Subject Consistency Constraint: During denoising, deep feature matching is used to forcibly lock the physical material and print layout of the repair area to prevent attribute shift.

[0045] 2. Inference Acceleration Module: Loads the Turbo network, compressing the original dozens of sampling and inference steps to 4-8 steps, meeting the second-level low latency requirements of e-commerce for large-scale SKU concurrent processing.

[0046] Adaptive edge soft blending and iterative verification: Breaking away from the pain points of traditional image restoration's "one-way blind box" generation, a dynamic closed-loop verification system with "evaluation-correction" capabilities is constructed to achieve zero human intervention.

[0047] Automatic quality inspection: Compare the features of the initial repaired image with the reference image and calculate the overall perceptual similarity. If the score meets the standard, the image is output; otherwise, it is automatically intercepted and parameter readjustment and secondary repair are triggered.

[0048] Adaptive error correction strategy: a. For "harsh edges or residual imperfections": Dynamically expand the Mask expansion radius and capture more surrounding textures as context to achieve smooth edge transitions.

[0049] b. For "inaccurate identification restoration (illusion)": Dynamically increase the weight of visual feature injection (λ), force the diffusion model to strictly follow the physical characteristics of the original image, and remove its divergent randomness. The system converges quickly through automatic iteration to ensure that the output image is 100% qualified.

[0050] On the other hand, Embodiment 1 of the present invention also discloses a high-fidelity fully automatic restoration system for commercial photography images, comprising: Input module: used to obtain the original reference image and the image to be repaired; Intelligent detection and mask generation module: used to input the original reference image and the image to be repaired into a multimodal visual large model to output a spatial bounding box, and input the spatial bounding box into a zero-sample image segmentation model to generate a pixel-level target mask; Visual feature extraction module: used to crop local image patches from the original reference image using the target mask and convert them into high-dimensional visual feature vectors; Multimodal joint redrawing module: used to input the image to be repaired, the target mask and the high-dimensional visual feature vector into the latent diffusion model, and perform weighted fusion in the cross attention layer to generate the initial redrawing image; Closed-loop verification and iteration module: used to calculate the perceptual similarity score between the initial redrawn image and the reference original image in the target area, and automatically adjust parameters and trigger the redrawing module to perform secondary repair when the score is lower than the threshold; otherwise, output the current image.

[0051] Example 2: To demonstrate the technical effectiveness of the present invention, Embodiment 2 of the present invention discloses an experimental process for a high-fidelity fully automatic restoration method and system for commercial photography images, specifically including: Experimental setup: Dataset Construction: A dataset containing 5,000 real e-commerce commercial photography failure images was constructed, covering a variety of typical commercial defect scenarios such as logo deformation, missing prints, and material distortion.

[0052] Multidimensional evaluation system: To comprehensively demonstrate the repair performance, a four-dimensional quantitative evaluation system was constructed, ranging from bottom-level pixels to high-level semantics. PSNR (Peak Signal-to-Noise Ratio): Measures the accuracy and fidelity of color and brightness reconstruction at the pixel level.

[0053] SSIM (Structural Similarity): Evaluates the geometric and structural consistency of physical topological features such as clothing folds and logo outlines.

[0054] LPIPS (Perceptual Patch Similarity): Measures the naturalness of the image in accordance with human visual perception, and examines the edge blending effect and whether there are any artificial composite traces.

[0055] CLIP-I (Semantic Consistency Score): Evaluates the semantic reconstruction capability in a high-dimensional vector space to ensure high-precision alignment between brand identity and the commercial attributes of specific textures.

[0056] Qualitative comparative experiment: This embodiment compares the visual effects of the present invention with those of current mainstream inpainting methods. See details... Figure 2 As shown, the left side is a magnified view of the defective area in the original reference image or the image to be repaired; the middle side shows the repair result of a conventional inpainting method (such as the diffusion model driven by this invention), which produces unrecognizable gibberish, random patterns, or texture distortion in the logo or printed area, i.e., a "visual illusion" phenomenon; the right side shows the repair result of the method of this invention (labeled Ours), which highly restores the font shape, letter arrangement, and detailed texture of the original logo and special print in the same area, and the repaired area and the surrounding background show natural continuity in light and shadow and edge transition, without obvious splicing artifacts. Figure 2 As can be seen, conventional inpainting produces unrecognizable gibberish (AI illusion) when dealing with complex English logos. The method proposed in this invention not only highly reproduces the original logo's font and special printing, but also achieves extremely natural light and shadow and texture fusion at the mask edges, possessing visual quality suitable for direct commercial use.

[0057] Quantitative comparative experiment: The quantitative assessment results are shown in Table 1.

[0058] Table 1 Comparison of Quantitative Assessment Results

[0059] As shown in Table 1, the framework of this invention significantly outperforms mainstream baseline models in all key indicators, successfully achieving a balance between repair fidelity and engineering efficiency. Pure invention-driven model (SDXL): bottom-ranked (SSIM: 0.814, CLIP-I: 0.788). Pure invention is difficult to represent complex patterns, easily produces "illusory textures", and cannot meet commercial fidelity requirements.

[0060] Multimodal model (Qwen-Image-Edit-2511): good semantic alignment (CLIP-I:0.865), but with severe inference latency (14.2s / img) and insufficient local accurate reconstruction (SSIM:0.856).

[0061] Industrial-grade baseline (NanoPro): Good fusion and structure performance (LPIPS: 0.048, SSIM: 0.902), but lacks strong local pixel-level constraints, and the reproduction accuracy is still insufficient when dealing with specific logos and materials.

[0062] Invention framework (comprehensively optimal): Ultimate fidelity: The PSNR (31.88dB) and CLIP-I (0.968) are significantly higher, proving that the direct injection mechanism of visual features effectively breaks through the characterization bottleneck of this invention and achieves near-lossless restoration of product details.

[0063] Natural fusion: Achieving the lowest LPIPS (0.035), we have verified the significant effect of the adaptive edge soft fusion mechanism in eliminating harsh synthetic traces.

[0064] Ultra-fast reasoning: Thanks to the reasoning acceleration module, the time taken for a single graph is reduced to 4.2 seconds, which is nearly 50% faster than SDXL and more than 3 times faster than Qwen, demonstrating extremely high potential for industrial-grade real-time processing.

[0065] Ablation Study: To verify the effectiveness of each module proposed in this invention, an ablation experiment was designed in this embodiment, as shown in Table 2.

[0066] Table 2 Comparison of Ablation Tests

[0067] As can be seen from Table 2, the Baseline (pure invention + manual masking) lacks physical priors, resulting in low accuracy (SSIM: 0.814, CLIP-I: 0.788), and relies on manual selection, resulting in a single image taking up to 18.5 seconds, making it unsuitable for automated production.

[0068] + LVLM & Mask (Intelligent Detection and Automatic Masking): Completely eliminates the bottleneck of manual frame drawing, greatly improves the system's automation level, and reduces processing time to 12.8s.

[0069] + Visual Prompt (Visual Feature Extraction and Injection): A qualitative leap in restoration fidelity (CLIP-I jumps to 0.942, SSIM to 0.915). This confirms that visual priors have successfully overcome the limitations of this invention and are a decisive factor in accurately reproducing complex logos and specific materials.

[0070] + Dual Modules (Complete Framework / Dual Fine-tuning): Subject consistency constraint: Further optimize edge lighting and shadow soft blending to eliminate visual disharmony (LPIPS reaches the optimal 0.038).

[0071] Inference acceleration module: Significantly compresses the number of denoising sampling steps, reducing the time taken from 14.5s to 4.2s.

[0072] Conclusion: Visual feature injection is the core support for ensuring "high-fidelity restoration", while LVLM automatic detection and dual acceleration constraint mechanism are the key cornerstones for realizing the "fully automatic and low-latency" industrial application of the system.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0074] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-fidelity fully automatic image restoration method for commercial photography, characterized in that, include: S1: Obtain a reference original image and a generated image to be repaired, wherein the reference original image contains the correct physical properties, and the generated image to be repaired is a generated image with local defects; S2: Input the original reference image and the image to be repaired into the multimodal visual large model, output a spatial bounding box describing the defect area through semantic comparison, and use the spatial bounding box as a prompt to input the zero-sample image segmentation model to generate a pixel-level target mask on the image to be repaired. S3: Use the target mask to crop out the corresponding local image patch from the original reference image, and convert the local image patch into a high-dimensional visual feature vector through a visual feature encoding network; S4: Input the image to be repaired, the target mask, and the high-dimensional visual feature vector into the latent diffusion model. In the cross-attention layer of the latent diffusion model, the high-dimensional visual feature vector and the feature vector of the present invention are weighted and fused to guide the latent diffusion model to redraw within the coverage area of ​​the target mask and generate the initial redraw image. S5: Calculate the perceptual similarity score between the initial redrawn image and the reference original image within the area corresponding to the target mask. If the score is lower than a preset threshold, automatically adjust the redrawing parameters and return to S4 for a second redraw. Otherwise, output the current redrawn image as the final high-fidelity repaired image.

2. The high-fidelity fully automatic restoration method for commercial photography images according to claim 1, characterized in that, The spatial bounding box is a coarse-grained bounding box.

3. The high-fidelity fully automatic restoration method for commercial photography images according to claim 1, characterized in that, The zero-sample image segmentation model is the Segment Anything Model.

4. The high-fidelity fully automatic restoration method for commercial photography images according to claim 1, characterized in that, The target mask is a binary pixel-level mask.

5. The high-fidelity fully automatic restoration method for commercial photography images according to claim 1, characterized in that, The weighted fusion is performed according to a dynamic fusion weight λ, which is adaptively adjusted based on the area and / or texture complexity of the target mask: the λ value is increased when the target mask covers high-density identifiers or complex textures, and the λ value is decreased when the target mask covers smooth areas.

6. The high-fidelity fully automatic restoration method for commercial photographs according to claim 1, characterized in that, The automatic adjustment of redrawing parameters includes: Increase the expansion radius of the target mask to capture more of the surrounding context texture; Increase the injection weight of the high-dimensional visual feature vector in the cross-attention layer.

7. The high-fidelity fully automatic restoration method for commercial photographs according to claim 1, characterized in that, The potential diffusion model also includes a dual fine-tuning module, which comprises: The main body consistency constraint submodule is used to forcibly lock the material and structural properties of the repair area during the noise reduction process; The inference acceleration submodule is used to compress the number of denoising sampling steps to a preset low step range.

8. A high-fidelity fully automatic image restoration system for commercial photography, characterized in that, include: Input module: used to obtain the original reference image and the image to be repaired; Intelligent detection and mask generation module: used to input the original reference image and the image to be repaired into a multimodal visual large model to output a spatial bounding box, and input the spatial bounding box into a zero-sample image segmentation model to generate a pixel-level target mask; Visual feature extraction module: used to crop local image patches from the original reference image using the target mask and convert them into high-dimensional visual feature vectors; Multimodal joint redrawing module: used to input the image to be repaired, the target mask and the high-dimensional visual feature vector into the latent diffusion model, and perform weighted fusion in the cross attention layer to generate the initial redrawing image; Closed-loop verification and iteration module: used to calculate the perceptual similarity score between the initial redrawn image and the reference original image in the target area, and automatically adjust parameters and trigger the redrawing module to perform secondary repair when the score is lower than the threshold; otherwise, output the current image.

Images