Document semantic repair and stroke reconstruction method, system and device based on conditional control diffusion architecture and medium

Abstract

The invention discloses a document semantic restoration and stroke reconstruction method, system and device based on a condition control diffusion architecture and a medium, and belongs to the technical field of document image restoration, and the method comprises the following steps: extracting a visual structure condition, a semantic condition and a degradation condition of a degraded document image; respectively inputting the three types of conditions into corresponding encoders to obtain structure encoding features, semantic encoding features and degradation encoding features; inputting the three types of coding features into a diffusion model, respectively injecting the coding features at different network hierarchies through a layered injection mechanism, dynamically adjusting the weight according to the denoising progress, and generating a reconstructed image; performing coarse-grained restoration, fine-grained restoration and quality refining on the reconstructed image in sequence; semantic verification and visual evaluation are carried out on the repaired image, and a substandard region is positioned for key optimization; and outputting the optimized document image. According to the method, semantic understanding and stroke reconstruction are synchronously carried out through a multi-modal condition control mechanism and a layered injection strategy.

Description

Technical Field

[0001] This invention relates to the field of document image restoration technology, specifically to a method, system, device, and medium for document semantic restoration and stroke reconstruction based on a conditional controlled diffusion architecture. Background Technology

[0002] Handwritten document restoration technology is an important branch of computer vision. Taking the power system as an example, documents that need to be digitized and restored include on-site inspection records, equipment commissioning reports, hand-drawn circuit diagrams, operation tickets, work tickets, and historical archives. These documents generally suffer from problems such as ink fading, yellowing paper, mold stains, physical wrinkles, and even damage. In the early digitization process, due to limitations in scanning equipment and technology, secondary degradation such as image blurring, low contrast, and complex background noise occurred, making it difficult to identify key information in the documents.

[0003] Traditional image processing algorithms and early deep learning models struggle to effectively distinguish noise from real strokes in complex scenarios, or they may erase subtle, discernible strokes during denoising, let alone understand the contextual semantics of technical terms. While convolutional neural networks and encoder-decoder architectures can handle complex degradation, they lack semantic constraints and may generate visually plausible but semantically incorrect characters. Generative adversarial networks suffer from training instability and are prone to producing illusory textures, potentially leading to the forgery of strokes or symbols. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention provides a method, system, device and medium for document semantic repair and stroke reconstruction based on a conditional controlled diffusion architecture.

[0005] Therefore, the technical problem solved by this invention is that existing document repair technologies suffer from poor semantic consistency, distorted stroke structure, reliance on large amounts of labeled data, and low computational efficiency. Existing technologies cannot simultaneously guarantee visual quality, semantic accuracy, and the integrity of stroke structure. In professional document repair scenarios, when large areas are missing or severely blurred, semantically incorrect characters may be generated, affecting accuracy in professional applications.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture, comprising, The stroke skeleton features of the input degraded document image are extracted by edge detection as visual structural conditions, the document semantics of the degraded document image are obtained by text recognition and semantic encoding as semantic conditions, and the degradation features of the degraded document image are evaluated by a degradation estimation network as degradation conditions. The visual structural conditions, semantic conditions, and degradation conditions are respectively input into the corresponding conditional encoders for encoding to obtain structural coding features, semantic coding features, and degradation coding features; The structural coding features, semantic coding features, and degenerate coding features are input into the diffusion model. During the denoising iteration of the diffusion model, the structural coding features, semantic coding features, and degenerate coding features are injected into different network layers through a hierarchical injection mechanism. The weights of the structural coding features, semantic coding features, and degenerate coding features are dynamically adjusted according to the denoising progress. The reconstructed image is generated through the reverse diffusion process. The reconstructed image is then subjected to coarse-grained repair, fine-grained repair, and quality refinement in sequence to obtain a repaired image; The repaired image is subjected to semantic verification and visual evaluation. Based on the results of the semantic verification and visual evaluation, the substandard areas are located and the substandard areas are optimized in a focused manner. Output the optimized document image.

[0007] As a preferred embodiment of the document semantic repair and stroke reconstruction method based on a conditional controlled diffusion architecture described in this invention, wherein: the step of evaluating the degradation features of the degraded document image as degradation conditions through a degradation estimation network includes, Construct a multi-scale feature representation of the degraded document image; The multi-scale feature representation is input into the degradation classification network to extract degradation features and output a degradation representation vector; The degradation representation vector is input into a multilayer perceptron for dimensional mapping to generate a high-dimensional degradation embedding vector as the degradation condition.

[0008] As a preferred embodiment of the document semantic repair and stroke reconstruction method based on a conditional controlled diffusion architecture described in this invention, wherein: the conditional encoder corresponding to the input performs encoding including, The visual structural conditions are input into a visual encoder to extract structural feature representations, and the structural encoded features are output. The semantic conditions are input into the semantic encoder for semantic feature extraction and enhanced with domain knowledge to output semantic encoded features. The degradation conditions are input into the degradation encoder for degradation feature encoding, and the degradation encoded features are output.

[0009] As a preferred embodiment of the document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture described in this invention, wherein: the obtained repaired image includes, The reconstructed image is downsampled to a preset resolution, and a first denoising reconstruction is performed using a diffusion model to obtain a coarsely repaired image. The coarse-restored image is upsampled to the original resolution, and a second denoising and reconstruction is performed using a diffusion model to obtain the fine-restored image. The finely restored image is input into an adversarial network for quality enhancement to obtain the restored image.

[0010] As a preferred embodiment of the document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture described in this invention, wherein: the injection of the structural coding features, the semantic coding features, and the degenerate coding features at different network levels through a hierarchical injection mechanism during the denoising iteration of the diffusion model includes, The degradation coding feature and the structural coding feature are injected into the first network layer of the diffusion model; The semantic encoding features and the structural encoding features are injected into the second network layer of the diffusion model; The structural coding features, the semantic coding features, and the degenerate coding features are comprehensively injected into the third network layer of the diffusion model.

[0011] The beneficial effects of this preferred technical solution are as follows: by injecting different modal coding features into different network layers of the diffusion model, injecting degenerate coding features and structural coding features into the first network layer to constrain the global content and layout outline, injecting semantic coding features and structural coding features into the second network layer to standardize stroke direction and character shape, and injecting all coding features into the third network layer to refine endpoint transitions and fine strokes, a progressive reconstruction from coarse to fine is achieved, ensuring the unity of global semantic accuracy and local stroke integrity, and reducing the risk of false strokes and deformed characters.

[0012] As a preferred embodiment of the document semantic repair and stroke reconstruction method based on conditional control diffusion architecture described in this invention, wherein: the degradation evaluation result output by the degradation classification network is a degradation vector, the degradation vector contains six dimensions of degradation degree, the six dimensions being ink diffusion, paper wrinkles, uneven lighting, noise pollution, blur degree and local occlusion in order. The step of generating a degradation embedding vector by feature mapping of the degradation assessment result includes: The degradation vector is input into a three-layer perceptron, which expands the dimension sequentially through three fully connected layers, mapping the degradation vector into a high-dimensional degradation embedding vector.

[0013] The beneficial effects of this preferred technical solution are as follows: by refining the degradation features into six dimensions—ink diffusion, paper wrinkles, uneven lighting, noise pollution, blurriness, and local occlusion—it accurately represents the diversity and complexity of document degradation. This enables the degradation estimation network to comprehensively evaluate different types of degradation features. Through a three-layer perceptron, the dimensions are expanded and mapped to a high-dimensional degradation embedding vector, providing accurate degradation prior information for the subsequent conditionally controlled diffusion process. This improves the adaptability of the restoration method in various degradation scenarios and ensures the stability of restoration quality under different degradation conditions.

[0014] In a preferred embodiment of the document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture described in this invention, the feature map resolution corresponding to the first network layer is greater than that corresponding to the second network layer, and the feature map resolution corresponding to the second network layer is greater than that corresponding to the third network layer. The degenerate coding features and the structural coding features are injected into the first network layer of the diffusion model, and the degenerate coding features and the structural coding features are integrated into the feature map of the first network layer through conditional normalization. The semantic encoding features and the structural encoding features are injected into the second network layer of the diffusion model, and the semantic encoding features and the structural encoding features are integrated into the feature map of the second network layer through a cross-attention mechanism. The structural coding features, semantic coding features, and degenerate coding features are comprehensively injected into the third network layer of the diffusion model, and the structural coding features, semantic coding features, and degenerate coding features are integrated into the feature map of the third network layer through a multi-head cross-attention mechanism.

[0015] This invention provides a document semantic repair and stroke reconstruction system based on a condition-controlled diffusion architecture.

[0016] To solve the above technical problems, the present invention provides the following technical solution: a document semantic repair and stroke reconstruction system based on a conditional control diffusion architecture, comprising: a multimodal conditional coding module, used to extract and encode the input degraded document image under conditions, and output structural coding features, semantic coding features and degraded coding features, wherein the multimodal conditional coding module includes a visual structure coding unit, a semantic conditional coding unit and a degraded feature coding unit; The diffusion reconstruction module is used to perform a condition-controlled reverse diffusion process and perform multi-stage repair. The diffusion reconstruction module includes a condition injection unit, a multi-stage reconstruction unit, and an adaptive sampling unit. The quality assessment module is used to perform semantic verification and visual evaluation on the repaired image. The quality assessment module includes a semantic consistency verification unit, a visual quality assessment unit, and a quality report generation unit. An iterative optimization module is used to perform targeted optimization and repair on areas that do not meet the standards. The iterative optimization module includes a region positioning unit, a targeted optimization unit, and a convergence judgment unit. The training management module is used to manage the training process of the model. The training management module includes a course learning scheduling unit, a multi-task loss calculation unit, and a model optimization unit.

[0017] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the document semantic repair and stroke reconstruction method based on a conditional control diffusion architecture.

[0018] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the document semantic repair and stroke reconstruction method based on a conditional control diffusion architecture.

[0019] The beneficial effects of this invention are as follows: By combining multimodal conditional control with a diffusion model, a technological leap from traditional image processing to intelligent semantic understanding is achieved in document restoration. Through a multimodal conditional perception and encoding mechanism, three types of conditional information—visual structure, domain semantics, and degradation characteristics—are extracted in parallel, providing multi-dimensional prior knowledge for document restoration. Through a conditionally controlled inverse diffusion architecture, a hierarchical injection mechanism is embedded in the denoising network, achieving progressive and refined reconstruction from global semantics to local strokes. Through an end-to-end training and inference system, combined with progressive multi-stage restoration, multi-loss function optimization, and adaptive inference strategies, computational efficiency is improved while ensuring restoration quality. A semantically driven reconstruction paradigm of restoration-as-recognition is realized, ensuring the accurate recovery of key information in professional documents. The modular design facilitates expansion to document restoration tasks in other professional fields. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 The above is a flowchart of a document semantic repair and stroke reconstruction method based on a conditional controlled diffusion architecture, which is provided as an embodiment of the present invention.

[0022] Figure 2 This is a structural diagram of a multimodal conditional coding module for a document semantic repair and stroke reconstruction system based on a conditional control diffusion architecture, provided as an embodiment of the present invention.

[0023] Figure 3 This is a structural diagram of a conditional diffusion reconstruction module in a document semantic repair and stroke reconstruction system based on a conditional control diffusion architecture, provided as an embodiment of the present invention. Detailed Implementation

[0024] To make the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides a document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture, including: Step 1: Extract the stroke skeleton features of the input degraded document image through edge detection as a visual structure condition; obtain the document semantics of the degraded document image through text recognition and semantic encoding as a semantic condition; and evaluate the degradation features of the degraded document image through a degradation estimation network as a degradation condition. Step 2: Input the visual structural conditions, the semantic conditions, and the degradation conditions into the corresponding conditional encoders for encoding to obtain structural coding features, semantic coding features, and degradation coding features; Step 3: Input the structural coding features, semantic coding features, and degenerate coding features into the diffusion model. During the denoising iteration of the diffusion model, inject the structural coding features, semantic coding features, and degenerate coding features into different network layers through a hierarchical injection mechanism. Dynamically adjust the weights of the structural coding features, semantic coding features, and degenerate coding features according to the denoising progress. Generate a reconstructed image through the reverse diffusion process. Step 4: Perform coarse-grained repair, fine-grained repair, and quality refinement on the reconstructed image in sequence to obtain the repaired image; Step 5: Perform semantic verification and visual evaluation on the repaired image, locate the substandard areas based on the results of the semantic verification and visual evaluation, and focus on optimizing the substandard areas. Step 6: Output the optimized document image.

[0026] The core technical challenge in handwritten document restoration lies in simultaneously ensuring semantic accuracy and stroke structure integrity. Traditional methods rely on single visual cues for restoration. When a document exhibits extensive degradation, they fail to understand the contextual semantics of technical terms, easily leading to errors in the restoration of critical equipment numbers or parameter values—for example, mistakenly restoring 101 as 181. This can have serious consequences in applications with extremely high accuracy requirements, such as power systems. While generative adversarial networks (GANs) can improve texture realism, they suffer from training instability and the potential to generate illusory textures, potentially generating strokes or symbols that do not actually exist. Existing diffusion models lack effective integration of multimodal prior knowledge from the professional domain, failing to achieve synergistic restoration driven by semantics and vision. This method constructs a multimodal conditional control mechanism, uniformly encoding three types of conditional information—visual structure, document semantics, and degradation characteristics—and injecting them layer by layer into the diffusion model, achieving simultaneous semantic understanding and stroke reconstruction.

[0027] This embodiment presents a complete document restoration process through six steps, addressing the technical challenges of poor semantic consistency and stroke structure distortion. Step 1 extracts three types of prior knowledge in parallel using a multimodal conditional awareness mechanism, providing multi-dimensional constraints for subsequent restoration. Step 2 converts the original conditions into high-dimensional feature representations using a conditional encoder, facilitating deep integration with the diffusion model. Step 3 introduces different modal conditions at different levels of the denoising network through a hierarchical injection mechanism: shallow layers inject degradation and structural conditions to constrain the global contour; mid-layers inject semantic and structural conditions to standardize character morphology; and deep layers inject all conditions to refine details. The weights of each modality are dynamically adjusted according to the denoising progress, achieving a progressive reconstruction from coarse to fine. Step 4 improves computational efficiency while maintaining quality through a multi-stage restoration strategy. Step 5 ensures the reliability of the restoration results through a dual mechanism of semantic verification and visual evaluation, performing targeted iterative optimization on areas that do not meet the standards. The overall process realizes a semantically driven paradigm of restoration equal to recognition, ensuring the accurate recovery of key information in professional documents.

[0028] Example 2, an embodiment of the present invention, provides a document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture, based on the previous embodiment, including: In step 1, the stroke skeleton features of the input degraded document image are extracted through edge detection as visual structural conditions, the document semantics of the degraded document image are obtained through text recognition and semantic encoding as semantic conditions, and the degradation features of the degraded document image are evaluated through a degradation estimation network as degradation conditions. This includes the following steps A1-A3: A1: Constructing multi-scale feature representations of degraded document images; A2: Input the multi-scale feature representation into the degradation classification network to extract degradation features and output a degradation representation vector; A3: Input the degenerate representation vector into the multilayer perceptron for dimension mapping to generate a high-dimensional degenerate embedding vector as the degeneracy condition.

[0029] In this embodiment of the application, in step A2, the degradation classification network is constructed by: inputting multi-scale feature representations into the degradation classification network architecture, which includes an input convolutional layer, four residual blocks and a fully connected layer. The input convolutional layer performs initial feature extraction on the multi-scale feature representation, each residual block learns features through two convolutional layers and skip connections, and the fully connected layer maps the extracted features into a degradation representation vector.

[0030] In an optional implementation, in step A2, the degradation classification network can: input multi-scale feature representations into a lightweight convolutional neural network architecture, the degradation classification network uses depthwise separable convolutional layers to reduce the number of parameters, aggregate spatial features through global average pooling layers, and output a degradation representation vector.

[0031] In another alternative implementation, in step A2, the degradation classification network can also: input multi-scale feature representations into the degradation classification network of the visual transformer architecture, the degradation classification network converts features into sequence representations through image segmentation and position encoding, captures global degradation patterns through a multi-head self-attention mechanism, and outputs a degradation representation vector.

[0032] Furthermore, the multilayer perceptron contains three fully connected layers. The first fully connected layer maps the degenerate representation vector from 6 dimensions to 64 dimensions and uses the ReLU activation function. The second fully connected layer maps the 64-dimensional features to 256 dimensions and uses the ReLU activation function. The third fully connected layer maps the 256-dimensional features to 512 dimensions and outputs a high-dimensional degenerate embedding vector.

[0033] In step 2, the visual structural conditions, semantic conditions, and degradation conditions are input into the corresponding conditional encoders for encoding to obtain structural coding features, semantic coding features, and degradation coding features, including the following steps B1-B3: B1: Input the visual structural conditions into the visual encoder to extract structural feature representations and output structural encoded features; B2: Input semantic conditions into the semantic encoder to extract semantic features and enhance them with professional domain knowledge, and output semantic encoded features; B3: Input the degradation conditions into the degradation encoder to encode the degradation features, and output the degradation encoded features.

[0034] Furthermore, the visual encoder adopts the U-Net architecture, which concatenates the visual structural conditions and the degraded document image in the channel dimension and inputs them into the U-Net. The encoder part of the U-Net contains four downsampling layers to gradually reduce the spatial resolution and increase the number of feature channels, while the decoder part contains four upsampling layers to gradually restore the spatial resolution. Corresponding layers of the encoder and decoder pass features to preserve detailed information through skip connections. A self-attention mechanism is introduced in the bottleneck layer between the encoder and decoder to enhance the modeling ability of long-distance stroke dependencies.

[0035] Furthermore, domain-specific knowledge enhancement is achieved by establishing a domain-specific dictionary, which contains mappings of equipment models, parameter units, and technical terms. During semantic encoding, word class masks and attention weight enhancements are applied to key terms in the text sequence that match the domain-specific dictionary, enabling key terms to obtain higher representation strength in semantic encoding features and suppressing interference from non-key text.

[0036] In step 3, structural coding features, semantic coding features, and degenerate coding features are input into the diffusion model. During the denoising iteration of the diffusion model, structural coding features, semantic coding features, and degenerate coding features are injected into different network layers through a hierarchical injection mechanism. The weights of structural coding features, semantic coding features, and degenerate coding features are dynamically adjusted according to the denoising progress. The reconstructed image is generated through a reverse diffusion process, including the following steps C1-C3: C1: Injecting degenerate coding features and structural coding features into the first network layer of the diffusion model; C2: Inject semantic and structural coding features into the second network layer of the diffusion model; C3: Inject structural coding features, semantic coding features, and degenerate coding features into the third network layer of the diffusion model.

[0037] In this embodiment, step 3, the hierarchical injection mechanism works as follows: at the first network layer of the diffusion model, degenerate coding features and structural coding features are scaled and translated through a conditional normalization layer and then integrated into the feature map; at the second network layer, semantic coding features and structural coding features are used as key-value pairs and queried and integrated into the feature map through a cross-attention layer; at the third network layer, structural coding features, semantic coding features, and degenerate coding features are integrated into the feature map through a multi-head cross-attention layer that performs parallel computation of multiple attention heads.

[0038] In an optional implementation, in step 3, the hierarchical injection mechanism can be achieved by: generating modulation parameters for each modality-coded feature through a feature modulation layer at different network layers of the diffusion model. The modulation parameters include scaling coefficients and offset coefficients. The modulation parameters are then applied to the feature map through affine transformation to achieve conditional injection.

[0039] In another alternative implementation, in step 3, the hierarchical injection mechanism can also achieve conditional injection by concatenating the modality-encoded features and feature maps in the channel dimension at different network layers of the diffusion model, and fusing the concatenated features through convolutional layers.

[0040] In step 4, the reconstructed image undergoes coarse-grained restoration, fine-grained restoration, and quality refinement sequentially to obtain the restored image, including the following steps D1-D3: D1: The reconstructed image is downsampled to a preset resolution, and the first denoising reconstruction is performed using a diffusion model to obtain a coarsely repaired image; D2: Upsample the coarsely restored image to the original resolution, and perform a second denoising and reconstruction using a diffusion model to obtain the finely restored image; D3: Input the finely restored image into the adversarial network for quality enhancement to obtain the restored image.

[0041] In this embodiment of the application, in step D3, the adversarial network performs the following steps: inputting the finely restored image into a generator for feature enhancement, the generator containing multiple convolutional layers and upsampling layers; inputting the image output by the generator and the real, clear image into a discriminator, the discriminator containing multiple convolutional layers and fully connected layers, and outputting a realism score; calculating the adversarial loss based on the realism score, and optimizing the generator parameters through backpropagation to obtain the restored image.

[0042] In an optional implementation, in step D3, the adversarial network can: input the finely repaired image into the generator, downsample the image output by the generator to multiple resolution levels, each resolution level corresponding to a discriminator, the multiple discriminators perform authenticity discrimination from the global, local and stroke levels respectively, calculate the adversarial loss by combining the scores of the multiple discriminators, and optimize the generator.

[0043] In another optional implementation, in step D3, the adversarial network can also: input the finely restored image into the generator, extract multi-layer features from the image output by the generator and the real clear image respectively, use a feature discriminator to judge the authenticity of features at different levels, calculate the perceptual adversarial loss based on the discrimination results of the feature levels, and optimize the generator.

[0044] Furthermore, the preset resolution is determined based on the original resolution and computational efficiency requirements. For an original resolution image of 512×512 pixels, the preset resolution is set to 112×112 pixels. The reconstructed image is downsampled to the preset resolution through bilinear interpolation or max pooling operations. Coarse repair at low resolution can use a larger denoising step size to quickly complete the reconstruction of global structure and semantic content.

[0045] Furthermore, in this embodiment, in step 5, locating the substandard regions involves: inputting the repaired image into a text recognition model to obtain the recognition result; calculating the character accuracy between the recognition result and the recognition result of the degraded document image as a semantic consistency score; calculating the peak signal-to-noise ratio, structural similarity, and perceptual loss of the repaired image as a visual quality score; and marking image regions with semantic consistency scores below a first threshold or visual quality scores below a second threshold as substandard regions.

[0046] In an optional implementation, in step 5, locating the non-compliant areas can be achieved by: detecting stroke break points and character deformation areas in the repaired image through a stroke continuity detection network, counting the number of stroke break points and the degree of character deformation as a structural integrity score, and marking areas with a structural integrity score lower than a third threshold as non-compliant areas.

[0047] In another optional implementation, in step 5, locating the substandard areas can also be achieved by: constructing a quality heatmap, calculating a local quality score for each image block of the repaired image, combining the local semantic confidence and local visual quality, and marking the areas corresponding to the image blocks with scores below a threshold in the quality heatmap as substandard areas.

[0048] It should be noted that the degradation evaluation result output by the degradation classification network is a degradation vector, which contains six dimensions of degradation degree. The six dimensions are, in order, ink spread, paper wrinkles, uneven lighting, noise pollution, blur degree and local occlusion. The degradation assessment results are used to generate degradation embedding vectors by feature mapping, including: inputting the degradation vector into a three-layer perceptron, and the three-layer perceptron is expanded in dimension by three fully connected layers in sequence to map the degradation vector into a high-dimensional degradation embedding vector.

[0049] Furthermore, the feature map resolution corresponding to the first network layer is greater than that corresponding to the second network layer, and the feature map resolution corresponding to the second network layer is greater than that corresponding to the third network layer. Degenerate coding features and structural coding features are injected into the first network layer of the diffusion model. Conditional normalization is then used to integrate these features into the feature map of the first network layer. Semantic and structural encoding features are injected into the second network layer of the diffusion model, and these features are integrated into the feature map of the second network layer through a cross-attention mechanism. In the third network layer of the diffusion model, structural coding features, semantic coding features, and degenerate coding features are comprehensively injected. Through a multi-head cross-attention mechanism, structural coding features, semantic coding features, and degenerate coding features are integrated into the feature map of the third network layer.

[0050] Example 3, an embodiment of the present invention, provides a document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture, based on the previous embodiment, including: In step A1, constructing the multi-scale feature representation of the degraded document image specifically includes: The degraded document image is constructed into a multi-scale feature pyramid, which includes original resolution features, half-resolution features, and quarter-resolution features. Global structural analysis is performed on the features at each resolution level to extract the degraded feature representations at different scales.

[0051] In step 1, the stroke skeleton features of the input degraded document image are extracted as visual structural conditions through edge detection, specifically including: An improved Canny edge detection algorithm is used to extract edges from degraded document images. The improved Canny edge detection algorithm adds adaptive threshold determination and non-maximum suppression optimization to the standard Canny framework. The edge features extracted by the Canny edge detection algorithm are fused with the edge features extracted by the deep learning edge enhancement network. The deep learning edge enhancement network learns multi-scale edge representations through a convolutional neural network. The stroke continuity enhancement process is applied to the fused edge features, with a focus on preserving the stroke continuity of the device number area and parameter reading area, and the output stroke skeleton features are used as visual structure conditions.

[0052] In step 2, the visual structural conditions are input into the visual encoder to extract structural feature representations, specifically including: The visual structural conditions are stitched together with the original image of the degraded document image in the channel dimension to form a multi-channel input; The visual encoder is a multi-channel input encoder-decoder architecture. The encoder part of the visual encoder extracts multi-scale structural features through multiple downsampling convolutional layers, and the decoder part of the visual encoder restores spatial resolution through multiple upsampling layers. A self-attention mechanism is introduced at the bottleneck layer of the visual encoder. The self-attention mechanism enhances the ability to model long-distance stroke dependencies by calculating the correlation between different positions in the feature map. The decoder part of the visual encoder outputs multi-scale structure-encoded features.

[0053] In step 2, the semantic conditions are input into the semantic encoder for semantic feature extraction and enhanced with domain-specific knowledge, specifically including: Text sequence is obtained by performing text recognition on degraded document images using an optical character recognition model. The text sequence is input into a pre-trained language model for deep semantic encoding. The pre-trained language model extracts the semantic representation of the text through a multi-layer transformer structure. Key terms in the text sequence are annotated based on a professional domain dictionary, which includes a mapping relationship between equipment model, parameter unit and professional terminology; During the encoding process of the pre-trained language model, attention enhancement is applied to the key terms annotated by the professional domain dictionary to increase the weight of key information in the semantic encoding features and output semantic encoding features.

[0054] In step 2, the degradation conditions are input into the degradation encoder for degradation feature encoding, specifically including: A six-dimensional degradation feature enhancement and conditional fusion unit is used as the degradation encoder. The six-dimensional degradation feature enhancement and conditional fusion unit contains six parallel processing branches. Each processing branch completes channel compression and scale alignment through a one-to-one convolutional layer; The degradation conditions are mapped to high-dimensional degradation embeddings through a multilayer perceptron, which contains three fully connected layers. The high-dimensional degenerate embedding is applied to the backbone features and then scaled and translated using conditional normalization. The conditional normalization generates scaling and translation parameters based on the degenerate embedding. The degradation-aware attention mechanism performs adaptive modulation in the channel and spatial dimensions, and dynamically adjusts the fusion weights and enhancement strength according to different degradation types. To reduce computational complexity, a two-level 3x3 convolutional concatenation is used instead of a 5x5 convolution, and multi-scale contextual features are constructed in conjunction with dilated convolution and pyramid pooling. Depth-separable convolution is used instead of standard convolution to further reduce computational cost; At the end of each processing branch, dilated convolutions with different dilation coefficients are used to converge features and obtain degenerate coding features.

[0055] In step 2, the fusion process of the encoded features output by the conditional encoder specifically includes: An edge enhancement prior is introduced into the edge feature channel, and an edge confidence map is generated by adaptive thresholding and optimized non-maximum suppression. The edge confidence map and structural coding features are fused at multiple scales to maintain the continuity of the stroke skeleton. In the text semantic channel, part-of-speech masking is applied to the semantic encoding features through a professional domain dictionary. The part-of-speech masking marks key information such as device model and unit symbol to suppress interference from non-key text. Residual connections are preserved before and after the fusion of each coding feature, and a self-attention mechanism is added to the bottleneck layer of the fusion network to enhance long-distance dependency modeling.

[0056] In step 3, the weights of structural coding features, semantic coding features, and degenerate coding features are dynamically adjusted according to the denoising progress, specifically including: Design a conditional attention gating mechanism to calculate the contribution weight of each modal condition based on the current noise level and repair progress in each denoising iteration. In the early stages of denoising, when the noise level is high, increase the weight of structural coding features and degenerate coding features, and decrease the weight of semantic coding features to stabilize the global outline and coarse-grained content. In the later stages of denoising, when the noise level is low, the weight of degenerate coding features is reduced and the weight of semantic coding features is increased to guide detail convergence and content accuracy. The weight adjustment adopts a noisy scheduling strategy with a larger step size in the early stage and a smaller step size in the later stage.

[0057] In step 3, the process of generating the reconstructed image through reverse diffusion specifically includes: A denoising diffusion probability model framework is used for reverse diffusion. The denoising diffusion probability model gradually denoises Gaussian noise to generate a clear image through a time stepping method. At each time step, the time step information is encoded into a time embedding vector through a time embedding layer; The temporal embedding vector and the modality-coded features are jointly injected into the denoising network through conditional normalization and cross-attention mechanism to ensure channel alignment and controllable intensity. Degradation coding features serve as a unified prior throughout the entire denoising process. Adaptive modulation is implemented in both the channel and spatial dimensions. The fusion ratio and gain limit of the structural branch, semantic branch, and edge branch are dynamically adjusted for different degradation types such as ink diffusion, uneven illumination, blurring, noise, wrinkles, and occlusion. To prevent feature collapse, residual connections and skip connections are maintained in the denoising network. Cross-scale consistency constraints and edge confidence graph priors are introduced to improve convergence stability and reconstruction robustness.

[0058] In step D1, the process of downsampling the reconstructed image to a preset resolution for coarse restoration specifically includes: The reconstructed image is downsampled to a preset resolution of 112×112 using bilinear interpolation or pooling operations; The downsampled image is denoised and reconstructed using a diffusion model. The first denoising and reconstruction uses a large denoising step size to quickly complete the reconstruction of the global structure and semantic content. In the first denoising and reconstruction process, structural coding features and degenerative coding features are used as strong constraints to stabilize the pattern and contour, thus obtaining a coarsely repaired image.

[0059] Step D2, which involves upsampling the coarsely repaired image to its original resolution for fine repair, specifically includes: The coarse-repaired image is upsampled to its original resolution using bilinear interpolation or deconvolution. A detail enhancement diffusion model is introduced to perform a second denoising reconstruction on the upsampled image. The detail enhancement diffusion model focuses on processing stroke connections, character intersections, occlusion edges, and breakage areas. In the second denoising and reconstruction process, perceptual loss, edge consistency loss and stroke continuity constraints are combined to improve the realism of details and reading coherence; Adaptive region refinement is performed based on edge confidence maps and character confidence maps. For high-risk regions with low edge confidence or low character confidence, the number of denoising steps is increased to obtain a finely restored image.

[0060] In step D3, the process of inputting the finely restored image into the adversarial network for quality enhancement specifically includes: The finely restored image is input into the adversarial refining network, which contains a generator and a multi-scale discriminator. The multi-scale discriminator includes a global discriminator, a local discriminator, and a stroke-level discriminator. The global discriminator judges the authenticity of the entire image, the local discriminator judges the authenticity of local regions, and the stroke-level discriminator judges the authenticity of stroke details. The adversarial loss is calculated based on the discrimination results of the multi-scale discriminator, and the generator parameters are optimized through adversarial training. Cross-scale consistency regularization constraints and domain-specific dictionary constraints are incorporated into the adversarial training process. Cross-scale consistency regularization constraints ensure the consistency of features at different scales, while domain-specific dictionary constraints reduce semantic bias, thereby further improving visual naturalness and text readability.

[0061] Step 5, the process of semantic verification of the repaired image, specifically includes: The repaired image is input into the optical character recognition system to obtain the repaired recognition result; The restored recognition results are compared with the original recognition results of the degraded document image; Design a semantic confidence scoring function that comprehensively considers character recognition confidence, contextual semantic coherence, and domain knowledge conformity. Character recognition confidence is calculated using the probability value output by the optical character recognition model; Contextual semantic coherence is evaluated using language models to assess the reasonableness of text sequences; Domain knowledge compliance is calculated using the matching rate of a professional domain dictionary. The semantic consistency score is calculated based on the semantic confidence scoring function.

[0062] Step 5, the process of visually evaluating the restored image, specifically includes: A full-reference image quality assessment algorithm is used to calculate the similarity between the restored image and the real clear image; Calculate the peak signal-to-noise ratio as a pixel-level similarity index; Calculate structural similarity as an indicator of structural preservation; The perceptual loss is calculated as a perceptual quality indicator. The perceptual loss is calculated by extracting features and calculating feature distances through a pre-trained convolutional neural network. The stroke continuity evaluation subnetwork detects the number of stroke breakpoints and the degree of character deformation. The stroke continuity evaluation subnetwork learns stroke structure features through a convolutional neural network. The visual quality score is calculated by combining the above indicators.

[0063] Step 5, the process of focusing on optimizing areas that do not meet the standards, specifically includes: A regionalized repair mask is generated based on the semantic consistency score and visual quality score. The repair mask marks image regions whose semantic consistency score is lower than a first threshold or whose visual quality score is lower than a second threshold. In the next round of diffusion, the region marked by the repair mask will be optimized in a key way, and the repair effect will be enhanced by increasing the number of denoising steps and increasing the semantic condition weights. Monitor the quality improvement during the optimization process and calculate the quality score increment between two consecutive iterations; The iterative optimization is terminated when the quality score increment is less than the preset threshold or the number of iterations reaches the maximum number of iterations.

[0064] Furthermore, the method also supports the following optional enhancements: The real-time interactive repair function receives user annotations of key areas through the user interface and applies stronger semantic and structural constraints to the user-annotated areas as high-priority areas during the diffusion process. The multi-version output function provides three repair results: fast version, standard version, and fine version. The fast version uses fewer denoising steps, the standard version uses a medium number of denoising steps, and the fine version uses more denoising steps and performs iterative optimization. The repair history database function stores the mapping relationship between degradation feature vectors and optimal hyperparameter combinations. After extracting degradation features from newly input degradation document images, it quickly initializes repair parameters by matching historical records through nearest neighbor search.

[0065] It should be noted that the degradation classification network specifically adopts the ResNet-34 residual network architecture, which achieves deep feature learning through residual block structure and skip connections; the visual encoder specifically adopts the U-Net architecture, which includes symmetrical encoder and decoder paths, and features are transferred between the encoder and decoder through skip connections; the pre-trained language model specifically adopts a lightweight version of the BERT model, which extracts the semantic representation of text through a multi-layer transformer structure.

[0066] It should be noted that the equipment models in the professional domain dictionary include power equipment models such as LW36-126, and the parameter units include power parameter units such as kV and A. During the encoding process, attention enhancement is applied to the key terms marked in the professional domain dictionary to increase the weight of key information in semantic encoding features.

[0067] It should be noted that the preset resolution is 112×112 pixels, and the original resolution is 512×512 pixels; the multi-scale feature pyramid includes three levels: original resolution, half resolution, and quarter resolution.

[0068] It should be noted that the dynamic adjustment range of the number of denoising steps is 25 to 50 steps, and the appropriate number of denoising steps is dynamically selected according to the degree of image degradation; the total number of time steps is 1000 steps, and a linear noise scheduling strategy is adopted, with the noise variance increasing linearly from 0.0001 to 0.02.

[0069] It should be noted that the dimensional mapping relationship of the multilayer perceptron is 6-dimensional input, 64-dimensional first hidden layer, 256-dimensional second hidden layer, and 512-dimensional output layer; the six-dimensional degradation vector includes the degradation degree of six dimensions: ink spread, paper wrinkles, uneven lighting, noise pollution, blur degree, and local occlusion.

[0070] It should be noted that conditional normalization incorporates degradation embeddings into the backbone features through scaling and translation operations; the degradation-aware attention mechanism adaptively modulates the channel and spatial dimensions, dynamically adjusting the fusion weights and enhancement intensity according to different degradation types.

[0071] It should be noted that dilated convolution uses different dilation coefficients to converge features, obtaining diverse multi-scale feature information and a wider effective receptive field; depthwise separable convolution includes two steps: depthwise convolution and pointwise convolution, reducing computational cost; pyramid pooling constructs multi-scale contextual features.

[0072] It should be noted that the self-attention mechanism enhances the ability to model long-distance stroke dependencies by calculating the correlation between different positions in the feature map; the multi-head cross-attention mechanism contains 8 attention heads, and the outputs of multiple attention heads are fused.

[0073] It should be noted that the stroke continuity constraint is achieved by detecting the continuity of stroke connections and character intersections; the edge consistency loss is achieved by calculating the difference between the edge maps of the repaired image and the real image; and the perceptual loss is achieved by extracting features and calculating feature distances through a pre-trained VGG network.

[0074] It should be noted that the adversarial loss optimizes the generator parameters through adversarial training; the cross-scale consistency regularization constraint ensures the consistency of features at different scales.

[0075] It should be noted that the character confidence map is generated by the recognition confidence score output by the optical character recognition model; the edge confidence map is generated by the edge detection algorithm; and the region adaptive refinement identifies high-risk areas based on the edge confidence map and the character confidence map, and increases the number of denoising steps for high-risk areas.

[0076] It should be noted that the real-time interactive repair function receives user annotations of key regions through the user interface, and applies stronger semantic and structural constraints to the user-annotated regions as high-priority regions during the diffusion process; the multi-version output function provides three repair results: fast version, standard version, and fine version, with different versions using different denoising steps; the repair history library function stores the mapping relationship between degraded feature vectors and optimal hyperparameter combinations, and quickly initializes repair parameters by matching historical records through nearest neighbor search.

[0077] It should be noted that residual connections achieve feature reuse by adding input features to output features; skip connections achieve detail information transfer by concatenating features from the encoder layer with features from the corresponding decoder layer; and two-stage 3×3 convolutional cascades replace 5×5 convolutions to reduce computational complexity.

[0078] It should be noted that the diffusion model, conditional encoder, and adversarial network involved in the method are optimized using a multi-stage training strategy.

[0079] In the training data preparation phase, a document image dataset covering multiple degradation types is constructed, encompassing professional documents such as handwritten records of power systems and equipment drawings. For each type of document, a complete image is paired with a corresponding simulated damaged version to form sample pairs that can be used for training. The training dataset is divided into different difficulty levels according to the degree of degradation, which is used to implement a progressive course learning and training strategy from simple degradation to complex degradation.

[0080] In the pre-training stage of the basic diffusion model, the basic diffusion model is first pre-trained on a large-scale synthetic degraded-clear document image pair. The basic diffusion model gradually adds noise to the clear image through a forward diffusion process, and then trains a denoising network to recover the clear image from the noise through a reverse diffusion process. The pre-training stage uses pixel-level reconstruction loss to calculate the pixel difference between the predicted image and the ground truth image. An adaptive momentum optimizer is used for gradient backpropagation, and a model checkpoint mechanism is established to save the model parameters with the best performance on the validation set.

[0081] In the multimodal conditional fine-tuning stage, a multimodal conditional vector is introduced on the pre-trained basic diffusion model for fine-tuning. The multimodal conditional vector includes structural coding features, semantic coding features, and degradation coding features. The fine-tuning stage adopts a composite loss function, which combines pixel-level L1 loss and perceptual loss LPIPS, and also introduces semantic loss. The perceptual loss is calculated by extracting features through a pre-trained VGG network to calculate the perceptual distance in the feature space. The semantic loss is evaluated by an optical character recognition model to assess the OCR accuracy of the restored image, and the accuracy difference between the restored image recognition result and the ground truth image recognition result is calculated.

[0082] In the adversarial training refinement stage, adversarial training is introduced to further improve visual quality. This adversarial training includes a generator and a multi-scale discriminator. The generator enhances the quality of the repaired image output from the fine-tuning stage. The multi-scale discriminator performs realism judgment and detail refinement at three levels: global image, local region, and stroke level. The generator parameters are optimized through adversarial training between the generator and the discriminator to improve visual naturalness and text readability. Cross-scale consistency regularization and domain dictionary constraints are added during the adversarial training process. The domain dictionary constraints ensure that the repaired device model and parameter units conform to domain specifications by matching the OCR recognition results with a professional domain dictionary, thereby reducing artifacts and semantic deviations.

[0083] The progressive course learning scheduling module implements a progressive training strategy from simple degradation to complex degradation. The course learning scheduling unit automatically adjusts the difficulty level of the training data. In the early stage of training, simple degradation samples are used to enable the model to quickly learn basic repair capabilities. In the middle stage of training, medium degradation samples are gradually introduced to enhance the model's generalization ability. In the later stage of training, complex degradation samples are used to improve the model's repair capabilities under extreme degradation conditions.

[0084] The model is integrated with power industry scenarios. The conditional encoder incorporates power industry knowledge into the semantic encoding process through a professional domain dictionary. This dictionary includes mappings between power equipment models, parameter units, and technical terms. A domain dictionary specifically for power industry technical terms is established to strengthen the feature representation of key information such as equipment models and parameter units during the encoding process. The degradation estimation network outputs a six-dimensional degradation vector, representing ink diffusion, paper wrinkles, uneven lighting, noise pollution, blurring, and partial occlusion, respectively. This six-dimensional degradation vector is mapped according to its numerical range to generate repair strategy parameters, providing quantitative input for subsequent conditional control. The model is trained on typical degradation types of power documents. The diffusion model takes a degraded document image and its corresponding structural, semantic, and degradation coding features as input and outputs a clear, repaired document image. Through multimodal conditional guidance, the integrity of the stroke structure and the semantic accuracy of technical terms are ensured simultaneously during the repair process.

[0085] To ensure stability during training and inference, residual connections are preserved in each branch before and after fusion, and self-attention is added to the bottleneck layer to enhance long-distance stroke dependency modeling. The denoising network maintains residual connections and skip connections to prevent feature collapse, and introduces cross-scale consistency constraints and edge confidence map priors to improve convergence stability and reconstruction robustness. The model checkpoint mechanism saves the model parameters with the best performance on the validation set during training.

[0086] Example 4, refer to Figures 2-3 This is one embodiment of the present invention, which provides a document semantic repair and stroke reconstruction system based on a conditionally controlled diffusion architecture, comprising: The multimodal conditional coding module is used to conditionally extract and encode the input degraded document image, and output structural coding features, semantic coding features, and degradation coding features; The diffusion reconstruction module is used to perform a condition-controlled reverse diffusion process and carry out multi-stage repair. The quality assessment module is used to perform semantic verification and visual evaluation on the repaired image; The iterative optimization module is used to perform targeted optimization and repair on areas that do not meet the standards. The training management module is used to manage the model training process.

[0087] Furthermore, the multimodal conditional coding module includes a visual structure coding unit, a semantic conditional coding unit, and a degradation feature coding unit; The visual structure encoding unit adopts a multi-scale feature pyramid network, takes a degraded document image as input, and outputs stroke skeleton features and layout features. Through improved Canny edge detection combined with deep learning edge enhancement algorithm, it accurately extracts the stroke continuity features of key areas. The semantic condition coding unit uses an optical character recognition model to perform text recognition, inputs the recognized text sequence into a pre-trained language model for deep semantic encoding to obtain semantic conditions, and incorporates a professional domain dictionary to enhance the feature representation of key information such as device model and parameter units. The degradation feature encoding unit evaluates six types of degradation features through a degradation estimation network. These six types of degradation features include ink diffusion, paper wrinkles, uneven lighting, noise pollution, blurring degree, and local occlusion. The unit outputs a degradation feature vector and maps it to a high-dimensional conditional space.

[0088] Furthermore, the diffusion reconstruction module includes a conditional injection unit, a multi-stage reconstruction unit, and an adaptive sampling unit; The conditional injection unit adopts a denoising network with an encoder-decoder architecture, injecting the structural coding features, semantic coding features and degenerate coding features at different depths of the network respectively, and designs a conditional attention gating mechanism to dynamically adjust the contribution weight of each modality condition according to the current noise level. The multi-stage reconstruction unit implements a three-stage progressive repair strategy, including a coarse repair stage, a fine repair stage, and a post-processing stage. The coarse repair stage performs rapid reconstruction at low resolution, the fine repair stage performs detail optimization at high resolution, and the post-processing stage refines the quality through adversarial training. The adaptive sampling unit dynamically adjusts the number of denoising steps according to the degree of image degradation, optimizing computational efficiency while ensuring quality.

[0089] Furthermore, the quality assessment module includes a semantic consistency verification unit, a visual quality assessment unit, and a quality report generation unit; The semantic consistency verification unit inputs the repaired image into the optical character recognition system and comprehensively evaluates the character recognition confidence, contextual semantic coherence, and domain knowledge conformity through the semantic confidence scoring function. The visual quality assessment unit adopts a full-reference image quality assessment algorithm to quantify the repair effect from three dimensions: peak signal-to-noise ratio, structural similarity, and perceptual loss. The stroke continuity assessment sub-network is used to specifically detect problems such as stroke breakage and character deformation. The quality report generation unit generates a detailed evaluation report that includes before-and-after comparisons, quality scores, and annotations of problem areas.

[0090] Furthermore, the iterative optimization module includes a region positioning unit, a targeted optimization unit, and a convergence judgment unit; The regional positioning unit locates the substandard areas based on the quality assessment results and generates a regional repair mask; The targeted optimization unit focuses on optimizing low-quality areas during the next round of diffusion, employing a locally enhanced diffusion sampling strategy. The convergence judgment unit monitors the quality improvement during the optimization process and terminates the optimization when the quality requirements are met or the maximum number of iterations is reached.

[0091] Furthermore, the training management module includes a course learning scheduling unit, a multi-task loss calculation unit, and a model optimization unit; The course learning scheduling unit implements a progressive training strategy from simple degradation to complex degradation, and automatically adjusts the difficulty level of the training data; The multi-task loss calculation unit calculates a composite loss function, which includes reconstruction loss, perceptual loss, adversarial loss, and semantic loss. The model optimization unit uses an adaptive momentum optimizer to perform gradient backpropagation based on the composite loss function and establishes a model checkpoint mechanism to save the model parameters with the best performance on the validation set.

[0092] This system, through the collaborative work of multiple modules, realizes a complete process from condition awareness to high-quality document reconstruction. While ensuring semantic accuracy, it significantly improves the reconstruction quality at the stroke level, providing reliable technical support for the digital protection of documents in professional fields.

[0093] This embodiment also provides an electronic device applicable to a document semantic repair and stroke reconstruction method based on a conditional control diffusion architecture, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the document semantic repair and stroke reconstruction method based on a conditional control diffusion architecture as proposed in the above embodiment.

[0094] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a document semantic repair and stroke reconstruction method based on a conditional control diffusion architecture as proposed in the above embodiments.

[0095] The storage medium proposed in this embodiment and the document semantic repair and stroke reconstruction method based on conditional control diffusion architecture proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0096] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0097] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture, characterized in that: include, The stroke skeleton features of the input degraded document image are extracted by edge detection as visual structural conditions, the document semantics of the degraded document image are obtained by text recognition and semantic encoding as semantic conditions, and the degradation features of the degraded document image are evaluated by a degradation estimation network as degradation conditions. The visual structural conditions, semantic conditions, and degradation conditions are respectively input into the corresponding conditional encoders for encoding to obtain structural coding features, semantic coding features, and degradation coding features; The structural coding features, semantic coding features, and degenerate coding features are input into the diffusion model. During the denoising iteration of the diffusion model, the structural coding features, semantic coding features, and degenerate coding features are injected into different network layers through a hierarchical injection mechanism. The weights of the structural coding features, semantic coding features, and degenerate coding features are dynamically adjusted according to the denoising progress. The reconstructed image is generated through the reverse diffusion process. The reconstructed image is then subjected to coarse-grained repair, fine-grained repair, and quality refinement in sequence to obtain a repaired image; The repaired image is subjected to semantic verification and visual evaluation. Based on the results of the semantic verification and visual evaluation, the substandard areas are located and the substandard areas are optimized in a focused manner. Output the optimized document image.

2. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 1, characterized in that: The step of evaluating the degradation features of the degraded document image as degradation conditions using a degradation estimation network includes... Construct a multi-scale feature representation of the degraded document image; The multi-scale feature representation is input into the degradation classification network to extract degradation features and output a degradation representation vector; The degradation representation vector is input into a multilayer perceptron for dimensional mapping to generate a high-dimensional degradation embedding vector as the degradation condition.

3. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 2, characterized in that: The conditional encoder corresponding to the input performs encoding, including... The visual structural conditions are input into a visual encoder to extract structural feature representations, and structural encoded features are output. The semantic conditions are input into the semantic encoder for semantic feature extraction and enhanced with domain knowledge to output semantic encoded features. The degradation conditions are input into the degradation encoder for degradation feature encoding, and the degradation encoded features are output.

4. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 3, characterized in that: The obtained restored image includes, The reconstructed image is downsampled to a preset resolution, and a first denoising reconstruction is performed using a diffusion model to obtain a coarsely repaired image. The coarse-restored image is upsampled to the original resolution, and a second denoising and reconstruction is performed using a diffusion model to obtain the fine-restored image. The finely restored image is input into an adversarial network for quality enhancement to obtain the restored image.

5. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 4, characterized in that: The step of injecting the structural coding features, the semantic coding features, and the degenerate coding features at different network levels through a hierarchical injection mechanism during the denoising iteration of the diffusion model includes: The degradation coding feature and the structural coding feature are injected into the first network layer of the diffusion model; The semantic encoding features and the structural encoding features are injected into the second network layer of the diffusion model; The structural coding features, the semantic coding features, and the degenerate coding features are comprehensively injected into the third network layer of the diffusion model.

6. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 5, characterized in that: The degradation assessment result output by the degradation classification network is a degradation vector, which contains six dimensions of degradation degree. The six dimensions are, in order, ink diffusion, paper wrinkles, uneven lighting, noise pollution, blur degree and local occlusion. The step of generating a degradation embedding vector by feature mapping of the degradation assessment result includes: The degradation vector is input into a three-layer perceptron, which expands the dimension sequentially through three fully connected layers, mapping the degradation vector into a high-dimensional degradation embedding vector.

7. The document semantic repair and stroke reconstruction method based on a conditionally controlled diffusion architecture as described in claim 6, characterized in that: The feature map resolution corresponding to the first network layer is greater than that corresponding to the second network layer, and the feature map resolution corresponding to the second network layer is greater than that corresponding to the third network layer. The degenerate coding features and the structural coding features are injected into the first network layer of the diffusion model, and the degenerate coding features and the structural coding features are integrated into the feature map of the first network layer through conditional normalization. The semantic encoding features and the structural encoding features are injected into the second network layer of the diffusion model, and the semantic encoding features and the structural encoding features are integrated into the feature map of the second network layer through a cross-attention mechanism. The structural coding features, semantic coding features, and degenerate coding features are comprehensively injected into the third network layer of the diffusion model, and the structural coding features, semantic coding features, and degenerate coding features are integrated into the feature map of the third network layer through a multi-head cross-attention mechanism.

8. A document semantic repair and stroke reconstruction system based on a conditional controlled diffusion architecture, employing the document semantic repair and stroke reconstruction method based on a conditional controlled diffusion architecture as described in any one of claims 1 to 7, characterized in that, include: A multimodal conditional coding module is used to extract and encode the input degraded document image under conditions, and output structural coding features, semantic coding features and degraded coding features. The multimodal conditional coding module includes a visual structural coding unit, a semantic conditional coding unit and a degraded feature coding unit. The diffusion reconstruction module is used to perform a condition-controlled reverse diffusion process and perform multi-stage repair. The diffusion reconstruction module includes a condition injection unit, a multi-stage reconstruction unit, and an adaptive sampling unit. The quality assessment module is used to perform semantic verification and visual evaluation on the repaired image. The quality assessment module includes a semantic consistency verification unit, a visual quality assessment unit, and a quality report generation unit. An iterative optimization module is used to perform targeted optimization and repair on areas that do not meet the standards. The iterative optimization module includes a region positioning unit, a targeted optimization unit, and a convergence judgment unit. The training management module is used to manage the training process of the model. The training management module includes a course learning scheduling unit, a multi-task loss calculation unit, and a model optimization unit.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the document semantic repair and stroke reconstruction method based on the conditional control diffusion architecture according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the document semantic repair and stroke reconstruction method based on the conditional control diffusion architecture as described in any one of claims 1 to 7.

Images