High-light fuzzy text image semi-supervised restoration method and system based on dark channel guidance

Abstract

The invention discloses a highlight fuzzy text image restoration method and system based on dark channel prior guidance. The method comprises the following steps: firstly, constructing a highlight-fuzzy degraded text image training set containing synthetic data and real shooting data, simulating different illumination forms through a renderer, and collecting real samples under indoor and outdoor multi-equipment and multi-illumination conditions to form labeled and non-labeled data; according to the image restoration network, an encoder-decoder structure is adopted, Haar wavelet downsampling, dark channel guide feature extraction and a YZMama multi-scale long-range dependence modeling module are introduced, and text detail reconstruction is achieved through jump connection and multi-scale output in the decoding stage. Through a twin network training strategy, different degradation versions of the same scene are input into a weight sharing network, and through combination of color consistency loss, adversarial loss, feature space loss and dark channel consistency loss, supervision and self-supervision joint optimization is realized.

Description

A semi-supervised method and system for inpainting specularly blurred text images based on dark channel guidance Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and more specifically, to a semi-supervised method and system for inpainting specularly blurred text images based on dark channel guidance. Background Technology

[0002] Text images captured in natural scenes often exhibit localized highlights and quality degradation due to uneven lighting, specular reflection, motion blur, noise and compression artifacts, device limitations, or material surface characteristics. These factors can impair the extraction of critical information. Such quality impairments not only affect human reading but also significantly reduce the performance of downstream applications such as optical character recognition (OCR). Therefore, developing methods for highlight removal and blur removal in text images is crucial.

[0003] Early specular highlight removal methods primarily relied on mathematical optimization models or chromaticity propagation techniques. Some methods proposed optimization frameworks based on the observation that the dark channel of a natural image can approximate a specular-free image. Other methods, based on the assumptions of specular sparsity and that diffuse components can be represented as sparse linear combinations of primary colors, further proposed a joint specular-diffuse separation framework for single images. However, these methods often struggle to effectively reconstruct missing details in specular regions, limiting their practical application value. With the advancement of deep learning, numerous specular highlight removal methods have emerged in recent years, but these methods mainly target small areas of specular highlights in natural scene images, performing poorly when dealing with large areas of specular highlights in text images. Some methods propose text image specular highlight removal methods based on text synthesis datasets built using rendering software. However, because their training data mainly comes from synthetic datasets and employs fully supervised learning with real-world annotations, the trained models have limited generalization ability on real-world scene data.

[0004] Significant progress has been made in the field of image deblurring primarily through two research paradigms: (1) optimization-based methods, which utilize mathematical priors and iterative optimization for image restoration; and (2) learning-based methods, which use deep neural networks to directly learn deblurring maps from data. Existing methods typically treat image deblurring and highlight removal as independent tasks, and their models are trained only on synthetic datasets within a supervised learning framework. This limitation severely restricts the model's ability to generalize to real-world images, often resulting in poor restoration performance.

[0005] Therefore, there is an urgent need for a semi-supervised method and system for inpainting specularly blurred text images based on dark channel guidance to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to solve the technical problems mentioned in the background art above, and to provide a method for inpainting specularly blurred text images based on dark channel prior guidance. The method comprises the following steps: Step 1: Constructing a training set, which includes synthetic data and real-world shooting data. The synthetic data simulates specular blur effects of different positions, shapes, and intensities using a renderer. The real-world shooting data includes specular, blurred, and specular-blur composite degraded text images collected in indoor and outdoor environments, and is divided into labeled and unlabeled data based on whether a high-definition reference image is available; Step 2: Designing an image inpainting network, which adopts an encoder-decoder structure, introducing Haar wavelet downsampling and dark channel prior guidance during the encoding stage. The feature extraction mechanism is guided by the principle of multi-scale upsampling and skip connections for image reconstruction during the decoding stage. A twin network training strategy is constructed to input different degraded versions of the same high-definition text image into a neural network with shared weights to constrain the consistency of the restored images. Step 3: Design loss functions. For labeled data, a supervised loss is constructed that includes appearance color consistency loss, adversarial loss, high-dimensional space feature consistency loss, and dark channel consistency loss. For unlabeled data, a consistency loss based on the restoration results of multiple degraded versions is constructed. Step 4: Train the network. First, an initial model is obtained by training only labeled data. Then, semi-supervised training is performed by alternating between labeled and unlabeled data until the total loss on the training set no longer decreases.

[0007] As a preferred technical solution of the present invention, step 1, constructing the training set, includes: Step 1-1: Obtaining high-definition text image materials through mobile phone photography or web crawling. The text image materials include common text carriers such as bank cards, certificates, and billboards. Mobile phone photography uses professional mode, and image clarity is ensured by adjusting parameters such as ISO, shutter speed, and white balance. Web crawling obtains images from public datasets or compliant websites through automated scripts, and performs preprocessing such as noise reduction, sharpening, and format unification on the images; Step 1-2: Generating synthetic data using Blender or Unity renderers, simulating the highlight blur effect of different positions and shapes by adjusting parameters such as light source angle, light intensity, highlight coverage, and blur degree; Step 1-3: Collecting real scene data in indoor and outdoor environments, and dividing the real scene data into labeled data and unlabeled data according to the presence of high-definition ground truth images.

[0008] As a preferred technical solution of the present invention, in steps 1-2, the light source angle is adjustable within the range of 0°-360°, the light intensity is adjustable within the range of 100-1000 lumens, the highlight range occupies 5%-50% of the image area, and the blur degree is adjusted by the Gaussian blur radius of 1-10 pixels, combined with the material reflectivity of 0.1-0.9 and the ambient light occlusion parameter to enhance the realism of the synthesized highlight blur effect; in steps 1-3, in the indoor environment, a tripod is used to fix the SLR camera and shoot with an F8 aperture, and in the outdoor environment, a handheld gimbal stabilizer is used in conjunction with a mobile phone to collect data from multiple angles, with the shooting distance controlled within the range of 0.1-0.5 meters, and at least 1000 sets of samples are collected for natural light, artificial light source and mixed lighting conditions respectively.

[0009] As a preferred technical solution of the present invention, the training set constitutes the first real scene specular-blurred degraded text image dataset. The labeled data consists of degraded-clear paired images that simultaneously have high-definition reference images, and are accompanied by annotation information such as text location, text content, and blur level. The unlabeled data consists of text images that only contain different regions and different intensities of specular highlights and blur degrades.

[0010] As a preferred technical solution of the present invention, the image restoration network in step 2 has the following features: In the encoding stage, the Haar wavelet downsampling module is used to replace max pooling and average pooling to preserve high-frequency detail information, and 1×1, 2×2 and 3×3 convolutional kernels are used in parallel at each scale to extract local point features, small region features and larger neighborhood features, and then multi-scale feature representations are obtained through summation or fusion operations; the multi-scale features are further input into the YZMamba module based on the state space model to model long-range dependencies; in the decoding stage, the spatial resolution is gradually restored by upsampling, and skip connections are established with the features corresponding to each encoding scale, and the image is iteratively restored using a multi-scale output form.

[0011] As a preferred technical solution of the present invention, in each scale of encoding and decoding, the dark channel image corresponding to the input image is encoded as a high-dimensional feature and fused with the backbone feature as additional guiding information to highlight specular highlights and blurred areas; in the twin network training strategy, multiple degraded versions are constructed for the same real text scene, and the multiple degraded versions adopt different highlight intensities, positions and different blur kernel parameters. The corresponding restored images are output by the image inpainting network with weight sharing, and the robustness and generalization ability of the network to different degrees of degradation are improved by output consistency constraints in color space, feature space and dark channel space.

[0012] As a preferred technical solution of the present invention, the supervised loss of the labeled data in step (3) includes: Step 3-1: appearance color consistency loss, used to constrain the pixel-wise norm difference between the repaired image and the high-definition label image in the RGB color space; Step 3-2: adversarial loss, by constructing a generator and a discriminator, adversarial learning is performed on the authenticity between the repaired image and the real label image to improve the subjective visual quality of the repaired image; Step 3-3: high-dimensional space feature consistency loss, using a pre-trained feature extraction network to extract high-dimensional features of the repaired image and the label image, of which some features are used to measure content consistency and others are used to measure style consistency; Step 3-4: dark channel consistency loss, the repaired image and the label image are mapped to the corresponding dark channel images respectively, and the consistency of the two in the dark channel space is constrained by the norm to strengthen the sparsity prior of the highlight area and avoid text blurring caused by color loss.

[0013] As a preferred technical solution of the present invention, adversarial loss is not used for unlabeled data. Instead, a self-supervised loss is constructed between the outputs of different degraded images of the same real subject after network restoration. This includes color consistency loss, high-dimensional space feature consistency loss, and dark channel consistency loss. By constraining the consistency of multiple restoration results in pixel space, feature space, and dark channel space, self-supervised optimization without real and clear labels is achieved.

[0014] As a preferred technical solution of the present invention, the specific process of network training in step (4) is as follows: First, only labeled data is used to supervise the training of the network until the supervised loss on the training set converges to obtain the initial model; on the basis of the initial model, unlabeled data is introduced, and labeled and unlabeled data are used alternately for semi-supervised training, wherein labeled data participates in the optimization of both supervised loss and self-supervised loss, and unlabeled data only participates in the optimization of self-supervised loss; when the comprehensive loss function on the training set tends to be stable and no longer decreases significantly, training is stopped.

[0015] This invention also provides a specular blur text image restoration system based on dark channel prior guidance, comprising: a training data construction module for constructing a training set; an image restoration network module for performing encoder-decoder structure, multi-scale convolution, Haar wavelet downsampling, dark channel guided feature extraction, and YZMamba long-range dependency modeling; a loss function and optimization module for calculating the supervised loss and self-supervised loss, and updating the network parameters accordingly; and a training control module for controlling the training process and supervising the training process.

[0016] Compared with existing technologies, this invention has the following advantages: By constructing a dataset of specular-blurred composite degradation text images in real-world scenes, this invention effectively overcomes the shortcomings of traditional methods, which rely excessively on synthetic data and struggle to cover the complexity of real lighting and material reflections. The established dataset includes various shooting devices, different lighting conditions, text carriers with multiple materials, and multiple levels of degradation, enabling the model to learn realistic specular morphology, reflection patterns, and blur distributions in natural scenes. This achieves comprehensive coverage of multiple light sources, multiple scenes, and multiple degradation conditions, fundamentally improving the realism and representativeness of the training data.

[0017] The image inpainting network structure proposed in this invention combines Haar wavelet downsampling, dark channel guided features, YZMamba long-range dependency modeling, and multi-scale convolutional fusion mechanisms to simultaneously capture sparse structural features in highlight areas and texture details in blurred areas. The introduction of multi-scale output and high-dimensional dark channel features during decoding enhances the network's ability to recognize and reconstruct details in text edges, stroke structures, and reflected areas. Compared to methods relying solely on convolutions or Transformers, the network structure of this invention combines high efficiency with lightweight design, significantly reducing computational costs while improving image quality.

[0018] The semi-supervised Siamese network training mechanism constructed in this invention further breaks through the dependence of traditional fully supervised models on a large number of high-definition paired labels. By applying consistency constraints to different degraded versions of the same text scene and combining the synergistic optimization of supervised and self-supervised losses, this invention can continuously improve model performance even with abundant real unlabeled data. The introduction of dark channel priors enhances the network's physical understanding of specular highlight regions, making the restoration results significantly superior to existing methods in terms of structural consistency, brightness rationality, and the naturalness of highlight removal. Finally, this invention achieves significant improvements in PSNR, SSIM, and OCR recognition rates in real-world scenes, demonstrating good application prospects and engineering value. Attached Figure Description

[0019] Figure 1 is an overall flowchart of the present invention.

[0020] Figure 2 shows the overall training framework of this invention.

[0021] Figure 3 shows the network details of the present invention.

[0022] Figure 4 shows the details of the YZMamba network module proposed in this invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and Figures 1-4. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the present invention is not limited to these embodiments. Equivalent modifications made by those skilled in the art without departing from the principles of the present invention should fall within the protection scope of the present invention.

[0025] Example 1: This invention proposes a semi-supervised inpainting method for specularly blurred text images based on dark channel guidance, as shown in Figure 1. The method includes the following steps: Step 1, constructing a training set, specifically as follows: We constructed the first specular highlight-blur text dataset (SHBT) containing both real and synthetic data. The synthetic data part generates a progressive blur effect by applying adjustable Gaussian noise (σ∈[1.5,3.0]) to a publicly available virtual specular dataset, simulating the common specular-blur composite degradation conditions during mobile shooting.

[0026] Poisson noise is introduced to simulate sensor noise, enhancing the physical realism of the synthesized data. The real dataset contains three classes of rigorously labeled samples: 1) labeled highlight images (Highlight, H), containing 2,150 pairs of data (highlight input / clear ground truth); 2) low-quality images (Blur, B), containing 1,800 motion-blurred samples; 3) low-quality highlight images (Highlight-Blur, HB), containing 1,200 composite degradation samples. Each class of samples is accompanied by a quality assessment label (0-5 levels) according to the ISO / IEC 29112 standard.

[0027] In addition, 3,500 unlabeled real-world images covering these three categories were collected to form the basic data for the semi-supervised learning module. The dataset adopted a multi-device acquisition strategy: it included automatic mode shooting samples from mainstream smartphones such as iPhone 13 / 14 and Huawei Mate 40 series, as well as baseline images acquired by Canon EOS R5 with EF 24-70mm lens under fixed tripod conditions.

[0028] Indoor scene acquisition employs a strictly controlled scheme: using high-definition text images under natural light as the reference ground truth (GT), and illuminating the target from multiple angles (15°-75° incident angle) with Bowens 500W film and television lights to generate highlights of different intensities (10-95% pixel ratio) and spatial distribution as network input.

[0029] For outdoor scenes, a differential acquisition method was used: a highlight-blurred image under direct sunlight (illuminance > 80,000 lux) was used as input, while a clear reference image without highlights was obtained simultaneously using a professional sun umbrella to ensure geometric consistency of the scene. To enrich the highlight characteristics, additional samples were collected during dawn and dusk (illuminance 5,000-30,000 lux), covering the warm-toned highlight phenomenon of golden sunlight.

[0030] Strict quality control was implemented during the data preparation phase: invalid images caused by camera shake or text movement were removed using OpenCV's FAST feature point detection, and bicubic interpolation was used to uniformly crop all images to a resolution of 512x352 (maintaining an aspect ratio of 0.9), achieving a 92% improvement in video memory utilization on the Tesla V100 GPU.

[0031] Data augmentation strategies include: 1) random rotation (±30°) to enhance orientation robustness; 2) horizontal flipping to simulate a mirrored scene; and 3) adaptive histogram equalization (CLAHE) combined with random brightness adjustment (±20%) to enhance illumination robustness. The final dataset contains 8,650 training samples and 1,850 test samples.

[0032] Step 2: Image inpainting network design.

[0033] As shown in Figure 2, a twin network training strategy is constructed by inputting the degraded image corresponding to the unified high-definition image into a neural network with shared weights, thus constraining the consistency of the restored images.

[0034] As shown in Figure 3, we employ an encoder-decoder-based deep learning network architecture to recover text images with specular highlights and blurriness. In the encoding stage, the network replaces traditional max pooling and average pooling methods with a Haar wavelet downsampling (HWD) module, effectively preserving high-frequency detail information. Simultaneously, a dark channel-guided feature extraction mechanism is introduced, using a YZMamba module for multi-scale long-range dependency modeling. In the decoding stage, resolution is gradually restored through upsampling, and features corresponding to the encoder scale are fused using skip connections, ultimately generating multi-scale output.

[0035] We first encode the input image to extract features. By using 1x1, 2x2, and 3x3 convolutional kernels in parallel at three different scales, we extract local point features, small region features, and slightly larger neighborhood features, respectively. These multi-scale features are then fused through a simple summation operation. The 1x1 convolution emphasizes cross-channel information interaction, while the 2x2 convolution (with 1-pixel padding) and 3x3 convolution (with standard 1-pixel padding) preserve the spatial dimensions of the input and output. The final summed output integrates feature representations from different receptive fields while retaining the original resolution, thereby enhancing the model's ability to capture multi-scale visual patterns.

[0036] We propose YZMamba as the core network component, an optimized variant of MobileMamba that achieves superior feature representation by integrating the CBAM attention module (as shown in Figure 4). Mamba is a recently proposed state-space model (SSM) that can effectively model long-range dependencies with linear computational complexity, significantly reducing computational overhead compared to the traditional Transformer architecture while achieving a global receptive field.

[0037] To adapt to visual tasks, MobileMamba innovatively proposes a Multi-Receptive Field Feature Interaction (MRFFI) module: it processes input features through three parallel paths: (1) a long-range wavelet transform enhanced Mamba (WTEMamba) branch, which extracts global features while preserving fine-grained edge details through wavelet transform enhancement; (2) a multi-kernel deep convolution (MK-DeConv) branch, which captures multi-scale local features; and (3) an efficient channel processing branch, which reduces the redundancy of high-dimensional feature space by eliminating redundant identity mappings. The MRFFI module collaboratively integrates these complementary feature representations to achieve simultaneous modeling of global context and multi-scale local patterns, significantly enhancing the model's ability to extract high-frequency edge details while maintaining computational efficiency.

[0038] We introduce parallel CBAM blocks after the branches of MK-DeConv and MRFFI modules to achieve a double improvement: (1) The dual-channel-spatial attention mechanism of CBAM effectively compensates for the inherent limitations of MobileMamba in explicit attention modeling and greatly improves the ability to select discriminative features; (2) CBAM can optimally coordinate the fusion of global representation of WTEMamba and multi-scale local features of MK-DeConv, and while maintaining the complete long-range dependency relationship, it significantly improves the perception sensitivity of key fine-grained patterns (especially edge structures and texture details).

[0039] Step 3: Loss Function Design. The loss function includes supervised testing with labeled data pairs and self-supervised testing with unlabeled data pairs. Supervised objective function. Constraining the consistency between the predicted image and the label data, specifically including color loss. Combating losses Feature space loss Dark channel loss Combining the four loss functions:

[0040] in The parameters represent the weights assigned to each loss function. Each loss function will be described in detail below.

[0041] Color loss measures the difference between multi-scale labeled images and output images:

[0042] Where K=3 represents the scale level. This represents the label image at scale k. This represents the output image at scale k. This represents the number of image elements at the k-th scale.

[0043] The adversarial loss function we employ not only evaluates the absolute discriminant scores between the real and output images, but also considers their relative differences:

[0044] Here, σ represents the Sigmoid activation function, BCE represents the binary cross-entropy, and g / d represents the training objective (g=1, d=0 represents the generator update, g=0, d=1 represents the discriminator update). This relative evaluation mechanism enhances the model's ability to discern subtle quality differences, thereby helping to generate more realistic images.

[0045] Feature space loss represents the difference between the labeled image and the generated image in high-dimensional space:

[0046] Where φ represents the first layer features of the pre-trained VGG19 model, and the corresponding loss represents the content loss; G=φφT, and the corresponding loss represents the style loss.

[0047] Color loss Only pixel-level brightness errors are constrained, while the dark channel loss explicitly enforces the physical sparsity of the highlight region through local minimum filtering (min(R,G,B)), thereby ensuring that the restoration result is more consistent with prior knowledge, that is, the highlight occupies only a very small local area in the natural scene (as described in He's dark channel prior).

[0048] By processing local extrema rather than pixel intensity, dark channel loss avoids the blurring effect caused by color loss on text images:

[0049] in This represents the dark channel image corresponding to the k-th scale image. This represents the dark channel image corresponding to the label image at scale k.

[0050] For self-supervised training of unlabeled data, the loss function includes color consistency loss between the two output images, feature space loss, and dark channel loss:

[0051] in The parameter represents the weight of each loss function.

[0052] Step 4: Model Training. During the network training phase, an initial model is first trained on a labeled dataset containing synthetic data and real-world indoor and outdoor scene images, serving as the base model.

[0053] Subsequently, a semi-supervised learning strategy was employed, using labeled and unlabeled data alternately for joint training. To avoid overfitting, a random sampling strategy was used to dynamically select training images during training. Specifically, training was first conducted on labeled data until convergence, and then unlabeled data was introduced for further training to improve the model's generalization ability. Finally, the training process was terminated when the training loss function stabilized and no longer decreased significantly.

[0054] Example 2: In actual deployment, the system runs on a training server equipped with an NVIDIA RTX A6000 or Tesla V100 GPU, using Ubuntu 20.04 as the operating system and PyTorch and CUDA 11.8 as the deep learning framework. This example describes the overall system workflow.

[0055] The system first prepares the training set using a training data construction module. To cover various degradation scenarios in real-world scenes, such as specular highlights, local reflections, material bright spots, and motion blur, this module constructs two types of samples: synthetic data and real-world shooting data. The synthetic data is generated using Blender or Unity renderers. By adjusting parameters such as light source angle (0°–360°), light intensity (100–1000 lumens), highlight coverage (5%–50% of image area), and Gaussian blur kernel (radius 1–10 pixels), it simulates specular reflections and bright spots of different shapes (circular, striped, irregular) on the text surface. The realism of the synthetic image is further enhanced by combining material reflectivity settings (0.1–0.9) and ambient occlusion (AO) parameters. The real-world data was collected in both indoor and outdoor environments: indoor data was captured using a Canon 5D Mark IV DSLR camera with a tripod and an aperture of F8 to ensure clear and stable text images; outdoor samples were captured using a smartphone with a gimbal from different angles at distances ranging from 0.1 to 0.5 meters, and samples were taken under natural light, artificial light, and mixed light conditions to obtain images with highlights, striped reflections, and motion blur. If the scene allowed, high-resolution reference images without highlights were simultaneously acquired using a light-blocking device and used as labeled data; samples obtained through ordinary shooting but without high-resolution ground truth images were used as unlabeled data. After processing, the training set includes various text carriers such as bank cards, ID cards, and billboards, covering thousands of images, and underwent preprocessing such as cropping, normalization, and random enhancement before being imported into the system.

[0056] The image inpainting network module is built on an encoder-decoder structure. In the encoding stage, the system uses Haar wavelet downsampling (HWD) instead of traditional max pooling, allowing the network to retain high-frequency texture features while compressing resolution. This is crucial for removing highlight edges and restoring text details. The input image is processed at multiple scales using 1×1, 2×2, and 3×3 convolutional kernels to extract cross-channel, local region, and large receptive field features, which are then uniformly fused. The fused features are input into the optimized YZMamba module of this invention. The YZMamba module integrates multi-kernel deep convolution (MK-DeConv) for local detail modeling, a wavelet-enhanced WTEMamba branch for long-range dependency modeling, and introduces a CBAM channel-spatial attention mechanism after the outputs of both branches, thereby simultaneously enhancing feature contrast and structural expressiveness in both highlight and blurred regions. In the decoding stage, spatial resolution is restored through progressive upsampling, detail compensation is achieved by combining skip connections from each scale of the encoder, and corresponding dark channel features are embedded in the output of each scale to guide the network to focus on sparse highlight regions.

[0057] The dark channel guidance mechanism is a key component of the system. During the input phase, the system first calculates the dark channel of the image (i.e., selects the local minimum component in the RGB three channels) to obtain the "inverse saliency map" of the highlight region. The dark channel features are encoded as independent high-dimensional features and input together with the backbone network features into the multi-scale module. This enables the network to automatically distinguish between specular highlights and normal brightness, improving the structure restoration effect of the highlight region and avoiding the blurring problem caused by simple brightness regression.

[0058] The training control module employs a twin network training strategy to enhance the model's adaptive capability. Specifically, for the same clear text image, the system inputs multiple pairs of highlight and blurry images with different degrees of degradation into two network branches that share weights. Since the inputs to the two branches differ only in the degree of degradation while the underlying real text content remains the same, the system uses consistency constraints to make the two output images as close as possible, thus learning the mapping between degradation causes and structural restoration even without a real, clear label. This strategy is particularly effective in real-world unlabeled scenarios.

[0059] The loss function and optimization module support both supervised and self-supervised modes. In supervised mode, the system uses color consistency loss to constrain pixel differences, employs GAN adversarial loss to enhance image realism, and calculates the consistency of content features and style features through a pre-trained VGG19 network, thereby improving the restoration of text contours and textures. Simultaneously, dark channel consistency loss utilizes sparse priors in highlight regions to prevent the network from producing "diffuse highlight residue," ensuring the restoration results conform to the lighting patterns of natural images. In self-supervised mode, since unlabeled data lacks truly high-resolution images, adversarial loss is not used. Instead, the network updates weights based on color, feature, and dark channel consistency constraints between restoration results of the same scene's degraded versions.

[0060] The system training process is divided into two stages: In the first stage, only labeled data is used to train the model, enabling the system to have basic text structure recovery capabilities.

[0061] In the second stage, the training control module initiates a semi-supervised mechanism, adding a large number of unlabeled real images to the training dataset. The supervisory signal alternates with the consistent self-supervised signal corresponding to the same high-resolution image at different degradation levels, enabling the model to have stronger generalization ability when facing complex ambient lighting, motion blur, and material reflections in real-world scenarios. When the overall loss function on the training set stabilizes and no longer decreases, the system automatically terminates training and exports the final model.

[0062] After training according to this embodiment, the system can perform highlight removal and blur restoration on images such as scanned documents, street photography text, ID photos, reflective billboards, labels, and tickets in practical applications, outputting clear, readable text images with significantly improved OCR recognition performance. Because the system relies on real datasets, dark channel priors, and a twin-supervised semi-supervised mechanism, it possesses stronger adaptability and robustness than traditional methods.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for inpainting specularly blurred text images based on dark channel prior guidance, characterized in that, Includes the following steps: Step 1: Construct a training set, which includes synthetic data and real-world shooting data. The synthetic data simulates specular blur effects of different positions, shapes, and intensities using a renderer. The real-world shooting data includes specular, blurred, and specular-blur composite degraded text images captured in indoor and outdoor environments, and is divided into labeled and unlabeled data based on whether a high-resolution reference image is available. Step 2: Design an image inpainting network. The image inpainting network adopts an encoder-decoder structure. In the encoding stage, Haar wavelet downsampling and dark channel-guided feature extraction mechanisms are introduced. In the decoding stage, image reconstruction is performed through multi-scale upsampling and skip connections. The process involves several steps: 1) Constructing a twin network training strategy, where different degraded versions of the same high-resolution text image are input into a weight-sharing neural network to ensure consistent restored images; 2) Designing a loss function, where a supervised loss is constructed for labeled data, including appearance color consistency loss, adversarial loss, high-dimensional space feature consistency loss, and dark channel consistency loss; and for unlabeled data, a consistency loss based on the restoration results of multiple degraded versions is constructed; 3) Training the network, first using only labeled data to obtain an initial model, then alternating between labeled and unlabeled data for semi-supervised training until the total loss on the training set no longer decreases.

2. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 1, characterized in that, Step 1, constructing the training set, includes: Step 1-1: Obtaining high-definition text image materials through mobile phone photography or web scraping. The text image materials include common text carriers such as bank cards, certificates, and billboards. Mobile phone photography uses professional mode, and parameters such as ISO, shutter speed, and white balance are adjusted to ensure image clarity. Web scraping uses automated scripts to obtain images from public datasets or compliant websites, and the images are preprocessed by denoising, sharpening, and format unification. Step 1-2: Generating synthetic data using Blender or Unity renderers. By adjusting parameters such as light source angle, light intensity, highlight coverage, and blur degree, highlight blur effects at different positions and shapes are simulated. Step 1-3: Collecting real-world scene data in indoor and outdoor environments, and dividing the real-world scene data into labeled data and unlabeled data based on the presence of high-definition ground truth images.

3. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 2, characterized in that: In steps 1-2, the light source angle is adjustable within the range of 0°-360°, the light intensity is adjustable within the range of 100-1000 lumens, the highlight area occupies 5%-50% of the image area, and the blur level is adjusted by the Gaussian blur radius of 1-10 pixels, combined with the material reflectivity of 0.1-0.9 and the ambient light occlusion parameter to enhance the realism of the synthesized highlight blur effect; In steps 1-3, in the indoor environment, a tripod is used to fix the SLR camera and shoot with an F8 aperture, and in the outdoor environment, a handheld gimbal stabilizer is used in conjunction with a mobile phone to collect data from multiple angles, with the shooting distance controlled within the range of 0.1-0.5 meters, and at least 1000 sets of samples are collected for natural light, artificial light source and mixed lighting conditions respectively.

4. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 1, characterized in that, The training set constitutes the first real-world specular-blurred degraded text image dataset. The labeled data consists of degraded-sharp paired images that also have a high-resolution reference image, along with annotation information such as text location, text content, and blur level. The unlabeled data consists of text images containing only different regions and intensities of specular highlights and blurred degrades.

5. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 1, characterized in that, The image inpainting network described in step 2 has the following characteristics: In the encoding stage, Haar wavelet downsampling is used to replace max pooling and average pooling to preserve high-frequency detail information. At each scale, 1×1, 2×2, and 3×3 convolutional kernels are used in parallel to extract local point features, small region features, and larger neighborhood features. Then, multi-scale feature representations are obtained through summation or fusion operations. The multi-scale features are further input into the YZMamba module based on the state-space model to model long-range dependencies. During the decoding stage, spatial resolution is gradually restored through upsampling, and skip connections are established with features corresponding to each coding scale. The image is then iteratively repaired using a multi-scale output format.

6. The method for inpainting specularly blurred text images based on dark channel prior guidance according to claim 5, characterized in that: In each of the encoding and decoding scales, the dark channel image corresponding to the input image is encoded as a high-dimensional feature and fused with the backbone feature as additional guiding information to highlight specular highlights and blurred areas. In the twin network training strategy, multiple degraded versions of the same real text scene are constructed. These multiple degraded versions use different highlight intensities, positions, and different blur kernel parameters. The corresponding restored images are output by the image inpainting network with shared weights. By constraining the output consistency in color space, feature space, and dark channel space, the robustness and generalization ability of the network to different degrees of degradation are improved.

7. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 1, characterized in that, The supervised loss for the labeled data in step (3) includes: Step 3-1: Appearance color consistency loss, used to constrain the pixel-wise norm difference between the repaired image and the high-resolution label image in the RGB color space; Step 3-2: Adversarial loss, by constructing a generator and a discriminator, adversarial learning is performed on the authenticity between the repaired image and the real label image to improve the subjective visual quality of the repaired image; Step 3-3: High-dimensional space feature consistency loss, using a pre-trained feature extraction network to extract high-dimensional features of the repaired image and the label image, of which some features are used to measure content consistency and others are used to measure style consistency; Step 3-4: Dark channel consistency loss, the repaired image and the label image are mapped to the corresponding dark channel images respectively, and the norm constrains the consistency of the two in the dark channel space to strengthen the sparsity prior of the highlight area and avoid text blurring caused by color loss.

8. The method for inpainting specularly blurred text images based on dark channel prior guidance according to claim 7, characterized in that: Instead of using adversarial loss for unlabeled data, a self-supervised loss is constructed between the outputs of different degraded images of the same real subject after network restoration. This includes color consistency loss, high-dimensional space feature consistency loss, and dark channel consistency loss. By constraining the consistency of multiple restoration results in pixel space, feature space, and dark channel space, self-supervised optimization without real, clear labels is achieved.

9. The method for restoring specularly blurred text images based on dark channel prior guidance according to claim 1, characterized in that, The specific process of network training in step (4) is as follows: First, supervised training of the network is performed using only labeled data until the supervised loss on the training set converges, and an initial model is obtained; based on the initial model, unlabeled data is introduced, and semi-supervised training is performed by alternating between labeled and unlabeled data. Labeled data participates in the optimization of both supervised and self-supervised loss, while unlabeled data only participates in the optimization of self-supervised loss; training is stopped when the comprehensive loss function on the training set tends to stabilize and no longer decreases significantly.

10. A system for restoring specularly blurred text images based on dark channel prior guidance, the system being configured to implement the method of any one of claims 1-9, characterized in that, include: The training data construction module is used to perform training set construction; The image inpainting network module is used to perform encoder-decoder structure, multi-scale convolution, Haar wavelet downsampling, dark channel guided feature extraction, and YZMamba long-range dependency modeling; the loss function and optimization module is used to calculate the supervised loss and self-supervised loss, and update the network parameters accordingly. The training control module is used to control the training process and supervise the training process.

Images