Degenerated image recovery processing method applied to liquid crystal display

Abstract

According to the degraded image restoration processing method applied to liquid crystal display, on one hand, an integrated image restoration model, namely a Smart-IR model, is provided, and compared with an existing method, the calculation efficiency is remarkably improved on the aspect of achieving multiple tasks of integrated image restoration; and on the other hand, by designing a novel hybrid expert layer, expert models with different complexities can be dynamically activated to process the image through a complexity sensing router according to the requirement of the input content, and fusion of extracting global features and local features at the same time is realized in a unified architecture for the first time.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, specifically applicable to image processing of LCD TVs or intelligent driving systems, and particularly relates to a method for restoring degraded images applied to LCD displays. Background Technology

[0002] Current popular image restoration methods often only achieve single functions such as deraining, dehazing, and denoising. Therefore, to achieve restoration of multiple degraded images, multiple models need to be integrated, which not only increases the workload but also significantly increases the computational burden on edge devices. While some single models have recently achieved simultaneous deraining, dehazing, and denoising, their algorithms are often very complex, making it difficult to integrate and run them on edge devices. Summary of the Invention

[0003] Technical Solution: To address the shortcomings of existing detection methods, this invention provides a method for restoring degraded images applied to liquid crystal displays. The specific steps include: first, inputting the degraded image and extracting shallow feature information through convolution operations; then, encoding and decoding the feature signals multiple times; the encoding and decoding modules mainly use a complexity-aware router to perform expert modeling, and the selected experts perform deep feature processing on the image; finally, after global residual connections, the image is reconstructed and a high-quality image is output. The encoding skips connections to the decoding, and both encoding and decoding use transformer modules with doubled embedding dimensions.

[0004] As an improvement, both encoding and decoding employ dynamic gradient adjustment processing technology. The structure of this technology includes a complexity-aware router, which dynamically activates expert models of different degrees based on the requirements of the input content. The specific algorithm is... (1) In the formula: The length of the routing gate vector is n , is a one-dimensional vector that ultimately determines which expert model to choose for image processing; This is a linear transformation operation used to project the input into a new vector space. It is a weight matrix. It is the input feature vector; This is a bias term used to adjust the output; To transform the input vector into a probability distribution function, where each element has a value between 0 and 1, and the sum of the values ​​is 1, To select the one with the highest probability k An expert model.

[0005] As an improvement, the specific steps of the dynamic gradient adjustment processing technique for image processing include: first, the input image is regularized and divided into two paths, one of which is then processed by... QKV The image is constructed using two methods: projection and attention extraction; another method fuses the encoder output sequence, passes through a router, selects the corresponding expert model for processing, and then adds the processed data pixel by pixel; the two methods are then fused through spatial and frequency domains, regularized again, and finally passed through a multilayer perceptron to recover a clear image.

[0006] As an improvement, the attention mechanism employs the following algorithm: ; in the formula : The output features calculated through the attention mechanism; Q The query matrix is ​​obtained by performing a linear transformation on the input. K : This is the key matrix, obtained by performing a linear transformation on the input; d : is the dimension of the key vector; , No. i Each position corresponds to value The first vector j Each component.

[0007] As an improvement, L1 regularization is adopted, and the specific calculation process of the spatial-frequency domain fusion mechanism is as follows: (2) (3) (4) (5) Constraints: ≥0 (6) In the above formula: For the input image, H is the image height. w C is the image width, and C is the number of image channels; It is the Fourier transform function. It is the inverse Fourier transform function; For space and expert networks; For frequency domain expert networks; This is a global average pooling process; These are the parameters for linear transformation; This is the selection function for the first K terms.

[0008] Beneficial Effects: This invention proposes a method for restoring degraded images applied to liquid crystal displays. On one hand, by proposing an integrated image restoration model, the Smart-IR model, it significantly improves computational efficiency compared to existing methods in achieving multiple tasks of integrated image restoration. On the other hand, by designing a novel hybrid expert layer, it can dynamically activate expert models of different complexities through a complex router to process images based on the input content, achieving for the first time the simultaneous extraction of global and local features in a unified architecture. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the overall framework of the Smart-IR model of this invention.

[0010] Figure 2 This is a schematic diagram of the DGM technology of the present invention.

[0011] Figure 3 This is a schematic diagram showing the effect before and after defogging in Embodiment 1 of the present invention.

[0012] Figure 4 This is a schematic diagram showing the effects of rain removal before and after rain removal treatment in Embodiment 1 of the present invention.

[0013] Figure 5 This is a schematic diagram showing the effect before and after noise reduction in Embodiment 1 of the present invention. Detailed Implementation

[0014] The technical solutions in the embodiments of the present invention will be clearly and completely described below, so that those skilled in the art can better understand the advantages and features of the present invention, thereby making a clearer definition of the scope of protection of the present invention. The embodiments described in this invention are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0015] See Figure 1 As shown, this is the single-model Smart-IR structure used in this invention. The SmartIR uses a U-shaped architecture. The input degraded image undergoes shallow feature extraction, followed by four layers of DGM encoding and then four layers of DGM decoding. One of the DGM layers performs image reconstruction, and convolution and residual connections are used for data stabilization. Finally, a convolution operation outputs the restored high-quality image.

[0016] The Smart-IR model of this invention is an efficient, integrated image restoration model that utilizes complexity experts—specialized units with varying computational requirements and receptive fields. By matching the degradation task with appropriate resources, Smart-IR overcomes the limitations of previous methods. Its built-in preference for simple paths enhances task-discriminative learning, enabling the model to bypass irrelevant experts and achieve rapid inference while maintaining restoration quality.

[0017] Furthermore, the present invention has been extensively tested under various degradation settings, including more challenging multi-complex degradation scenarios, demonstrating the computational efficiency of the model by selectively utilizing experts, and outperforming state-of-the-art methods in all experiments.

[0018] See Figure 2 As shown, in a specific embodiment of the present invention, the specific implementation steps of the DGM technology are as follows: The input image, after regularization, is divided into two paths. One path is processed through QKV projection to extract attention mechanism, and the other path is fused with the encoder output sequence. After passing through a router, the corresponding expert model is selected for processing, and the data processed by the relevant experts are added pixel by pixel. Then, the two paths are processed through spatial and frequency domain fusion mechanisms and regularized again. Finally, after passing through a multilayer perceptron, a clear image can be recovered.

[0019] Simultaneously, this invention proposes a Dynamic Gradient Adjustment (DGM) technique, which includes a Complexity Aware Router and a Hybrid Expert Layer. The Complexity Aware Router, based on an innovative dynamic computation allocation technique within the Smart-IR model, intelligently allocates computational resources according to the processing difficulty of the input data and the computational cost of the expert network. This allows for the dynamic activation of expert models with varying degrees of complexity based on the needs of the input content. Unlike traditional hybrid expert models that only consider "content matching," the MOCE (Multi-Level Computational) hybrid expert model simultaneously considers "complexity matching," achieving an optimal balance between computational efficiency and model performance. Dynamically selecting the appropriate expert model for the corresponding image degradation processing achieves an intelligent balance between restoration quality and computational efficiency.

[0020] As one specific embodiment of the present invention, the attention mechanism employs the following algorithm: ; in the formula : The output features calculated through the attention mechanism; Q The query matrix is ​​obtained by performing a linear transformation on the input. K : This is the key matrix, obtained by performing a linear transformation on the input; d : is the dimension of the key vector; , No. i The value vector corresponding to the i-th position is the first...j Each component.

[0021] In this embodiment of the invention, the attention mechanism is standard. transform r mechanism, Transformer Image restoration involves enabling each "part" in an image to directly communicate with all other "parts" and collaboratively determine the best solution for repairing each local area. This mechanism excels at tasks that require global information to guide local reconstruction, such as restoring large areas of damage, reconstructing coherent structural textures, and removing large areas of blur.

[0022] As a specific embodiment of the present invention, the DGM technology includes a complexity-aware router. This complexity-aware router dynamically activates expert models of different degrees based on the requirements of the input content. In other words, if there are N experts, after passing through the complexity-aware router of the present invention, the top expert can be selected. k The expert model processes the image. k When the value is less than N, the specific algorithm for the structure to operate in this invention is as follows: (1) In the formula: The length of the routing gate vector is n , is a one-dimensional vector that ultimately determines which expert model to choose for image processing; This is a linear transformation operation used to project the input into a new vector space. It is a weight matrix. It is the input feature vector; This is a bias term used to adjust the output; To convert the input vector into a probability distribution function, each element has a value between 0 and 1, and the sum is 1. Further, in the Smart-IR of this invention, this typically represents the probability that each "expert" model is selected. To select the one with the highest probability k In a hybrid expert model (MOCE), for the sake of efficiency and diversity, not all experts are typically selected; instead, the most suitable expert is chosen for the output.

[0023] In another specific embodiment of the present invention, L1 regularization is adopted, and the specific calculation process of the spatial-frequency domain fusion mechanism is as follows: (2) (3) (4) (5) Constraints: ≥0 (6) In the above formula: For the input image, H is the image height. w C is the image width, and C is the number of image channels; It is the Fourier transform function. It is the inverse Fourier transform function; For space and expert networks; For frequency domain expert networks; This is a global average pooling process; For linear transformation parameters, ; For the front K Item selection function.

[0024] Furthermore, formulas 2-6 above enable the fusion of the frequency and spatial domains. Simply put, spatial domain processing directly manipulates pixels and excels at handling local and structural information; while frequency domain processing manipulates the transformed coefficients and excels at capturing global, periodic patterns, and textures. This fusion mechanism allows the system to utilize both perspectives simultaneously. Example

[0025] This embodiment applies the technical solution of the present invention, specifically using a single, efficient model to simultaneously restore multiple degraded images such as rain removal, fog removal, and noise reduction. The method includes the following steps: S1, prepare some photos with rain, fog, and noise; S2, input these photos into the model; S3, the photos processed by the model will automatically complete the restoration, removing the rain, fog, and noise from the photos; S4, the effect after model processing is as follows.

[0026] See Figure 3 As shown, this is the effect of dehazing. The left side is an image with fog in front of the model, and the right side is a restored photo after the fog has been removed by model processing. The effect is very good.

[0027] See Figure 4 As shown, this is the effect of removing rain. The left side is an image with rain in front of the model, and the right side is a restored photo after the rain has been removed through model processing. The effect is very good.

[0028] See Figure 5 As shown, this is the effect of noise reduction. The left side is the image with noise in front of the model, and the right side is the restored photo after the noise has been removed by the model. The effect is very good.

[0029] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for restoring degraded images applied to liquid crystal displays, characterized in that, The specific steps include: first, the input degraded image is processed through convolution to extract shallow feature information; then, the feature signals are encoded and decoded multiple times; the encoding and decoding modules mainly use a complexity-aware router to select the appropriate expert model, which performs deep feature processing on the image; finally, after global residual connections, the image is reconstructed and a high-quality image is output. The encoding skips connections to the decoding, and both encoding and decoding employ... transformer The module is embedded, and the dimension is doubled.

2. The method for restoring degraded images applied to liquid crystal displays according to claim 1, characterized in that, Both encoding and decoding employ dynamic gradient adjustment technology. The structure of this technology includes a complexity-aware router, which dynamically activates expert models of varying degrees based on the requirements of the input content. The specific algorithm is... （1） In the formula: The length of the routing gate vector is n , is a one-dimensional vector that ultimately determines which expert model to choose for image processing; For linear transformation operations, It is a weight matrix. It is the input feature vector. For bias terms, To transform the input vector into a probability distribution function, where each element has a value between 0 and 1, and the sum of the values ​​is 1, To select the one with the highest probability k An expert model.

3. The method for restoring degraded images applied to liquid crystal displays according to claim 1, characterized in that, The dynamic gradient adjustment processing technique for image processing includes the following steps: First, the input image is regularized and split into two paths; one path is then processed by... QKV The image is constructed using two methods: projection and attention extraction; another method fuses the encoder output sequence, passes through a router, selects the corresponding expert model for processing, and then adds the processed data pixel by pixel; the two methods are then fused through spatial and frequency domains, regularized again, and finally passed through a multilayer perceptron to recover a clear image.

4. The method for restoring degraded images applied to liquid crystal displays according to claim 1, characterized in that, The attention mechanism uses the following algorithm: ; in the formula : The output features calculated through the attention mechanism; Q The query matrix is ​​obtained by performing a linear transformation on the input. K : This is the key matrix, obtained by performing a linear transformation on the input; d : is the dimension of the key vector; , No. i Each position corresponds to value The first vector j Each component.

5. The method for restoring degraded images applied to liquid crystal displays according to claim 1, characterized in that, The regularization uses L1 regularization, and the specific calculation process of the spatial-frequency domain fusion mechanism is as follows: （2） （3） （4） （5） Constraints: ≥0 (6) In the above formula: For the input image, H is the image height. w C is the image width, and C is the number of image channels; It is the Fourier transform function. It is the inverse Fourier transform function; For space and expert networks; For frequency domain expert networks; This is a global average pooling process; For linear transformation parameters, ; For the front K Item selection function.

Images