Deep learning enhancement model updating method and system based on image quality assessment

Abstract

The application provides a deep learning enhancement model updating method and system based on image quality evaluation, which comprises the following steps: integrating a ControlNet architecture into an image enhancement model, improving the image enhancement model by adding an auxiliary module and a conditional perception cross-entropy loss function, using the auxiliary module to introduce an additional condition to generate a control signal to dynamically adjust the generation strategy, and using the improved image enhancement model to generate an enhanced image according to the input additional condition and input image; inputting the enhanced image into an image quality evaluation model to output a quality score and a text description; if the quality score is lower than a preset threshold, designing a multi-modal loss function according to the quality score and the description text output by the image quality evaluation model, and adjusting the weight parameters of the improved image enhancement model in real time through back propagation. According to the evaluation result, the weight information of the image enhancement model is adjusted in real time, so that the generalization ability of the model and the image enhancement effect are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image quality evaluation, and in particular to a deep learning enhancement model updating method and system based on image quality evaluation. BACKGROUND

[0002] The power system is an important part of national infrastructure, and its safe and stable operation depends on efficient equipment inspection technology. In recent years, with the wide application of unmanned aerial vehicles, robots and other intelligent inspection equipment, power inspection has gradually shifted from manual mode to automated and intelligent mode. However, the inspection images of power equipment often appear blurred, low contrast or noise interference due to complex environment, which directly affects the accuracy of subsequent fault detection. At the same time, with the wide application of deep learning technology in the field of image enhancement, there is often a significant difference between the existing deep learning model in training and actual use scene, because the image distribution of the data set used in training is inconsistent with that in the actual application scene, which leads to insufficient generalization performance of the model.

[0003] In order to solve this problem, the research in recent years mainly focuses on static image enhancement algorithm and fixed parameter deep learning model. For example, the enhancement model based on convolutional neural network (CNN) improves the image quality through end-to-end training, but its weight is fixed after deployment and cannot be dynamically adjusted according to the actual scene; while the adaptive method introduced by reinforcement learning can partially solve the generalization problem, but it relies on the manually designed reward function and a large number of trial and error costs, which is low in efficiency. In addition, the existing technology relies on manually labeled "true value" as the optimization target, but this method not only consumes time and effort, but also makes it difficult to achieve real-time updating and acquisition of high-quality reference images, thus leading to lack of effective basis for model updating. These defects make it difficult for the existing technology to balance real-time, robustness and automation requirements in complex power scenarios, limiting the further application of intelligent inspection technology. SUMMARY

[0004] The purpose of the present application is to solve the existing problems, and to provide a real-time updating method and system of deep learning enhancement model based on image quality evaluation, which aims to automatically evaluate the quality of enhanced images by introducing an image quality evaluation model, and to adjust the weight information of the image enhancement model in real time according to the evaluation results, so as to improve the generalization ability and image enhancement effect of the model.

[0005] A deep learning enhancement model updating method based on image quality evaluation, comprising the following steps:

[0006] The ControlNet architecture is integrated into the image enhancement model, the ControlNet architecture improves the image enhancement model by adding an auxiliary module and a conditional perception cross-entropy loss function, the auxiliary module is used to introduce an additional condition to generate a control signal to dynamically adjust the generation strategy, and the improved image enhancement model generates an enhanced image according to the input additional condition and the input image;

[0007] The enhanced image is input into an image quality evaluation model to output a quality score and a text description;

[0008] If the quality score is lower than a preset threshold, a multi-modal loss function is designed according to the quality score and the description text output by the image quality evaluation model, and the weight parameters of the improved image enhancement model are adjusted in real time through back propagation.

[0009] Further, the auxiliary module includes a locked copy and a trainable copy, the locked copy is used to copy the structure and weight of the image enhancement model, and the parameters are fixed during the training process, and the generation ability of the image enhancement model is retained; the trainable copy has the same structure as the image enhancement model, and the parameters can be fine-tuned, and the generation strategy is dynamically adjusted through the additional condition to generate a control signal.

[0010] Further, the trainable copy generates a control signal through an additional condition, specifically including:

[0011]

[0012] wherein, is a generated control signal tensor, are the height, width and channel number of the feature map, respectively, is a fine-tuned model, is an input image;

[0013] control signal intermediate feature map of the main model interact through adaptive feature fusion, so that the model can dynamically adjust the generation process according to the additional condition when generating the image:

[0014]

[0015] wherein, is a learnable scaling coefficient used to dynamically adjust the strength of the control signal.

[0016] Further, the loss function of the image enhancement model is a pixel-level loss or a perception loss, a conditional perception cross-entropy loss function forms a hybrid loss function:

[0017]

[0018] wherein, is a dynamically adjusted weight coefficient, which is larger in the initial stage and gradually decreases later;

[0019] The conditional perception cross-entropy loss function is designed as follows: assuming that the control signal output by the ControlNet is The target distribution is defined as The cross-entropy loss between the model prediction distribution and the target distribution is:

[0020]

[0021] wherein, The probability map output by the ControlNet is normalized by Softmax to obtain which is dynamically generated according to the quality evaluation result;

[0022] In the pre-training of the main model, only is used to train the original image enhancement model to ensure the basic generation ability; secondly, the main model is frozen again, and the main model weight is fixed when training the ControlNet model, and the trainable copy of the ControlNet is optimized through to learn to generate control signals that adapt to additional conditions; finally, in the joint fine-tuning stage, the main model is unfrozen, and the hybrid loss is used to jointly optimize the main model and the ControlNet parameters, thereby improving the overall performance of the image enhancement model.

[0023] Further, the enhanced image is input into an image quality evaluation model to output a quality score and a text description, which includes:

[0024] The text guidance mechanism is introduced, so that the image quality evaluation model can output the quality score and detailed text description of the image at the same time, helping to understand the problems existing in the image;

[0025] The texture perception module based on the Transformer structure analyzes the weight of each Token in the image to judge the contribution weight of each Token to the image quality evaluation to evaluate the local features of the image;

[0026] The global quality evaluation module fuses all Tokens to generate a global weight to evaluate the local features of the image;

[0027] The local and global weights are combined to generate the quality score and the text description of the image.

[0028] Further, the additional condition is a multi-dimensional external information for dynamically controlling the image generation process, for guiding the model to generate an enhanced image more in line with the target according to the specific scene requirements.

[0029] Further, the quality score and the description text output by the image quality evaluation model are used to design a multi-modal loss function, specifically including:

[0030] The multi-modal loss function is designed in combination with the quality score loss and the text generation loss to guide the model update:

[0031]

[0032] wherein, the quality score loss, the text generation loss, and the dynamic weight coefficient for balancing the optimization direction of the quality score and the text description;

[0033]

[0034] wherein, the model prediction score, the true quality label, and are their mean values, respectively;

[0035] The text description loss encodes the text description into a feature vector and aligns it with the image features :

[0036]

[0037] The improved image enhancement model weight is updated according to the multi-modal loss function, and the gradient of the parameter of the improved image enhancement model is calculated by the chain rule, and the model weight is updated using the optimizer:

[0038] .

[0039] A real-time updating system of a deep learning enhancement model based on image quality evaluation, comprising:

[0040] An image enhancement module for integrating a ControlNet architecture into an original image enhancement model, the ControlNet architecture improves the original image enhancement model by adding an auxiliary module and a cross-entropy-based loss function, the auxiliary module is used to introduce an additional condition to generate a control signal to dynamically adjust the generation strategy, and the improved image enhancement model generates an enhanced image according to the input additional condition and the input image; ​

[0041] a quality evaluation module, configured to input the enhanced image into an image quality evaluation model, and output a quality score and a text description;

[0042] a dynamic updating module, configured to, if the quality score is lower than a preset threshold, design a multi-modal loss function according to the quality score and the description text output by the image quality evaluation model, and adjust the weight parameters of the improved image enhancement model in real time through back propagation.

[0043] Further, the auxiliary module includes a locked copy and a trainable copy. The locked copy is used to copy the structure and weights of the image enhancement model, and the parameters are fixed and unchanged during training, so as to retain the generation ability of the image enhancement model. The trainable copy has the same structure as the image enhancement model, and the parameters can be fine-tuned. An extra condition generation control signal is used to dynamically adjust the generation strategy.

[0044] The trainable copy generates the control signal through an extra condition, and the extra condition includes:

[0045]

[0046] wherein, is the generated control signal tensor, are the height, width and channel number of the feature map respectively, is the fine-tuned model, is the input image;

[0047] control signal intermediate feature map of the main model The model can dynamically adjust the generation process according to the extra condition when generating an image through adaptive feature fusion interaction:

[0048]

[0049] wherein, is a learnable scaling coefficient used to dynamically adjust the strength of the control signal.

[0050] Further, the loss function of the image enhancement model is a pixel-level loss or a perception loss, and a conditional perception cross-entropy loss function forms a hybrid loss function:

[0051]

[0052] wherein, is a dynamically adjusted weight coefficient, which is larger in the initial stage and gradually decreases in the later stage;

[0053] The conditional perception cross-entropy loss function is designed as follows: assuming that the control signal output by ControlNet is The guided model generates images that meet quality conditions, defining a target distribution The cross-entropy loss between the model prediction distribution and the target distribution is defined as:

[0054]

[0055] where, The probability map output by the ControlNet is normalized by Softmax , and is dynamically generated according to the quality evaluation results.

[0056] In the pre-training of the main model phase, only The original image enhancement model is trained to ensure basic generation capabilities. Secondly, the main model is frozen, and the main model weight is fixed when training the ControlNet model. The trainable copy of the ControlNet is optimized through to learn to generate control signals that adapt to additional conditions. Finally, in the joint fine-tuning phase, the main model is unfrozen, and the hybrid loss is used to jointly optimize the main model and ControlNet parameters, thereby improving the overall performance of the image enhancement model.

[0057] Compared with the prior art, the present application improves the effect and generalization ability of image enhancement by optimizing the weight of the enhancement model in real time in actual use scenarios, and has the following beneficial effects:

[0058] 1. The ControlNet architecture is introduced into the existing image enhancement model, and the weight of the copy is locked and does not participate in updating at all times, ensuring that the core capabilities of the original model are not damaged and improving the stability of the model. At the same time, the improved hybrid loss function further improves the performance of the model. This scheme can dynamically adjust the model weight in actual application scenarios, improve the image enhancement effect, reduce the dependence on manually labeled data, and significantly improve the automation level and adaptability of the model.

[0059] 2. The image quality evaluation model outputs the quality score and text description of the image by evaluating the enhanced image in real time, and dynamically adjusts the model weight according to the evaluation results, realizing the continuous optimization of the model in actual use scenarios and improving the generalization ability of the model and the enhancement effect of the image.

[0060] 3. The image enhancement model can be continuously optimized in actual scenarios, gradually improving the adaptability to complex environments (such as low light and high noise), and more carefully evaluating the local and global quality of the image, ensuring the adaptability of the model in different scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0061] Fig. 1 ​A structural schematic diagram of an image quality evaluation model in an embodiment of the present application is shown in FIG. 1.

[0062] Fig. 2 A structural schematic diagram of a global quality evaluation module in an embodiment of the present application is shown in FIG. 2.

[0063] Fig. 3 A flowchart of a deep learning enhanced model updating method based on image quality evaluation in an embodiment of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in detail with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0065] Referring to Figs. 1-3 The embodiment of the present application provides a deep learning enhanced model updating method based on image quality evaluation, which comprises the following steps:

[0066] Step 1: integrate the ControlNet architecture into the image enhancement model, the ControlNet architecture improves the image enhancement model by adding an auxiliary module and a conditional perception cross-entropy loss function, the auxiliary module is used to introduce an additional condition to generate a control signal to dynamically adjust the generation strategy, and the improved image enhancement model generates an enhanced image according to the input additional condition and input image. The specific implementation process is as follows:

[0067] ControlNet is an auxiliary neural network structure, which introduces an additional condition to control the generation process of the image by adding an auxiliary module to the model, and mainly consists of two parts: the first part is a locked copy, which copies the structure and weights of the original image enhancement model, but the parameters are fixed during the training process, and the generation ability of the original model is retained; the second part is a trainable copy, which has the same structure as the original model, but the parameters can be fine-tuned, and a control signal is generated through an additional condition (such as a quality score or a text description).

[0068] After the input image and the additional condition are input, the process of outputting the control signal generated by the trainable copy is as follows:

[0069]

[0070] wherein, is the generated control signal tensor, are the height, width, and channel number of the feature map, respectively, is a fine-tuned model, for the input image;

[0071] control signal intermediate feature map of the main model Through adaptive feature fusion interaction, the model can dynamically adjust the generation process according to additional conditions (such as image quality feedback) when generating images:

[0072]

[0073] wherein, is a learnable scaling coefficient for dynamically adjusting the strength of the control signal. The main model refers to the original, unimproved image enhancement model based on the ControlNet architecture; the improved image enhancement model is the version of the main model combined with the ControlNet architecture.

[0074] In order to further improve the performance of the model, a cross-entropy-based loss function is used for improvement. The loss function of the original model is usually a pixel-level loss (such as loss) or perceptual loss, and the conditional perceptual cross-entropy loss is introduced in the present application to form a hybrid loss function:

[0075]

[0076] wherein, is a dynamically adjusted weight coefficient, which is larger in the initial stage and gradually decreases later.

[0077] The design of the conditional perceptual cross-entropy loss is as follows: assuming that the control signal output by the ControlNet needs to guide the model to generate images that meet the quality conditions, define the cross-entropy loss between the target distribution (such as the distribution of high-quality images) and the model prediction distribution :

[0078]

[0079] wherein, is the probability map output by the ControlNet, which is normalized by Softmax to obtain , which is dynamically generated according to the quality evaluation results (such as the probability of high-score areas is 1 and the probability of low-score areas is 0).

[0080] In the pre-training main model stage, only is used to train the original image enhancement model to ensure the basic generation ability; secondly, when the main model is frozen and the ControlNet model is trained, the main model weight is fixed, and Optimize the trainable copy of ControlNet to learn to generate control signals that adapt to additional conditions; finally, in the joint fine-tuning stage, unfreeze the main model and use a hybrid loss Jointly optimize the main model and ControlNet parameters to further improve the overall performance of the image enhancement model.

[0081] Step 2: input the enhanced image into the image quality assessment model, output the quality score and text description, and provide more comprehensive quality feedback. The specific implementation process of step 2 is as follows:

[0082] Step 2.1: Introduce a text-guided mechanism so that the image quality assessment model can not only output a quality score but also generate natural language text describing the image's problems, achieving dual output of image quality score and text description:

[0083] Introduce a text generation branch in the image quality assessment model, and the model body consists of two branches: the scoring branch is based on the CLS Token in the ViT model, which outputs the quality score through a fully connected layer S ∈ [ 0 , 1 ] The text generation branch is based on the Transformer decoder, taking the Token sequence of ViT as input to generate natural language descriptions .

[0084] Align image features and text features using a pre-trained visual-linguistic model CLIP.

[0085] Define the contrastive loss function:

[0086]

[0087] where, is the image feature, is the text feature, is the temperature coefficient.

[0088] Use the autoregressive decoder to generate the text description, and define the text-guided loss function as the cross-entropy:

[0089]

[0090] where, is the th word generated, is the Token sequence feature output by ViT.

[0091] Step 2.2: Based on the texture perception module of the Transformer structure, analyze the weight of each Token in the image to determine the local contribution weight of each Token in the image quality assessment :

[0092] Token sequence of input ViT , the weight of each Token is calculated by self-attention mechanism :

[0093]

[0094] wherein, and are the query vector and key vector of the i-th Token, is the feature dimension; The multi-scale features extracted by the pre-trained model are compressed in channel number by the convolution layer to obtain local texture feature information

[0095] , and are weighted and fused with the ViT Token:

[0096]

[0097] wherein, the MLP is used to project the local features to the dimension of the ViT.

[0098] Step 2.3: Introduce a global quality evaluation module to fuse all Token information (as shown in Fig. 2 ), generate a global weight to evaluate the overall quality of the image;

[0099] The Token sequence of the ViT is averaged and pooled to obtain a global feature vector :

[0100]

[0101] The global weight is generated by a fully connected layer and a Sigmoid activation function:

[0102]

[0103] wherein, , are learnable parameters, is the Sigmoid function.

[0104] Step 2.4: Combine the local weight and the global weight for analysis, output the final image quality score and text description:

[0105] Adjust the score branch using the global weight , output the final quality score:

[0106]

[0107] ​wherein, is the historical average score for stabilization training;

[0108] The update formula of is:

[0109]

[0110] wherein, is the number of all .

[0111] The question description text is generated by the Transformer decoder with the fused Token sequence as input:

[0112]

[0113] Step 3: According to the quality score output by the image quality evaluation model and the text description, dynamically adjust the weight parameters of the image enhancement model.

[0114] According to the application scenario (such as power inspection, safety supervision), set the quality score threshold S th ∈ [ 0 , 1 ] , judge whether the quality of the enhanced image reaches the preset threshold, if the final quality score , then keep the current enhancement model weight; if , design a multi-modal loss function according to the score output by the image quality evaluation model and the text description, and update the weight of the image enhancement model through back propagation.

[0115] Combine the quality score loss and the text generation loss to design a multi-modal loss function to guide the model update:

[0116]

[0117] wherein, is the quality evaluation loss, is the text generation loss, , is a dynamic weight coefficient to balance the optimization direction of quality score and text description. In the initial stage , it is higher to strengthen the text guidance, and gradually decays in the later stage.

[0118] For example: When a keyword triggers a certain defect label, the corresponding sub-loss weight will increase, so that the enhancement model will be punished more severely in this regard, thereby improving these deficiencies.

[0119] The quality score loss PLCC (Pearson Linear Correlation Coefficient) loss measures the consistency between the predicted score and the manual annotation, defined as:

[0120]

[0121] wherein, is the model predicted score, is the true quality label, and are their mean values respectively.

[0122] The text description loss encodes the text description into a feature vector and aligns it with the image feature :

[0123]

[0124] The image enhancement model weights are updated according to the loss function, the gradient of the image enhancement model parameters is calculated by the chain rule, and the model weights are updated using an optimizer:

[0125]

[0126] The "locked copy" weights of ControlNet are fixed, only the "trainable copy" and the backbone parameters of the image enhancement model are updated, ensuring that the basic generation capability is not damaged.

[0127] The embodiment of the application also provides a real-time updating system of a deep learning enhancement model based on image quality evaluation, comprising:

[0128] An image enhancement module is used to integrate the ControlNet architecture into the original image enhancement model, the ControlNet architecture improves the original image enhancement model by adding an auxiliary module and a cross-entropy-based loss function, the auxiliary module is used to introduce an additional conditional generation control signal to dynamically adjust the generation strategy, and the improved image enhancement model generates an enhanced image according to the input additional condition and input image;

[0129] A quality evaluation module is used to input the enhanced image into an image quality evaluation model to output a quality score and a text description;

[0130] A dynamic updating module is used to design a multi-modal loss function according to the quality score and description text output by the image quality evaluation model if the quality score is lower than a preset threshold, and the weight parameters of the improved image enhancement model are adjusted in real time through back propagation.

[0131] The application has the following characteristics and effects:

[0132] Dynamic adjustment technology based on ControlNet architecture: By introducing auxiliary modules of locked copy and trainable copy, the locked copy fixes the original model weight to retain the basic generation ability, and the trainable copy generates dynamic control signals combined with additional conditions (such as quality score), and adjusts the generation strategy through adaptive feature fusion. This technology realizes real-time optimization of model weights according to actual scenes after deployment, significantly improves the adaptability and stability of image enhancement, and reduces the dependence on manual annotation data.

[0133] Mixed loss function optimization technology: Combine conditional perception cross-entropy loss function with traditional pixel-level / perception loss, dynamically adjust weight coefficient (initially focus on condition guidance, later strengthen generation ability), and optimize model in stages (pre-train main model, train ControlNet, joint fine-tune). This technology enhances the model's fitting ability for high-quality image distribution, solves the generalization problem caused by fixed loss function in traditional methods, and significantly improves the enhancement effect in complex scenes (such as low light, high noise).

[0134] Quality evaluation and text guidance technology: The image quality evaluation model outputs quality score and text description simultaneously through the Transformer structure, combines local texture weight analysis and global feature fusion, and designs a multi-modal loss function (quality score loss + text generation loss) to guide model updating. This technology uses text feedback to accurately locate image defects (such as blur, low contrast), and optimizes model parameters directionally, achieving more fine-grained quality control and improving the accuracy and automation level of fault detection in power inspection and other scenarios.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by any person skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A deep learning-based augmentation model update method based on image quality assessment, characterized in that, Includes the following steps: The ControlNet architecture is integrated into the image augmentation model. The ControlNet architecture improves the image augmentation model by adding an auxiliary module and a conditional cross-entropy loss function. The auxiliary module is used to introduce additional conditions to generate control signals to dynamically adjust the generation strategy. The improved image augmentation model generates an enhanced image based on the additional input conditions and the input image. The enhanced image is input into the image quality assessment model, which outputs a quality score and a text description. If the quality score is lower than the preset threshold, a multimodal loss function is designed based on the quality score output by the image quality assessment model and the descriptive text, and the weight parameters of the improved image enhancement model are adjusted in real time through backpropagation. Loss function of image enhancement model To address pixel-level or perceptual loss, a conditional perceptual cross-entropy loss function is introduced. Forming a hybrid loss function: ； in, It is a dynamically adjusted weighting coefficient, which is larger in the initial stage and gradually decreases in the later stage; The conditional cross-entropy loss function is designed as follows: assuming the control signal output by ControlNet... The model needs to be guided to generate images that meet quality requirements, and the target distribution needs to be defined. Distribution predicted by the model Cross-entropy loss between: ； in, The probability map output by ControlNet is obtained by Softmax normalization. Dynamically generated based on quality assessment results; During the pre-training of the main model, only use First, train the original image enhancement model to ensure basic generation capabilities; then, freeze the main model and fix its weights while training the ControlNet model. The trainable copy of ControlNet is optimized to learn and generate control signals adapted to additional conditions; finally, in the joint fine-tuning stage, the main model is unfrozen and a hybrid loss is adopted. By jointly optimizing the parameters of the main model and ControlNet, the overall performance of the image enhancement model can be improved.

2. The method according to claim 1, characterized in that, The auxiliary module includes a locked copy and a trainable copy. The locked copy is used to replicate the structure and weights of the image enhancement model, and the parameters remain fixed during training to retain the generative capability of the image enhancement model. The trainable copy has the same structure as the image enhancement model, but the parameters can be fine-tuned. It generates control signals through additional conditions to dynamically adjust the generation strategy.

3. The method according to claim 2, characterized in that, The trainable copy generates control signals through additional conditions, specifically including: ； in, For the generated control signal tensor, These are the height, width, and number of channels of the feature map, respectively. To fine-tune the model, Input image; control signals Intermediate feature maps of the main model Interacting through adaptive feature fusion allows the model to dynamically adjust the generation process based on additional conditions when generating images. ； in, It is a learnable scaling factor used to dynamically adjust the strength of the control signal.

4. The method according to claim 1, characterized in that, The process of inputting the enhanced image into the image quality assessment model and outputting a quality score and text description includes: By introducing a text-guided mechanism, the image quality assessment model can simultaneously output the image quality score and a detailed text description, which helps to understand the problems existing in the image; The texture perception module based on the Transformer structure performs weight analysis on each token in the image to determine the contribution weight of each token to the image quality assessment in order to evaluate the local features of the image. The global quality assessment module fuses all tokens to generate global weights to evaluate the local features of the image; The image quality score and text description are generated by combining local and global weights.

5. The method according to claim 1, characterized in that, The additional conditions are multi-dimensional external information used to dynamically control the image generation process, guiding the model to generate enhanced images that better meet the target requirements based on specific scenario needs.

6. The method according to claim 1, characterized in that, The design of a multimodal loss function based on the quality score output by the image quality assessment model and the descriptive text specifically includes: By combining quality scoring loss and text generation loss, a multimodal loss function is designed to guide model updates: ； in, For quality score loss, For text generation loss, and These are dynamic weighting coefficients used to balance the optimization direction of quality scoring and text description; ； in, The model predicts the score. This is a genuine quality label. and These are their respective means; Text generation loss describes the text. Encode as feature vector and with image features Alignment: ； The weights of the improved image enhancement model are updated based on the multimodal loss function, and the parameters of the improved image enhancement model are calculated using the chain rule. The gradient is used to update the model weights using the optimizer: 。 7. A real-time update system for a deep learning-based enhancement model based on image quality assessment, characterized in that, include: An image enhancement module is used to integrate the ControlNet architecture into the original image enhancement model. The ControlNet architecture improves the original image enhancement model by adding an auxiliary module and a cross-entropy-based loss function. The auxiliary module is used to introduce additional conditions to generate control signals to dynamically adjust the generation strategy. The improved image enhancement model generates an enhanced image based on the additional input conditions and the input image. The quality assessment module is used to input the enhanced image into the image quality assessment model and output a quality score and text description. The dynamic update module is used to design a multimodal loss function based on the quality score and descriptive text output by the image quality assessment model if the quality score is lower than a preset threshold, and adjust the weight parameters of the improved image enhancement model in real time through backpropagation. Loss function of image enhancement model To address pixel-level or perceptual loss, a conditional perceptual cross-entropy loss function is introduced. Forming a hybrid loss function: ； in, It is a dynamically adjusted weighting coefficient, which is larger in the initial stage and gradually decreases in the later stage; The conditional cross-entropy loss function is designed as follows: assuming the control signal output by ControlNet... The model needs to be guided to generate images that meet quality requirements, and the target distribution needs to be defined. Distribution predicted by the model Cross-entropy loss between: ； in, The probability map output by ControlNet is obtained by Softmax normalization. Dynamically generated based on quality assessment results; During the pre-training of the main model, only use First, train the original image enhancement model to ensure basic generation capabilities; then, freeze the main model and fix its weights while training the ControlNet model. The trainable copy of ControlNet is optimized to learn and generate control signals adapted to additional conditions; finally, in the joint fine-tuning stage, the main model is unfrozen and a hybrid loss is adopted. By jointly optimizing the parameters of the main model and ControlNet, the overall performance of the image enhancement model can be improved.

8. The system according to claim 7, characterized in that, The auxiliary module includes a locked copy and a trainable copy. The locked copy is used to replicate the structure and weights of the image enhancement model, and the parameters remain fixed during training to retain the generative capability of the image enhancement model. The trainable copy has the same structure as the image enhancement model, but the parameters can be fine-tuned. It generates control signals through additional conditions to dynamically adjust the generation strategy. The trainable copy generates control signals through additional conditions, specifically including: ； in, For the generated control signal tensor, These are the height, width, and number of channels of the feature map, respectively. To fine-tune the model, Input image; control signals Intermediate feature maps of the main model Interacting through adaptive feature fusion allows the model to dynamically adjust the generation process based on additional conditions when generating images. ； in, It is a learnable scaling factor used to dynamically adjust the strength of the control signal.

Images