Adaptive image quality enhancement method and device, equipment and medium
By combining the CLIP model and parameter mapping network, the differences between image and text features are extracted, which solves the problem of lack of semantic information in traditional image quality assessment, realizes adaptive image quality enhancement, and improves the image quality assessment and enhancement effect in the medical and financial fields.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-06
- Publication Date
- 2026-04-17
AI Technical Summary
Traditional image quality assessment methods fail to provide semantic information, resulting in a lack of rationale for image quality enhancement, which affects the accuracy of medical diagnosis and the verification of the authenticity of financial documents.
The CLIP model is used to extract the differences between image and text feature vectors. The optimal control parameter set is obtained through dimensionality reduction processing via a parameter mapping network. Finally, the image quality is enhanced using an adaptive histogram equalization algorithm.
This approach enables the understanding of image quality assessment from multiple semantic dimensions, improving the accuracy and efficiency of image quality enhancement, reducing repeated parameter tuning steps, and ultimately enhancing image quality.
Smart Images
Figure CN121883282A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and visual technology, and in particular to an adaptive image quality enhancement method, apparatus, device, and medium. Background Technology
[0002] Traditional image quality assessment methods mostly output a single quality score that lacks semantic information. This fails to explain the rationale behind the image quality assessment and cannot guide subsequent image quality enhancement. In the field of smart healthcare, the failure to better enhance the image quality of medical images poses a risk to the accuracy of medical diagnosis. Similarly, in the financial sector, the failure to better enhance the image quality of scanned images of signed documents poses a risk of missing counterfeit documents. Summary of the Invention
[0003] This invention provides an adaptive image quality enhancement method, apparatus, computer device, and storage medium to solve the technical problem that traditional image quality assessment methods cannot explain the rationale for image quality assessment and cannot guide subsequent image quality enhancement.
[0004] Firstly, an adaptive image quality enhancement method is provided, including: The image feature vector of the input image is extracted by the CLIP model image encoder, and the text feature vector of the target image quality text prompt is extracted by the CLIP model text encoder. The difference between the text feature vector and the image feature vector is calculated to obtain the semantic residual vector. The semantic residual vector is reduced in dimensionality using a parameter mapping network to obtain the optimal set of control parameters. The input image is subjected to image quality enhancement processing based on the optimal control parameter set to obtain an enhanced image.
[0005] Secondly, an adaptive image quality enhancement device is provided, comprising: The semantic residual calculation module is used to extract the image feature vector of the input image through the CLIP model image encoder, extract the text feature vector of the target image quality text prompt through the CLIP model text encoder, calculate the difference between the text feature vector and the image feature vector, and obtain the semantic residual vector. The parameter mapping module is used to perform dimensionality reduction processing on the semantic residual vector using a parameter mapping network to obtain the optimal set of control parameters; The image quality enhancement module is used to perform image quality enhancement processing on the input image according to the optimal control parameter set to obtain a quality-enhanced image.
[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described adaptive image quality enhancement method.
[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described adaptive image quality enhancement method.
[0008] In the aforementioned adaptive image quality enhancement method, apparatus, device, and medium, the solution involves receiving an input image and a target image quality text prompt via a client. An image encoder using the CLIP model extracts the image feature vector of the input image, and a text encoder using the CLIP model extracts the text feature vector of the target image quality text prompt. The difference between the text feature vector and the image feature vector is calculated to obtain a semantic residual vector. A parameter mapping network is used to reduce the dimensionality of the semantic residual vector to obtain an optimal control parameter set. Based on the optimal control parameter set, the input image undergoes image quality enhancement processing to obtain an enhanced image, which is then fed back to the client. In this invention, for a surgical video recording and reporting intelligent management system in the field of smart healthcare, or for a transaction process video recording and reporting intelligent management system in the field of finance, an adaptive... The image quality enhancement scheme extracts image feature vectors from the input image using the CLIP model image encoder and extracts text feature vectors from the target image quality text prompts using the CLIP model text encoder. The difference between the text feature vectors and the image feature vectors is calculated to obtain a semantic residual vector, enabling a deeper understanding of the image quality assessment rationale from multiple semantic dimensions. This provides profound insights for accurate image quality enhancement and facilitates subsequent image quality enhancement processing. A parameter mapping network is used to reduce the dimensionality of the semantic residual vectors to obtain an optimal set of control parameters, transforming the abstract semantic residual vectors into precise mathematical instructions, thus improving the image quality enhancement effect. By performing image quality enhancement processing on the input image based on the optimal control parameters, adaptive image quality enhancement is achieved without repeated parameter tuning, improving image quality enhancement efficiency and resulting in high-quality enhanced images. Attached Figure Description
[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1This is a schematic diagram of an application environment for an adaptive image quality enhancement method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an adaptive image quality enhancement method according to an embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of a specific implementation method for step S20; Figure 4 This is a schematic diagram of the structure of an adaptive image quality enhancement device according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 6 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] The adaptive image quality enhancement method provided in this embodiment of the invention can be applied to, for example... Figure 1In application environments such as smart healthcare and finance, intelligent management systems are typically implemented through a server-side architecture, where the client communicates with the server via a network. The server receives input images and target image quality text prompts from the client. It extracts image feature vectors from the input image using an image encoder based on the CLIP model, and extracts text feature vectors from the target image quality text prompts using a text encoder based on the CLIP model. It calculates the difference between the text feature vectors and the image feature vectors to obtain a semantic residual vector. A parameter mapping network is then used to reduce the dimensionality of the semantic residual vectors to obtain an optimal set of control parameters. Based on the optimal control parameter set, the input image undergoes image quality enhancement processing to obtain an enhanced image, which is then fed back to the client. In this invention, for medical image intelligent management systems in the smart healthcare field, or for signature document image intelligent management systems in the financial field, an adaptive image quality enhancement scheme can be utilized, using a CLIP model's image... The process involves an encoder extracting image feature vectors from the input image, and a text encoder using the CLIP model extracting text feature vectors for the target image quality cues. The difference between the text feature vectors and the image feature vectors is calculated to obtain a semantic residual vector. This allows for understanding the rationale behind image quality assessment from multiple semantic dimensions, providing deep insights for precise image quality enhancement and facilitating subsequent image quality enhancement processing. A parameter mapping network is used to reduce the dimensionality of the semantic residual vectors, obtaining an optimal set of control parameters. This transforms the abstract semantic residual vectors into precise mathematical instructions, improving the image quality enhancement effect. By applying the optimal control parameters to the input image for image quality enhancement, adaptive image quality enhancement is achieved without repeated parameter tuning, improving efficiency and resulting in high-quality enhanced images. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster. The following detailed description of specific embodiments further illustrates this invention.
[0013] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the adaptive image quality enhancement method provided in this embodiment of the invention includes the following steps: S10: Extract the image feature vector of the input image using the CLIP model image encoder, extract the text feature vector of the target image quality text prompt using the CLIP model text encoder, calculate the difference between the text feature vector and the image feature vector, and obtain the semantic residual vector.
[0014] CLIP (Contrastive Language-Image Pre-training) models embed images and text into the same feature space through contrastive learning, achieving cross-modal semantic alignment. Target image quality text cues refer to textual prompts related to ideal image quality. These cues may include information related to image quality such as brightness, contrast, and sharpness. The CLIP model extracts image feature vectors from the input image using its image encoder and extracts text feature vectors from the target image quality text cues using its text encoder. The difference between the text feature vectors and the image feature vectors is calculated to obtain a semantic residual vector. This allows for understanding the rationale behind image quality assessment from multiple semantic dimensions, providing deep insights for precise image quality enhancement and facilitating subsequent image quality enhancement processing. The adaptive image quality enhancement method provided by this invention can be applied to intelligent management systems in various application scenarios such as healthcare and finance. Intelligent management systems are typically implemented through a server that can receive input images and target image quality text cues in real time. For example, in the medical field, users often manage the storage of medical images via computers and frequently require intelligent management systems to analyze and process these images to improve diagnostic efficiency.
[0015] For example, in the medical field, the input image could be a medical image such as a CT scan or X-ray, and the target image quality text prompt could be a clear, well-exposed, and detailed professional photograph. Alternatively, in the financial field, the input image could be a scanned image of a signed transaction document, and the target image quality text prompt could be a clear and well-exposed photograph.
[0016] Preferably, in some embodiments, the extraction of image feature vectors from the input image using the CLIP model image encoder in step S10 specifically involves: The CLIP model image encoder maps the input image to a low-dimensional continuous vector space and extracts the image feature vector of the input image.
[0017] The resolution of the obtained image feature vector can be 224×224.
[0018] Preferably, in some embodiments, the extraction of text feature vectors of target image quality text prompts by the text encoder using the CLIP model in step S10 specifically involves: The CLIP model's text encoder maps the target image quality text prompt to a low-dimensional continuous vector space, extracting the text feature vector of the target image quality text prompt.
[0019] The CLIP model maps the input image and target image quality text prompts to a low-dimensional continuous vector space to make the size of the obtained image feature vector equal to the size of the text feature vector, so as to calculate the difference between the text feature vector and the image feature vector.
[0020] Specifically, in some embodiments, step S10, calculating the difference between the text feature vector and the image feature vector to obtain a semantic residual vector, includes: The semantic residual vector is obtained by calculating the difference between the text feature vector and the image feature vector using formula (1): v R =E t (T in )-E i (I in (1) In the formula, v R E represents the semantic residual vector. t (●) indicates a text encoder, T in Text prompts indicating target image quality, E t (T in E represents the extracted text feature vector of the target image quality text prompt. i (●) indicates the image encoder, I in E represents the input image. i (I in ) represents the extracted image feature vector of the input image. The direction and magnitude of the semantic residual vector precisely characterize the quality difference between the input image and the target image across multiple semantic dimensions, such as sharpness, contrast, and brightness.
[0021] S20: The semantic residual vector is reduced in dimensionality using a parameter mapping network to obtain the optimal set of control parameters.
[0022] Preferably, in step S20, the parameter mapping network used to reduce the dimensionality of the semantic residual vector is a feedforward neural network, which includes a multilayer perceptron (MLP) to convert the high-dimensional semantic residual vector into low-dimensional specific enhancement algorithm parameters.
[0023] Specifically, such as Figure 3 As shown, step S20, which involves using a parameter mapping network to reduce the dimensionality of the semantic residual vector to obtain the optimal set of control parameters, includes the following steps: S21: Set the network structure of the parameter mapping network; Preferably, step S21, which is setting the network structure of the parameter mapping network, includes the following steps: Set the input layer, and set the number of neurons in the input layer according to the dimension of the semantic residual vector; where the dimension of the semantic residual vector can be 512-dimensional or 768-dimensional, then the number of neurons in the input layer can be set to 512 or 768, so that the number of neurons in the input layer corresponds to the dimension of the semantic residual vector.
[0024] At least one fully connected layer is set as a hidden layer, and a nonlinear activation function is connected after each fully connected layer; wherein, the nonlinear activation function can be a ReLU function (Linear rectification function); by setting a hidden layer including at least one fully connected layer, and each fully connected layer is connected to a ReLU function, the nonlinear relationship between the semantic residual vector and the optimal parameters is learned.
[0025] The output layer is configured with its number of neurons determined by the number of parameters in the image quality enhancement module. This output layer serves as the output of the parameter mapping network, connecting to the image quality enhancement module to output corresponding parameters for image quality enhancement operations. The image quality enhancement module may employ an Adaptive Histogram Equalization (AHE) algorithm, resulting in two parameters and two neurons in the output layer. Each neuron outputs a specific parameter value. The parameters corresponding to the two neurons in the output layer can be the clipping limit parameter and the image patch size parameter, denoted as `clipLimit` and `gridSize`, respectively. The activation function of the output layer can be set according to the parameter value range. For the clipping limit parameter, which ranges from 1.0 to 40.0, a scaled sigmoid function can be used as the activation function for the neurons in the output layer corresponding to the clipping limit parameter. For the image patch size parameter, which ranges from positive integers, the product of a floating-point number and a scaling factor, rounded to the nearest integer, can be used as the activation function for the neurons in the output layer corresponding to the image patch size parameter.
[0026] S22: Set the trainable parameters of the parameter mapping network, and train the parameter mapping network based on the trainable parameters.
[0027] The trainable parameters are obtained by supervised learning on a dataset consisting of quality image pairs. A quality image pair is an image pair consisting of a low-quality image and a high-quality reference image paired with that low-quality image.
[0028] Preferably, step S22, training the parameter mapping network based on the trainable parameters, specifically includes: Low-quality images are obtained from the training set of the dataset and used as training images; The difference between the quality text prompts of the training image and the target image is calculated as the training semantic residual vector; The training semantic residual vector is input into the value parameter mapping network to obtain the prediction parameter set; The prediction parameter set is used to enhance the image quality of the training image using an adaptive histogram equalization algorithm to obtain the enhanced training image. The enhanced training image is compared with a high-quality reference image paired with the training image in the dataset, and the loss between the enhanced training image and the high-quality reference image paired with the training image in the dataset is calculated as the quality loss. The gradient of the trainable parameters is calculated using the backpropagation algorithm based on the quality loss, and the trainable parameters are updated by an optimizer along the gradient descent direction.
[0029] Preferably, after step S22, i.e., setting the trainable parameters of the parameter mapping network and training the parameter mapping network according to the trainable parameters, the adaptive image quality enhancement method further includes: Parameter mapping network mapping inference verification.
[0030] The parameter mapping network mapping inference verification specifically includes the following steps: The semantic residual vector between the features of the mapped image and the features of the mapped quality text is calculated based on the input low-quality mapped image and the quality text of the mapped target image as the mapping semantic residual vector; The dimensionality reduction of the mapping semantic residual vector is performed using the trained parameter mapping network to obtain the mapping quality parameter set.
[0031] Specifically, when the input low-quality image suffers from severe underexposure and lack of contrast, while the target image is well-exposed and rich in detail, the magnitude of the resulting semantic residual vector is larger, indicating a significant quality gap with the target image. Consequently, the parameters in the mapping quality parameter set will be larger to enhance local contrast and make overall adjustments to larger areas of the image. Conversely, when the input low-quality image has good exposure and contrast, while the target image is well-exposed and rich in detail, the magnitude of the resulting semantic residual vector is smaller. Consequently, the parameters in the mapping quality parameter set will be smaller to avoid over-enhancing image quality and amplifying image noise.
[0032] S30: Perform image quality enhancement processing on the input image according to the optimal control parameter set to obtain a quality-enhanced image.
[0033] In this embodiment of the invention, step S30, namely, performing image quality enhancement processing on the input image according to the optimal control parameter set to obtain a quality-enhanced image, specifically involves: Based on the optimal control parameter set, an adaptive histogram equalization algorithm is used to perform image quality enhancement processing on the input image to obtain a quality-enhanced image.
[0034] When using an adaptive histogram equalization algorithm for image quality enhancement, the optimal control parameter set includes two parameters: a clipping limit parameter (clipLimit) and an image patch size parameter (gridSize). Specifically, the process of using the adaptive histogram equalization algorithm to enhance the image quality of the input image based on the optimal control parameter set includes the following steps: The input image is segmented into grids based on the image block size parameters of the optimal control parameter set to obtain image grid blocks. The image segmented into grids can be divided into multiple non-overlapping and continuous rectangular tile grids. The image grid block is each rectangular tile in the rectangular tile grid. The image block size parameter is the number of rows and columns of the corresponding grid in the image. When the image block size parameter is 8, the image will be segmented into an 8×8 grid, that is, the image will be segmented into 64 non-overlapping and continuous rectangular tile grids.
[0035] The gray level of each pixel in the input image is calculated based on the pixels in each image grid block, thus obtaining the image gray level histogram; this image gray level histogram describes the pixel intensity distribution of the corresponding region of the input image.
[0036] The cropping threshold is calculated based on the cropping constraint parameter. The gray levels of the image gray-level histogram are traversed. When the number of pixels corresponding to a gray level is greater than the cropping threshold, the region corresponding to the number of pixels exceeding the threshold is cropped. The higher the value of the cropping constraint parameter, the greater the degree of local contrast stretching is allowed.
[0037] Obtain the total amount of all cropped pixels, distribute the total amount of all cropped pixels evenly to each gray level of the image gray-level histogram, update the image gray-level histogram, and obtain the updated gray-level histogram; by distributing the total amount of all cropped pixels evenly to each gray level, the excessive amplification of noise in flat areas of the image can be suppressed. Based on the updated grayscale histogram, the cumulative distribution function of each image grid block is calculated; where the cumulative integral function is the pixel intensity mapping function of the image grid block.
[0038] Obtain the distance from each pixel in the input image to the center point of the four nearest image grid blocks around that pixel, and use a bilinear interpolation algorithm combined with the distance to calculate the final output grayscale of that pixel; The final output grayscale of each pixel in the input image is used as the grayscale of each pixel in the quality-enhanced image.
[0039] As can be seen, in the above scheme, for medical image intelligent management systems in the field of smart healthcare, or for signature document image intelligent management systems in the financial field, an adaptive image quality enhancement scheme can be used. This involves extracting image feature vectors from the input image using a CLIP model image encoder, extracting text feature vectors from the target image quality text prompts using a CLIP model text encoder, and calculating the difference between the text feature vectors and the image feature vectors to obtain semantic residual vectors. This allows for understanding the rationale for image quality assessment from multiple semantic dimensions, providing deep insights for precise image quality enhancement and facilitating subsequent image quality enhancement processing. A parameter mapping network is used to reduce the dimensionality of the semantic residual vectors to obtain the optimal set of control parameters, transforming the abstract semantic residual vectors into precise mathematical instructions, thus improving the image quality enhancement effect. By performing image quality enhancement processing on the input image based on the optimal control parameters, repeated parameter tuning is unnecessary, achieving adaptive image quality enhancement, improving image quality enhancement efficiency, and obtaining high-quality enhanced images.
[0040] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0041] In one embodiment, an adaptive image quality enhancement device is provided, which corresponds one-to-one with the adaptive image quality enhancement method in the above embodiments. For example... Figure 4 As shown, the adaptive image quality enhancement device includes a semantic residual calculation module 101, a parameter mapping module 102, and an image quality enhancement module 103. Detailed descriptions of each functional module are as follows: The semantic residual calculation module 101 is used to extract the image feature vector of the input image through the image encoder of the CLIP model, extract the text feature vector of the target image quality text prompt through the text encoder of the CLIP model, calculate the difference between the text feature vector and the image feature vector, and obtain the semantic residual vector. Parameter mapping module 102 is used to perform dimensionality reduction processing on the semantic residual vector using a parameter mapping network to obtain the optimal control parameter set; The image quality enhancement module 103 is used to perform image quality enhancement processing on the input image according to the optimal control parameter set to obtain a quality-enhanced image.
[0042] In one embodiment, the parameter mapping module 102 is specifically used for: Configure the network structure of the parameter mapping network; Set the trainable parameters of the parameter mapping network, and train the parameter mapping network based on the trainable parameters.
[0043] In one embodiment, the parameter mapping module 102 is specifically used for: Set the input layer, and set the number of neurons in the input layer according to the dimension of the semantic residual vector; Set at least one fully connected layer as a hidden layer, and connect a non-linear activation function after each fully connected layer; Configure the output layer by setting the number of neurons in the output layer based on the number of parameters of the image quality enhancement module.
[0044] In one embodiment, the parameter mapping module 102 is further configured to: Low-quality images are obtained from the training set of the dataset and used as training images; The difference between the quality text prompts of the training image and the target image is calculated as the training semantic residual vector; The training semantic residual vector is input into the value parameter mapping network to obtain the prediction parameter set; The prediction parameter set is used to enhance the image quality of the training image using an adaptive histogram equalization algorithm to obtain the enhanced training image. The enhanced training image is compared with a high-quality reference image paired with the training image in the dataset, and the loss between the enhanced training image and the high-quality reference image paired with the training image in the dataset is calculated as the quality loss. The gradient of the trainable parameters is calculated using the backpropagation algorithm based on the quality loss, and the trainable parameters are updated by an optimizer along the gradient descent direction.
[0045] In one embodiment, the image quality enhancement module 103 is specifically used for: Based on the optimal control parameter set, an adaptive histogram equalization algorithm is used to perform image quality enhancement processing on the input image to obtain a quality-enhanced image.
[0046] In one embodiment, the image quality enhancement module 103 is specifically used for: The input image is divided into grids based on the image block size parameters of the optimal control parameter set to obtain image grid blocks. The grayscale of each pixel in the input image is calculated based on the pixels in each image grid block, and the image grayscale histogram is obtained. The cropping threshold is calculated based on the cropping constraint parameters. The gray levels of the image gray level histogram are traversed. When the number of pixels corresponding to a gray level is greater than the cropping threshold, the region corresponding to the number of pixels exceeding the threshold is cropped. Get the total amount of all cropped pixels, distribute the total amount of all cropped pixels evenly to each gray level of the image gray level histogram, update the image gray level histogram, and obtain the updated gray level histogram. Based on the updated grayscale histogram, calculate the cumulative distribution function of each image grid block; Obtain the distance from each pixel in the input image to the center point of the four nearest image grid blocks around that pixel, and use a bilinear interpolation algorithm combined with the distance to calculate the final output grayscale of that pixel; The final output grayscale of each pixel in the input image is used as the grayscale of each pixel in the quality-enhanced image.
[0047] This invention provides an adaptive image quality enhancement device. It extracts image feature vectors from the input image using a CLIP model image encoder and extracts text feature vectors from the target image quality text prompts using a CLIP model text encoder. The difference between the text feature vectors and the image feature vectors is calculated to obtain a semantic residual vector. This allows for understanding the rationale for image quality assessment from multiple semantic dimensions, providing deep insights for precise image quality enhancement and facilitating subsequent image quality enhancement processing. A parameter mapping network is used to reduce the dimensionality of the semantic residual vectors to obtain an optimal set of control parameters, transforming the abstract semantic residual vectors into precise mathematical instructions, thus improving the image quality enhancement effect. By performing image quality enhancement processing on the input image based on the optimal control parameters, it eliminates the need for repeated parameter tuning, achieving adaptive image quality enhancement, improving image quality enhancement efficiency, and obtaining high-quality enhanced images.
[0048] Specific limitations regarding the adaptive image quality enhancement device can be found in the limitations of the adaptive image quality enhancement method described above, and will not be repeated here. Each module in the aforementioned adaptive image quality enhancement device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0049] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of an adaptive image quality enhancement method on the server side.
[0050] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of an adaptive image quality enhancement method.
[0051] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The image feature vector of the input image is extracted by the CLIP model image encoder, and the text feature vector of the target image quality text prompt is extracted by the CLIP model text encoder. The difference between the text feature vector and the image feature vector is calculated to obtain the semantic residual vector. The semantic residual vector is reduced in dimensionality using a parameter mapping network to obtain the optimal set of control parameters. The input image is subjected to image quality enhancement processing based on the optimal control parameter set to obtain an enhanced image.
[0052] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The image feature vector of the input image is extracted by the CLIP model image encoder, and the text feature vector of the target image quality text prompt is extracted by the CLIP model text encoder. The difference between the text feature vector and the image feature vector is calculated to obtain the semantic residual vector. The semantic residual vector is reduced in dimensionality using a parameter mapping network to obtain the optimal set of control parameters. The input image is subjected to image quality enhancement processing based on the optimal control parameter set to obtain an enhanced image.
[0053] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0054] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0055] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0056] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method of adaptive image quality enhancement, characterized in that, include: The image feature vector of the input image is extracted by the CLIP model image encoder, and the text feature vector of the target image quality text prompt is extracted by the CLIP model text encoder. The difference between the text feature vector and the image feature vector is calculated to obtain the semantic residual vector. The semantic residual vector is reduced in dimensionality using a parameter mapping network to obtain the optimal set of control parameters. The input image is subjected to image quality enhancement processing based on the optimal control parameter set to obtain an enhanced image.
2. The adaptive image quality enhancement method as described in claim 1, characterized in that, The step of using a parameter mapping network to reduce the dimensionality of the semantic residual vector to obtain the optimal set of control parameters includes the following steps: Configure the network structure of the parameter mapping network; Set the trainable parameters of the parameter mapping network, and train the parameter mapping network based on the trainable parameters.
3. The adaptive image quality enhancement method as described in claim 2, characterized in that, The network structure of the parameter mapping network includes: Set the input layer, and set the number of neurons in the input layer according to the dimension of the semantic residual vector; Set at least one fully connected layer as a hidden layer, and connect a non-linear activation function after each fully connected layer; Configure the output layer by setting the number of neurons in the output layer based on the number of parameters of the image quality enhancement module.
4. The adaptive image quality enhancement method as described in claim 2, characterized in that, The training of the parameter mapping network based on trainable parameters includes: Low-quality images are obtained from the training set of the dataset and used as training images; The difference between the quality text prompts of the training image and the target image is calculated as the training semantic residual vector; The training semantic residual vector is input into the value parameter mapping network to obtain the prediction parameter set; The prediction parameter set is used to enhance the image quality of the training image using an adaptive histogram equalization algorithm to obtain the enhanced training image. The enhanced training image is compared with a high-quality reference image paired with the training image in the dataset, and the loss between the enhanced training image and the high-quality reference image paired with the training image in the dataset is calculated as the quality loss. The gradient of the trainable parameters is calculated using the backpropagation algorithm based on the quality loss, and the trainable parameters are updated by an optimizer along the gradient descent direction.
5. The adaptive image quality enhancement method as described in claim 2, characterized in that, The adaptive image quality enhancement method further includes setting the trainable parameters of the parameter mapping network, training the parameter mapping network based on the trainable parameters, and then: Parameter mapping network mapping inference verification.
6. The adaptive image quality enhancement method as described in claim 1, characterized in that, The step of performing image quality enhancement processing on the input image based on the optimal control parameter set to obtain a quality-enhanced image specifically involves: Based on the optimal control parameter set, an adaptive histogram equalization algorithm is used to perform image quality enhancement processing on the input image to obtain a quality-enhanced image.
7. The adaptive image quality enhancement method as described in claim 6, characterized in that, When using an adaptive histogram equalization algorithm for image quality enhancement, the optimal control parameter set includes two parameters: a cropping constraint parameter and an image block size parameter. The process of using the adaptive histogram equalization algorithm to enhance the input image based on the optimal control parameter set includes the following steps: The input image is divided into grids based on the image block size parameters of the optimal control parameter set to obtain image grid blocks. The grayscale of each pixel in the input image is calculated based on the pixels in each image grid block, and the image grayscale histogram is obtained. The cropping threshold is calculated based on the cropping constraint parameters. The gray levels of the image gray level histogram are traversed. When the number of pixels corresponding to a gray level is greater than the cropping threshold, the region corresponding to the number of pixels exceeding the threshold is cropped. Get the total amount of all cropped pixels, distribute the total amount of all cropped pixels evenly to each gray level of the image gray level histogram, update the image gray level histogram, and obtain the updated gray level histogram. Based on the updated grayscale histogram, calculate the cumulative distribution function of each image grid block; Obtain the distance from each pixel in the input image to the center point of the four nearest image grid blocks around that pixel, and use a bilinear interpolation algorithm to calculate the final output grayscale of that pixel in combination with the distances corresponding to that pixel. The final output grayscale of each pixel in the input image is used as the grayscale of each pixel in the quality-enhanced image.
8. An adaptive image quality enhancement device, characterized in that, include: The semantic residual calculation module is used to extract the image feature vector of the input image through the CLIP model image encoder, extract the text feature vector of the target image quality text prompt through the CLIP model text encoder, calculate the difference between the text feature vector and the image feature vector, and obtain the semantic residual vector. The parameter mapping module is used to perform dimensionality reduction processing on the semantic residual vector using a parameter mapping network to obtain the optimal set of control parameters; The image quality enhancement module is used to perform image quality enhancement processing on the input image according to the optimal control parameter set to obtain a quality-enhanced image.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the adaptive image quality enhancement method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the adaptive image quality enhancement method as described in any one of claims 1 to 7.