Image quality enhancement model training method and device, computer device, and storage medium

Abstract

The application relates to a picture quality enhancement model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining an initial training image and an application scene identifier, and obtaining corresponding image application parameters based on the application scene identifier; generating an applied image from the initial training image according to the image application parameters to obtain a target training image matched with the application scene identifier; inputting the target training image into an initial picture quality enhancement model to perform image picture quality enhancement and obtain an initial picture quality enhancement image; performing loss calculation based on the initial picture quality enhancement image and the initial training image to obtain loss information, updating the initial picture quality enhancement model based on the loss information to obtain an updated picture quality enhancement model; taking the updated picture quality enhancement model as the initial picture quality enhancement model and performing cyclic iteration until training is completed, and obtaining a target picture quality enhancement model corresponding to the application scene identifier. The method can improve the generalization performance of the model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for training image quality enhancement models. Background Technology

[0002] With the development of image processing technology, image enhancement techniques have emerged, such as image super-resolution (GSL). GSL refers to enhancing the image quality of a low-resolution image or image sequence to create a high-resolution image. Currently, deep learning algorithms from artificial intelligence are commonly used to build image enhancement models. This involves obtaining high-resolution image data from open-source datasets, then generating corresponding low-resolution images as training samples using methods such as Gaussian blurring, upsampling, Gaussian noise, and quality compression. The image enhancement model is then trained using these training samples and the high-resolution image data, and finally, it is used to enhance the image quality.

[0003] However, current image enhancement models trained using training samples and high-resolution image data suffer from poor generalization performance. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for training image quality enhancement models that can improve generalization performance in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training an image quality enhancement model. The method includes:

[0006] Obtain the initial training images and application scenario identifiers, and obtain the corresponding image application parameters based on the application scenario identifiers;

[0007] The initial training images are processed according to the image application parameters to generate application images, resulting in target training images that match the application scenario identifier.

[0008] The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in the initial image quality enhanced image;

[0009] Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model.

[0010] The updated image enhancement model is used as the initial image enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met. Then, the target image enhancement model corresponding to the application scene identifier is obtained. The target image enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0011] Secondly, this application also provides an image quality enhancement model training device. The device includes:

[0012] The parameter acquisition module is used to acquire the initial training image and application scenario identifier, and to acquire the corresponding image application parameters based on the application scenario identifier;

[0013] The image generation module is used to generate application images from the initial training images according to the image application parameters, so as to obtain target training images that match the application scene identifier.

[0014] The image quality enhancement module is used to input the target training image into the initial image quality enhancement model to enhance the image quality and obtain the initial enhanced image.

[0015] The update module is used to calculate the loss based on the initial image enhancement image and the initial training image to obtain loss information, and then update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model.

[0016] The iteration module is used to take the updated image quality enhancement model as the initial image quality enhancement model and return to the step of obtaining the initial training image until the training completion condition is met, and then obtain the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0018] Obtain the initial training images and application scenario identifiers, and obtain the corresponding image application parameters based on the application scenario identifiers;

[0019] The initial training images are processed according to the image application parameters to generate application images, resulting in target training images that match the application scenario identifier.

[0020] The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in the initial image quality enhanced image;

[0021] Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model.

[0022] The updated image enhancement model is used as the initial image enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met. Then, the target image enhancement model corresponding to the application scene identifier is obtained. The target image enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0023] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0024] Obtain the initial training images and application scenario identifiers, and obtain the corresponding image application parameters based on the application scenario identifiers;

[0025] The initial training images are processed according to the image application parameters to generate application images, resulting in target training images that match the application scenario identifier.

[0026] The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in the initial image quality enhanced image;

[0027] Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model.

[0028] The updated image enhancement model is used as the initial image enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met. Then, the target image enhancement model corresponding to the application scene identifier is obtained. The target image enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0029] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0030] Obtain the initial training images and application scenario identifiers, and obtain the corresponding image application parameters based on the application scenario identifiers;

[0031] The initial training images are processed according to the image application parameters to generate application images, resulting in target training images that match the application scenario identifier.

[0032] The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in the initial image quality enhanced image;

[0033] Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model.

[0034] The updated image enhancement model is used as the initial image enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met. Then, the target image enhancement model corresponding to the application scene identifier is obtained. The target image enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0035] The aforementioned image quality enhancement model training method, apparatus, computer equipment, storage medium, and computer program product acquire an initial training image and an application scene identifier, and obtain corresponding image application parameters based on the application scene identifier; generate an application image from the initial training image according to the image application parameters to obtain a target training image matching the application scene identifier; input the target training image into the initial image quality enhancement model for image quality enhancement to obtain an initial image quality enhanced image; perform loss calculation based on the initial image quality enhanced image and the initial training image to obtain loss information; update the initial image quality enhancement model based on the loss information to obtain an updated image quality enhancement model; use the updated image quality enhancement model as the initial image quality enhancement model, and return to the step of acquiring the initial training image. The process continues until the training completion condition is met, resulting in a target image quality enhancement model corresponding to the application scenario identifier. This model is used to enhance the image quality of the image corresponding to the application scenario identifier. Specifically, it obtains the corresponding image application parameters through the application scenario identifier, then uses these parameters to obtain a target training image that matches the application scenario identifier. Finally, it uses the target training image to train the initial image quality enhancement model, thus obtaining the target image quality enhancement model. This allows the trained target image quality enhancement model to be applicable to the corresponding application scenario, thereby improving its generalization performance. Finally, the target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scenario identifier, improving the image quality enhancement effect in the application scenario corresponding to the application scenario identifier. Attached Figure Description

[0036] Figure 1 This is a diagram illustrating the application environment of an image quality enhancement model training method in one embodiment.

[0037] Figure 2 This is a flowchart illustrating a method for training an image quality enhancement model in one embodiment;

[0038] Figure 3 This is a flowchart illustrating the process of establishing a first association in one embodiment;

[0039] Figure 4 This is a flowchart illustrating the process of establishing a second association in one embodiment;

[0040] Figure 5 This is a schematic diagram of the process for obtaining an initial image quality enhancement image in one embodiment;

[0041] Figure 6This is a schematic diagram of the image enhancement model architecture in a specific embodiment;

[0042] Figure 7 This is a schematic diagram illustrating the process of obtaining the target image quality enhancement model in one embodiment;

[0043] Figure 8 This is a schematic diagram of the training process for an image enhancement model in a specific embodiment;

[0044] Figure 9 This is a schematic diagram comparing test results in a specific embodiment;

[0045] Figure 10 This is a schematic diagram comparing test results in another specific embodiment;

[0046] Figure 11 This is a structural block diagram of an image quality enhancement model training device in one embodiment;

[0047] Figure 12 This is an internal structural diagram of a computer device in one embodiment;

[0048] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0051] The solutions provided in this application involve artificial intelligence technologies such as image processing and video processing, which are specifically illustrated through the following embodiments:

[0052] The image quality enhancement model training method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed in the cloud or on another server. The process involves: acquiring initial training images and application scene identifiers; obtaining corresponding image application parameters based on the application scene identifiers; generating application images from the initial training images according to the image application parameters to obtain target training images matching the application scene identifiers; inputting the target training images into the initial image quality enhancement model for image quality enhancement to obtain an initial image quality enhanced image; calculating the loss between the initial image quality enhanced image and the initial training image to obtain loss information; updating the initial image quality enhancement model based on the loss information to obtain an updated image quality enhancement model; using the updated image quality enhancement model as the initial image quality enhancement model and returning to the step of acquiring the initial training images, until the training completion condition is met, resulting in the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0053] In one embodiment, such as Figure 2 As shown, a method for training an image quality enhancement model is provided, and this method is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0054] Step 202: Obtain the initial training image and application scenario identifier, and obtain the corresponding image application parameters based on the application scenario identifier.

[0055] Here, the initial training image refers to the high-quality image initially acquired for training; this image serves as the training label. The application scenario identifier identifies the application scenario for which image quality enhancement will be performed; different application scenarios have different identifiers. Image application parameters are the parameters used when generating images within the application scenario corresponding to the identifier; different application scenarios have different image application parameters. These parameters can be those used by the application scenario device during image acquisition, encoding, and preprocessing. The image application parameters match the application scenario identifier.

[0056] Specifically, the server can obtain initial training images from a database, from a data service provider, or from images uploaded by the terminal. Simultaneously, the server obtains the application scenario identifier for generating the corresponding image enhancement model; this identifier can be obtained from the terminal, the database, or the internet. Then, the server retrieves the corresponding image application parameters based on the application scenario identifier. This can be done by searching the database for already saved image application parameters, or by using a pre-defined association between application scenario identifiers and image application parameters. Alternatively, the server can retrieve image application parameters from the corresponding application scenario device based on the application scenario identifier.

[0057] Step 204: Generate application images from the initial training images according to the image application parameters to obtain target training images that match the application scenario identifier.

[0058] The target training image refers to the image used when training the image quality enhancement model. This target training image is matched with the application scenario corresponding to the application scenario identifier. Different application scenarios have different image application parameters, resulting in different target training images. These target training images are encoded and compressed images, and are low-quality images.

[0059] Specifically, the server applies image application parameters to the initial training images. This means that the initial training images are used to simulate the generation of application scenario images using the image application parameters. Specifically, the parameters used when acquiring, encoding, and preprocessing images using the application scenario device are adjusted to obtain target training images that match the application scenario identifier. In one embodiment, when the initial training image set is obtained, each initial training image in the initial training image set can be adjusted using the image application parameters to obtain the target training image set. Then, the target training image set can be split into a training image subset, a validation image subset, and a test image subset. The target training images in the training image subset are then used to train the image quality enhancement model. The trained image quality enhancement model is then validated using the target training images in the validation image subset. Finally, the validated image quality enhancement model is tested using the test image subset. When the test is passed, the final trained image quality enhancement model is obtained.

[0060] Step 206: Input the target training image into the initial image quality enhancement model to enhance the image quality and obtain the initial image quality enhanced image.

[0061] The initial image enhancement model refers to the image enhancement model whose parameters are initialized. This model is used to enhance a low-quality image to a corresponding high-quality image. This image enhancement model is built using a deep learning network from artificial intelligence. The initial image enhancement image refers to the image with enhanced quality output from the initial image enhancement model during training. This initial image enhancement image is obtained by enhancing the target training image.

[0062] Specifically, the server inputs the target training image into the initial image enhancement model for forward propagation. This means the image quality is enhanced through the deep learning neural network within the initial image enhancement model, resulting in the output image with enhanced initial quality. The deep learning neural network mimics the human brain through a combination of data inputs, weights, and biases. These elements work together to accurately identify, classify, and describe objects in the data.

[0063] Step 208: Calculate the loss based on the initial image enhancement image and the initial training image to obtain loss information. Update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model.

[0064] The loss information is used to characterize the error between the initial image enhancement image and the initial training image; the smaller the error, the more accurate the trained model. Updating the image enhancement model refers to the image enhancement model after updating its parameters.

[0065] Specifically, the server uses a loss function to calculate the error between the initial image enhancement image and the initial training image to obtain loss information. Then, it uses the loss information to update the initial parameters in the initial image enhancement model in reverse using a gradient descent algorithm. When the parameter update is complete, the updated image enhancement model is obtained.

[0066] Step 210: Use the updated image quality enhancement model as the initial image quality enhancement model, and return to the step of obtaining the initial training image until the training completion condition is met, to obtain the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0067] The training completion conditions refer to the conditions under which model training is completed, including but not limited to reaching the maximum number of iterations, the model loss information reaching a preset threshold, or the model parameters no longer changing. The target image quality enhancement model refers to the trained image quality enhancement model, which is applied to the application scenario corresponding to the application scenario identifier.

[0068] Specifically, when the server determines whether the training completion condition has been met, if the condition has not been met, it updates the image quality enhancement model as the initial image quality enhancement model and returns to the step of obtaining the initial training image for iterative execution until the server determines that the training completion condition has been met. At this point, the image quality enhancement model at the time of meeting the training completion condition is used as the final target image quality enhancement model, which is the target image quality enhancement model corresponding to the application scenario identifier. Then, the target image quality enhancement model can be deployed to the application scenario corresponding to the application scenario identifier. When image quality enhancement is needed, the image in the application scenario is obtained, and the image is input into the target image quality enhancement model for image quality enhancement to obtain the enhanced image.

[0069] The above-described image quality enhancement model training method involves: acquiring an initial training image and an application scene identifier; obtaining corresponding image application parameters based on the application scene identifier; generating an application image from the initial training image according to the image application parameters to obtain a target training image matching the application scene identifier; inputting the target training image into the initial image quality enhancement model for image quality enhancement to obtain an initial image quality enhanced image; calculating a loss between the initial image quality enhanced image and the initial training image to obtain loss information; updating the initial image quality enhancement model based on the loss information to obtain an updated image quality enhancement model; using the updated image quality enhancement model as the initial image quality enhancement model, and returning the result of acquiring the initial training image. The process is repeated until the training completion condition is met, resulting in the target image quality enhancement model corresponding to the application scenario identifier. This involves obtaining the corresponding image application parameters through the application scenario identifier, using these parameters to obtain the target training image that matches the application scenario identifier, and then using the target training image to train the initial image quality enhancement model. This process ensures that the trained target image quality enhancement model is applicable to the corresponding application scenario, thereby improving its generalization performance. Finally, the target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scenario identifier, thus improving the image quality enhancement effect in the application scenario corresponding to the application scenario identifier.

[0070] In one embodiment, such as Figure 3 As shown, before step 202, before obtaining the initial training image and application scenario identifier, and before obtaining the corresponding image application parameters based on the application scenario identifier, the following steps are also included:

[0071] Step 302: Obtain the application scenario video corresponding to each application scenario identifier.

[0072] The application scenario identifier identifies the specific application scenario in which the image or video is used. This application scenario can be an entertainment application, such as a live video streaming platform, short video platform, or video platform. It can also be a video service scenario based on the open internet, such as television services or movie services. The application scenario video refers to the actual video of the application scenario corresponding to the application scenario identifier. This application scenario video is a low-resolution video, representing the encoded and compressed video obtained during the actual application.

[0073] Specifically, the server can retrieve application scenario videos corresponding to each application scenario identifier from the database. The server can also retrieve application scenario videos from the corresponding application scenarios based on the application scenario identifier, for example, from the application scenario's business server. Another example is retrieving application scenario videos uploaded through application scenario devices.

[0074] Step 304: Analyze the application scenario video using a preset video analysis command to obtain the video generation parameters corresponding to the application scenario video, and use the video generation parameters as image application parameters.

[0075] Among them, the preset video analysis command pair refers to the pre-set commands used for video analysis. The video generation parameters refer to the parameters used by the application scenario when generating application scenario videos.

[0076] Specifically, the server uses pre-configured video analysis commands to invoke video analysis tools to analyze the application scenario videos. These video analysis tools are used for video analysis and can be tools such as FFmpeg (a multimedia codec framework) or Intel's Video Pro Analyzer (an audio and video analysis tool) to obtain the parameters set during the encoding of the application scenario video. Then, the video generation parameters corresponding to the application scenario video are obtained, and these video generation parameters are directly used as image application parameters. The server analyzes the application scenario video corresponding to each application scenario identifier to obtain the image application parameters corresponding to each application scenario identifier.

[0077] Step 306: Establish the first association between each application scenario identifier and the corresponding image application parameters.

[0078] Specifically, the server associates each application scenario identifier with its corresponding image application parameters to obtain a first association relationship, and then saves this first association relationship to the database. This first association relationship is established based on the image application parameters obtained from video analysis using video analysis tools.

[0079] Step 202: Obtain the corresponding image application parameters based on the application scenario identifier, including:

[0080] Find the image application parameters corresponding to the application scenario identifier based on the first association relationship.

[0081] Specifically, when retrieving image application parameters, the server can directly search the database for image application parameters associated with the application scenario identifier. Alternatively, the server can find the image application parameters corresponding to the application scenario identifier in the first association relationship.

[0082] In one embodiment, such as Figure 4 As shown, before step 202, that is, before obtaining the initial training image and application scene identifier, and before obtaining the corresponding image application parameters based on the application scene identifier, the following steps are also included:

[0083] Step 402: Obtain the application scenario video corresponding to each application scenario identifier.

[0084] Step 404: Obtain the corresponding video encoding format based on the application scenario video, determine the video generation parameters corresponding to the application scenario video based on the video encoding format, and use the video generation parameters as image application parameters.

[0085] Among them, video encoding format refers to the format used when encoding video. Different application scenarios select the corresponding video encoding format to obtain videos for different application scenarios.

[0086] Specifically, the server can retrieve the application scenario videos corresponding to each application scenario identifier from the database. Then, it obtains the corresponding video encoding format based on the application scenario video, performs video analysis based on the video encoding format to determine the video generation parameters corresponding to the application scenario video, and then uses the video generation parameters as image application parameters. By analyzing the application scenario video corresponding to each application scenario identifier using the corresponding video encoding format, the image application parameters corresponding to each application scenario identifier are obtained.

[0087] Step 406: Establish a second association between each application scenario identifier and the corresponding image application parameters.

[0088] Specifically, the server associates each application scenario identifier with its corresponding image application parameters to obtain a second association relationship, which is then saved to the database. This second association relationship is established based on the image application parameters obtained through video analysis of the video encoding format.

[0089] Step 202: Obtain the corresponding image application parameters based on the application scenario identifier, including:

[0090] The image application parameters corresponding to the application scenario identifier are found based on the second association relationship.

[0091] Specifically, when retrieving image application parameters, the server can directly search the database for image application parameters associated with the application scenario identifier. Alternatively, the server can find the image application parameters corresponding to the application scenario identifier in the second association relationship.

[0092] In the above embodiments, by acquiring the application scenario videos corresponding to each application scenario identifier, analyzing the application scenario videos to obtain video generation parameters, and then using the video generation parameters as image application parameters for subsequent use, the accuracy of the obtained image application parameters is improved.

[0093] In one embodiment, step 202, obtaining the corresponding image application parameters based on the application scenario identifier, includes the following steps:

[0094] When the application scenario identifier is a specific application scenario identifier, obtain the image generation device identifier corresponding to the specific application scenario identifier; obtain the image generation parameters corresponding to the image generation device identifier, and use the image generation parameters as the image application parameters corresponding to the specific application scenario identifier.

[0095] The specific application scenario identifier is a unique identifier for a specific application scenario, which refers to a pre-defined application scenario that requires the use of its own device to obtain image generation parameters. The image generation device identifier uniquely identifies the device used for image generation. This image generation device is used within the specific application scenario identified by the specific application scenario identifier.

[0096] Specifically, when training an image enhancement model for a specific application scenario, the obtained application scenario identifier is called the specific application scenario identifier. At this time, the image generation device identifier corresponding to this specific application scenario identifier is obtained, and then the image generation parameters corresponding to this image generation device identifier are obtained according to a pre-set correspondence between image generation device identifiers and image generation parameters. The server can pre-save the correspondence between image generation device identifiers and image generation parameters in a database. When needed, the server looks up the image generation parameters corresponding to the image generation device identifier in the database and then uses these image generation parameters as the image application parameters corresponding to the specific application scenario identifier.

[0097] In one embodiment, the target training image may also be obtained by inputting the initial training image into the image generation device corresponding to the image generation device identifier for image generation.

[0098] In the above embodiments, the image generation device identifier corresponding to the specific application scenario identifier is obtained; then the corresponding image generation parameters are obtained according to the image generation device identifier, and the image generation parameters are used as the image application parameters corresponding to the specific application scenario identifier, so that the obtained image application parameters are more accurate.

[0099] In one embodiment, image application parameters include image preprocessing parameters and image compression coding parameters;

[0100] Step 204: Generate application images from the initial training images according to the image application parameters to obtain target training images that match the application scene identifier. This includes the following steps:

[0101] The initial training image is preprocessed using image preprocessing parameters to obtain a preprocessed image; the preprocessed image is then compressed and encoded using image compression coding parameters to obtain the target training image for application scenario identification matching.

[0102] Image preprocessing parameters refer to the parameters used during image preprocessing. These parameters are used when generating images in the application scenario, including the acquisition parameters used when collecting images and the parameters used for manual manipulation of the images after acquisition. A preprocessed image is the image obtained after processing the initial training image using these image preprocessing parameters.

[0103] Specifically, the server simulates the environment in which the application scenario generates images, that is, it processes the initial training image using image application parameters. Specifically, the initial training image can be preprocessed using image preprocessing parameters to obtain a preprocessed image, and then the preprocessed image can be compressed and encoded using image compression coding parameters to obtain the target training image that matches the application scenario identifier. The main parameters of image compression coding include: coding kernel algorithm, profile (related to coding speed and quality), ref (controls the size of the decoded image buffer), bitrate (target bitrate), width, height, frame rate (video frame rate), color space, bit depth, qp (fixed quantization value), keyint (keyframe interval), bframes (B-frames), aq-mode (adaptive quantizer mode), me / subme (subpixel estimation complexity), aq-strength (bitrate control parameter), rc-lookahead (the number of frames used for mb-tree bitrate control and vbv-lookahead), deblock (deblocking filtering), partitions, b-adapt (setting the flexible B-frame configuration decision algorithm), sao (transmission parameter), wpp (parallelism), crf (fixed bitrate coefficient), ipratio (I-frame quantization value), pbratio (bitrate control parameter), vbv-maxrate (maximum bitrate to fill the VBV buffer), etc.

[0104] In the above embodiments, by applying image preprocessing parameters and image compression coding parameters to the initial training image, a target training image matching the application scenario identifier is obtained. Since the image preprocessing parameters and image compression coding parameters are consistent with the image generation parameters in the application scenario corresponding to the application scenario identifier, the obtained target training image conforms to the actual production environment of the application scenario. Then, the initial image quality enhancement model trained using the target training image improves the generalization performance of the trained image quality enhancement model.

[0105] In one embodiment, image preprocessing parameters include image degradation parameters and object setting parameters;

[0106] The initial training images are preprocessed using image preprocessing parameters to obtain preprocessed images, including the following steps:

[0107] The initial training image is degraded using image degradation parameters to obtain a degraded image; the degraded image is then processed using object setting parameters to obtain a preprocessed image.

[0108] Image degradation refers to the decline in image quality during the formation, recording, processing, and transmission of images due to imperfections in the imaging system, recording equipment, transmission medium, and processing methods. Image degradation algorithms can be used to degrade the initial training images. These algorithms typically include adding noise, blurring the image using a blur kernel, upsampling / downsampling, and image compression.

[0109] Object setting parameters refer to the parameters used by an object when setting an image. For example, an object can perform operations such as beautification, image quality enhancement, and adding special effects on an image, and the parameters corresponding to these operations are used as object setting parameters. This object can be a real object, such as a person, or a virtual object, such as a virtual streamer.

[0110] Specifically, the server uses image degradation parameters and an image degradation algorithm to degrade the initial training image. This involves applying a degradation operator or degradation system to the initial training image, then superimposing it with noise to form the degraded image, i.e., the recommended image. The degradation operator, degradation system, and noise are obtained based on the image degradation parameters. Then, object setting parameters are used to set the image in the degraded image to obtain the preprocessed image.

[0111] In the above embodiments, by applying image degradation parameters, object setting parameters, and image compression coding parameters to the initial training image, it is possible to simulate the generation of images in the application scenario, thereby improving the accuracy of the target training image.

[0112] In one embodiment, the initial image enhancement model includes an initial super-resolution model, which includes a feature extraction network, a mapping network, and a reconstruction network.

[0113] like Figure 5 As shown, step 206 involves inputting the target training image into the initial image quality enhancement model for image quality enhancement, resulting in an initial enhanced image, including:

[0114] Step 502: Input the target training image into the feature extraction network to extract features and obtain the initial image features.

[0115] The target training images are generated using image application parameters applied to the initial training images. These initial training images can be obtained from various high-definition data sources, such as open-source datasets including FDDB (Face Image Dataset), lickr2K (Super-Resolution Image Enhancement Dataset), div2K (Super-Resolution Image Enhancement Dataset), Vimeo8K (Image Super-Resolution Dataset), Middlebury (Binocular Stereo Matching Test Dataset), and UCF101 (Action Recognition Dataset). These open-source data sources contain high-quality images, which can refer to 2K / 4K / 8K resolution images or images from videos at 30FPS / 60FPS / 120FPS (frames per second). The images from the open-source datasets can then be processed using image application parameters to obtain the target training image set. This target training image set consists of low-quality images, which can be 480P / 720P / 1080P images. These low-quality images can also be images with low bitrates, blurry noise, jitter, compression block artifacts, and other image quality losses. A feature extraction network is a neural network used for image feature extraction; this neural network can be a convolutional neural network. The initial image features refer to the image features extracted during training. The low-resolution image has lower image quality compared to the high-resolution image. For example, a low-resolution image might be a 1080P image, while a high-resolution image might be an 8K resolution image; conversely, a low-resolution image might be a 2K resolution image, while a high-resolution image might be an 8K resolution image.

[0116] Specifically, the target training image of the server is input into the initial image quality enhancement model. The image features are extracted by the feature extraction network in the initial image quality enhancement model to obtain the initial image features, which can be represented by a feature map.

[0117] Step 504: Input the initial image features into the mapping network for feature mapping to obtain the mapped features.

[0118] The mapping network is used for feature mapping, which is achieved through convolutional operations. This mapping can be either non-linear or linear. The mapped features are the features obtained after mapping through convolutional operations during training.

[0119] Specifically, the server inputs the initial image features into the mapping network in the initial image enhancement model. This mapping network obtains the output mapping features by performing convolution operations on the output initial image features.

[0120] Step 506: Input the mapped features into the reconstruction network to reconstruct the image and obtain the initial image quality enhancement image.

[0121] Image reconstruction is used to process a low-quality image or image sequence to recover a high-quality image. The initial image enhancement image refers to the high-quality image obtained during training.

[0122] Specifically, the server uses the mapped features as input to the reconstruction network, and performs image reconstruction through the reconstruction network to obtain the initial image quality enhancement output.

[0123] In the above embodiments, the accuracy of the obtained initial image enhancement image can be improved by using the feature extraction network, mapping network and reconstruction network in the initial image enhancement model to train the initial image enhancement image.

[0124] In one embodiment, step 502, which involves inputting the target training image into a feature extraction network for feature extraction to obtain initial image features, includes the following steps:

[0125] When the size of the target training image is smaller than the preset target size, the target training image is enlarged to obtain a preprocessed image that conforms to the preset target size; the preprocessed image that conforms to the preset target size is input into the feature extraction network for feature extraction to obtain the initial image features.

[0126] The size of the target training image can be the same as or different from the size of the initial training image. The preset target size refers to the pre-set image size, which is the fixed size of the input image to the image quality enhancement model.

[0127] Specifically, the server determines the relationship between the size of the target training image and the preset target size. When the size of the target training image is smaller than the preset target size, it means that the target training image cannot be directly used as the input of the image quality enhancement model. In this case, the target training image is enlarged using image enlargement algorithms, such as bicubic interpolation or neural network algorithms, to enlarge the target training image to the preset target size, resulting in a preprocessed image that conforms to the preset target size. This preprocessed image is then used as the input of the feature extraction network in the initial image quality enhancement model, and features are extracted through the feature extraction network to obtain the initial image features, thereby making the obtained initial image features more accurate.

[0128] In a specific embodiment, such as Figure 6 As shown, a schematic diagram of the architecture of a super-resolution model is provided. The super-resolution model built by a convolutional neural network (CNN) includes a three-layer fully convolutional neural network. Through block extraction and representation layers, features are extracted from the output low-resolution image to obtain a series of feature maps, as shown in the following formula (1):

[0129] F1(Y)=max(0, W1*Y+B1) Formula (1)

[0130] Where W1 and B1 represent the weights and biases of the filter (convolution kernel), the max operation corresponds to the ReLU activation function, the convolution and activation of the CNN, F1 is the feature map of the output, and Y refers to the input.

[0131] Then, nonlinear mapping is performed through a nonlinear mapping layer, which is implemented using a convolutional neural network, as shown in formula (2) below:

[0132] F2(Y)=max(0, W2*F1(Y)+B2) Formula (2)

[0133] Here, W2 and B2 still represent the filter weights and biases, the max operation corresponds to the ReLU activation function, and the convolution and activation in a CNN. F2 is the output mapping feature, and Y is the input.

[0134] Then, image reconstruction is performed through a reconstruction layer, which also involves convolution operations. This can be achieved using the following formula (3).

[0135] F(Y)=W3*F2(Y)+B3 Formula (3)

[0136] Here, W3 and B3 still represent the filter weights and biases, F is the high-resolution output image, and Y refers to the input. By using block extraction and representation layers to extract image features, performing nonlinear mapping through nonlinear mapping layers, and finally reconstructing the image through reconstruction layers, the accuracy of the output image is improved.

[0137] In one embodiment, after step 210, i.e., after the step of using the updated image quality enhancement model as the initial image quality enhancement model and returning to obtain the initial training image is executed, until the training completion condition is met and the target image quality enhancement model corresponding to the application scenario identifier is obtained, the method further includes:

[0138] Obtain the application image corresponding to the application scenario identifier, input the application image into the target image quality enhancement model corresponding to the application scenario identifier for image quality enhancement, and obtain the target image quality enhancement image corresponding to the application scenario identifier.

[0139] The application image refers to the actual image generated in the application scenario corresponding to the application scenario identifier. This image is of low quality and can be obtained by capturing the image in the application scenario using an image acquisition device, performing preprocessing, and finally compressing and encoding the image. Preprocessing refers to image processing before encoding, such as background blurring, adding effects, and image quality enhancement. In other words, the application image is generated using the image generation parameters of the application scenario corresponding to the application scenario identifier. The target image quality enhancement image refers to the image obtained after image quality enhancement.

[0140] Specifically, when the server needs to enhance the image quality of an image in an application scenario, it can obtain the application image corresponding to the application scenario identifier from the application scenario server. The server can also directly obtain the application image corresponding to the application scenario identifier from the database. Alternatively, the server can obtain the application image corresponding to the application scenario identifier from the business server. The server can also obtain the application image corresponding to the application scenario identifier uploaded by the terminal.

[0141] After training the target image quality enhancement models for each application scenario, the server deploys these models for subsequent use. When a target image quality enhancement model is needed, the server locates the model to be used based on the identifier of the application scenario, invokes that model, and inputs the application image into the target image quality enhancement model corresponding to the application scenario identifier for image quality enhancement, resulting in the output target image quality enhanced image corresponding to the application scenario identifier. In one specific embodiment, the image enhancement model can be applied to image enhancement scenarios such as compression repair, super-resolution frame interpolation, face, font, and ROI (region of interest) processing, and can obtain an image with enhanced image quality. Specifically, the image to be compressed and repaired can be input into the target image enhancement model for compression modification to obtain the image before compression; the image to be super-resolution frame interpolation can be input into the target image enhancement model for compression modification to obtain the super-resolution frame interpolation image; the face image can be input into the target image enhancement model for image quality enhancement to obtain a high-definition face image; or the image can be input into the target image enhancement model for font and ROI (region of interest) processing to enhance image quality, to obtain the image with enhanced font and ROI (region of interest) image.

[0142] In the above embodiments, by using the target image quality enhancement model trained under the same application scenario to enhance the image quality of the application image, the accuracy of the obtained target image quality enhancement image can be improved.

[0143] In one embodiment, obtaining the application image corresponding to the application scenario identifier includes:

[0144] The application video corresponding to the application scenario identifier is obtained. The application video is generated by processing the video application parameters corresponding to the application scenario identifier. The application video is then divided into frames, and each video frame is used as an application image in sequence.

[0145] Application video refers to the video generated after applying the video application parameters corresponding to the application scenario identifier. Video application parameters are the parameters required to generate video within the application scenario corresponding to the application scenario identifier. These parameters can be those used during video encoding and compression, or they can include parameters such as jitter, blur, and noise from the application scenario device when capturing the video.

[0146] Specifically, when the server needs to enhance the image quality of a video in a video application scenario, it first retrieves the application video corresponding to the application scenario identifier from the database, or it can retrieve the application video corresponding to the application scenario identifier from the device in that video application scenario. The server can also retrieve the application video corresponding to the uploaded application scenario identifier from the terminal. The application video is generated by processing the video application parameters corresponding to the application scenario identifier. Then, the server divides the application video into frames, obtaining individual video frames, and uses each video frame sequentially as the application image.

[0147] After step 210, that is, after inputting the application image into the target image quality enhancement model corresponding to the application scene identifier for image quality enhancement, and obtaining the target image quality enhancement image corresponding to the application scene identifier, the process further includes:

[0148] The target image quality enhancement images corresponding to each video frame are combined to obtain the target image quality enhancement video corresponding to the application scene identifier.

[0149] Specifically, the server uses each video frame as an application image and inputs these application images into a target image quality enhancement model for enhancement, resulting in a target image quality enhanced image corresponding to each application image. These target image quality enhanced images are then combined to obtain the target image quality enhanced video corresponding to the application scene identifier. For example, in a live video streaming platform, the live streaming client captures the video of the live stream object and transmits it to the platform. The platform uses each video frame as an application image and inputs it into its corresponding target image quality enhancement model. This model is trained using the platform's video generation parameters to weight the initial training data, resulting in a target training image. The target image enhancement model enhances the image quality to obtain individual high-quality images. These high-quality images are then merged to obtain an enhanced video, which is then sent to the viewing device for playback.

[0150] In the above embodiment, the video is generated using video application parameters. The application video is divided into frames to obtain individual video frames. Each video frame is then used sequentially as an application image for image quality enhancement to obtain individual target image quality enhancement images. Finally, the target image quality enhancement images corresponding to each video frame are combined to obtain the target image quality enhancement video corresponding to the application scene identifier, thereby improving the accuracy of the obtained target image quality enhancement video.

[0151] In one embodiment, the application scenario identifier includes at least two; such as Figure 7 As shown, image enhancement model training methods also include:

[0152] Step 702: Obtain the initial training image and at least two application scenario identifiers, and obtain the corresponding image application parameters based on the at least two application scenario identifiers.

[0153] Specifically, when generating image enhancement models for multiple application scenarios, the server can first obtain the identifiers of each application scenario, and then obtain the image application parameters corresponding to each application scenario identifier. This can be achieved by obtaining the initial training image and at least two application scenario identifiers from a database, from a business server, or by uploading the initial training image and at least two application scenario identifiers from a terminal. Then, the corresponding image application parameters can be obtained based on the application scenario identifiers. These image application parameters can be obtained from a database, from an application scenario server, or uploaded from a terminal.

[0154] Step 704: Generate application images from the initial training images according to the image application parameters to obtain target training images that match at least two application scenario identifiers.

[0155] Specifically, the server uses image application parameters for different application scenarios to weight the initial training images, obtaining target training images matching each application scenario. The server can weight multiple initial training images to obtain a target training image dataset matching each application scenario.

[0156] Step 706: Input the target training image into the initial image quality enhancement model to enhance the image quality and obtain the initial image quality enhanced image.

[0157] Step 708: Calculate the loss based on the initial image enhancement image and the initial training image to obtain loss information. Update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model.

[0158] Specifically, the server trains the initial image enhancement model using target training images from different application scenarios, and then uses the gradient descent algorithm to update the initial image enhancement model in reverse based on the loss information obtained during training, thus obtaining the updated image enhancement model.

[0159] Step 710: Use the updated image quality enhancement model as the initial image quality enhancement model, and return to the step of obtaining the initial training image until the training completion condition is met, so as to obtain at least two target image quality enhancement models corresponding to the application scenario identifiers respectively.

[0160] Specifically, the server uses target training images for each application scenario to iteratively train the initial image quality enhancement model. When the training completion condition is met, the initial image quality enhancement model at that point is used as the target image quality enhancement model, thus obtaining the target image quality enhancement model for each application scenario. The target image quality enhancement model for each application scenario can be trained sequentially or simultaneously. Then, the target image quality enhancement model can be deployed and used. Application scenarios for this target image quality enhancement model include, but are not limited to, short video platforms, live streaming platforms, video platforms, and various video and data service devices based on the open internet.

[0161] In one embodiment, a total dataset containing target training images for all application scenarios can be obtained using target training image datasets matched to various application scenarios. Then, an image quality enhancement model is trained using this total dataset, resulting in a total image quality enhancement model applicable to all application scenarios. This total image quality enhancement model can then be used to enhance the image quality in each application scenario, thereby improving the generalization performance of the image quality enhancement model.

[0162] In the above embodiments, different target training images are generated for different application scenarios, and then target image quality enhancement models for different application scenarios are trained using the target training images of different application scenarios, thereby improving the generalization performance of the target image quality enhancement models.

[0163] In a specific embodiment, such as Figure 8 As shown, a method for training an image quality enhancement model is provided, which specifically includes the following steps:

[0164] Step 802: Obtain the application scenario video corresponding to each application scenario identifier; analyze the application scenario video using a preset video analysis command to obtain the video generation parameters corresponding to the application scenario video, use the video generation parameters as image application parameters, and establish the first association relationship between each application scenario identifier and the corresponding image application parameters.

[0165] Step 804: Obtain the initial training image and the target application scene identifier, and find the image application parameters corresponding to the target application scene identifier according to the first association relationship; perform image degradation on the initial training image through the image degradation parameters in the image application parameters to obtain the degraded image;

[0166] Step 806: Apply the object setting parameters in the image application parameters to the degraded image to obtain a preprocessed image; apply the image compression coding parameters in the image application parameters to compress and encode the preprocessed image to obtain a target training image for target application scene identifier matching.

[0167] Step 808: Input the target training image into the initial image quality enhancement model, extract features through the feature extraction network in the initial image quality enhancement model to obtain initial image features, input the initial image features into the mapping network in the initial image quality enhancement model to perform feature mapping to obtain mapped features, input the mapped features into the reconstruction network in the initial image quality enhancement model to perform image reconstruction to obtain the initial image quality enhancement image.

[0168] Step 810: Calculate the loss based on the initial image enhancement image and the initial training image to obtain loss information; update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model.

[0169] Step 812: Use the updated image quality enhancement model as the initial image quality enhancement model, and return to the step of obtaining the initial training image until the training completion condition is met, and obtain the target image quality enhancement model corresponding to the target application scene identifier.

[0170] In the above embodiments, by finding the image application parameters corresponding to the target application scenario, generating the target training image using the image application parameters from the initial training image, and then training the target image quality enhancement model using the target training image, the trained target image quality enhancement model can improve the accuracy of image quality enhancement for the target application scenario, thereby improving the generalization performance of the target image quality enhancement model.

[0171] In one specific embodiment, the trained target image quality enhancement model can be compared and tested in aspects such as compression restoration, super-resolution frame interpolation, face, font, and ROI processing. The comparison test results of the target image quality enhancement model's generalization performance in compression restoration with related technologies are as follows: Figure 9 As shown in the figure. Peak signal-to-noise ratio (PSRN) and structural similarity (SSIM) were used as evaluation metrics. The LIVE1 (image dataset 1), BSDS (image dataset 2), and Urban (image dataset 3) datasets were used as the initial training image datasets. Image quality enhancement was compared with existing related techniques A, B, C, D, and E. It is evident that the generalization performance of this application is significantly better than existing related techniques. Furthermore, the comparative test results of the target image quality enhancement model's generalization performance in super-resolution frame interpolation with related techniques are shown below. Figure 10 As shown. Among them, the generalization performance of the target image quality enhancement model 1 and target image quality enhancement model 2 established by this application using different neural network algorithms is significantly better than that of existing related technologies.

[0172] In one specific embodiment, the method of using this image quality enhancement model is applied to a television broadcasting platform. Specifically, it involves obtaining the hardware device parameters and configuration parameters used during the frequency source editing and encoding of the television broadcasting platform equipment. Then, it obtains an initial training image. Using the same hardware device parameters and configuration parameters used during the frequency source editing and encoding of the television broadcasting platform equipment, it applies the same hardware device parameters and configuration parameters to the initial training image to obtain a target training image matching the television broadcasting platform. This target training image is then input into the initial image quality enhancement model for image quality enhancement, resulting in an initial enhanced image. The initial enhanced image is then compared with the initial training image to calculate the loss information. Based on this loss information, the initial image quality enhancement model is updated in reverse, resulting in an updated image quality enhancement model. This updated model is then used as the initial enhancement model, and the process of obtaining the initial training image is repeated until the training completion condition is met, thus obtaining the target image quality enhancement model corresponding to the television broadcasting platform. When a television broadcasting platform wants to push a video, it can enhance the video's quality. This involves acquiring the video, dividing it into frames, and then sequentially inputting each frame into the corresponding target quality enhancement model of the television broadcasting platform for enhancement. The resulting target video frames are then merged to obtain the enhanced target video. The television broadcasting platform then pushes the enhanced target video to the target user's terminal, which plays the enhanced video, thereby improving the user experience of the television broadcasting platform.

[0173] In one specific embodiment, the method of using the image quality enhancement model is applied to a video cloud media platform. That is, by acquiring cloud media videos from the video cloud media platform, inputting the cloud media videos into the target image quality enhancement model corresponding to the video cloud media application scenario for image quality enhancement, obtaining image quality enhanced cloud media videos, and then publishing the image quality enhanced cloud media videos through the video cloud media platform, thereby improving the video image quality and thus enhancing the user experience of the video cloud media platform.

[0174] In one specific embodiment, the method of using this image quality enhancement model is applied to a compression and restoration scenario. Specifically, it involves obtaining the compression parameters used during image compression. Then, an initial training image is acquired and compressed using the same compression parameters to obtain a compressed image. This compressed image is then input into the initial image quality enhancement model for image quality enhancement, i.e., compression and restoration, resulting in an initial compressed and restored image. Next, loss calculation is performed between the initial compressed and restored image and the initial training image to obtain loss information. Based on this loss information, the initial image quality enhancement model is updated, resulting in an updated image quality enhancement model. This updated model is then used as the initial image quality enhancement model, and the process returns to the step of acquiring the initial training image. This process continues until the training completion condition is met, resulting in the compression and restoration model corresponding to the compression and restoration scenario. When a video needs compression and restoration, the video to be restored is acquired, divided into video frames, and each video frame is sequentially input into the compression and restoration model corresponding to the compression and restoration scenario for image compression and restoration, resulting in output target video frames. These target video frames are then merged to obtain the compressed and restored target video.

[0175] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0176] Based on the same inventive concept, this application also provides an image quality enhancement model training device for implementing the image quality enhancement model training method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the image quality enhancement model training device provided below can be found in the limitations of the image quality enhancement model training method described above, and will not be repeated here.

[0177] In one embodiment, such as Figure 11 As shown, a training device 1000 for an image quality enhancement model is provided, comprising: a parameter acquisition module 1002, an image generation module 1004, an image quality enhancement module 1006, an update module 1008, and an iteration module 1010, wherein:

[0178] The parameter acquisition module 1002 is used to acquire the initial training image and the application scene identifier, and to acquire the corresponding image application parameters based on the application scene identifier;

[0179] The image generation module 1004 is used to generate an application image from the initial training image according to the image application parameters, so as to obtain a target training image that matches the application scene identifier.

[0180] The image quality enhancement module 1006 is used to input the target training image into the initial image quality enhancement model to enhance the image quality and obtain the initial image quality enhanced image.

[0181] The update module 1008 is used to perform loss calculation based on the initial image enhancement image and the initial training image to obtain loss information, and to update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model.

[0182] The iteration module 1010 is used to take the updated image quality enhancement model as the initial image quality enhancement model and return to the step of obtaining the initial training image until the training completion condition is met, and then obtain the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

[0183] In one embodiment, the image quality enhancement model training device 1000 further includes:

[0184] The first relationship establishment module is used to obtain the application scenario videos corresponding to each application scenario identifier; analyze the application scenario videos using preset video analysis commands to obtain the video generation parameters corresponding to the application scenario videos, and use the video generation parameters as image application parameters; and establish the first association relationship between each application scenario identifier and the corresponding image application parameters.

[0185] The parameter acquisition module 1002 is also used to find the image application parameters corresponding to the application scenario identifier based on the first association relationship.

[0186] In one embodiment, the image quality enhancement model training device 1000 further includes:

[0187] The second relationship establishment module is used to obtain the application scenario video corresponding to each application scenario identifier; obtain the corresponding video encoding format based on the application scenario video; determine the video generation parameters corresponding to the application scenario video based on the video encoding format; use the video generation parameters as image application parameters; and establish a second association relationship between each application scenario identifier and the corresponding image application parameters.

[0188] The parameter acquisition module 1002 is also used to find the image application parameters corresponding to the application scenario identifier based on the second association relationship.

[0189] In one embodiment, the parameter acquisition module 1002 is further configured to acquire the image generation device identifier corresponding to the specific application scenario identifier when the application scenario identifier is a specific application scenario identifier; acquire the image generation parameters corresponding to the image generation device identifier; and use the image generation parameters as the image application parameters corresponding to the specific application scenario identifier.

[0190] In one embodiment, image application parameters include image preprocessing parameters and image compression coding parameters;

[0191] The image generation module 1004 is also used to preprocess the initial training image using image preprocessing parameters to obtain a preprocessed image; and to compress and encode the preprocessed image using image compression coding parameters to obtain a target training image for application scenario identification matching.

[0192] In one embodiment, image preprocessing parameters include image degradation parameters and object setting parameters;

[0193] The image generation module 1004 is also used to degrade the initial training image using image degradation parameters to obtain a degraded image; and to set the image using object setting parameters to obtain a preprocessed image.

[0194] In one embodiment, the initial image enhancement model includes an initial super-resolution model, which includes a feature extraction network, a mapping network, and a reconstruction network.

[0195] The image quality enhancement module 1006 is also used to input the target training image into the feature extraction network for feature extraction to obtain initial image features; input the initial image features into the mapping network for feature mapping to obtain mapped features; and input the mapped features into the reconstruction network for image reconstruction to obtain the initial image quality enhanced image.

[0196] In one embodiment, the image quality enhancement module 1006 is further configured to enlarge the target training image to obtain a preprocessed image that conforms to the preset target size when the size of the target training image is smaller than the preset target size; and input the preprocessed image that conforms to the preset target size into the feature extraction network for feature extraction to obtain initial image features.

[0197] In one embodiment, the image quality enhancement model training device 1000 further includes:

[0198] The image quality enhancement module is used to obtain the application image corresponding to the application scene identifier, input the application image into the target image quality enhancement model corresponding to the application scene identifier for image quality enhancement, and obtain the target image quality enhancement image corresponding to the application scene identifier.

[0199] In one embodiment, the image quality enhancement module is further used to obtain the application video corresponding to the application scenario identifier. The application video is obtained by generating the video after applying the video application parameters corresponding to the application scenario identifier. The application video is divided into frames to obtain each video frame, and each video frame is used as the application image in sequence.

[0200] The image enhancement model training device 1000 also includes:

[0201] The image quality enhancement video acquisition module is used to combine the target image quality enhancement images corresponding to each video frame to obtain the target image quality enhancement video corresponding to the application scene identifier.

[0202] In one embodiment, the application scenario identifiers include at least two; the image quality enhancement model training device 1000 further includes:

[0203] Multiple model training modules are used to acquire initial training images and at least two application scene identifiers, and to acquire corresponding image application parameters based on the at least two application scene identifiers; to generate application images from the initial training images according to the image application parameters, resulting in target training images that match the at least two application scene identifiers; to input the target training images into an initial image quality enhancement model for image quality enhancement, resulting in an initial image quality enhancement image; to perform loss calculation based on the initial image quality enhancement image and the initial training image, to obtain loss information, and to update the initial image quality enhancement model in reverse based on the loss information, resulting in an updated image quality enhancement model; to use the updated image quality enhancement model as the initial image quality enhancement model, and to return to the step of acquiring the initial training images, until the training completion condition is met, resulting in target image quality enhancement models corresponding to the at least two application scene identifiers respectively.

[0204] Each module in the aforementioned image enhancement model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0205] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores initial training images, image application parameters, application scene identifiers, and other data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for training an image quality enhancement model.

[0206] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for training an image quality enhancement model. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0207] Those skilled in the art will understand that Figure 12 or Figure 13The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0208] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0209] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0210] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0211] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0212] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0213] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training an image quality enhancement model, characterized in that, The method includes: Obtain the initial training image and application scenario identifier, and obtain the corresponding image application parameters based on the application scenario identifier; The initial training image is used to generate an application image according to the image application parameters to obtain a target training image that matches the application scene identifier. The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in an initial enhanced image. Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model. The updated image quality enhancement model is used as the initial image quality enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met, so as to obtain the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

2. The method according to claim 1, characterized in that, Before obtaining the initial training image and application scenario identifier, and obtaining the corresponding image application parameters based on the application scenario identifier, the method further includes: Obtain the application scenario video corresponding to each application scenario identifier; The application scenario video is analyzed using a preset video analysis command to obtain the video generation parameters corresponding to the application scenario video, and the video generation parameters are used as the image application parameters. Establish a first association relationship between each application scenario identifier and its corresponding image application parameters; The step of obtaining the corresponding image application parameters based on the application scenario identifier includes: Based on the first association relationship, find the image application parameters corresponding to the application scenario identifier.

3. The method according to claim 1, characterized in that, Before obtaining the initial training image and application scenario identifier, and obtaining the corresponding image application parameters based on the application scenario identifier, the method further includes: Obtain the application scenario video corresponding to each application scenario identifier; Based on the application scenario video, obtain the corresponding video encoding format, determine the video generation parameters corresponding to the application scenario video based on the video encoding format, and use the video generation parameters as the image application parameters. Establish a second association relationship between each application scenario identifier and its corresponding image application parameters; The step of obtaining the corresponding image application parameters based on the application scenario identifier includes: Based on the second association relationship, find the image application parameters corresponding to the application scenario identifier.

4. The method according to claim 1, characterized in that, The step of obtaining the corresponding image application parameters based on the application scenario identifier includes: When the application scenario identifier is a specific application scenario identifier, the image generation device identifier corresponding to the specific application scenario identifier is obtained; Obtain the image generation parameters corresponding to the image generation device identifier, and use the image generation parameters as the image application parameters corresponding to the specific application scenario identifier.

5. The method according to claim 1, characterized in that, The image application parameters include image preprocessing parameters and image compression coding parameters; The step of generating an application image from the initial training image according to the image application parameters to obtain a target training image that matches the application scene identifier includes: The initial training image is preprocessed using the image preprocessing parameters to obtain a preprocessed image; The preprocessed image is compressed and encoded using the image compression coding parameters to obtain the target training image for matching the application scenario identifier.

6. The method according to claim 5, characterized in that, The image preprocessing parameters include image degradation parameters and object setting parameters; The step of preprocessing the initial training image using the image preprocessing parameters to obtain a preprocessed image includes: The initial training image is degraded using the image degradation parameters to obtain a degraded image; The degraded image is processed using the object setting parameters to obtain the preprocessed image.

7. The method according to claim 1, characterized in that, The initial image quality enhancement model includes an initial super-resolution model, which includes a feature extraction network, a mapping network, and a reconstruction network. The step of inputting the target training image into the initial image quality enhancement model for image quality enhancement to obtain the initial image quality enhanced image includes: The target training image is input into the feature extraction network for feature extraction to obtain initial image features; The initial image features are input into the mapping network for feature mapping to obtain the mapped features; The mapping features are input into the reconstruction network to reconstruct the image, resulting in the initial image quality enhancement image.

8. The method according to claim 7, characterized in that, The step of inputting the target training image into the feature extraction network for feature extraction to obtain initial image features includes: When the size of the target training image is smaller than the preset target size, the target training image is enlarged to obtain a preprocessed image that meets the preset target size; The preprocessed image that meets the preset target size is input into the feature extraction network for feature extraction to obtain the initial image features.

9. The method according to claim 1, characterized in that, After the step of using the updated image quality enhancement model as the initial image quality enhancement model and returning to obtain the initial training image is executed until the training completion condition is met, and the target image quality enhancement model corresponding to the application scenario identifier is obtained, the method further includes: Obtain the application image corresponding to the application scenario identifier, input the application image into the target image quality enhancement model corresponding to the application scenario identifier to perform image quality enhancement, and obtain the target image quality enhancement image corresponding to the application scenario identifier.

10. The method according to claim 9, characterized in that, The step of obtaining the application image corresponding to the application scenario identifier includes: Obtain the application video corresponding to the application scenario identifier. The application video is generated by processing the video application parameters corresponding to the application scenario identifier. The application video is divided into frames to obtain individual video frames, and each video frame is used as an application image in sequence. After inputting the application image into the target image quality enhancement model corresponding to the application scene identifier for image quality enhancement, and obtaining the target image quality enhancement image corresponding to the application scene identifier, the method further includes: The target image quality enhancement images corresponding to each video frame are combined to obtain the target image quality enhancement video corresponding to the application scene identifier.

11. The method according to claim 9, characterized in that, The application scenario identifiers include at least two; the method further includes: Obtain initial training images and at least two application scenario identifiers, and obtain corresponding image application parameters based on the at least two application scenario identifiers; The initial training image is used to generate an application image according to the image application parameters to obtain a target training image that matches the at least two application scenario identifiers; The target training image is input into the initial image quality enhancement model to enhance the image quality, resulting in an initial enhanced image. Loss information is calculated based on the initial image enhancement image and the initial training image. The initial image enhancement model is then updated based on the loss information to obtain the updated image enhancement model. The updated image quality enhancement model is used as the initial image quality enhancement model, and the step of obtaining the initial training image is returned to be executed until the training completion condition is met, so as to obtain the target image quality enhancement model corresponding to the at least two application scenario identifiers respectively.

12. A training device for an image quality enhancement model, characterized in that, The device includes: The parameter acquisition module is used to acquire the initial training image and the application scenario identifier, and to acquire the corresponding image application parameters based on the application scenario identifier; The image generation module is used to generate an application image from the initial training image according to the image application parameters, so as to obtain a target training image that matches the application scene identifier; The image quality enhancement module is used to input the target training image into the initial image quality enhancement model to enhance the image quality and obtain the initial image quality enhanced image. The update module is used to perform loss calculation based on the initial image enhancement image and the initial training image to obtain loss information, and to update the initial image enhancement model in reverse based on the loss information to obtain the updated image enhancement model; The iteration module is used to take the updated image quality enhancement model as the initial image quality enhancement model and return to the step of obtaining the initial training image until the training completion condition is met, so as to obtain the target image quality enhancement model corresponding to the application scene identifier. The target image quality enhancement model is used to enhance the image quality of the image corresponding to the application scene identifier.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images