Video quality determination method, device and equipment and readable storage medium

Abstract

The invention discloses a video quality determination method, device and equipment and a readable storage medium, and the method comprises the steps: migrating an HDR domain to an SDR domain, combining training optimization, taking a model obtained through SDR video training as a backbone network, and combining a domain migration module to construct a model capable of carrying out the quality evaluation of an HDR video, thereby achieving the quality evaluation of the HDR video. The complexity of model development and maintenance can be reduced, pre-training knowledge of a large-scale standard dynamic range data set can be fully utilized, and dependence on high dynamic range data is reduced.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, apparatus, device and readable storage medium for determining video quality. Background Technology

[0002] Full-reference video quality assessment refers to a method that learns to quantify video quality by comparing the differences between the original reference video and the distorted video. That is, the original reference video and the distorted video are used as inputs, and the resulting quality score is used as the output. The obtained result should be as close as possible to the subjective perception of the human eye.

[0003] However, current high dynamic range (HDR) / standard dynamic range (SDR) cross-domain adaptability is insufficient. For example, many full-reference quality assessment algorithms are designed for SDR videos, while HDR videos differ significantly from SDR videos in pixel distribution, luminance dynamic range, and color space. This often leads to problems such as sluggish detail response and distorted scoring when directly transferring SDR models. Designing different models for HDR and SDR videos separately increases the complexity of model development and maintenance.

[0004] Furthermore, the scarcity of high dynamic range (HDR) data makes training difficult. HDR video quality assessment datasets are relatively scarce, with limited data volume. Training models directly on small-scale HDR data can easily lead to overfitting and poor generalization performance. At the same time, the high cost of annotating HDR videos and the difficulty in obtaining large-scale labeled data further limit the improvement of model performance.

[0005] In conclusion, how to effectively solve problems such as HDR video quality assessment is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this application is to provide a video quality determination method, apparatus, device, and readable storage medium, which uses a model trained on SDR video as the backbone network and combines a domain transfer module to construct a model that can evaluate the quality of HDR video through domain transfer and training optimization. This not only reduces the complexity of model development and maintenance, but also makes full use of the pre-trained knowledge of large-scale standard dynamic range datasets, reducing the dependence on high dynamic range data.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution: A video quality assessment method, comprising: Acquire a target video and determine the dynamic range of the target video; wherein the dynamic range includes HDR and SDR, HDR being high dynamic range and SDR being standard dynamic range; When the dynamic range of the target video is HDR, the target video is mapped using the domain transfer module in the trained video quality assessment model, and then the mapped video is evaluated using the backbone model of the video quality assessment model to obtain the quality assessment result. The process of training the video quality assessment model includes: The backbone network was trained using an SDR video training set for quality assessment. After completing the training of the backbone network, the domain migration module is added to the backbone network. During the quality assessment training of the video quality assessment model using a mixed video set including SDR and HDR videos, if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network is used to process the mapped video. The video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model.

[0008] Preferably, the video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: Freeze the model parameters of the backbone network, and fine-tune the domain transfer module in the video quality assessment model using the HDR video training set to obtain the video quality assessment model; Correspondingly, it also includes: With the target video having a dynamic range of SDR, the target video is evaluated using the backbone model of the trained video quality assessment model to obtain the quality assessment result.

[0009] Preferably, the video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: The domain transfer module and the backbone network are fine-tuned using the HDR video training set to obtain the video quality assessment model.

[0010] Preferably, the domain migration module includes a plurality of residual blocks, wherein the residual blocks include convolutional layers, activation functions and residual connections.

[0011] Preferably, the video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: Using the aforementioned HDR video training set, training samples and validation samples are obtained; The video quality assessment model was fine-tuned using the training samples. The video quality assessment model was validated using the validation samples. After successful verification, the current video quality assessment model will be determined as the video quality assessment model.

[0012] Preferably, using the SDR video training set, training samples and validation samples are obtained, including: Randomly extract frames from the SDR videos in the SDR video training set to obtain the training samples; The verification samples are obtained by performing fixed frame extraction on the SDR videos in the SDR video training set.

[0013] Preferably, after mapping the target video using the domain transfer module in the trained video quality assessment model, the backbone model of the video quality assessment model is used to evaluate the mapped video to obtain a quality assessment result, including: The current frame is obtained by extracting the reference video and the distorted video corresponding to the target video frame by frame; The domain migration module is used to map the high dynamic range features of the current frame to the standard dynamic range domain. The backbone network is used to perform feature extraction, feature fusion, and quality regression processing on the image features mapped to the standard dynamic range domain to obtain the single-frame evaluation result of the current frame; The quality assessment result is obtained by aggregating the single-frame evaluation results corresponding to each frame of the target video.

[0014] A video quality determination device, comprising: A video acquisition module is used to acquire a target video and determine the dynamic range of the target video; wherein the dynamic range includes HDR and SDR, HDR being high dynamic range and SDR being standard dynamic range; The HDR evaluation module is used to map the target video using the domain transfer module in the trained video quality evaluation model when the target video has a dynamic range of HDR, and then evaluate the mapped video using the backbone model of the video quality evaluation model to obtain a quality evaluation result. The model training module includes: a first-stage training unit, a second-stage training unit, and a third-stage training unit; The first training unit is used to perform quality evaluation training on the backbone network using the SDR video training set. The second-stage training unit is used to add the domain transfer module to the backbone network after the backbone network training is completed; during the quality assessment training of the video quality assessment model using a mixed video set including SDR video and HDR video, if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network is used to process the mapped video. The third-stage training unit is used to fine-tune the video quality assessment model using the HDR video training set to obtain the video quality assessment model.

[0015] An electronic device, comprising: Memory, used to store computer programs; A processor is used to implement the steps of the video quality determination method described above when executing the computer program.

[0016] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the video quality determination method described above.

[0017] The method provided in this application provides an example of acquiring a target video and determining its dynamic range. The dynamic range includes HDR and SDR, where HDR is High Dynamic Range and SDR is Standard Dynamic Range. When the target video's dynamic range is HDR, the target video is mapped using the domain transfer module in the trained video quality assessment model. The mapped video is then evaluated using the backbone model of the video quality assessment model to obtain a quality assessment result. The process of training the video quality assessment model includes: training the backbone network for quality assessment using an SDR video training set; adding a domain transfer module to the backbone network after training; training the video quality assessment model using a mixed video set including SDR and HDR videos; if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network processes the mapped video; and the video quality assessment model is fine-tuned using the HDR video training set to obtain the final video quality assessment model.

[0018] In this application, a video quality assessment model can be pre-trained. The backbone network of this model can be trained on an SDR video training set. Then, to enable this backbone network to process DHR videos, a domain transfer module can be set up to map HDR videos from a mixed video set including SDR and HDR videos to SDR videos before processing. Finally, the video quality assessment model is fine-tuned based on the HDR video training set to obtain the final model. After obtaining the target video, its dynamic range can be determined, i.e., whether the target video corresponds to HDR or SDR. If the target video corresponds to HDR, the domain transfer module in the trained video quality assessment model can be used to map the target video, and then the backbone model of the video quality assessment model can be used to evaluate the mapped video to obtain the quality assessment result.

[0019] By using a model trained on SDR video as the backbone network through domain transfer and training optimization, and combining it with a domain transfer module to build a model that can evaluate the quality of HDR video, we can not only reduce the complexity of model development and maintenance, but also make full use of the pre-trained knowledge of large-scale standard dynamic range datasets and reduce the dependence on high dynamic range data.

[0020] Accordingly, embodiments of this application also provide a video quality determination apparatus, device, and readable storage medium corresponding to the above-described video quality determination method, which have the aforementioned technical effects, and will not be elaborated further here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the implementation of a video quality determination method in this application. Figure 2 This is a schematic diagram of model training in an embodiment of this application; Figure 3 This is a schematic diagram of a domain migration module structure in an embodiment of this application; Figure 4 This is a flowchart illustrating a specific implementation of a video quality determination method in this application. Figure 5 This is a schematic diagram of the structure of a video quality determination device according to an embodiment of this application; Figure 6This is a schematic diagram of the structure of an electronic device according to an embodiment of this application; Figure 7 This is a schematic diagram of the specific structure of an electronic device in an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Please refer to Figure 1 , Figure 1 This is a flowchart of a video quality determination method according to an embodiment of this application. The method includes the following steps.

[0025] S101. Acquire the target video and determine the dynamic range of the target video.

[0026] Dynamic range includes HDR and SDR, where HDR stands for High Dynamic Range and SDR stands for Standard Dynamic Range.

[0027] It should be noted that, in this embodiment, the video quality is evaluated using a full reference video quality method. Therefore, the target video essentially includes both a reference video and a distorted video.

[0028] When video quality assessment is required, the target video to be assessed can be obtained, and its dynamic range can be determined, which is also the dynamic range of the reference video. The dynamic ranges of the reference video and the distorted video are consistent.

[0029] S102. When the target video has a dynamic range of HDR, the target video is mapped using the domain transfer module in the trained video quality assessment model. Then, the mapped video is evaluated using the backbone model of the video quality assessment model to obtain the quality assessment result.

[0030] The domain transfer module maps the target video, resulting in a video that possesses similar features in the SDR domain. This video can be considered an SDR-like feature video. In other words, the subsequent backbone network can process the mapped video as an SDR video.

[0031] When the dynamic range of the target video corresponds to HDR, in the video quality analysis, the domain transfer module in the trained video quality assessment model can be used to map the target video, and then the backbone model of the video quality assessment model can be used to evaluate the mapped video to obtain the quality assessment result.

[0032] Please refer to Figure 2 The process of training the video quality assessment model includes the following steps.

[0033] The first stage involves using the SDR video training set to train the backbone network for quality assessment.

[0034] In the second stage, after completing the training of the backbone network, a domain migration module is added to the backbone network.

[0035] During the training of the video quality assessment model using a mixed video set including SDR and HDR videos, if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network is used to process the mapped video.

[0036] The third stage involves fine-tuning the video quality assessment model using the HDR video training set to obtain the final video quality assessment model.

[0037] The following section provides a detailed explanation of the specific implementation for different training stages.

[0038] Phase 1: Pre-training the backbone network.

[0039] In the first phase of training, the primary objective is to evaluate the backbone network using a large-scale standard dynamic range quality assessment dataset. Figure 2 The main quality evaluation model shown is pre-trained to establish basic quality evaluation capabilities.

[0040] Training objective: Train the backbone network (including feature extraction and fusion modules) to extract effective quality-aware features from standard dynamic range videos.

[0041] Training data: A large-scale standard dynamic range video quality assessment dataset is used, which contains a large number of standard dynamic range video samples and their corresponding quality scores, providing rich training samples for the model.

[0042] Training method: End-to-end training is performed on all parameters of the backbone network. By minimizing the mean squared error (MSE Loss) between the predicted quality score and the true quality score, the model learns the quality evaluation rules of standard dynamic range videos.

[0043] Output: After training is complete, the pre-trained weights of the backbone network are saved as initialization parameters for subsequent training stages, providing a good foundation for feature extraction of the model.

[0044] Phase 2: Joint training of standard dynamic range and high dynamic range.

[0045] In the second phase of training, high dynamic range data is introduced, and the model is jointly trained using standard dynamic range data and high dynamic range data, enabling the model to process both types of video simultaneously.

[0046] Pre-trained weight loading: Load the pre-trained weights of the backbone network obtained in the first stage of training as model initialization, and utilize the learned standard dynamic range quality assessment knowledge.

[0047] Domain transfer module introduction: The domain transfer module is introduced to map high dynamic range image features to the standard dynamic range domain, so that high dynamic range data and standard dynamic range data can be processed uniformly in the same backbone network.

[0048] Mixed data training: Using a video dataset that includes both standard dynamic range and high dynamic range, the model is trained jointly to learn the quality evaluation rules of both types of data simultaneously.

[0049] Training strategy: The domain transfer module and the backbone network are trained simultaneously. Through end-to-end backpropagation, the overall model's ability to evaluate the quality of mixed data is optimized.

[0050] Training objective: To minimize the mean squared error (MSE Loss) between the predicted quality score and the true quality score, so that the model can accurately predict the quality of standard dynamic range and high dynamic range videos.

[0051] Phase 3: Fine-tuning of high dynamic range data.

[0052] In the third phase of training, the model is fine-tuned for high dynamic range data to further improve its performance on high dynamic range video quality evaluation tasks.

[0053] Pre-trained weight loading: Load the optimal model weights obtained from the second stage of training, including the parameters of the domain transfer module and the backbone network, as the initialization for fine-tuning.

[0054] Training data: High dynamic range video quality assessment dataset is used, focusing on the quality assessment task of high dynamic range data, so that the model can better adapt to the characteristics of high dynamic range videos.

[0055] Fine-tuning strategy: Use a small learning rate to fine-tune the model, and perform fine optimization for high dynamic range data while maintaining existing knowledge.

[0056] Training objective: To further optimize the model's performance on high dynamic range video quality assessment tasks and improve its ability to perceive high dynamic range video quality features.

[0057] This three-stage progressive training strategy enables the model to start with basic quality assessment capabilities on standard dynamic range data, gradually expand to mixed data, and finally perform fine-tuning on high dynamic range data, achieving a progressive learning process from general to specific. The first stage fully utilizes large-scale standard dynamic range datasets to establish basic quality assessment capabilities; the second stage uses a domain transfer module to unify the processing of two data types, allowing the model to learn the quality assessment rules of both types of data simultaneously; the third stage performs specialized fine-tuning for high dynamic range data, further improving the model's performance on high dynamic range tasks. Furthermore, a random frame sampling strategy improves data utilization efficiency: N frames are randomly sampled from each video during the training phase, and M frames are fixed during the validation phase, improving both data utilization efficiency and evaluation stability. This staged training strategy avoids the overfitting problem that may result from training directly on small-scale high dynamic range data, and improves training stability and final performance through a gradual learning process from general to specific.

[0058] In other words, in this embodiment, a model capable of quality assessment for HDR videos can be pre-trained. To reduce the difficulty of model development and the dependence on HDR samples, a model capable of quality assessment for SDR videos can be directly used as the backbone network; that is, the backbone network can be trained for quality assessment using an SDR video training set.

[0059] After training the backbone network, a domain transfer module can be added to it. Then, a mixed video set is used to train the video quality assessment model for quality evaluation. For SDR videos, the training process is the same as in the first stage. For HDR videos, the domain transfer module is used to map the current HDR video, and the backbone network is used to process the mapped video.

[0060] Finally, the video quality assessment model is fine-tuned using an HDR video training set to obtain the final video quality assessment model. This model can then be used to assess the quality of HDR videos.

[0061] The domain transfer module comprises several residual blocks, each consisting of convolutional layers, activation functions, and residual connections. Its primary function is to address the domain differences between high dynamic range (HDR) and standard dynamic range (LVR) videos. This module uses a lightweight convolutional network structure to transform the feature representations of HDR images into feature representations similar to those of standard LVR images.

[0062] Please refer to Figure 3 The domain transfer module employs a lightweight design based on basic residual blocks, consisting of multiple stacked basic residual blocks. Each basic residual block contains convolutional layers, activation functions, and residual connections, mapping high dynamic range image features to the standard dynamic range domain through residual learning. The module maintains a consistent input and output dimension of 3×H×W, ensuring compatibility with the backbone quality assessment model.

[0063] The domain transfer module works by learning the mapping relationship from a high dynamic range (HDR) image to a standard dynamic range (LVR) domain through stacked basic residual blocks. Each basic residual block contains a convolutional layer and an activation function. Residual connections are used to add the original input to the result of the convolution, achieving residual learning. This design allows the module to learn the difference between the HDR and standard LVR, achieving domain transfer by adding the original input to the learned difference. The module maintains a consistent input and output dimension (3×H×W), ensuring compatibility with the backbone quality assessment model. This lightweight design ensures effective domain transfer while keeping the model's parameter count and computational cost within acceptable limits.

[0064] The method provided in this application provides an example of acquiring a target video and determining its dynamic range. The dynamic range includes HDR and SDR, where HDR is High Dynamic Range and SDR is Standard Dynamic Range. When the target video's dynamic range is HDR, the target video is mapped using the domain transfer module in the trained video quality assessment model. The mapped video is then evaluated using the backbone model of the video quality assessment model to obtain a quality assessment result. The process of training the video quality assessment model includes: training the backbone network for quality assessment using an SDR video training set; adding a domain transfer module to the backbone network after training; training the video quality assessment model using a mixed video set including SDR and HDR videos; if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network processes the mapped video; and the video quality assessment model is fine-tuned using the HDR video training set to obtain the final video quality assessment model.

[0065] In this application, a video quality assessment model can be pre-trained. The backbone network of this model can be trained on an SDR video training set. Then, to enable this backbone network to process DHR videos, a domain transfer module can be set up to map HDR videos from a mixed video set including SDR and HDR videos to SDR videos before processing. Finally, the video quality assessment model is fine-tuned based on the HDR video training set to obtain the final model. After obtaining the target video, its dynamic range can be determined, i.e., whether the target video corresponds to HDR or SDR. If the target video corresponds to HDR, the domain transfer module in the trained video quality assessment model can be used to map the target video, and then the backbone model of the video quality assessment model can be used to evaluate the mapped video to obtain the quality assessment result.

[0066] By using a model trained on SDR video as the backbone network through domain transfer and training optimization, and combining it with a domain transfer module to build a model that can evaluate the quality of HDR video, we can not only reduce the complexity of model development and maintenance, but also make full use of the pre-trained knowledge of large-scale standard dynamic range datasets and reduce the dependence on high dynamic range data.

[0067] It should be noted that, based on the above embodiments, the embodiments of this application also provide corresponding improvement schemes. In the preferred / improved embodiments, the same or corresponding steps as in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other; however, these will not be elaborated upon in the preferred / improved embodiments herein.

[0068] In one specific embodiment of this application, after mapping the target video using the domain transfer module in the trained video quality assessment model, the backbone model of the video quality assessment model is used to evaluate the mapped video to obtain a quality assessment result. This includes: extracting the reference video and distorted video corresponding to the target video frame by frame to obtain the current frame; mapping the high dynamic range features of the current frame to the standard dynamic range domain using the domain transfer module; performing feature extraction, feature fusion, and quality regression processing on the image features mapped to the standard dynamic range domain using the backbone network to obtain a single-frame evaluation result for the current frame; and aggregating the single-frame evaluation results corresponding to each frame of the target video to obtain the quality assessment result.

[0069] Please refer to Figure 4The input consists of a reference video sequence and a distorted video sequence. Both video sequences undergo preprocessing, followed by frame-by-frame image extraction to convert the video sequences into a sequence of image frames. For each extracted frame, it is first determined whether it is a high dynamic range (HMR) frame. If yes, the frame enters the domain transfer module to map HMR features to the standard dynamic range domain; otherwise, the domain transfer module is skipped. The frames that have undergone domain transfer or are skipped then enter a unified backbone quality assessment model for feature extraction and quality evaluation. The backbone quality assessment model extracts multi-scale features and performs fusion processing, calculating a single-frame quality score (a specific single-frame evaluation result) through a quality regression network. Finally, the quality scores of all frames are aggregated, and the average of all frame scores is calculated to obtain the overall video quality score (a specific quality assessment result) as the final output.

[0070] In one specific embodiment of this application, a video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: The model parameters of the backbone network are frozen, and the domain transfer module in the video quality assessment model is fine-tuned using the HDR video training set to obtain the video quality assessment model. Correspondingly, it also includes: Given that the target video has a dynamic range of SDR, the backbone model of the trained video quality assessment model is used to evaluate the target video and obtain the quality assessment result.

[0071] In practical applications, video quality assessment models can also be used to evaluate the quality of SDR videos. To ensure the reliability of quality assessment for both SDR and HDR videos, when fine-tuning the video quality assessment model, adjustments can be made only to the domain transfer module. This involves freezing the model parameters of the backbone network to maintain the reliability of the video quality assessment model for SDR videos, while fine-tuning the domain transfer module ensures the reliability of its quality assessment when processing HDR videos.

[0072] Furthermore, during the quality assessment process, the SDR video is not processed by the domain migration module. Therefore, fine-tuning the domain migration module will not affect the reliability of the SDR video quality assessment.

[0073] In one specific embodiment of this application, a video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: The domain transfer module and backbone network were fine-tuned using an HDR video training set to obtain a video quality assessment model.

[0074] In other words, when the video quality assessment requirement is mainly for HDR videos, meaning the trained video quality assessment model does not need to assess SDR videos, then the domain transfer module and the backbone network can be fine-tuned using the HDR video training set, making them more suitable for HDR video quality assessment processing.

[0075] In one specific embodiment of this application, a video quality assessment model is fine-tuned using an HDR video training set to obtain the video quality assessment model, including: Training and validation samples are obtained using the HDR video training set; Fine-tuning of the video quality assessment model using training samples; The video quality assessment model was validated using validation samples. After successful verification, the current video quality assessment model will be adopted as the final video quality assessment model.

[0076] The training and validation samples are obtained using the SDR video training set, including: Randomly extract frames from the SDR videos in the SDR video training set to obtain training samples; Frames are extracted from the SDR videos in the SDR video training set to obtain validation samples.

[0077] In this embodiment, to improve data utilization efficiency and model generalization ability during training, a random frame sampling strategy for video data can be adopted. During the training phase, training samples are selected for each video using random frame sampling. Specifically, for the reference video (the video training set includes both the reference video and distorted videos), N frames are randomly selected from the video sequence as training samples. This random selection method increases the diversity of training data and improves the model's generalization ability. The random frame sampling strategy is essentially a data augmentation method; by selecting frames from different positions within the same video, it increases the diversity of training samples, helping the model learn more robust feature representations.

[0078] During the verification phase, a fixed frame sampling method is used to ensure the consistency of the evaluation. Verification involves sampling a fixed number of M frames from each video for evaluation, ensuring the consistency and repeatability of the verification process.

[0079] Corresponding to the above method embodiments, this application also provides a video quality determination device, and the video quality determination device described below can be referred to in correspondence with the video quality determination method described above.

[0080] See Figure 5 As shown, the device includes the following modules: The video acquisition module 101 is used to acquire the target video and determine the dynamic range of the target video; wherein, the dynamic range includes HDR and SDR, HDR is high dynamic range and SDR is standard dynamic range; The HDR evaluation module 102 is used to evaluate the mapped video by using the domain transfer module in the trained video quality evaluation model when the dynamic range of the target video is HDR, and then using the backbone model of the video quality evaluation model to obtain the quality evaluation result. The model training module 103 includes: a first-stage training unit, a second-stage training unit, and a third-stage training unit; The first training unit 1031 is used to train the backbone network for quality evaluation using the SDR video training set. The second-stage training unit 1032 is used to add a domain transfer module to the backbone network after the backbone network training is completed. During the quality assessment training of the video quality assessment model using a mixed video set including SDR and HDR videos, if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network is used to process the mapped video. The third-stage training unit 1033 is used to fine-tune the video quality assessment model using the HDR video training set to obtain the video quality assessment model.

[0081] Using the apparatus provided in the embodiments of this application, a target video is acquired, and the dynamic range of the target video is determined. The dynamic range includes HDR and SDR, where HDR is High Dynamic Range and SDR is Standard Dynamic Range. When the target video's dynamic range is HDR, the target video is mapped using the domain transfer module in the trained video quality assessment model. The mapped video is then evaluated using the backbone model of the video quality assessment model to obtain a quality assessment result. The process of training the video quality assessment model includes: training the backbone network for quality assessment using an SDR video training set; adding a domain transfer module to the backbone network after training; training the video quality assessment model using a mixed video set including SDR and HDR videos; if the current video is an HDR video, the domain transfer module is used to map the current HDR video, and the backbone network processes the mapped video; the video quality assessment model is fine-tuned using the HDR video training set to obtain the final video quality assessment model.

[0082] In this application, a video quality assessment model can be pre-trained. The backbone network of this model can be trained on an SDR video training set. Then, to enable this backbone network to process DHR videos, a domain transfer module can be set up to map HDR videos from a mixed video set including SDR and HDR videos to SDR videos before processing. Finally, the video quality assessment model is fine-tuned based on the SDR video training set to obtain the final model. After obtaining the target video, its dynamic range can be determined, i.e., whether the target video corresponds to HDR or SDR. If the target video corresponds to HDR, the domain transfer module in the trained video quality assessment model can be used to map the target video, and then the backbone model of the video quality assessment model can be used to evaluate the mapped video to obtain the quality assessment result.

[0083] By using a model trained on SDR video as the backbone network through domain transfer and training optimization, and combining it with a domain transfer module to build a model that can evaluate the quality of HDR video, we can not only reduce the complexity of model development and maintenance, but also make full use of the pre-trained knowledge of large-scale standard dynamic range datasets and reduce the dependence on high dynamic range data.

[0084] In one specific embodiment of this application, the third-stage training unit is specifically used to freeze the model parameters of the backbone network and fine-tune the domain transfer module in the video quality assessment model using the HDR video training set to obtain the video quality assessment model.

[0085] Accordingly, it also includes an SDR evaluation module, which is used to evaluate the target video using the backbone model in the trained video quality evaluation model when the target video has a dynamic range of SDR, and obtain the quality evaluation result.

[0086] In one specific embodiment of this application, the third-stage training unit is specifically used to fine-tune the domain transfer module and the backbone network using the HDR video training set to obtain a video quality assessment model.

[0087] In one specific embodiment of this application, the domain migration module includes several residual blocks, each of which includes a convolutional layer, an activation function, and residual connections.

[0088] In one specific embodiment of this application, the third-stage training unit is specifically used to obtain training samples and validation samples using an HDR video training set; to fine-tune the video quality assessment model using the training samples; to validate the video quality assessment model using the validation samples; and, after successful validation, to determine the current video quality assessment model as the video quality assessment model.

[0089] In one specific embodiment of this application, the third-stage training unit is specifically used to randomly extract frames from the SDR videos in the SDR video training set to obtain training samples; and to perform fixed frame extraction from the SDR videos in the SDR video training set to obtain verification samples.

[0090] In one specific embodiment of this application, the HDR evaluation module is specifically used to extract the reference video and the distorted video corresponding to the target video frame by frame to obtain the current frame; to use the domain transfer module to map the high dynamic range features of the current frame to the standard dynamic range domain; to use the backbone network to perform feature extraction, feature fusion and quality regression processing on the image features mapped to the standard dynamic range domain to obtain the single-frame evaluation result of the current frame; and to aggregate the single-frame evaluation results corresponding to each frame of the target video to obtain the quality evaluation result.

[0091] As can be seen, the technical solution provided in this application overcomes the insufficient cross-domain adaptability of high dynamic range (HDR) / standard dynamic range (SVR) data. Specifically, a domain transfer module is introduced in the model design stage. This module employs a lightweight residual convolutional network structure to convert the feature representation of HDR images into feature representations similar to those of SVR images. Specifically, for an input video frame, the system first determines whether the frame is HDR. If the determination result is yes, the frame image enters the domain transfer module for processing, mapping the HDR features to the SVR domain; if the determination result is no, the domain transfer module is skipped. Image frames that have undergone domain transfer processing or are skipped directly then enter a unified backbone network for feature extraction and quality evaluation. This allows HDR and SVR data to be processed uniformly within the same backbone network, avoiding the complexity of designing separate models for different types of data and improving the algorithm's versatility and deployment efficiency.

[0092] The technical solution provided in this application also overcomes the training difficulties caused by the scarcity of high dynamic range data by adopting a three-stage progressive training strategy. The first stage uses a large-scale standard dynamic range quality assessment dataset to pre-train the backbone network, establishing basic quality assessment capabilities. The second stage introduces a domain transfer module, using a mixed dataset of standard and high dynamic range data for joint training. This module unifies the processing of the two data types, enabling the model to learn the quality assessment rules of both types of data simultaneously. The third stage involves specialized fine-tuning for high dynamic range data, using a smaller learning rate for refined optimization. While maintaining existing knowledge, optimization is performed specifically for the characteristics of high dynamic range data.

[0093] Furthermore, a random frame sampling strategy is employed to improve data utilization efficiency: N frames are randomly sampled from each video during the training phase, while M frames are fixed during the validation phase. This approach improves data utilization efficiency while ensuring evaluation stability. This phased training strategy avoids overfitting issues that may result from training directly on small-scale, high dynamic range data. By gradually moving from general to specific learning processes, it enhances training stability and final performance. M and N can be set according to actual training needs and are not specifically limited in this application.

[0094] Corresponding to the above method embodiments, this application also provides an electronic device. The electronic device described below and the video quality determination method described above can be referred to each other.

[0095] See Figure 6 As shown, the electronic device includes: Memory 332 is used to store computer programs; The processor 322 is used to implement the steps of the video quality determination method of the above method embodiments when executing a computer program.

[0096] For details, please refer to Figure 7 , Figure 7 This is a schematic diagram of the specific structure of an electronic device provided in this embodiment. The electronic device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the electronic device 301.

[0097] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0098] The steps in the video quality determination method described above can be implemented by the structure of an electronic device.

[0099] Corresponding to the above method embodiments, this application also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the video quality determination method described above.

[0100] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the video quality determination method described in the above method embodiments.

[0101] The readable storage medium can specifically be a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or any other readable storage medium capable of storing program code.

[0102] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0103] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0105] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0106] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of video quality determination, characterized by, The method comprises the following steps: acquiring a target video and determining a dynamic range of the target video; wherein the dynamic range comprises an HDR (High Dynamic Range) and an SDR (Standard Dynamic Range); in a case where the dynamic range of the target video is the HDR, performing mapping processing on the target video by using a domain migration module in a trained video quality evaluation model, performing evaluation processing on a video obtained by the mapping by using a backbone model of the video quality evaluation model, and obtaining a quality evaluation result; wherein a process of training the video quality evaluation model comprises: performing quality evaluation training on the backbone network by using an SDR video training set; after the training of the backbone network is completed, adding the domain migration module on the basis of the backbone network; in a process of performing quality evaluation training on the video quality evaluation model by using a mixed video set comprising SDR videos and HDR videos, if a current video is an HDR video, performing mapping processing on the current HDR video by using the domain migration module, and processing a video obtained by the mapping by using the backbone network; performing fine-tuning on the video quality evaluation model by using an HDR video training set, and obtaining the video quality evaluation model.

2. The method of claim 1, wherein, performing fine-tuning on the video quality evaluation model by using an HDR video training set, and obtaining the video quality evaluation model, comprises: freezing model parameters of the backbone network, performing fine-tuning on a domain migration module in the video quality evaluation model by using the HDR video training set, and obtaining the video quality evaluation model; correspondingly, the method further comprises: in a case where the dynamic range of the target video is the SDR, performing evaluation processing on the target video by using a backbone model in a trained video quality evaluation model, and obtaining a quality evaluation result.

3. The method of claim 1, wherein, performing fine-tuning on the video quality evaluation model by using an HDR video training set, and obtaining the video quality evaluation model, comprises: performing fine-tuning on the domain migration module and the backbone network by using the HDR video training set, and obtaining the video quality evaluation model.

4. The method of claim 1, wherein, the domain migration module comprises a plurality of residual blocks, and each residual block comprises a convolution layer, an activation function and a residual connection.

5. The method of claim 1, wherein, performing fine-tuning on the video quality evaluation model by using an HDR video training set, and obtaining the video quality evaluation model, comprises: obtaining training samples and verification samples by using the HDR video training set; performing fine-tuning on the video quality evaluation model by using the training samples; verifying the video quality evaluation model by using the verification samples; after the verification is passed, determining a current video quality evaluation model as the video quality evaluation model.

6. The method of claim 5, wherein, obtaining training samples and verification samples by using the SDR video training set, comprises: randomly extracting frames from SDR videos in the SDR video training set, and obtaining the training samples; fixedly extracting frames from the SDR videos in the SDR video training set, and obtaining the verification samples.

7. The method according to any one of claims 1 to 6, characterized in that, performing mapping processing on the target video by using a domain migration module in a trained video quality evaluation model, performing evaluation processing on a video obtained by the mapping by using a backbone model of the video quality evaluation model, and obtaining a quality evaluation result, comprises: extracting a current frame by frame by frame from a reference video and a distortion video corresponding to the target video; mapping high dynamic range features of the current frame to a standard dynamic range domain by using the domain migration module; extracting image features, fusing features and quality regression processing by using the backbone network on the mapped image features to a standard dynamic range domain, to obtain a single frame evaluation result of the current frame; aggregating single frame evaluation results corresponding to each frame of the target video to obtain the quality evaluation result.

8. A video quality determination apparatus characterized by comprising: It comprises: a video acquisition module for acquiring a target video and determining the dynamic range of the target video; wherein the dynamic range includes HDR and SDR, HDR is high dynamic range, and SDR is standard dynamic range; an HDR evaluation module for performing mapping processing on the target video by using a domain migration module in a trained video quality evaluation model, and performing evaluation processing on the mapped video by using a backbone model of the video quality evaluation model to obtain a quality evaluation result when the dynamic range of the target video is HDR; a model training module comprising a first stage training unit, a second stage training unit and a third stage training unit; the first stage training unit is used for quality evaluation training of the backbone network by using an SDR video training set; the second stage training unit is used for adding the domain migration module on the basis of the backbone network after the training of the backbone network is completed; during the quality evaluation training of the video quality evaluation model by using a mixed video set including SDR video and HDR video, if the current video is an HDR video, the current HDR video is mapped and processed by using the domain migration module, and the mapped video is processed by using the backbone network; the third stage training unit is used for fine-tuning the video quality evaluation model by using an HDR video training set to obtain the video quality evaluation model.

9. An electronic device, comprising: It comprises: a memory for storing a computer program; a processor for executing the computer program to realize the steps of the video quality determination method according to any one of claims 1 to 7.

10. A readable storage medium, characterized by, The computer program is stored on the readable storage medium, and the computer program is executed by the processor to realize the steps of the video quality determination method according to any one of claims 1 to 7.

Images