Video image quality scoring method and device, electronic equipment and readable storage medium

Abstract

The invention provides a video image quality scoring method and apparatus, an electronic device and a readable storage medium. The method comprises the steps of obtaining a to-be-evaluated video; performing frame sampling on the to-be-evaluated video according to a predetermined optimal frame sampling interval to obtain a frame sequence; the target image quality evaluation network determines the score of each frame according to the frame sequence, and outputs the average value of the scores of each frame; wherein the target image quality evaluation network is obtained by performing structure simplification on a trained image quality evaluation network and then performing training again; and taking the average value of the scores of each frame as the image quality score of the to-be-evaluated video. According to the method, the accuracy of a video image quality evaluation result can be improved while the reasoning time consumption of an image quality evaluation network is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a video quality scoring method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] With the rapid development of live streaming, short video and online video services, objective, efficient and accurate evaluation of video quality has become a key link to ensure user experience. The common video quality evaluation inference process is as shown in the following figure: Figure 1 First, the video to be measured is frame-sampled according to the frame rate to form a frame sequence; then, the trained image quality evaluation network is used to independently predict the quality score of each frame image; finally, the average value of the quality scores of all frames is taken to obtain the overall video quality score.

[0003] However, the existing image quality evaluation network still faces significant challenges. On the one hand, the mainstream model usually has a complex structure and large amount of calculation, and the inference time is high when processing concurrently on the server side or deploying on the mobile side, which makes it difficult to meet the demand for real-time evaluation of video quality in live streaming scenarios; on the other hand, the video quality data used for training often has an uneven distribution of scores, i.e. some quality intervals (such as medium quality) have too many samples, while extreme quality intervals (such as excellent or poor) have sparse samples, resulting in insufficient prediction accuracy of the model during the training process.

[0004] Therefore, how to improve the video quality evaluation speed and the accuracy of the evaluation results based on the image quality evaluation network is a technical problem to be solved. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a video quality scoring method, device, electronic equipment and readable storage medium, which can reduce the inference time of the image quality evaluation network while improving the accuracy of the video quality evaluation results. In order to achieve the above purpose, the technical solutions adopted by the present application are as follows: In a first aspect, the present application provides a video quality scoring method, which comprises: obtaining a video to be evaluated; frame sampling the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence; determining the score of each frame according to the frame sequence by a target image quality evaluation network, and outputting the average value of the scores of each frame; wherein the target image quality evaluation network is obtained by simplifying the structure of a trained image quality evaluation network and then retraining it; and taking the average value of the scores of each frame as the quality score of the video to be evaluated.

[0006] In a second aspect, the present application provides a video quality scoring device, comprising: an obtaining module configured to obtain a video to be evaluated; a sampling module configured to sample frames of the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence; an evaluation module configured to determine scores of each frame according to the frame sequence by a target image quality evaluation network, and output an average value of the scores of each frame; wherein the target image quality evaluation network is obtained by simplifying the structure of a trained image quality evaluation network and then retraining; and the evaluation module is further configured to take the average value of the scores of each frame as a quality score of the video to be evaluated.

[0007] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program capable of being executed by the processor, and the processor can execute the computer program to implement the video quality scoring method according to any one of the preceding aspects.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the video quality scoring method according to any one of the preceding aspects.

[0009] The video quality scoring method, device, electronic device and readable storage medium provided by the embodiments of the present application, the method comprises: after obtaining a video to be evaluated, frame sampling of the video to be evaluated is performed through the optimal frame sampling interval to obtain a frame sequence, so that the data amount input into the image quality evaluation network can be reduced, the inference time of the image quality evaluation network can be reduced, and the scoring accuracy can be ensured. Then, the frame sequence obtained by using the optimal frame sampling interval is input into a target image quality evaluation network, the network is obtained by simplifying the structure of a trained image quality evaluation network and then retraining, has inference acceleration capability and accurate prediction performance, and finally the quality score of the video to be evaluated can be quickly and accurately obtained.

[0010] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following preferred embodiments are specifically described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0012] Figure 1 The inference process of the traditional video quality evaluation is shown; Figure 2A schematic flow chart of a video quality scoring method provided by an embodiment of the present application is shown; Figure 3 An example diagram of determining an optimal frame sampling interval provided by an embodiment of the present application is shown; Figure 4 A schematic flow chart of an image quality evaluation network training method provided by an embodiment of the present application is shown; Figure 5 An example diagram of constructing a training sample set provided by an embodiment of the present application is shown; Figure 6 An example schematic diagram of an image quality evaluation model training process provided by an embodiment of the present application is shown; Figure 7 A functional module diagram of a video quality scoring device provided by an embodiment of the present application is shown; Figure 8 A block schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0014] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative labor are within the scope of protection of the present application.

[0015] It should be noted that the relational terms such as "first" and "second" and the like are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations exist in any such actual relationship or order. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a series of elements includes not only those elements, but also other elements not explicitly listed or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus including the element.

[0016] In view of the fact that the prior art has low accuracy and efficiency of scoring results when scoring video quality based on an image quality evaluation network, the embodiment of the present application provides a video quality scoring method, which can re-train a trained image quality evaluation network and use a pre-determined optimal frame sampling interval to perform video frame extraction as the input of the trained image quality evaluation network, thereby improving the model inference speed and ensuring the accuracy of the video quality evaluation results.

[0017] Please refer to Figure 2 , Figure 2 The embodiment of the present application provides a schematic flowchart of a video quality scoring method, which comprises steps S201 to S203, and the explanations are as follows: S201: obtaining a video to be evaluated; S202: frame sampling the video to be evaluated according to a pre-determined optimal frame sampling interval to obtain a frame sequence; S203: determining the scores of each frame according to the frame sequence by a target image quality evaluation network, and outputting the average value of the scores of each frame; wherein the target image quality evaluation network is obtained by simplifying the structure of a trained image quality evaluation network and re-training it; S204: taking the average value of the scores of each frame as the quality score of the video to be evaluated.

[0018] Unlike the existing video quality scoring technology, the embodiment of the present application first performs frame sampling on the video to be evaluated by using the optimal frame sampling interval to obtain a frame sequence, which can reduce the amount of data input into the image quality evaluation network, reduce the inference time of the image quality evaluation network, and ensure the accuracy of subsequent scoring. Then, the frame sequence obtained by using the optimal frame sampling interval is input into the target image quality evaluation network, which is obtained by simplifying the structure of a trained image quality evaluation network and re-training it, has inference acceleration capability and accurate prediction performance, and can finally quickly and accurately obtain the accurate quality score of the video to be evaluated.

[0019] Next, the embodiment of the present application will describe in detail the video quality scoring process shown in the related drawings. Figure 2

[0020] In an embodiment of the present application, the video to be evaluated in step S201 is obtained by a terminal device or a collection device from a local storage medium, a network platform (such as a video website, a social media platform or a cloud storage service), and a real-time data stream (such as a monitoring camera, a live broadcast signal or a real-time shooting of a mobile terminal); or the video can also be uploaded by a user, automatically grabbed by a system, or transmitted in a manner of calling by a third-party interface. The video to be evaluated can include but is not limited to various video forms such as live video, short video, user-generated content, etc. ​

[0021] In an embodiment of the present application, in order to reduce the inference time of the image quality evaluation network and the accuracy of the evaluation results, in step S202, an optimal frame sampling interval can be determined in advance. Specifically, step S202 can be implemented according to the following implementation: Step a1: obtaining a test video and a preset frame sampling interval sequence; In an embodiment of the present application, the number of test videos can be one or more, which is not limited here.

[0022] In an embodiment of the present application, the frame sampling interval sequence can be set based on the video frame rate (fps) of the test video. For ease of understanding, the frame sampling interval can be represented as: , which means taking one frame every n frames. This can be understood as: when , that is, taking one frame every 30 frames, which is equivalent to taking one frame per second, and the frequency is 1 Hz; when , it means sampling once every frame, that is, taking one frame every two frames, when , it means continuous frame-by-frame sampling. With the decrease of , the actual sampling frequency gradually increases.

[0023] It can be understood that by adjusting the frame sampling interval, a trade-off between time resolution and data volume can be made: a larger value reduces the sampling frequency and reduces data redundancy and computational overhead; a smaller value retains more timing details, which is suitable for application scenarios sensitive to dynamic changes.

[0024] Step a2: determining the picture quality scores of the frame sequence corresponding to the test video under each frame sampling interval; In an embodiment of the present application, each frame sampling interval can be used in turn to perform frame extraction processing on the test video to obtain the corresponding frame sequence. For example, if the frame sampling interval sequence is , then the frame extraction can be performed on the test video using to obtain the frame sequence.

[0025] Step a3: taking the frame sampling interval corresponding to the picture quality score with a difference less than or equal to a preset threshold from the picture quality score corresponding to the minimum frame sampling interval as the optimal frame sampling interval.

[0026] In an embodiment of the present application, the minimum frame sampling interval is , that is, frame-by-frame sampling is performed on the test video to retain all original frames. For the frame extraction frame sequence under different frame sampling intervals, the corresponding picture quality scores are obtained through the image quality evaluation network. The image quality evaluation network can use the model trained in the embodiment of the present application, or a model trained in other ways.

[0027] For the convenience of intuitive understanding, the frame sequence quality score obtained under the condition of The difference value between the quality score of each frame sampling interval and the reference benchmark is calculated. When the difference between the quality score of a certain frame sampling interval and the reference benchmark is less than or equal to a preset threshold (for example, which can be set to 0.01), it is considered that there is no significant degradation in visual quality compared with the frame-by-frame sampling result. At this time, the frame sampling interval can maximize the reduction of data volume and processing time under the premise of ensuring the quality, thereby obtaining the optimal speed-quality trade-off. Therefore, the frame sampling parameter that meets the above conditions and has the largest sampling interval (i.e., the lowest sampling frequency) is determined as the optimal frame sampling interval, which is used for subsequent video processing procedures.

[0028] In the above embodiment, to avoid redundant processing of invalid parameter combinations, frame-by-frame sampling (i.e., frame sampling interval ) can be performed on the test video first, the frame sequence under this condition is obtained, and the quality score thereof is calculated as a reference benchmark. Subsequently, for other frame sampling intervals , the test video is sequentially frame-extracted, the corresponding frame sequence is generated, and the difference between the quality score and the quality score under the frame-by-frame sampling condition is evaluated. If the difference is greater than the preset threshold, the next smaller sampling interval is tried; if the difference is less than or equal to the preset threshold, it is considered that the current sampling interval can sufficiently retain the key visual information of the video, and the subsequent processing procedure of higher density sampling can be terminated in advance. By introducing this early termination mechanism, unnecessary computational overhead can be effectively reduced under the premise of ensuring the quality performance, thereby improving the overall evaluation efficiency.

[0029] For the convenience of overall understanding of the above process of determining the optimal frame sampling interval, please refer to Figure 3 , Figure 3 An example diagram for determining the optimal frame sampling interval provided by the embodiment of the present application is shown. By pre-determining the optimal frame sampling interval, the input data volume of the image quality evaluation network in the process of processing the video to be evaluated can be reduced, and the accuracy of the evaluation result can be ensured.

[0030] Further, after obtaining the optimal frame sampling interval based on the above embodiment, the video to be evaluated in step S201 can be frame-sampled to obtain a frame sequence. Taking this frame sequence as the input of the target image quality evaluation network in subsequent step S203 can reduce the input volume and thus improve the inference speed of the model.

[0031] In step S203 of the embodiment of the present application, the target image quality evaluation network is obtained by the training method provided by the embodiment of the present application, which not only has a fast inference speed but also has good inference performance, thereby providing a guarantee for accurately realizing the video quality score. Please refer to Figure 4 ,Figure 4 A schematic flowchart of the image quality evaluation network training method provided by the embodiment of the present application is shown, comprising steps S401 to S403, which are described as follows: S401: constructing a plurality of quality level training sample sets; wherein the sample density of each training sample set is consistent; S402: taking the image quality evaluation network that has been trained as a target model, and simplifying the structure of the target model to obtain a to-be-trained model; S403: training the to-be-trained model using the training sample set and the target model until a training end condition is reached; wherein the model parameters of the target model remain unchanged during the training of the to-be-trained model; S404: taking the trained to-be-trained model as a target image quality evaluation network.

[0032] The lightweight image quality evaluation network with inference acceleration capability and good prediction accuracy is obtained through the above training method, and the video picture quality score is obtained in combination with the optimal frame sampling interval determined in advance in step S202, so that the accuracy of the score result can be improved.

[0033] In step S401 of the embodiment of the present application, it is considered that in a real video quality dataset (such as a public database or user uploaded content), the picture quality score often presents a skewed distribution of “few at both ends and many in the middle”, which leads to weak ability of the model to distinguish between high quality and low quality during the training process. Therefore, the embodiment of the present application first proposes an implementation manner of constructing a plurality of uniform training sample sets of multiple quality levels, as shown in steps b1 to b4, which are described as follows: Step b1: obtaining a high-definition original video; In the embodiment of the present application, the high-definition original video refers to a high-quality digital video with a resolution not lower than 1080p (1940x1080), a frame rate usually of 30fps or above, and recorded or stored by using a lower compression ratio or a lossless encoding method. The high-definition original video can be obtained from a public video dataset, professional camera equipment collected content, or a raw streaming signal in a live streaming scene, and the present application does not limit this.

[0034] Step b2: transcoding the high-definition original video using a plurality of preset encoding parameters to generate a plurality of sets of low-definition videos with decreasing picture quality; In the embodiment of the present application, the same segment of high-definition original video can be re-encoded using a plurality of preset encoding parameters CRF (Constant Rate Factor), for example, CRF = {10, 12, 14,..., 36}, a total of 14 levels, to obtain a plurality of versions of low-definition videos, which have the same content but gradually decreasing picture quality.

[0035] It can be understood that the above process can ensure that, in addition to the picture quality, the video content, the length and other factors remain consistent, and a systematic degradation video set is constructed, covering the complete picture quality spectrum from "close to the original picture" to "severe distortion", avoiding the interference of the model caused by content differences.

[0036] Step b3: determining the picture quality score of each group of low-definition videos; In the embodiment of the application, different scoring strategies can be used to score the picture quality of each group of low-definition videos, and then the picture quality scores under each scoring strategy are weighted and fused to obtain the final picture quality score of the low-definition video.

[0037] Optionally, the scoring strategy can include but is not limited to objective VMAF (Video Multimethod Assessment Fusion) scoring (with reference to high-definition video evaluation) and subjective MOS scoring. The VMAF score can reflect the objective distortion degree in multiple dimensions; the subjective MOS score refers to organizing a subjective evaluation experiment, inviting no less than 15 observers to score in a standard viewing environment to obtain the average opinion score.

[0038] Subsequently, the normalized VMAF and MOS are fused in a weighted manner. The weights of the VMAF score and the MOS score can be adjusted according to the data to improve the reliability of the label, so as to obtain a more stable and more representative "real" quality label.

[0039] Step b4: extracting a frame sequence from the low-definition video, and associating the extracted video frame with the picture quality score to construct a uniform training sample set.

[0040] In the embodiment of the application, the low-definition video can be extracted frame by frame to obtain a frame sequence, each frame in the frame sequence is associated with the picture quality score of the video, and a data pair (image, score) is obtained as a training sample of the image quality evaluation network.

[0041] By constructing a data set with complete picture quality coverage and balanced sample quantity at each level through the above steps b1 to b4, the distribution skewness problem of traditional picture quality data set, i.e. "too many intermediate quality samples and too few extreme quality samples", can be effectively alleviated, which helps to improve the sensitivity and prediction stability of the image quality evaluation model to subtle picture quality changes in the inference stage.

[0042] In order to intuitively understand the implementation process of the above step S401, please refer to Figure 5 , Figure 5An example diagram for constructing a training sample set provided by the embodiment of the present application is shown. By using the same high-definition video source, a series of low-definition videos with decreasing image quality are generated by transcoding with different encoding parameter values; then the VMAF score and the subjective MOS score are combined to score each low-definition video, and a weighted score is obtained; finally, these low-definition videos are extracted frame by frame into frame sequences, and are associated with the scores of the corresponding videos to form a large number of training sample sets with accurate labels (image, score), which lay a foundation for subsequent model distillation and inference acceleration.

[0043] Next, based on the above-constructed training sample set, in step S402, the image quality evaluation network that has been trained is used to guide the training of the image quality evaluation network after structure simplification, so as to obtain an image quality evaluation network that is lightweight and has good prediction accuracy.

[0044] For ease of description, the image quality evaluation network that has been trained is referred to as a "target model" in the embodiment of the present application. Since the target model has a relatively complex network structure, the embodiment of the present application simplifies the structure of the target model to obtain a model with a lighter structure as a "to-be-trained model".

[0045] Optionally, during the process of structure simplification, the embodiment of the present application can retain only one residual block at each stage of the target model, and the remaining network architecture remains unchanged, so as to construct the simplified to-be-trained model. The to-be-trained model significantly reduces the parameter quantity and the calculation overhead, and inherits the performance of the target model through knowledge distillation, and is suitable for low-delay and high-efficiency video image quality evaluation scenarios.

[0046] Before training, the embodiment of the present application also pre-constructs a training loss function. Specifically, the training loss function L can be constructed according to the real label regression loss L kd of the to-be-trained model, the knowledge distillation loss L AT between the to-be-trained model and the target model, and the attention transfer loss L total . Specifically: (1) (2) (3) (4) Where y is the real score of the training sample; y T and y S are the predicted scores output by the target model and the to-be-trained model respectively; the feature map is a 3-dimensional tensor, and C, H and W represent the number of channels, height and width respectively. and the intermediate layer feature maps output by the to-be-trained model and the target model respectively, represents flattening a 3-dimensional tensor into a 2-dimensional tensor, is the i-th channel; represents a vector normalization operation; represents a set of activation layers that need to be calculated .

[0047] When training the to-be-trained model based on the above training loss function , the parameters of the target model are kept fixed, the predicted scores of the training sample, the real scores of the training sample and the intermediate layer feature maps are obtained according to the to-be-trained model and the target model respectively, the loss value of is determined, and finally the loss value is back propagated to the to-be-trained model for model parameter iterative updating until the training end condition is reached. Therefore, the implementation process of the above step S403 is as follows: Step c1: input the training sample into the to-be-trained model and the target model to obtain the output results of the to-be-trained model and the target model respectively; It can be understood that the output results are the predicted scores of the training sample of the target model and the to-be-trained model respectively, and the intermediate layer feature maps output by the two models; Step c2: determine the total loss function value according to the output results and the real scores of the training sample; In combination with the above L1, and The execution process of this step is: determine the real label regression loss value L1 according to the predicted score and the real score output by the to-be-trained model; determine the knowledge distillation loss value and the attention transfer loss value respectively using the predicted scores and the intermediate layer feature maps output by the target model and the to-be-trained model respectively; then the real label regression loss value, the knowledge distillation loss value and the attention transfer loss value are weighted and fused to obtain the total loss function value according to the above formula (4).

[0048] Step c3: back propagate the total loss function value to the to-be-trained model for model parameter iterative updating until the training end condition is reached.

[0049] Optionally, the training end condition can be convergence or reaching a preset number of iterations.

[0050] In order to facilitate the overall understanding of the above training process, please refer to Figure 6 , Figure 6 ​This diagram illustrates an example of the training process for the image quality assessment model provided in an embodiment of the present invention. Figure 6 In this process, the network parameters of the target model are frozen and do not participate in gradient updates. They are only used for forward inference on the input image to output its predicted image quality score, which serves as a soft label to guide the learning of the model to be trained. The training of the model to be trained is guided by multiple loss functions: L1 loss is used to supervise the model to be trained to fit the real image quality score, ensuring that its output is consistent with the real perceived quality; By minimizing the difference between the prediction scores of the training model and the target model at the output layer, the training model is guided to learn the detailed knowledge contained in the target model. The loss is applied to the intermediate layer feature map. By comparing the spatial attention distribution of the target model and the model to be trained on the key activation layer, the model to be trained is prompted to imitate the target model's attention pattern to key regions of the image, so as to retain important semantic perception capabilities even with structural simplification.

[0051] In this way, through L1, and The synergistic optimization of these three elements allows the trained model to not only approximate the true quality label but also effectively inherit the output behavior and internal representation characteristics of the target model. This significantly reduces the number of model parameters and inference latency while maximizing the preservation of the original model's evaluation accuracy, laying the foundation for deploying high-performance image quality evaluation models in mobile or live streaming scenarios.

[0052] Unlike existing methods for training image quality assessment networks, the image quality assessment network provided in this embodiment of the invention first constructs multiple training sample sets covering different quality levels, ensuring consistent sample density across sets. This guarantees that the model to be trained learns on training samples with a balanced quality distribution, avoiding evaluation bias due to data skew, thus laying the foundation for subsequent efficient training. Based on this, the model to be trained is trained using these sample sets, guided by a pre-trained target model. During training, the target model's parameters are kept fixed, allowing the model to stably inherit the target model's ability to discriminate quality differences. Furthermore, due to the simplified structure of the model to be trained, it naturally possesses lower computational overhead and higher inference efficiency. As training gradually concludes, the model to be trained retains the fine-grained quality perception characteristics of the target model while significantly reducing model complexity. The resulting target image quality assessment network not only possesses fast inference capabilities but also maintains high prediction accuracy, meeting the dual requirements of efficiency and high quality in practical applications.

[0053] It should be noted that the aforementioned image quality assessment network can adopt different deployment and training strategies depending on the needs of the actual application scenario. In one implementation, the network can be pre-trained on the server side. After training, the optimized model to be trained is solidified into an inference model and deployed on a server or mobile device for real-time or near-real-time video quality assessment tasks. In another implementation, the image quality assessment network can also be locally trained or fine-tuned based on representative video data in the target environment before deployment, and then put into actual use.

[0054] In summary, the embodiments of the present invention have the following advantages: by using the video to be evaluated after pre-processing by sampling frames at the optimal frame sampling interval, the amount of input data to the network can be reduced. Furthermore, by using a lightweight image quality evaluation network with inference acceleration capabilities and good prediction accuracy to process the video to be evaluated after sampling frames at the optimal frame sampling interval, not only can the inference time of the network be effectively reduced, but the accuracy of the video quality evaluation results can also be guaranteed.

[0055] Based on and Figure 2 Following the same inventive concept, an implementation of the video quality scoring device 70 is also provided below. Please refer to... Figure 7 , Figure 7 The diagram shows a functional block diagram of a video quality scoring device provided in an embodiment of the present invention, including: an acquisition module 701, a sampling module 702, and an evaluation module 703.

[0056] Module 701 is used to acquire the video to be evaluated; The sampling module 702 is used to sample the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence; Evaluation module 703 is used to input the frame sequence into the target image quality assessment network and output the score of each frame in the frame sequence; wherein, the target image quality assessment network is obtained by retraining the already trained image quality assessment network after structural simplification; Evaluation module 703 also uses the average score as the picture quality score of the video to be evaluated.

[0057] It is understandable that the acquisition module 701, sampling module 702, and evaluation module 703 can execute collaboratively. Figure 2 Each step in the process is to achieve the corresponding technical effect.

[0058] It should be noted that the video quality scoring device 70 provided in this embodiment of the invention has the same basic principle and technical effect as the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment of the invention can be referred to the corresponding content in the above embodiments.

[0059] Optionally, the above Figure 7Modules can be stored in the form of software or firmware. Figure 8 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be used by... Figure 8 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0060] Please refer to Figure 8 , Figure 8 The diagram shows a block illustration of an electronic device provided in an embodiment of the present invention, including a memory 801, a processor 802, and a communication interface 803. The memory 801, processor 802, and communication interface 803 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0061] Optionally, the bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0062] In this embodiment of the invention, the processor 802 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this embodiment of the invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment of the invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The software modules may be located in the memory 801, and the processor 802 reads the program instructions from the memory 801 and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0063] In this embodiment of the invention, the memory 801 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as RAM. The memory can also be any other medium capable of carrying or storing desired executable program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. The memory in this embodiment of the invention can also be a circuit or any other device capable of implementing a storage function for storing instructions and / or data.

[0064] The memory 801 can be used to store software programs and modules, such as the instructions / modules of the video quality scoring device 70 provided in this embodiment of the invention. These can be stored in the memory 801 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device 80. The processor 802 executes various functional applications and data processing by executing the software programs and modules stored in the memory 801. The communication interface 803 can be used to communicate with other node devices for signaling or data.

[0065] It should be understood that, Figure 8 The structure shown is only a schematic diagram of the electronic device 80. The electronic device 80 may also include components that are larger than those shown. Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown. Figure 8 The components shown can be implemented using hardware, software, or a combination thereof.

[0066] Based on the above embodiments, the present invention also provides a storage medium storing a computer program. When the computer program is executed by a computer, it causes the computer to execute the video quality scoring method provided in the above embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0067] Based on the above embodiments, the present invention also provides a program product, which includes a computer program. The processor can execute the computer program to implement the video quality scoring method provided in the embodiments of the present invention. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0068] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0069] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0070] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0071] It should be noted that if the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of video quality scoring, the method comprising: The method comprises: obtaining a video to be evaluated; frame sampling the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence; determining scores of each frame according to the frame sequence by a target image quality evaluation network, and outputting an average value of the scores of each frame; wherein the target image quality evaluation network is obtained by simplifying the structure of a trained image quality evaluation network and then training again; taking the average value of the scores of each frame as a quality score of the video to be evaluated.

2. The video quality scoring method of claim 1, wherein, The method further comprises: constructing a plurality of training sample sets of different quality levels; wherein the sample density of each training sample set is consistent; taking a trained image quality evaluation network as a target model, and simplifying the structure of the target model to obtain a to-be-trained model; training the to-be-trained model using the training sample set and the target model until a training end condition is reached; wherein the model parameters of the target model remain unchanged during the training of the to-be-trained model; taking the trained to-be-trained model as the target image quality evaluation network.

3. The method of claim 2, wherein, Constructing a plurality of training sample sets of different quality levels comprises: obtaining a high-definition original video; transcoding the high-definition original video using a plurality of preset encoding parameters to generate a plurality of low-definition videos with decreasing quality; determining the quality scores of each group of low-definition videos; extracting a frame sequence from each group of low-definition videos, and associating the extracted video frames with the quality scores to construct the training sample set.

4. The method of claim 3, wherein, Determining the quality scores of each group of low-definition videos comprises: using different scoring strategies to score the quality of each group of low-definition videos respectively; weighting and fusing the quality scores under each scoring strategy to obtain the final quality score of the low-definition video.

5. The method of claim 2, wherein, Training the to-be-trained model using the training sample set and the target model until a training end condition is reached comprises: inputting the training sample into the to-be-trained model and the target model to obtain the output results of the to-be-trained model and the target model respectively; wherein the output results include predicted scores and intermediate layer feature maps of the training sample; determining a total loss function value according to the output results and the real scores of the training sample; performing model parameter iterative updating by back-propagating the total loss function value to the to-be-trained model until the training end condition is reached.

6. The method of claim 5, wherein, Determining the loss value of the training loss function according to the output results and the real scores of the training sample comprises: determining a real label regression loss value according to the predicted scores output by the to-be-trained model and the real scores; determining a knowledge distillation loss value and an attention transfer loss value respectively using the predicted scores and the intermediate layer feature maps output by the target model and the to-be-trained model respectively; weighting and fusing the real label regression loss value, the knowledge distillation loss value and the attention transfer loss value to obtain the total loss function value.

7. The method of claim 1-6, wherein, Frame sampling the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence comprises: obtaining a test video and a preset frame sampling interval sequence; determining the quality scores of the frame sequences corresponding to the test video under each frame sampling interval; The frame sampling interval corresponding to the quality score with a difference less than or equal to a preset threshold is taken as the optimal frame sampling interval.

8. A video quality scoring apparatus characterized by comprising: The method comprises the steps of: obtaining a video to be evaluated; sampling the video to be evaluated according to a predetermined optimal frame sampling interval to obtain a frame sequence; inputting the frame sequence into a target image quality evaluation network to output scores of each frame in the frame sequence; wherein the target image quality evaluation network is obtained by simplifying the structure of a trained image quality evaluation network and then retraining the simplified image quality evaluation network; the evaluation module also takes the average of the scores of each frame as the quality score of the video to be evaluated.

9. An electronic device, comprising: The computer program is executed by the processor to implement the video quality scoring method of any one of claims 1 to 7.

10. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the video quality scoring method of any one of claims 1 to 7.

Images