Video quality assessment method and its device, equipment, and medium

Abstract

This application relates to a video quality assessment method and its apparatus, device, and medium in the field of video transmission. The method comprises: obtaining a video stream to be evaluated; obtaining a plurality of consecutive image frames from the video stream and formatting them into standardized data; extracting image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model pre-trained to a convergent state; and obtaining a quality score of the video stream by performing classification mapping based on the image feature information using a prediction module of the video quality assessment model. This application can quantitatively represent the video quality obtained by evaluating a video without a reference using a quantitative score.

Description

Technical Field

[0001] The present application relates to the field of video transmission, and in particular to a video quality assessment method and its corresponding apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] Video quality assessment has been a research topic in academia and industry since the advent of video. It is primarily categorized into reference-based and non-reference-based assessments. Reference-based assessment compares the performance frame by frame against a given reference video, making it easier to obtain a relatively accurate assessment score. Non-reference-based video assessment provides a user-defined subjective quality score for the current video. Based on the non-reference-based assessment score, network transmission quality metrics such as bitrate, frame rate, and resolution can be dynamically adjusted to achieve cost control. These adjustments essentially maintain the user's subjective video viewing experience, thus satisfying user needs while conserving network transmission resources, resulting in considerable economic benefits.

[0003] No-reference video evaluation methods are extremely difficult, and traditional algorithms can only play a very limited role in no-reference video evaluation. Until the rapid development of artificial intelligence in recent years, no-reference video evaluation methods based on CNN (Convolutional Neural Network) have received attention and rapid development, becoming a major research and application direction, but there is still no mature solution. Summary of the Invention

[0004] The primary purpose of the present application is to solve at least one of the above problems and provide a video quality assessment method and its corresponding device, computer equipment, and computer-readable storage medium.

[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0006] A video quality assessment method provided for one of the purposes of this application includes the following steps:

[0007] Get the video stream to be evaluated;

[0008] Obtaining a plurality of consecutive image frames from the video stream and formatting them into standardized data;

[0009] Extracting image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model that has been pre-trained to a convergence state;

[0010] The prediction module of the video quality assessment model performs classification mapping according to the image feature information to obtain a quality score of the video stream.

[0011] In a further embodiment, the training process of the video quality assessment model includes the following steps:

[0012] Obtaining a training set, wherein the training set includes a plurality of training samples corresponding to video streams of different video qualities;

[0013] Batch calling multiple training samples corresponding to multiple video qualities in the training set to perform multi-task synchronous training on the video quality assessment model;

[0014] After the video quality assessment model extracts the image feature information corresponding to each training sample, the prediction module of the model predicts the quality score corresponding to each training sample;

[0015] Calculate the loss value of the quality score of each training sample in the same batch and its corresponding gradient update value, calculate the average of the multiple loss values, and terminate the training when the average reaches the preset threshold. Otherwise, implement gradient update on the model according to the gradient update value of each training sample, and continue to call the next batch of multiple training samples to implement iterative training until the model converges.

[0016] In a further embodiment, before the step of obtaining the training set, the following steps are also included:

[0017] Encoding a pre-collected high-definition video stream using a constant-quality encoding method to obtain a first encoded video stream;

[0018] Encoding the pre-collected high-definition video stream using a constant bit rate encoding method to obtain a second encoded video stream;

[0019] The first encoded video stream and the second encoded video stream are decoded accordingly to obtain the decoded video streams as training sets.

[0020] In a preferred embodiment, the step of predicting the quality score corresponding to each training sample by the prediction module of the model includes the following steps:

[0021] In the score prediction unit in the prediction module of the model, a preset score calculation function is called to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

[0022] In a further embodiment, the step of calculating the loss value of the quality score of each training sample in the same batch includes the following steps:

[0023] Calculating a regression loss value of a predicted quality score obtained by a video quality assessment model predicting the training sample based on a supervised score corresponding to the training sample, wherein the supervised score is a subjective quality score;

[0024] Calculate the corresponding cross entropy loss value according to the regression loss value;

[0025] The predicted quality score obtained by the video quality assessment model predicting the training sample and the supervisory score corresponding to the training sample are subjected to the same shuffling process respectively. The difference between the predicted quality score and the shuffled predicted quality score is used as the input of the preset function. The absolute value of the difference between the supervisory score and the shuffled supervisory score and the function output is calculated, and the maximum value between the sum and 0 is taken as the shuffling loss value;

[0026] The sum of the regression loss value, the cross entropy loss value, and the out-of-order loss value is calculated as the loss value.

[0027] In a preferred embodiment, the step of performing gradient updating on the model according to the gradient update value of each training sample includes the following steps:

[0028] Accumulate the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain the corresponding comprehensive update value;

[0029] Gradient updating is performed on the video quality assessment model according to the comprehensive update value.

[0030] In an extended embodiment, after obtaining the quality score of the video stream, the following steps are further included:

[0031] Adjusting the video stream according to the quality score corresponding to the video stream, and calling the video quality assessment model to obtain an adjusted quality score corresponding to the adjusted video stream;

[0032] Determine an adjusted video stream corresponding to the adjusted quality score that meets a preset indicator.

[0033] On the other hand, a video quality assessment device provided to meet one of the purposes of the present application includes a data acquisition module, a formatting module, a feature extraction module and a quality prediction module, wherein: the data acquisition module is used to acquire a video stream to be evaluated; the formatting module is used to obtain a plurality of consecutive image frames from the video stream and format them into standardized data; the feature extraction module is used to extract image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model that has been pre-trained to a convergence state; and the quality prediction module is used to perform classification mapping based on the image feature information by the prediction module of the video quality assessment model to obtain a quality score of the video stream.

[0034] In a further embodiment, the feature extraction module includes: a training set acquisition submodule, which is used to acquire a training set, and the training set contains multiple training samples corresponding to video streams with different video qualities; a synchronous training submodule, which is used to batch call multiple training samples corresponding to multiple video qualities in the training set to implement multi-task synchronous training on the video quality assessment model; a score prediction submodule, which is used to extract image feature information corresponding to each training sample from the video quality assessment model, and then predict the quality score corresponding to each training sample through the prediction module of the model; an iterative training submodule, which is used to calculate the loss value of the quality score of each training sample in the same batch, and its corresponding gradient update value, calculate the average of the multiple loss values, and terminate the training when the average reaches a preset threshold; otherwise, implement gradient update on the model according to the gradient update value of each training sample, and continue to call multiple training samples of the next batch to implement iterative training until the model converges.

[0035] In a further embodiment, before the training set acquisition submodule, it also includes: a first encoding submodule, used to encode the pre-collected high-definition video stream using a constant quality encoding method to obtain a first encoded video stream; a second encoding submodule, used to encode the pre-collected high-definition video stream using a constant bit rate encoding method to obtain a second encoded video stream; a decoding submodule, used to perform corresponding decoding on the first encoded video stream and the second encoded video stream to obtain the decoded video stream as a training set.

[0036] In a preferred embodiment, the score prediction submodule includes: a score calculation unit, which is used to call a preset score calculation function in the score prediction unit in the prediction module of the model to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

[0037] In a further embodiment, the iterative training submodule includes: a regression loss calculation unit, which is used to calculate the regression loss value of the predicted quality score obtained by the video quality assessment model predicting the training sample based on the supervision score corresponding to the training sample, and the supervision score is a subjective quality score; a cross entropy calculation unit, which is used to calculate the corresponding cross entropy loss value based on the regression loss value; a disorder loss calculation unit, which is used to perform the same disorder processing on the predicted quality score obtained by the video quality assessment model predicting the training sample and the supervision score corresponding to the training sample, and use the difference between the predicted quality score and the predicted quality score after disorder processing as the input of a preset function, calculate the absolute value of the difference between the supervision score and the supervision score after disorder processing and the sum of the function output result, and take the maximum value between the sum and 0 as the disorder loss value; a sum calculation unit, which is used to calculate the sum of the regression loss value, the cross entropy loss value and the disorder loss value as the loss value.

[0038] In a preferred embodiment, the iterative training submodule includes: a gradient accumulation unit for accumulating the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain a corresponding comprehensive update value; a gradient update unit for implementing gradient update on the video quality assessment model according to the comprehensive update value.

[0039] In an extended embodiment, the quality prediction module further includes: a quality adjustment module, which is used to adjust the video stream according to the quality score corresponding to the video stream, and call the video quality assessment model to obtain the adjusted quality score corresponding to the adjusted video stream; a video stream determination module, which is used to determine the adjusted video stream corresponding to the adjusted quality score that meets the preset indicators.

[0040] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the video quality assessment method described in the present application.

[0041] On the other hand, a computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the video quality assessment method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the method are executed.

[0042] The technical solution of this application has many advantages, including but not limited to the following:

[0043] On the one hand, the video quality assessment model can be used to perform reference-free video quality assessment on video streams, which is intelligent and efficient and does not require a lot of manpower costs.

[0044] On the other hand, the video quality of the video stream can be accurately reflected based on the use of a video quality assessment model. On the premise of ensuring the video quality of the video stream, the video stream can be adjusted according to actual business needs, so as to meet the user's viewing needs for the video stream and reduce the bandwidth required for video stream transmission without exceeding the user's bandwidth, thereby ensuring the smoothness and stability of video stream transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 A flowchart of a typical embodiment of the video quality assessment method of the present application is shown;

[0047] Figure 2 A schematic diagram illustrating an exemplary network structure of a video quality assessment model in an embodiment of the present application;

[0048] Figure 3 A schematic diagram illustrating an exemplary example of a backbone module of a network structure of a video quality assessment model in an embodiment of the present application;

[0049] Figure 4 Schematic diagram of the video quality assessment model training process in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the process of preparing a training set in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of a process for calculating loss value in an embodiment of the present application;

[0052] Figure 7 This is a flow chart of implementing gradient updating on a model in an embodiment of the present application;

[0053] Figure 8 This is a flow chart of adjusting a video stream using a video quality assessment model in an embodiment of the present application;

[0054] Figure 9 This is a principle block diagram of the video quality assessment device of this application;

[0055] Figure 10 This is a schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0056] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0057] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0058] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0059] It will be understood by those skilled in the art that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, or installed in a vehicle (air, sea, and / or land), or may be adapted and / or configured to operate locally and / or in a distributed manner at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or may include a smart TV, a set-top box, or other device.

[0060] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0061] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0062] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0063] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0064] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0065] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0066] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0067] A video quality assessment method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in the server of an e-commerce platform. The method can be executed by accessing the interface opened after the computer program product is run and performing human-computer interaction with the process of the computer program product through a graphical user interface.

[0068] See also Figure 1 The video quality assessment method of the present application, in its typical embodiment, comprises the following steps:

[0069] Step S1100: obtaining a video stream to be evaluated;

[0070] In one embodiment, generally speaking, in a live broadcast platform, multiple live broadcast room instances are run. After the corresponding anchor user records the video stream on his or her client device, each live broadcast room is pushed to the media server in the service cluster of the live broadcast platform. After being processed by the media server, it is pushed to the client device of the audience user in the live broadcast room to form a live video stream. In a large live broadcast platform, there are usually a large number of live broadcast room instances at the same time, and therefore there are also a large number of concurrent live broadcast video streams. The video quality assessment model of the present application can be deployed on a server in the service cluster, or directly deployed in a media server, to process the video streams of each live broadcast arriving at the media server to provide a video quality assessment service. The service can be used to assess the video quality of the live video stream generated by the live broadcast on the live broadcast platform and predict the corresponding no-reference score. The video stream can be understood as consisting of a series of image frames that are continuous in time, and the corresponding multiple image frames can be obtained by decoding the video stream. Accordingly, the live video stream generated by the live broadcast on the live broadcast platform can be obtained from the server or media server in the service cluster of the live broadcast platform as the video stream to be evaluated.

[0071] In another embodiment, the video quality assessment model of the present application can be deployed in a terminal device to process the live video stream that arrives at the terminal device or is directly captured by the terminal device to provide a video quality assessment service. Generally speaking, in a live broadcast scenario of a terminal device, the live video stream can be captured by the terminal device's camera unit or an independent camera device, and the video stream captured by the independent camera device usually needs to be transmitted to the terminal device. Accordingly, the live video stream generated by the live broadcast can be obtained from the terminal device as the video stream to be evaluated.

[0072] Step S1200: obtaining a plurality of consecutive image frames from the video stream and formatting them into standardized data;

[0073] In one embodiment, for the video stream that has been encoded, the video stream can be decoded by a codec implemented based on the FFmpeg audio and video processing library to obtain a plurality of continuous image frames corresponding to the video stream. The codec includes H.265, H.264, MPEG-4, DivX, etc. The specific implementation can be flexibly implemented by those skilled in the art based on the disclosure herein. For the video stream that has not been encoded, it means that the video stream has not been compressed and some image frames have not been lost, but is composed of complete image frames. Therefore, a plurality of complete continuous image frames in the video stream can be directly obtained.

[0074] Furthermore, the multiple consecutive image frames are formatted accordingly. For example, the image data type corresponding to the image frame is unit8, which needs to be formatted, converted to float data type, and then divided by 255 to normalize to the corresponding value range of [0, 1], thereby obtaining corresponding standardized data.

[0075] Step S1300: extracting image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model that has been pre-trained to a convergence state;

[0076] The network structure of the video quality assessment model is exemplified by Figure 2 As shown, it includes a convolutional layer 200, a backbone module 201, i.e., the feature extraction network, a global average pooling layer 202, and a prediction module 202. The convolutional layer is connected to the backbone module, the backbone module is connected to the global average pooling layer, and the global average pooling layer is connected to the prediction module.

[0077] In one embodiment, the convolution layer 200 is used to downsample the image frames of the video input to the video quality assessment model, and then input the sampling results to the backbone module connected to the convolution layer, and the convolution layer will only be enabled to perform the downsampling when the resolution corresponding to the input video is greater than a preset threshold. Otherwise, when the resolution is lower than the preset threshold, the convolution layer will not be enabled, and the input video skips the convolution layer and is directly input to the backbone module connected to the convolution layer. The preset threshold is the resolution corresponding to the high-resolution video, and 1080p or 720p is recommended. Those skilled in the art can flexibly set the threshold according to actual business needs. It can be seen that the application of the convolution layer can speed up the model's processing speed for high-resolution videos and improve the model's operating efficiency.

[0078] The prediction module 202 includes FC (Full Connection, fully connected layer) and a score prediction unit. The backbone module 200 includes multiple stages of convolution modules, where the convolution module of the previous stage is connected to the convolution module of the next stage, and the convolution module includes a convolution layer and a combination of BN and ReLU layers, such as Figure 3 As shown in the figure, the backbone module of the video quality assessment model includes five stages of convolution modules, and the number of channels corresponding to each convolution module is [32, 64, 128, 256, 512]. This structure is close to the VGG model, which helps to ensure the robustness and generalization ability of the model. Of course, those skilled in the art can flexibly set the convolution modules and their corresponding channel numbers according to the disclosure herein.

[0079] The BN (Batch Normalization) layer uses certain normalization methods to forcibly pull the distribution of input values ​​corresponding to the neurons in the convolutional layer connected to it back to a standard normal distribution with a mean of 0 and a variance of 1, that is, to force the increasingly biased distribution back to a relatively standard distribution, so that the activation input value falls in the area where the nonlinear function is more sensitive to the input. The output of the network will not be very large, and a relatively large gradient can be obtained, avoiding the problem of gradient disappearance. Moreover, a larger gradient means that the learning convergence speed is fast, which can greatly speed up the training speed. According to the principles disclosed herein, in another embodiment, those skilled in the art can flexibly choose the BN layer in the combination of the BN layer and the ReLU layer in the convolution module according to business needs, and can abandon the use of the BN layer and use the convolution layer and ReLU combination alone.

[0080] In one embodiment, a training set can be constructed in advance by collecting multiple video streams of varying video quality, with each video stream serving as a single training sample in the training set. Furthermore, each video stream is annotated with a corresponding true label, which can be obtained by manually scoring the video streams. The training set is then used to conduct supervised training on the video quality assessment model based on the true labels annotated for each sample in the training set, thereby obtaining a video quality assessment model that has been trained to convergence and is then put into use.

[0081] The backbone module in the video quality assessment model that has been pre-trained to convergence, namely the image feature extraction network, is used to extract the visual features of the standardized data and obtain corresponding image feature information.

[0082] Step S1400: The prediction module of the video quality assessment model performs classification mapping according to the image feature information to obtain a quality score of the video stream.

[0083] The global average pooling layer of the video quality assessment model receives the image feature information output by the image feature extraction network, performs spatial averaging on the image feature information and inputs it into the prediction module of the model. The fully connected layer therein is first expanded into a one-dimensional vector according to the corresponding channel direction, and then the one-dimensional vector is mapped to the corresponding classification space by the prediction scoring unit connected to the fully connected layer to obtain the quality score of the video stream.

[0084] It can be seen from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0085] On the one hand, the video quality assessment model can be used to perform reference-free video quality assessment on video streams, which is intelligent and efficient and does not require a lot of manpower costs.

[0086] On the other hand, the video quality of the video stream can be accurately reflected based on the use of a video quality assessment model. On the premise of ensuring the video quality of the video stream, the video stream can be adjusted according to actual business needs, so as to meet the user's viewing needs for the video stream and reduce the bandwidth required for video stream transmission without exceeding the user's bandwidth, thereby ensuring the smoothness and stability of video stream transmission.

[0087] See also Figure 4 In a further embodiment, step S1300, the training process of the video quality assessment model, includes the following steps:

[0088] Step S1310: Obtain a training set, wherein the training set includes a plurality of training samples corresponding to video streams of different video qualities;

[0089] In one embodiment, based on hard indicators of video quality such as bit rate, frame rate, and resolution, a sufficient number of video streams with different video qualities can be manually collected and organized as a training set, where each video stream is used as a single training sample. At the same time, the training samples in the training set are associated with their corresponding video qualities and stored. For example, video quality 1 includes training samples a, b, c, and d, and video quality 2 includes training samples e, f, and g. In addition, each individual training sample has a corresponding subjective quality score marked for it, which is used as a supervision score. The subjective quality score can be obtained by multiple people manually performing subjective quality scoring on the video quality of each image frame in the video stream corresponding to the training sample, and calculating the average of these subjective quality scores.

[0090] Furthermore, the video stream corresponding to the training sample is decoded to obtain multiple corresponding image frames, and data enhancement processing is appropriately performed on these multiple image frames. Flip and crop are recommended. Alternatively, the data enhancement processing can include mirroring, scaling, brightness and contrast adjustment, Gaussian noise, mosaic, mixup, cutout, cutmix, etc., and those skilled in the art can flexibly select and implement these methods based on actual business needs.

[0091] Step S1320: batch call multiple training samples corresponding to multiple video qualities in the training set to perform multi-task synchronous training on the video quality assessment model;

[0092] In one embodiment, partial training samples corresponding to various video qualities in the training set are called in small batches, wherein partial training samples of the same video quality are input into the video quality assessment model each time to implement multi-task synchronous training until partial training samples corresponding to various video qualities are input into the model. The various video qualities may be most or all of the video qualities corresponding to the training set, and the partial training samples may be multiple training samples of the same video quality, 8 of which are recommended. It can be understood that the more the number, the more difficult it is for the model to learn the differences in video quality in the video streams corresponding to the various training samples, which affects the accuracy of the subsequent model prediction quality score. Therefore, a smaller number can be set. Those skilled in the art should know that the multi-task synchronous training can be implemented through a parallel algorithm to achieve a given multiple input and return corresponding multiple output results.

[0093] Step S1330: After the video quality assessment model extracts the image feature information corresponding to each training sample, the prediction module of the model predicts the quality score corresponding to each training sample;

[0094] The backbone module of the video quality assessment model extracts the visual features corresponding to each training sample as the image feature information, wherein the corresponding initial convolution module extracts basic visual features such as details and edges from the training samples, and the corresponding final convolution module extracts deep semantic features and high-level logical features from the training samples. The model's average pooling layer then receives the image feature information output by the backbone module and performs spatial averaging on it. The fully connected layer in the model's prediction module then expands it into a one-dimensional vector along the corresponding channel direction. The prediction scoring unit then predicts a quality score based on this one-dimensional vector.

[0095] Step S1340: Calculate the loss value of the quality score of each training sample in the same batch and its corresponding gradient update value, calculate the average of the multiple loss values, and terminate the training when the average reaches a preset threshold. Otherwise, perform gradient update on the model according to the gradient update value of each training sample, and continue to call the next batch of multiple training samples to perform iterative training until the model converges.

[0096] A preset loss calculation function is used to calculate the loss values ​​of the quality scores corresponding to multiple training samples corresponding to various video qualities in the same batch. Furthermore, the average of the loss values ​​corresponding to the multiple training samples corresponding to the various video qualities is calculated. The loss calculation function includes the calculation of the regression loss, the out-of-order loss, and the cross-entropy loss of the video quality assessment model. The specific implementation is further disclosed in the subsequent embodiments, and this step is not discussed here for the time being.

[0097] It is understandable that multiple gradients corresponding to multiple samples corresponding to various video qualities can be calculated using a back propagation algorithm or a gradient descent algorithm to serve as the gradient update value.

[0098] Determine whether the average value reaches a preset threshold. When the preset threshold is not reached, the weight of the model is updated according to the gradient update value of each training sample, and the corresponding weight of the backbone module of the model is updated. In one embodiment, the gradient update value of each training sample is called one by one to update the weight of the model. The specific implementation can be flexibly implemented by those skilled in the art. In another embodiment, the gradient update value of each training sample is accumulated to update the weight of the model, which is further revealed by the subsequent embodiments. This step is not discussed for the time being. Further, multiple training samples of the next batch are called to implement iterative training until the model converges. When the preset threshold is reached, it is characterized that the model has been trained to convergence.

[0099] This example demonstrates the training process of the video quality assessment model. Once trained to convergence, the model can perform a reference-free assessment of video streams and predict corresponding quality scores. This approach can replace manual effort based on defined criteria in practical applications, enabling intelligent and rapid assessment of video stream quality, resulting in high efficiency.

[0100] See also Figure 5 In a further embodiment, before step S1310, the step of obtaining the training set, the following steps are also included:

[0101] Step S1301: Encode a pre-collected high-definition video stream using a constant-quality encoding method to obtain a first encoded video stream;

[0102] In one embodiment, the constant quality encoding method is CRF (constant rate factor), which is an encoding method with constant video quality and variable bit rate. Various CRF control parameters can be used to adjust the pre-collected high-definition video stream to obtain multiple video streams with similar visual quality as observed by the human eye but different objective video quality as the first encoded video stream. Recommended CRF control parameters include 0, 10, 20, 25, 30, 35, 40, and 45, with higher values ​​indicating higher video quality.

[0103] Step S1302: Encode the pre-collected high-definition video stream using a constant bit rate encoding method to obtain a second encoded video stream;

[0104] In one embodiment, the constant bitrate encoding method is CBR (constant bitrate), which is an encoding method with a constant bitrate and variable video quality. Preset types with different video quality effects and encoding speeds preset based on the FFmpeg audio and video processing library can be used to adjust the pre-captured high-definition video stream to obtain multiple video streams as the second encoded video stream. It is recommended to use all preset types, including ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, and verslow, and the corresponding bitrates for each preset are 100k, 250k, 500k, 750k, 1000k, 1500k, and 2000k.

[0105] Step S1303: Decode the first encoded video stream and the second encoded video stream accordingly to obtain the decoded video streams as training sets.

[0106] In one embodiment, the first encoded video stream and the second encoded video stream can be decoded accordingly by a codec implemented based on the FFmpeg audio and video processing library, thereby obtaining corresponding multiple video streams with different video qualities corresponding to multiple continuous image frames as training sets, wherein the multiple image frames corresponding to each video stream serve as a single training sample. The codecs include H.265, H.264, MPEG-4, DivX, etc. The specific implementation can be flexibly implemented by those skilled in the art based on the disclosure herein.

[0107] It can be understood that the video streams corresponding to a portion of the training samples in the training set are encoded using a constant-quality encoding method, i.e., this portion of the training samples contains video streams of different bitrates but the same video quality. It can be understood that this same video quality is an effect that cannot be distinguished by the human eye, and objectively, the video quality is different. Furthermore, the video streams corresponding to another portion of the training samples in the training set are encoded using a constant-rate encoding method, i.e., this portion of the training samples contains video streams of the same bitrate but different video quality. Based on this, it is not difficult to understand that the training set contains training samples corresponding to video streams of various video qualities. In one embodiment, a small portion of training samples can be randomly extracted from the training samples corresponding to each video quality in the training set as a batch, and then multiple batches can be extracted multiple times. When multiple training samples in the training set are subsequently batched to perform multi-task synchronous training on the video quality assessment model, a batch of training samples is called, and each time a portion of training samples of the same video quality from the same batch is input into the video quality assessment model for multi-task synchronous training, until all the training samples corresponding to each video quality in the batch are input into the model.

[0108] This embodiment, on the one hand, uses constant-quality encoding and constant-bitrate encoding to adjust the captured high-definition video stream, allowing for the rapid and simple acquisition of video streams of varying qualities as training sets. This eliminates the need for extensive human resources to capture these streams, thus reducing the difficulty of constructing the training set. Furthermore, this constant-quality, constant-bitrate encoding allows for the scientific and accurate encoding of video streams of varying qualities, helping to ensure the accuracy and applicability of subsequent video quality assessment models trained accordingly.

[0109] In a preferred embodiment, step S1330, the step of predicting the quality score corresponding to each training sample by the prediction module of the model, includes the following steps:

[0110] Step S1331: In the score prediction unit in the prediction module of the model, a preset score calculation function is called to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

[0111] The corresponding linear calculation in the score calculation function is calculated using the Clamp function, and the corresponding nonlinear calculation is calculated using the Sigmoid function. The exemplary formula is as follows:

[0112] Predict=λ1*Sigmoid(x)+λ2*Clamp(0,1,x)

[0113] Where: Predict is the quality score, λ1 and λ2 are adjustment coefficients that add up to 1. The recommended values ​​of λ1 and λ2 are both set to 0.5, and x is the input.

[0114] It can be understood that the Clamp function can limit the continuously increasing, decreasing or randomly changing numerical values ​​to a certain range of values. Accordingly, Clamp(0,1,x) applied to the scoring calculation function means that the continuously increasing, decreasing or randomly changing input value x is limited to the interval of [0,1], and the input x of the function returns a value within this interval.

[0115] Therefore, the quality score of each training sample can be calculated by the score calculation function.

[0116] In this embodiment, based on the nonlinear Sigmoid output, the linear Clamp output with truncation is combined to overcome the shortcomings of the Sigmoid function, ensure the effectiveness of high-segment and low-segment outputs, and improve the training speed.

[0117] See also Figure 6 In a further embodiment, step S1340, the step of calculating the loss value of the quality score of each training sample in the same batch, includes the following steps:

[0118] Step S1341: Calculate the regression loss value of the predicted quality score obtained by the video quality assessment model predicting the training sample according to the supervised score corresponding to the training sample, wherein the supervised score is a subjective quality score;

[0119] In one embodiment, the supervised score, ie, the subjective quality score, can be obtained by having multiple people perform subjective quality scoring on the video quality of each image frame in the video stream corresponding to the training sample, and calculating the average of the subjective quality scores.

[0120] Calculate the supervised score of each image frame in the video stream corresponding to the training sample, and the predicted quality score of each image frame in the video stream corresponding to the training sample predicted by the video quality assessment model. The absolute value of the difference between the two is used as the regression loss value. The exemplary formula is as follows:

[0121]

[0122] Among them: Lossmos is the regression loss value, n is the number of image frames in the video stream, equal to batch_size, y out The video quality assessment model predicts the predicted quality score of each image frame in the video stream corresponding to the training sample, y gt Provides supervised scores for each image frame in the video stream corresponding to the pre-labeled training samples.

[0123] Step S1342: Calculate a corresponding cross entropy loss value based on the regression loss value;

[0124] Based on the regression loss value, 1-Loss mos As the positive sample output probability, the corresponding cross entropy loss value is calculated. The exemplary formula is as follows:

[0125]

[0126] Among them: Loss ce is the cross entropy loss value, n is the number of image frames in the video stream, equal to batch_size, y out The video quality assessment model predicts the predicted quality score of each image frame in the video stream corresponding to the training sample, y gt Provides supervised scores for each image frame in the video stream corresponding to the pre-labeled training samples.

[0127] Step S1343: Perform the same shuffling process on the predicted quality scores obtained by the video quality assessment model predicting the training samples and the supervisory scores corresponding to the training samples. The difference between the predicted quality scores and the shuffled predicted quality scores is used as the input of a preset function. The sum of the absolute value of the difference between the supervisory scores and the shuffled supervisory scores and the function output is calculated, and the maximum value between the sum and 0 is taken as the shuffling loss value.

[0128] The accuracy of model prediction can be improved by increasing the difference between the predicted quality scores obtained by the video quality assessment model for predicting training samples. Accordingly, the predicted quality scores obtained by the video quality assessment model for each image frame in the video stream corresponding to the training sample, and the supervisory scores corresponding to the image frames in the video stream corresponding to the training sample, are subjected to the same shuffling process, and the predicted quality scores and supervisory scores after shuffling are obtained accordingly. For example, if the predicted quality scores are [0.1, 0.2, 0.3, 0.4, 0.5] and the supervisory scores are [0.2, 0.3, 0.3, 0.4, 0.3], the predicted quality scores after shuffling are [0.5, 0.4, 0.1, 0.3, 0.2], and the supervisory scores after shuffling are [0.3, 0.4, 0.2, 0.3, 0.3]. Furthermore, the difference between the quality scores and the shuffled predicted quality scores is used as the input of a preset function, the preset function being sgn, which is a sign function that can indicate the positive or negative value of the input content. Then calculate the absolute value of the difference between the supervision score and the supervision score after the shuffle and the sum of the function output results, and take the maximum value between the sum and 0 as the shuffle loss value. The exemplary formula is as follows:

[0129]

[0130] Among them: Loss rank is the out-of-order loss value, n is the number of image frames in the video stream, equal to batch_size, y gt is the supervision score corresponding to each image frame in the video stream corresponding to the pre-labeled training sample, r gt is the supervision score after disorder processing, y out The video quality assessment model predicts the predicted quality score of each image frame in the video stream corresponding to the training sample, r out Score the quality of the predictions after shuffling.

[0131] Those skilled in the art should know that in the formula of the example, |y gt -r gt |Provide the adaptive change of the prediction quality score difference after the disorder, sgn(y out -r out ) provides the direction of change in the difference in prediction quality scores after shuffling.

[0132] Step S1344: Calculate the sum of the regression loss value, the cross entropy loss value, and the out-of-order loss value as the loss value.

[0133] The sum of the regression loss value, the cross entropy loss value, and the disorder loss value is calculated as the loss value. The exemplary formula is:

[0134] Loss=Loss mos +Loss ce +Loss rank

[0135] Among them: Loss is the loss value corresponding to the training sample predicted by the video quality assessment model, Loss mos is the regression loss value, Loss ce is the cross entropy loss value, Loss rank is the out-of-order loss value.

[0136] In this embodiment, the calculation of the loss value corresponding to the training sample predicted by the video quality assessment model and the calculation of the three loss values ​​contained therein are disclosed. It can be understood that, on the one hand, the multi-dimensional loss value calculated by this embodiment can accurately reflect the accuracy of the model prediction, so that after the model is trained to convergence according to the multi-dimensional loss value, the accuracy of the model can be guaranteed. In actual business scenarios, the application of this model can accurately evaluate the video quality of the video stream, and then, under the premise of ensuring that the video quality remains unchanged, any one or more of the bit rate, frame rate, and resolution of the video stream can be reduced and adjusted to obtain a video stream with less transmission resources and unchanged video quality. It can be seen that the transmission bandwidth is saved, which can serve the live broadcast scenario with higher real-time requirements and ensure the stability and smoothness of the video stream during the live broadcast process. On the other hand, when training the video quality assessment model, the loss value calculated by applying this embodiment can accelerate the convergence of the model and improve the training efficiency.

[0137] See also Figure 7 In a preferred embodiment, step S1340, the step of performing gradient updating on the model according to the gradient update value of each training sample, includes the following steps:

[0138] Step S1345: Accumulate the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain a corresponding comprehensive update value;

[0139] Generally speaking, manually labeling video streams of varying video quality with scores can result in different video qualities but identical scores. This can prevent the video quality assessment model from converging. Therefore, the model needs to be updated synchronously based on the gradient update values ​​corresponding to the video streams of varying video quality. Accordingly, in one embodiment, the gradient update values ​​corresponding to multiple training samples of the same batch, corresponding to different video qualities, are accumulated to obtain a corresponding composite update value. This gradient update value can be calculated using a backpropagation algorithm or a gradient descent algorithm.

[0140] Step S1346: performing a gradient update on the video quality assessment model according to the comprehensive update value.

[0141] Furthermore, a gradient update is performed on the video quality assessment model according to the comprehensive update value, and the corresponding weights of the backbone modules of the model are updated, thereby ensuring that the model can effectively converge.

[0142] In this embodiment, the gradient of the video quality assessment model is implemented by accumulating the gradients corresponding to training samples of different video qualities, and the updated gradient is larger, which ensures that the model can converge effectively, improves the applicability and robustness of the model, and enables the model to accurately extract the image features corresponding to degraded videos of various video qualities, and then accurately predict the corresponding quality scores.

[0143] See also Figure 8 In an extended embodiment, after step S1400, obtaining the quality score of the video stream, the following steps are further included:

[0144] Step S1500: adjusting the video stream according to the quality score corresponding to the video stream, and calling the video quality assessment model to obtain an adjusted quality score corresponding to the adjusted video stream;

[0145] In one embodiment, a corresponding CRF control parameter is determined based on the quality score corresponding to the video stream, the video stream is encoded using a CRF encoding method based on the CRF control parameter, the bit rate of the video stream is adjusted, and an adjusted video stream is obtained. Furthermore, the video quality assessment model is called to predict the quality score corresponding to the video stream, i.e., the adjusted quality score. The CRF control parameter can be set as needed by those skilled in the art according to actual business needs combined with prior knowledge or experimental experience.

[0146] Step S1600: Determine the adjusted video stream corresponding to the adjusted quality score that meets the preset indicator.

[0147] In one embodiment, an indicator affecting the transmission bandwidth, such as bit rate, can be pre-set to assess whether the adjusted video stream meets the business requirements. Based on the disclosure herein, those skilled in the art can flexibly set the preset indicator. Thus, when the adjusted quality score corresponding to the adjusted video stream reaches the preset indicator, the adjusted video stream is obtained and saved to the media server for use in response to a request from a live broadcast room user. When the adjusted quality score corresponding to the adjusted video stream does not reach the preset indicator, the previous step S1500 is executed again until the preset indicator is reached.

[0148] In this embodiment, by adopting a video quality assessment model to accurately reflect the video quality of the video stream, the video stream can be effectively adjusted to reduce the bandwidth required for video stream transmission while ensuring the video quality of the video stream.

[0149] In an embodiment of deploying the video quality assessment model of the present application in a live broadcast scenario of the terminal device, reference can be made to the above steps S1400-1500. After making corresponding adjustments to the video stream generated by the live broadcast in the terminal device, a video stream that meets the bandwidth required for the actual transmission business is obtained, which can save bandwidth resources for network transmission and ensure stability and smoothness.

[0150] In an embodiment of deploying the video quality assessment model of the present application in a live broadcast scenario of the media server, the above steps S1400-1500 can also be referred to. After making corresponding adjustments to the video stream generated by the live broadcast in the media server, a video stream that meets the bandwidth required for the actual transmission business is obtained, which can save bandwidth resources for network transmission and ensure stability and smoothness.

[0151] See also Figure 9 A video quality assessment device provided to meet one of the purposes of the present application is a functional embodiment of the video quality assessment method of the present application. The device includes a data acquisition module 1100, a formatting module 1200, a feature extraction module 1300 and a quality prediction module 1400, wherein: the data acquisition module 1100 is used to obtain a video stream to be evaluated; the formatting module 1200 is used to obtain a plurality of consecutive image frames from the video stream and format them into standardized data; the feature extraction module 1300 is used to extract image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model pre-trained to a convergence state; the quality prediction module 1400 is used to obtain a quality score of the video stream by performing classification mapping according to the image feature information by the prediction module of the video quality assessment model.

[0152] In a further embodiment, the feature extraction module 1300 includes: a training set acquisition submodule for acquiring a training set, wherein the training set contains multiple training samples corresponding to video streams with different video qualities; a synchronous training submodule for batch calling multiple training samples corresponding to multiple video qualities in the training set to implement multi-task synchronous training on the video quality assessment model; a score prediction submodule for extracting image feature information corresponding to each training sample from the video quality assessment model, and then predicting the quality score corresponding to each training sample through the prediction module of the model; an iterative training submodule for calculating the loss value of the quality score of each training sample in the same batch, and its corresponding gradient update value, and calculating the average of the multiple loss values. When the average reaches a preset threshold, the training is terminated. Otherwise, the model is gradient updated according to the gradient update value of each training sample, and the next batch of training samples is continued to be called to implement iterative training until the model converges.

[0153] In a further embodiment, before the training set acquisition submodule, it also includes: a first encoding submodule, used to encode the pre-collected high-definition video stream using a constant quality encoding method to obtain a first encoded video stream; a second encoding submodule, used to encode the pre-collected high-definition video stream using a constant bit rate encoding method to obtain a second encoded video stream; a decoding submodule, used to perform corresponding decoding on the first encoded video stream and the second encoded video stream to obtain the decoded video stream as a training set.

[0154] In a preferred embodiment, the score prediction submodule includes: a score calculation unit, which is used to call a preset score calculation function in the score prediction unit in the prediction module of the model to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

[0155] In a further embodiment, the iterative training submodule includes: a regression loss calculation unit, which is used to calculate the regression loss value of the predicted quality score obtained by the video quality assessment model predicting the training sample based on the supervision score corresponding to the training sample, and the supervision score is a subjective quality score; a cross entropy calculation unit, which is used to calculate the corresponding cross entropy loss value based on the regression loss value; a disorder loss calculation unit, which is used to perform the same disorder processing on the predicted quality score obtained by the video quality assessment model predicting the training sample and the supervision score corresponding to the training sample, and use the difference between the predicted quality score and the predicted quality score after disorder processing as the input of a preset function, calculate the absolute value of the difference between the supervision score and the supervision score after disorder processing and the sum of the function output result, and take the maximum value between the sum and 0 as the disorder loss value; a sum calculation unit, which is used to calculate the sum of the regression loss value, the cross entropy loss value and the disorder loss value as the loss value.

[0156] In a preferred embodiment, the iterative training submodule includes: a gradient accumulation unit for accumulating the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain a corresponding comprehensive update value; a gradient update unit for implementing gradient update on the video quality assessment model according to the comprehensive update value.

[0157] In an extended embodiment, the quality prediction module 1400 further includes: a quality adjustment module, which is used to adjust the video stream according to the quality score corresponding to the video stream, and call the video quality assessment model to obtain the adjusted quality score corresponding to the adjusted video stream; a video stream determination module, which is used to determine the adjusted video stream corresponding to the adjusted quality score that meets the preset indicators.

[0158] The compression candidate model with comprehensive performance that meets the preset conditions is output as a video quality assessment model.

[0159] In order to solve the above technical problems, the embodiment of the present application also provides a computer device. Figure 10 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions, and the database may store a control information sequence, and when the computer-readable instructions are executed by the processor, the processor may implement a video quality assessment method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor may execute the video quality assessment method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0160] In this embodiment, the processor is used to execute Figure 9 The memory stores the program code and various data required to execute the modules and submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules and submodules in the video quality assessment device of this application. The server can call the server's program code and data to execute the functions of all submodules.

[0161] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video quality assessment method of any embodiment of the present application.

[0162] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0163] In summary, this application uses a highly applicable and accurate video quality assessment model to perform reference-free evaluation on videos and evaluate a quantitative quality score.

[0164] Those skilled in the art will appreciate that the steps, measures, and schemes in the various operations, methods, and processes discussed in this application may be interchanged, modified, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted.

[0165] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video quality assessment method, characterized in that: The steps include: Get the video stream to be evaluated; Obtaining a plurality of consecutive image frames from the video stream and formatting them into standardized data; Extracting image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model that has been pre-trained to a convergence state; The prediction module of the video quality assessment model performs classification mapping according to the image feature information to obtain a quality score of the video stream; The training process of the video quality assessment model includes: encoding a pre-collected high-definition video stream using a constant quality encoding method to obtain a first encoded video stream; encoding the pre-collected high-definition video stream using a constant bit rate encoding method to obtain a second encoded video stream; decoding the first encoded video stream and the second encoded video stream accordingly to obtain the decoded video streams as a training set, wherein the training set contains multiple training samples corresponding to video streams of different video qualities and their supervision scores, and the supervision scores are subjective quality scores; batch calling multiple training samples corresponding to multiple video qualities in the training set to perform multi-task synchronous training on the video quality assessment model, wherein the video quality of multiple training samples in the same batch is the same; after the video quality assessment model extracts image feature information corresponding to each training sample, the prediction module of the model predicts the quality score corresponding to each training sample; calculating the loss value between the supervision score and the quality score of each training sample in the same batch and the corresponding gradient update value, calculating the average of the multiple loss values, and terminating the training when the average value reaches a preset threshold; otherwise, performing gradient update on the model according to the gradient update value of each training sample, and continuing to call multiple training samples in the next batch to perform iterative training until the model converges; The step of calculating the loss value between the supervised score and the quality score of each training sample in the same batch includes: calculating the regression loss value of the predicted quality score obtained by the video quality assessment model predicting the training sample according to the supervised score corresponding to the training sample, and the supervised score is a subjective quality score; calculating the corresponding cross entropy loss value according to the regression loss value; performing the same shuffling processing on the predicted quality score obtained by the video quality assessment model predicting the training sample and the supervised score corresponding to the training sample, and taking the difference between the predicted quality score and the predicted quality score after shuffling processing as the input of a preset function, calculating the absolute value of the difference between the supervised score and the supervised score after shuffling processing and the sum of the function output result, and taking the maximum value between the sum and 0 as the shuffling loss value; calculating the sum of the regression loss value, the cross entropy loss value and the shuffling loss value as the loss value.

2. The video quality assessment method according to claim 1, wherein: The step of predicting the quality score corresponding to each training sample by the prediction module of the model includes the following steps: In the score prediction unit in the prediction module of the model, a preset score calculation function is called to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

3. The video quality assessment method according to claim 1, wherein: The steps of implementing gradient updating on the model according to the gradient update values ​​of each training sample include the following steps: Accumulate the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain the corresponding comprehensive update value; Gradient updating is performed on the video quality assessment model according to the comprehensive update value.

4. The video quality assessment method according to claim 1, wherein: After obtaining the quality score of the video stream, the following steps are further included: Adjusting the video stream according to the quality score corresponding to the video stream, and calling the video quality assessment model to obtain an adjusted quality score corresponding to the adjusted video stream; Determine an adjusted video stream corresponding to the adjusted quality score that meets a preset indicator.

5. A video quality assessment device, characterized in that: include: A data acquisition module, used to obtain the video stream to be evaluated; a formatting module, configured to obtain a plurality of consecutive image frames from the video stream and format them into standardized data; A feature extraction module is used to extract image feature information corresponding to the standardized data using an image feature extraction network in a video quality assessment model that has been pre-trained to a convergence state; a quality prediction module, configured to perform classification mapping by the prediction module of the video quality assessment model according to the image feature information to obtain a quality score of the video stream; The feature extraction module includes: a first encoding submodule, which is used to encode a pre-collected high-definition video stream in a constant quality encoding method to obtain a first encoded video stream; a second encoding submodule, which is used to encode a pre-collected high-definition video stream in a constant bit rate encoding method to obtain a second encoded video stream; a decoding submodule, which is used to decode the first encoded video stream and the second encoded video stream accordingly, and obtain the decoded video stream as a training set, wherein the training set contains multiple training samples corresponding to video streams of different video qualities and their supervision scores, and the supervision scores are subjective quality scores; a synchronous training submodule, which is used to batch call multiple training samples corresponding to multiple video qualities in the training set to perform quality control on the video quality. The quality assessment model implements multi-task synchronous training, and the video quality of multiple training samples in the same batch is the same; the score prediction submodule is used to extract the image feature information corresponding to each training sample by the video quality assessment model, and then predict the quality score corresponding to each training sample through the model's prediction module; the iterative training submodule is used to calculate the loss value between the supervision score and the quality score of each training sample in the same batch, and its corresponding gradient update value, and calculate the average of the multiple loss values. When the average value reaches a preset threshold, the training is terminated. Otherwise, the model is gradient updated according to the gradient update value of each training sample, and the next batch of training samples is continued to be called to perform iterative training until the model converges; The iterative training submodule includes: a regression loss calculation unit, which is used to calculate the regression loss value of the predicted quality score obtained by the video quality assessment model predicting the training sample according to the supervision score corresponding to the training sample, and the supervision score is a subjective quality score; a cross entropy calculation unit, which is used to calculate the corresponding cross entropy loss value according to the regression loss value; a disorder loss calculation unit, which is used to perform the same disorder processing on the predicted quality score obtained by the video quality assessment model predicting the training sample and the supervision score corresponding to the training sample, and use the difference between the predicted quality score and the predicted quality score after disorder processing as the input of a preset function, calculate the sum of the absolute value of the difference between the supervision score and the supervision score after disorder processing and the output of the function, and take the maximum value between the sum and 0 as the disorder loss value; a sum value calculation unit, which is used to calculate the sum of the regression loss value, the cross entropy loss value and the disorder loss value as the loss value.

6. The video quality assessment device according to claim 5, characterized in that: The score prediction submodule includes: a score calculation unit, which is used to call a preset score calculation function in the score prediction unit in the prediction module of the model to calculate the quality score corresponding to each training sample, and the score calculation function includes linear calculation and nonlinear calculation.

7. The video quality assessment device according to claim 5, characterized in that: The iterative training submodule includes: A gradient accumulation unit is used to accumulate the gradient update values ​​corresponding to the multiple training samples in the same batch to obtain a corresponding comprehensive update value; A gradient updating unit is configured to implement a gradient update on the video quality assessment model according to the comprehensive update value.

8. The video quality assessment device according to claim 5, wherein: After the quality prediction module, it also includes: A quality adjustment module, configured to adjust the video stream according to the quality score corresponding to the video stream, and call the video quality assessment model to obtain an adjusted quality score corresponding to the adjusted video stream; The video stream determination module is used to determine the adjusted video stream corresponding to the adjusted quality score that meets the preset indicator.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 4 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images