Live video quality evaluation method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for evaluating the quality of live video. The method comprises: periodically capturing video frames from a first live video at preset time intervals; for each captured video frame, inputting the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each training sample comprises a brightness component and a quality category label of a sample distorted image, and each quality category label is obtained by performing quality evaluation based on the corresponding sample distorted image and a standard definition image corresponding to the sample distorted image; and reporting the quality category of the first live video to a server based on the quality category of each video frame. The use of this method can improve the efficiency and accuracy of quality evaluation of live video on the viewer's terminal.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device, computer equipment and storage medium for evaluating the quality of live video. Background Art

[0002] With the rapid development of the live streaming industry, more and more users are satisfying their daily entertainment needs by watching live streaming.

[0003] During the live broadcast process, after the host terminal captures the video, it will pre-process and encode the video, and upload the encoded bit stream to the server. The server will then perform video transcoding and other operations on the encoded bit stream and send it to the audience terminal as a live video for users to watch.

[0004] However, during the above-mentioned live broadcast process, the live video received by the audience terminal may be distorted. How to quickly and accurately evaluate the quality of the live video received by the audience terminal has become a problem that needs to be solved urgently. Summary of the invention

[0005] Based on this, it is necessary to provide a live video quality evaluation method, device, computer equipment and storage medium that can improve the efficiency and accuracy of quality evaluation of live video on audience terminals in response to the above technical problems.

[0006] In a first aspect, an embodiment of the present application provides a method for evaluating the quality of a live video, the method comprising:

[0007] According to a preset time interval, periodically intercepting video frames from the first live video;

[0008] For each of the captured video frames, inputting the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, and each of the quality category labels is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image;

[0009] Report the quality category of the first live video to the server according to the quality category of each of the video frames.

[0010] In one of the embodiments, the training process of the image quality classification model includes:

[0011] Acquire a plurality of standard definition images and sample distorted images corresponding to each of the standard definition images;

[0012] For each of the sample distorted images, a quality evaluation algorithm is used to perform quality evaluation on the sample distorted image and the corresponding standard definition image to obtain a quality score corresponding to the sample distorted image;

[0013] Determining a score interval corresponding to the quality score, and determining a quality category label of the sample distorted image according to the score interval;

[0014] An initial image quality classification model is trained according to the brightness components and quality category labels of each of the sample distorted images to obtain the image quality classification model.

[0015] In one of the embodiments, before training an initial image quality classification model based on the brightness component and quality category label of each of the sample distorted images to obtain the image quality classification model, the method includes:

[0016] For each of the sample distorted images, a brightness component of the sample distorted image is extracted from the sample distorted image.

[0017] In one embodiment, the step of acquiring a plurality of standard definition images and a sample distorted image corresponding to each of the standard definition images includes:

[0018] Acquire multiple original images, and convert the image format of each of the original images to obtain a standard definition image corresponding to each of the original images;

[0019] Distortion processing is performed on each of the original images, and image format conversion is performed on each of the original images after the distortion processing to obtain a sample distorted image corresponding to each of the original images, wherein the distortion processing includes at least one of scaling processing, filtering processing and encoding processing.

[0020] In one embodiment, the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer, and the luminance component in the video frame is input into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model, including:

[0021] Inputting the brightness component in the video frame to the first feature extraction layer, and performing feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature;

[0022] Inputting the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracting the first image feature through the second feature extraction layer to obtain the second image feature, and extracting the first image feature through the third feature extraction layer to obtain the third image feature;

[0023] Inputting the second image feature and the third image feature into the feature stitching layer, and performing feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature;

[0024] The spliced ​​image features are input into the image quality classification layer to obtain the quality category of the video frame.

[0025] In one embodiment, reporting the quality category of the first live video to the server according to the quality category of each of the video frames includes:

[0026] Detecting whether the quality category of each of the video frames is a preset quality category;

[0027] If there is at least one video frame whose quality category is the preset quality category, the preset quality category is reported to the server as the quality category of the first live video.

[0028] In one embodiment, the method further comprises:

[0029] If the quality category of the first live video is a preset quality category, the first video bit rate of the first live video is reported to the server, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is chronologically subsequent to the first live video.

[0030] In a second aspect, an embodiment of the present application provides a live video quality evaluation device, the device comprising:

[0031] An interception module, used for periodically intercepting video frames from the first live video at preset time intervals;

[0032] A classification module, for inputting the brightness component in each captured video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, and each of the quality category labels is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image;

[0033] The reporting module is used to report the quality category of the first live video to the server according to the quality category of each of the video frames.

[0034] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method of the first aspect described above.

[0036] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0037] By periodically capturing video frames from the first live video at preset time intervals, for each captured video frame, inputting the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each training sample includes a brightness component and a quality category label of a sample distorted image, each quality category label is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image, and then, based on the quality category of each video frame, reporting the quality category of the first live video to a server; in this way, compared with manually scoring the quality of the sample distorted image based on experience and using the manually scored score as the quality category label of the sample distorted image, the quality category label of the embodiment of the present application is obtained by performing quality evaluation based on the corresponding sample distorted image and the standard definition image corresponding to the sample distorted image, which avoids the subjective error caused by determining the label by manual scoring, improves the accuracy of the quality category label, and thereby improves the accuracy of the image quality classification model and the accuracy of the quality category of the video frame. In addition, the embodiment of the present application inputs the brightness component in the video frame into the image quality classification model, that is, the quality category of the video frame can be obtained by processing the brightness component in the video frame through the image quality classification model only, without processing the entire video frame. This reduces the data processing amount of the image quality classification model, thereby improving the classification efficiency of the image quality classification model and improving the quality evaluation efficiency of the first live video. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram showing an application environment of a live video quality evaluation method in one embodiment;

[0039] Figure 2Schematic diagram of a flow chart of a method for evaluating the quality of a live video in one embodiment;

[0040] Figure 3 is a flowchart of a training process of an image quality classification model in one embodiment;

[0041] Figure 4 is a flow chart of step 301 in one embodiment;

[0042] Figure 5 is a network structure diagram of an exemplary initial image quality classification model in one embodiment;

[0043] Figure 6 A schematic diagram of a flow chart of a viewer terminal obtaining a quality category of a video frame through an image quality classification model in one embodiment;

[0044] Figure 7 Schematic diagram of a flow chart of a method for evaluating the quality of a live video in one embodiment;

[0045] Figure 8 Schematic diagram of a flow chart of a method for evaluating the quality of a live video in one embodiment;

[0046] Fig. 9 is a structural block diagram of a live video quality evaluation device in one embodiment;

[0047] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] Live broadcast refers to live video broadcasting over the Internet. During the live broadcast, the host's terminal (hereinafter referred to as the host terminal) can shoot video, which is then pre-processed and encoded, and the encoded bitstream is sent to the server. The server transcodes the bitstream and sends it to the audience's terminal (hereinafter referred to as the audience terminal). The audience terminal decodes the received bitstream and then renders it on the screen.

[0050] During the live broadcast process, the host terminal and the server may cause video distortion based on the processing of the captured video. When the distortion is large, it will seriously affect the live viewing experience of the audience terminal. Therefore, it is necessary to quickly and accurately evaluate the quality of the live video decoded by the audience terminal to improve the audience's live viewing experience.

[0051] In view of this, an embodiment of the present application proposes a method for evaluating the quality of live video, in which video frames are periodically captured from a first live video at preset time intervals, and for each captured video frame, the brightness component in the video frame is input into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each training sample comprising a brightness component and a quality category label of a sample distorted image, and each quality category label is based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image. The quality evaluation is obtained, and then, according to the quality category of each video frame, the quality category of the first live video is reported to the server; in this way, compared with the method of manually scoring the sample distorted image based on experience and using the manually scored score as the quality category label of the sample distorted image, the quality category label of the embodiment of the present application is obtained by quality evaluation based on the corresponding sample distorted image and the standard definition image corresponding to the sample distorted image, which avoids the subjective error caused by manual scoring to determine the label, improves the accuracy of the quality category label, thereby improving the accuracy of the image quality classification model and the accuracy of the quality category of the video frame. In addition, the embodiment of the present application inputs the brightness component in the video frame into the image quality classification model, that is, the image quality classification model only needs to process the brightness component in the video frame to obtain the quality category of the video frame, without having to process the entire video frame, which reduces the data processing amount of the image quality classification model, thereby improving the classification efficiency of the image quality classification model and improving the quality evaluation efficiency of the first live video.

[0052] Below, the implementation environment involved in the live video quality evaluation method provided in the embodiment of the present application will be briefly described.

[0053] like Figure 1 As shown, the implementation environment may include an anchor terminal 101, a server 102, and an audience terminal 103. The anchor terminal 101 may communicate with the server 102 via a wired or wireless network, and the audience terminal 103 may also communicate with the server 102 via a wired or wireless network.

[0054] The server 102 may be a single server or a server cluster consisting of multiple servers. The server 102 may be a tower server, a rack server, a blade server, a high-density server, a single-channel server, a dual-channel server, or a multi-channel server, etc. The embodiment of the present application does not specifically limit the type of the server 102. The anchor terminal 101 and the audience terminal 103 may be a personal computer, a laptop computer, a media player, a smart TV, a smart phone, a tablet computer, and a portable wearable device, etc. The embodiment of the present application does not specifically limit the type of the anchor terminal 101 and the audience terminal 103.

[0055] In one embodiment, Figure 2 As shown, a method for evaluating the quality of live video is provided. Figure 1 Taking the viewer terminal shown in FIG. 1 as an example, the method includes steps 201, 202 and 203:

[0056] Step 201: The viewer terminal periodically captures video frames from the first live video at a preset time interval.

[0057] During the live broadcast, the server performs operations such as transcoding on the encoded bitstream uploaded by the anchor terminal and sends it to the viewer terminal, and the viewer terminal decodes the received bitstream to obtain the first live video. The viewer terminal renders the first live video and then displays it on the screen for the viewer to watch.

[0058] The first live video is composed of continuous video frames. In the embodiment of the present application, the audience terminal can periodically capture video frames from the first live video according to a preset time interval. The preset time interval can be, for example, 2 seconds, 3 seconds, etc., and is not specifically limited here.

[0059] Step 202: For each captured video frame, the viewer terminal inputs the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model.

[0060] In an embodiment of the present application, the video frame may be image data in YUV format, and each video frame includes three components: Y, U, and V, wherein Y represents brightness, that is, grayscale value, and U and V represent chromaticity, which is used to specify the color of a pixel.

[0061] Every time the viewer terminal captures a video frame, the viewer terminal extracts the brightness component from the video frame, that is, extracts the Y component, and inputs the extracted brightness component into the image quality classification model. The image quality classification model performs feature extraction on the brightness component, and classifies based on the extracted features to obtain the quality category of the video frame output by the image quality classification model, such as excellent, good, medium, poor, etc. In this way, the viewer terminal only needs to process the Y component of the video frame through the image quality classification model to obtain the quality category of the video frame, without having to process the U and V components of the video frame, thereby reducing the data processing amount of the viewer terminal and improving the classification efficiency of the image quality classification model.

[0062] In an embodiment of the present application, the image quality classification model is trained based on multiple training samples, each training sample includes a brightness component and a quality category label of a sample distorted image, and each quality category label is obtained by performing quality evaluation based on the corresponding sample distorted image and a standard definition image corresponding to the sample distorted image. In one possible implementation, the image quality classification model may be pre-trained by the viewer terminal based on multiple training samples. In another possible implementation, the image quality classification model may also be trained by a server based on multiple training samples, and the server sends the trained image quality classification model to the viewer terminal, and so on. No specific limitation is made to the training subject of the image quality classification model herein.

[0063] Taking the server training of the image quality classification model as an example, the server obtains multiple sample distorted images and a standard definition image corresponding to each sample distorted image. The standard definition image can be image data in YUV format obtained by format conversion of the original image in the video shot by the anchor terminal, and the sample distorted image can be image data in YUV format obtained by distorting the original image and format conversion of the distorted image.

[0064] For each sample distorted image, the server uses quality evaluation algorithms such as PSNR (Peak Signal Noise Rate), SSIM (Structural SIMilarity) or VMAF (Video Multi-method Assessment Fusion) to perform quality evaluation on the sample distorted image and the corresponding standard definition image, and obtain a quality score corresponding to the sample distorted image. The server can then determine the quality category label corresponding to the sample distorted image based on the quality score. The server inputs the brightness component and the corresponding quality category label of each sample distorted image into the initial image quality classification model, iteratively trains the initial image quality classification model, and obtains the image quality classification model after the model converges. In an embodiment of the present application, the initial image quality classification model can be a convolutional neural network framework such as VggNet or ResNet, which is not specifically limited here.

[0065] In this way, compared with the method of manually scoring the quality of the sample distorted image based on experience and using the manually scored score as the quality category label of the sample distorted image, the quality category label in the embodiment of the present application is obtained by using a quality evaluation algorithm to perform quality evaluation on the sample distorted image and the standard definition image corresponding to the sample distorted image. This avoids the errors caused by manual scoring to determine the label due to inattention or subjective differences, improves the accuracy of the quality category label, and thus improves the accuracy of the image quality classification model and the accuracy of the quality category of the video frame.

[0066] Step 203: The viewer terminal reports the quality category of the first live video to the server according to the quality category of each video frame.

[0067] Every time a viewer terminal captures a video frame, the brightness component in the video frame is input into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model, such as excellent, good, medium or poor.

[0068] In one possible implementation, the viewer terminal can detect whether the quality category of each video frame is a preset quality category. The preset quality category may be, for example, poor. If there is at least one video frame with a quality category of the preset quality category, the viewer terminal reports the preset quality category as the quality category of the first live video to the server.

[0069] For example, if the audience terminal detects that the quality of a preset number of consecutive video frames is poor, such as the quality category of three consecutive video frames is poor, the audience terminal determines that the quality of the first live video is poor, and reports to the server that the quality category of the first live video is poor. In this way, the server can improve the quality of the live video of the audience terminal.

[0070] In another possible implementation, if the viewer terminal detects that the number of video frames with poor quality within a preset time period is greater than a preset number threshold, the viewer terminal may also determine that the quality of the first live video is poor, and report to the server that the quality category of the first live video is poor, etc. The manner in which the viewer terminal determines the quality category of the first live video according to the quality category of each video frame is not specifically limited.

[0071] As an implementation mode, if the viewer terminal determines that the quality category of the first live video is not bad, that is, the quality of the first live video is better, the viewer terminal does not report the quality category of the first live video to the server, thereby reducing the communication overhead between the viewer terminal and the server.

[0072] The above embodiment periodically captures video frames from the first live video at preset time intervals, and for each captured video frame, inputs the brightness component in the video frame into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on multiple training samples, each training sample includes a brightness component and a quality category label of a sample distorted image, and each quality category label is obtained by performing quality evaluation based on the corresponding sample distorted image and the standard definition image corresponding to the sample distorted image, and then, based on the quality category of each video frame, the quality category of the first live video is reported to the server; in this way, compared with the method of manually scoring the quality of the sample distorted image based on experience and using the manually scored score as the quality category label of the sample distorted image, the quality category label of the embodiment of the present application is obtained by performing quality evaluation based on the corresponding sample distorted image and the standard definition image corresponding to the sample distorted image, which avoids the subjective error caused by manual scoring to determine the label, improves the accuracy of the quality category label, and thus improves the accuracy of the image quality classification model and the accuracy of the quality category of the video frame. In addition, the embodiment of the present application inputs the brightness component in the video frame into the image quality classification model, that is, the quality category of the video frame can be obtained by processing the brightness component in the video frame through the image quality classification model only, without processing the entire video frame. This reduces the data processing amount of the image quality classification model, thereby improving the classification efficiency of the image quality classification model and improving the quality evaluation efficiency of the first live video.

[0073] In one embodiment, based on the above Figure 2 The embodiment shown in FIG. 1 is a training process of an image quality classification model. Figure 3 , the training process of the image quality classification model may include step 301, step 302, step 303, step 304 and step 305:

[0074] Step 301: Acquire a plurality of standard definition images and a sample distorted image corresponding to each standard definition image.

[0075] In this embodiment, the training process of the image quality classification model is described by taking the image quality classification model trained by the server as an example.

[0076] In a possible implementation of step 301, a plurality of standard definition images and sample distorted images corresponding to each standard definition image may be preset in the training sample database of the server. The standard definition image may be original image data in YUV format, and the sample distorted image may be distorted image data in YUV format corresponding to the original image data. The server may then obtain a plurality of standard definition images and sample distorted images corresponding to each standard definition image from the training sample database.

[0077] In another possible implementation of step 301, see Figure 4 , step 301 may include: Figure 4 Step 3011 and step 3012 shown:

[0078] Step 3011, obtain multiple original images, and convert the image format of each original image to obtain a standard definition image corresponding to each original image.

[0079] The server obtains multiple original images from the video captured by the anchor terminal. The original images are original image data in RGB format. For each original image, the server uses the following formula 1-3 to convert the format of the original image to obtain a standard definition image in YUV format corresponding to the original image:

[0080] Y=0.299*R+0.587*G+0.114*B Formula 1

[0081] U=-0.169*R-0.331*G+0.5*B Formula 2

[0082] V=0.5*R-0.419*G-0.081*B Formula 3

[0083] Among them, (R, G, B) is the RGB pixel value of a pixel in the original image, and (Y, U, V) is the YUV pixel value of the pixel at the corresponding position in the standard definition image. In this way, through formulas 1-3, the server can obtain the standard definition image in YUV format corresponding to each original image.

[0084] Step 3012: Perform distortion processing on each original image, and perform image format conversion on each original image after distortion processing to obtain a sample distorted image corresponding to each original image.

[0085] The server simulates the distortion that may exist in the transmission process of the original image in the live broadcast scenario, and performs distortion processing on each original image. The distortion processing includes at least one of scaling processing, filtering processing and encoding processing. After the distortion processing, the server obtains each original image after the distortion processing.

[0086] Each original image after distortion processing is distorted image data in RGB format. The server uses the above formulas 1-3 to convert the format of each original image after distortion processing to obtain a sample distorted image in YUV format corresponding to each original image after distortion processing.

[0087] In this way, the server obtains a plurality of standard definition images in YUV format and a sample distorted image in YUV format corresponding to each standard definition image.

[0088] Step 302: For each sample distorted image, a quality evaluation algorithm is used to perform quality evaluation on the sample distorted image and the corresponding standard definition image to obtain a quality score corresponding to the sample distorted image.

[0089] In the embodiment of the present application, the quality evaluation algorithm may be VMAF, which calculates a final image quality score by integrating multiple evaluation indicators, and can better reflect the subjective quality. For each sample distorted image, the server performs a quality evaluation on the sample distorted image and the corresponding standard definition image through the interface provided by VMAF, and obtains a quality score corresponding to the sample distorted image.

[0090] Step 303: determine a score interval corresponding to the quality score, and determine a quality category label of the sample distorted image according to the score interval.

[0091] The quality score obtained by VMAF ranges from 0 to 100, where 0 indicates that the quality of the sample distorted image is extremely poor, and 100 indicates that the quality of the sample distorted image is extremely good.

[0092] Since the quality score obtained by VMAF will fluctuate within a certain range, the embodiment of the present application divides the quality score into 4 intervals: 100-80, 80-60, 60-40, and 40-0, and these 4 intervals correspond to the quality category labels excellent, good, medium, and poor, respectively. That is, the quality category label of the sample distorted image with a quality score between 100-80 is determined as "excellent", the quality category label of the sample distorted image with a quality score between 80-60 is determined as "good", the quality category label of the sample distorted image with a quality score between 60-40 is determined as "medium", and the quality category label of the sample distorted image with a quality score between 0-40 is determined as "poor".

[0093] In this way, for each sample distorted image, the server determines the score interval corresponding to the quality score of the sample distorted image. If the score interval is 100-80, the quality category label of the sample distorted image is determined to be excellent. If the score interval is 60-80, the quality category label of the sample distorted image is determined to be good, and so on.

[0094] Step 304: for each sample distorted image, extract the brightness component of the sample distorted image from the sample distorted image.

[0095] After determining the quality category label of each sample distorted image, the server extracts the brightness component, that is, extracts the Y component, from each sample distorted image in the YUV format.

[0096] Step 305 : training an initial image quality classification model according to the brightness component and quality category label of each sample distorted image to obtain an image quality classification model.

[0097] In the embodiment of the present application, the initial image quality classification model may be a Resnet residual network structure. Figure 5 , Figure 5 is a network structure diagram of an exemplary initial image quality classification model. Figure 5 As shown, the convolutional layers of the initial image quality classification model may include conv0, conv1, conv2, conv3, conv4 and conv5, relu0, relu1 and relu2 are all activation layers, pool is a pooling layer, and fc is a fully connected layer. Due to the small number of layers, the resource consumption during model training and use is relatively small.

[0098] In a possible implementation, the convolution kernel size of conv0 can be 5*5, and the number of convolution kernels can be 32; the convolution kernel size of conv1 can be 1*1, and the number of convolution kernels can be 16; the convolution kernel size of conv2 can be 3*1, and the number of convolution kernels can be 8; the convolution kernel size of conv3 can be 1*3, and the number of convolution kernels can be 8; the convolution kernel size of conv4 can be 1*1, and the number of convolution kernels can be 8; the convolution kernel size of conv5 can be 1*1, and the number of convolution kernels can be 8. In this way, the convolution kernel size, the number of convolution kernels, and the number of network layers in the initial image quality classification model of the embodiment of the present application are all small, which reduces the resource consumption during model training and use, and the introduction of the residual structure can also improve the classification effect.

[0099] After the server resizes the brightness component of the sample distorted image according to the input size requirements of the initial image quality classification model, the brightness component and quality category label of each sample distorted image are input into the initial image quality classification model. After being processed by each convolution layer, activation layer, and pooling layer of the initial image quality classification model, the quality category of each sample distorted image is output through the fully connected layer. The server uses the cross entropy loss function to calculate the loss value between the quality category of each sample distorted image and the corresponding quality category label, and adjusts the model parameters of the initial image quality classification model according to the loss value corresponding to each sample distorted image. The server inputs the brightness component and quality category label of each sample distorted image as input again into the initial image quality classification model after parameter adjustment, and iterates repeatedly until the loss value is less than the preset threshold. The server determines that the initial image quality classification model converges and obtains the image quality classification model.

[0100] In a possible implementation, the server may also use an optimizer, such as an adaptive moment estimation Adam optimizer, to optimize the trained model to obtain a final image quality classification model.

[0101] In an embodiment of the present application, the server can integrate the image quality classification model into the installation package of the live broadcast application. In this way, if the viewer terminal requests the server to install the live broadcast application, the server will send the installation package of the live broadcast application integrated with the image quality classification model to the viewer terminal. After the viewer terminal installs the live broadcast application, when the viewer watches the live video through the installed live broadcast application, the viewer terminal periodically captures video frames from the first live video at a preset time interval. For each captured video frame, the brightness component in the video frame is input into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model; the viewer terminal reports the quality category of the first live video to the server based on the quality category of each video frame, if it is determined that the quality of the first live video is poor, and the server improves the quality of the live video of the viewer terminal, thereby ensuring the quality stability of the live video of the viewer terminal and improving the user's viewing experience. In addition, the viewer terminal of this embodiment does not need to train the image quality classification model by itself, thereby reducing the computing pressure of the viewer terminal.

[0102] In one embodiment, based on the above Figure 2 The embodiment shown, see Figure 6 , this embodiment relates to the process of how the viewer terminal obtains the quality category of the video frame through the image quality classification model. The image quality classification model of this embodiment includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer. The server can execute the following Figure 6Steps 601, 602, 603 and 604 shown in the figure implement the process of inputting the brightness component in the video frame into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model:

[0103] In step 601, the viewer terminal inputs the brightness component in the video frame to the first feature extraction layer, and the first feature extraction layer performs feature extraction on the brightness component in the video frame to obtain a first image feature.

[0104] In a possible implementation, the network structure of the image quality classification model can be as follows: Figure 5 As shown, the first feature extraction layer may include conv0, relu0 and pool layers. The viewer terminal inputs the brightness component in the video frame as input to the first feature extraction layer, and the first feature extraction layer extracts the brightness component in the video frame to obtain the first image feature.

[0105] In step 602, the viewer terminal inputs the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracts the first image feature through the second feature extraction layer to obtain the second image feature, and extracts the first image feature through the third feature extraction layer to obtain the third image feature.

[0106] Please continue to see Figure 5 The second feature extraction layer may include conv1, relu1, conv2, conv3, relu2, conv4, and the third feature extraction layer may include conv5. The audience terminal inputs the first image feature into the second feature extraction layer and the third feature extraction layer respectively to obtain the second image feature and the third image feature.

[0107] In step 603, the viewer terminal inputs the second image feature and the third image feature into the feature stitching layer, and the feature stitching layer stitches the second image feature and the third image feature to obtain a stitched image feature.

[0108] The audience terminal inputs the second image feature and the third image feature into the feature stitching layer, performs feature stitching on the second image feature and the third image feature, and after feature stitching, the feature can be pooled through the pooling layer to obtain the stitched image feature.

[0109] In step 604, the viewer terminal inputs the spliced ​​image features into the image quality classification layer to obtain the quality category of the video frame.

[0110] The viewer terminal inputs the spliced ​​image features into the image quality classification layer to obtain the quality category of the video frame. In this way, the convolution kernel size, the number of convolution kernels, and the number of network layers of the image quality classification model in the embodiment of the present application are all small, which reduces the computational pressure of the viewer terminal in the process of obtaining the quality category of the video frame, improves the classification efficiency of the quality category of the video frame, and introduces the residual structure to further improve the classification effect of the quality category of the video frame.

[0111] In one embodiment, based on the above Figure 2 The embodiment shown, see Figure 7 This embodiment relates to the process of how the server improves the quality of the live video of the viewer terminal. Figure 7 As shown, the quality evaluation method of live video in this embodiment may further include step 204:

[0112] Step 204: If the quality category of the first live video is a preset quality category, the viewer terminal reports the first video bit rate of the first live video to the server.

[0113] In an embodiment of the present application, for each captured video frame, the viewer terminal inputs the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, such as excellent, good, medium or poor.

[0114] The viewer terminal detects whether the quality category of each video frame is a preset quality category, such as poor. If there is at least one video frame with a quality category of the preset quality category, the viewer terminal determines that the quality category of the first live video is the preset quality category.

[0115] The viewer terminal reports the first video bit rate of the first live video to the server, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is after the first live video in time sequence.

[0116] After receiving the bitstream uploaded by the anchor terminal, the server will transcode the bitstream using different bitrates to obtain video data with multiple bitrates. In this way, when the server receives the first video bitrate sent by the viewer terminal, it determines that the quality of the live video of the viewer terminal is poor. When the server subsequently sends the live video to the viewer terminal, it selects the live video with a video bitrate greater than the first video bitrate, that is, the second live video data is sent to the viewer terminal. In this way, when the quality of the live video of the viewer terminal is poor, the bitrate of the live video of the viewer terminal is increased to improve the quality of the live video of the viewer terminal, so that the quality of the live picture of the viewer terminal remains stable and clear, improving the user's viewing experience.

[0117] In one embodiment, please refer to Figure 8 , which shows a flowchart of an exemplary live video quality evaluation method provided by an embodiment of the present application, which can be applied to Figure 1 In the implementation environment shown. Figure 8 As shown, the method may include the following steps:

[0118] Step 8001: The server sends a first live video to be decoded to a viewer terminal.

[0119] In step 8002, the viewer terminal decodes the first live video to be decoded to obtain the first live video, and periodically captures video frames from the first live video at preset time intervals.

[0120] Step 8003, for each captured video frame, the viewer terminal inputs the brightness component in the video frame to the first feature extraction layer of the image quality classification model, and extracts features of the brightness component in the video frame through the first feature extraction layer to obtain the first image feature.

[0121] The image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer. The image quality classification model is obtained by training based on multiple training samples, each training sample includes a brightness component and a quality category label of a sample distorted image, and each quality category label is obtained by quality evaluation based on the corresponding sample distorted image and the standard definition image corresponding to the sample distorted image.

[0122] In step 8004, the audience terminal inputs the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracts the first image feature through the second feature extraction layer to obtain the second image feature, and extracts the first image feature through the third feature extraction layer to obtain the third image feature.

[0123] Step 8005: The viewer terminal inputs the second image feature and the third image feature into the feature stitching layer, and the feature stitching layer performs feature stitching on the second image feature and the third image feature to obtain a stitched image feature.

[0124] Step 8006: The viewer terminal inputs the spliced ​​image features into the image quality classification layer to obtain the quality category of the video frame.

[0125] Step 8007: The viewer terminal detects whether the quality category of each video frame is a preset quality category.

[0126] Step 8008: If there is at least one video frame whose quality category is the preset quality category, the viewer terminal reports the preset quality category as the quality category of the first live video to the server.

[0127] Step 8009: If the quality category of the first live video is a preset quality category, the viewer terminal reports the first video bit rate of the first live video to the server.

[0128] The first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is after the first live video in terms of timing.

[0129] Step 8010: The server sends a second live video to the viewer terminal.

[0130] It should be understood that although Figure 2-8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0131] In one embodiment, Fig. 9 As shown, a live video quality evaluation device is provided, the device comprising:

[0132] The capture module 901 is used to periodically capture video frames from the first live video at a preset time interval;

[0133] A classification module 902 is used for inputting the brightness component in each captured video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is obtained by training based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, and each of the quality category labels is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image;

[0134] The reporting module 903 is used to report the quality category of the first live video to the server according to the quality category of each of the video frames.

[0135] In one embodiment, the apparatus further comprises:

[0136] An acquisition module, used for acquiring a plurality of standard definition images and a sample distorted image corresponding to each of the standard definition images;

[0137] An evaluation module, configured to, for each of the sample distorted images, perform quality evaluation on the sample distorted image and the corresponding standard definition image using a quality evaluation algorithm to obtain a quality score corresponding to the sample distorted image;

[0138] A determination module, used to determine a score interval corresponding to the quality score, and determine a quality category label of the sample distorted image according to the score interval;

[0139] The training module is used to train an initial image quality classification model according to the brightness component and quality category label of each sample distorted image to obtain the image quality classification model.

[0140] In one embodiment, the apparatus further comprises:

[0141] The extraction module is used for extracting the brightness component of the sample distorted image from each of the sample distorted images.

[0142] In one embodiment, the acquisition module includes:

[0143] A first acquisition unit is used to acquire a plurality of original images, and perform image format conversion on each of the original images to obtain a standard definition image corresponding to each of the original images;

[0144] The second acquisition unit is used to perform distortion processing on each of the original images and convert the image format of each of the original images after the distortion processing to obtain a sample distorted image corresponding to each of the original images, wherein the distortion processing includes at least one of scaling processing, filtering processing and encoding processing.

[0145] In one embodiment, the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer, and the classification module 902 includes:

[0146] A first input unit, configured to input a brightness component in the video frame into the first feature extraction layer, and perform feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature;

[0147] a second input unit, configured to input the first image feature to the second feature extraction layer and the third feature extraction layer respectively, extract the first image feature through the second feature extraction layer to obtain the second image feature, and extract the first image feature through the third feature extraction layer to obtain the third image feature;

[0148] A third input unit is used to input the second image feature and the third image feature into the feature stitching layer, and perform feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature;

[0149] The fourth input unit is used to input the spliced ​​image features into the image quality classification layer to obtain the quality category of the video frame.

[0150] In one embodiment, the reporting module 903 includes:

[0151] A detection unit, used to detect whether the quality category of each of the video frames is a preset quality category;

[0152] A reporting unit is configured to report the preset quality category as the quality category of the first live video to the server if there is at least one video frame whose quality category is the preset quality category.

[0153] In one embodiment, the reporting module is also used to report the first video bit rate of the first live video to the server if the quality category of the first live video is a preset quality category, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is chronologically subsequent to the first live video.

[0154] For the specific definition of the quality evaluation device for live video, please refer to the definition of the quality evaluation method for live video applied to the target terminal in the above text, which will not be repeated here. Each module in the above-mentioned quality evaluation device for live video can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for evaluating the quality of live video is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0156] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0158] According to a preset time interval, periodically intercepting video frames from the first live video;

[0159] For each of the captured video frames, inputting the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, and each of the quality category labels is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image;

[0160] Report the quality category of the first live video to the server according to the quality category of each of the video frames.

[0161] In one embodiment, the processor further implements the following steps when executing the computer program:

[0162] Acquire a plurality of standard definition images and sample distorted images corresponding to each of the standard definition images;

[0163] For each of the sample distorted images, a quality evaluation algorithm is used to perform quality evaluation on the sample distorted image and the corresponding standard definition image to obtain a quality score corresponding to the sample distorted image;

[0164] Determining a score interval corresponding to the quality score, and determining a quality category label of the sample distorted image according to the score interval;

[0165] An initial image quality classification model is trained according to the brightness components and quality category labels of each of the sample distorted images to obtain the image quality classification model.

[0166] In one embodiment, the processor further implements the following steps when executing the computer program:

[0167] For each of the sample distorted images, a brightness component of the sample distorted image is extracted from the sample distorted image.

[0168] In one embodiment, the processor further implements the following steps when executing the computer program:

[0169] Acquire multiple original images, and convert the image format of each of the original images to obtain a standard definition image corresponding to each of the original images;

[0170] Distortion processing is performed on each of the original images, and image format conversion is performed on each of the original images after the distortion processing to obtain a sample distorted image corresponding to each of the original images, wherein the distortion processing includes at least one of scaling processing, filtering processing and encoding processing.

[0171] In one embodiment, the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer, and the processor further implements the following steps when executing the computer program:

[0172] Inputting the brightness component in the video frame to the first feature extraction layer, and performing feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature;

[0173] Inputting the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracting the first image feature through the second feature extraction layer to obtain the second image feature, and extracting the first image feature through the third feature extraction layer to obtain the third image feature;

[0174] Inputting the second image feature and the third image feature into the feature stitching layer, and performing feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature;

[0175] The spliced ​​image features are input into the image quality classification layer to obtain the quality category of the video frame.

[0176] In one embodiment, the processor further implements the following steps when executing the computer program:

[0177] Detecting whether the quality category of each of the video frames is a preset quality category;

[0178] If there is at least one video frame whose quality category is the preset quality category, the preset quality category is reported to the server as the quality category of the first live video.

[0179] In one embodiment, the processor further implements the following steps when executing the computer program:

[0180] If the quality category of the first live video is a preset quality category, the first video bit rate of the first live video is reported to the server, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is chronologically subsequent to the first live video.

[0181] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0182] According to a preset time interval, periodically intercepting video frames from the first live video;

[0183] For each of the captured video frames, inputting the brightness component in the video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the image quality classification model is trained based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, and each of the quality category labels is obtained by performing quality evaluation based on a corresponding sample distorted image and a standard definition image corresponding to the sample distorted image;

[0184] Report the quality category of the first live video to the server according to the quality category of each of the video frames.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] Acquire a plurality of standard definition images and sample distorted images corresponding to each of the standard definition images;

[0187] For each of the sample distorted images, a quality evaluation algorithm is used to perform quality evaluation on the sample distorted image and the corresponding standard definition image to obtain a quality score corresponding to the sample distorted image;

[0188] Determining a score interval corresponding to the quality score, and determining a quality category label of the sample distorted image according to the score interval;

[0189] An initial image quality classification model is trained according to the brightness components and quality category labels of each of the sample distorted images to obtain the image quality classification model.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] For each of the sample distorted images, a brightness component of the sample distorted image is extracted from the sample distorted image.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] Acquire multiple original images, and convert the image format of each of the original images to obtain a standard definition image corresponding to each of the original images;

[0194] Distortion processing is performed on each of the original images, and image format conversion is performed on each of the original images after the distortion processing to obtain a sample distorted image corresponding to each of the original images, wherein the distortion processing includes at least one of scaling processing, filtering processing and encoding processing.

[0195] In one embodiment, the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer. When the computer program is executed by a processor, the following steps are also implemented:

[0196] Inputting the brightness component in the video frame to the first feature extraction layer, and performing feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature;

[0197] Inputting the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracting the first image feature through the second feature extraction layer to obtain the second image feature, and extracting the first image feature through the third feature extraction layer to obtain the third image feature;

[0198] Inputting the second image feature and the third image feature into the feature stitching layer, and performing feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature;

[0199] The spliced ​​image features are input into the image quality classification layer to obtain the quality category of the video frame.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] Detecting whether the quality category of each of the video frames is a preset quality category;

[0202] If there is at least one video frame whose quality category is the preset quality category, the preset quality category is reported to the server as the quality category of the first live video.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] If the quality category of the first live video is a preset quality category, the first video bit rate of the first live video is reported to the server, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is chronologically subsequent to the first live video.

[0205] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0206] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for evaluating the quality of live video, characterized in that: The method comprises: According to a preset time interval, periodically intercepting video frames from the first live video; For each of the captured video frames, the brightness component in the video frame is input into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model, wherein the brightness component is used to represent the brightness of the video frame, the image quality classification model is obtained by training based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, each of the quality category labels is obtained by performing quality evaluation on the corresponding sample distorted image and a standard definition image corresponding to the sample distorted image, the quality evaluation is performed on the sample distorted image and the standard definition image corresponding to the sample distorted image according to a quality evaluation algorithm, and the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer; Reporting the quality category of the first live video to the server according to the quality category of each of the video frames; The step of inputting the brightness component in the video frame into the image quality classification model to obtain the quality category of the video frame output by the image quality classification model includes: Inputting the brightness component in the video frame to the first feature extraction layer, and performing feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature; Inputting the first image feature into the second feature extraction layer and the third feature extraction layer respectively, extracting the first image feature through the second feature extraction layer to obtain the second image feature, and extracting the first image feature through the third feature extraction layer to obtain the third image feature; Inputting the second image feature and the third image feature into the feature stitching layer, and performing feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature; The spliced ​​image features are input into the image quality classification layer to obtain the quality category of the video frame.

2. The method according to claim 1, characterized in that The training process of the image quality classification model includes: Acquire a plurality of standard definition images and a sample distorted image corresponding to each of the standard definition images; For each of the sample distorted images, performing quality evaluation on the sample distorted image and the corresponding standard definition image using the quality evaluation algorithm to obtain a quality score corresponding to the sample distorted image; Determining a score interval corresponding to the quality score, and determining a quality category label of the sample distorted image according to the score interval; An initial image quality classification model is trained according to the brightness components and quality category labels of each of the sample distorted images to obtain the image quality classification model.

3. The method according to claim 2, characterized in that Before training the initial image quality classification model according to the brightness component and the quality category label of each sample distorted image to obtain the image quality classification model, the method comprises: For each of the sample distorted images, a brightness component of the sample distorted image is extracted from the sample distorted image.

4. The method according to claim 2, characterized in that: The step of acquiring a plurality of standard definition images and a sample distorted image corresponding to each of the standard definition images comprises: Acquire multiple original images, and convert the image format of each of the original images to obtain a standard definition image corresponding to each of the original images; Distortion processing is performed on each of the original images, and image format conversion is performed on each of the original images after the distortion processing to obtain a sample distorted image corresponding to each of the original images, wherein the distortion processing includes at least one of scaling processing, filtering processing and encoding processing.

5. The method according to claim 1, characterized in that: The quality assessment algorithm includes one of peak signal-to-noise ratio, structural similarity and video multi-method assessment fusion.

6. The method according to claim 1, characterized in that The reporting the quality category of the first live video to the server according to the quality category of each of the video frames includes: Detecting whether the quality category of each of the video frames is a preset quality category; If there is at least one video frame whose quality category is the preset quality category, the preset quality category is reported to the server as the quality category of the first live video.

7. The method according to claim 1, characterized in that The method further comprises: If the quality category of the first live video is a preset quality category, the first video bit rate of the first live video is reported to the server, and the first video bit rate is used to instruct the server to send a second live video with a video bit rate greater than the first video bit rate, and the second live video is chronologically subsequent to the first live video.

8. A device for evaluating the quality of live video, characterized in that: The device comprises: An interception module, used for periodically intercepting video frames from the first live video at preset time intervals; A classification module, for inputting the brightness component in each captured video frame into an image quality classification model to obtain a quality category of the video frame output by the image quality classification model, wherein the brightness component is used to represent the brightness of the video frame, the image quality classification model is obtained by training based on a plurality of training samples, each of the training samples includes a brightness component and a quality category label of a sample distorted image, each of the quality category labels is obtained by performing quality evaluation on the corresponding sample distorted image and a standard definition image corresponding to the sample distorted image, the quality evaluation is performed on the sample distorted image and the standard definition image corresponding to the sample distorted image according to a quality evaluation algorithm, and the image quality classification model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a feature splicing layer and an image quality classification layer; A reporting module, configured to report the quality category of the first live video to the server according to the quality category of each of the video frames The classification module comprises: A first input unit, configured to input a brightness component in the video frame into the first feature extraction layer, and perform feature extraction on the brightness component in the video frame through the first feature extraction layer to obtain a first image feature; a second input unit, configured to input the first image feature to the second feature extraction layer and the third feature extraction layer respectively, extract the first image feature through the second feature extraction layer to obtain the second image feature, and extract the first image feature through the third feature extraction layer to obtain the third image feature; A third input unit is used to input the second image feature and the third image feature into the feature stitching layer, and perform feature stitching on the second image feature and the third image feature through the feature stitching layer to obtain a stitched image feature; The fourth input unit is used to input the spliced ​​image features into the image quality classification layer to obtain the quality category of the video frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images