Video Quality Evaluation Method, Apparatus, Device, Storage Medium and Program Product

Abstract

An embodiment of the present application provides a video quality evaluation method, device, equipment, storage medium and program product, which relate to the fields of video, map, and traffic, and application scenarios include but are not limited to video image quality evaluation scenarios. The method is executed by a server and includes: determining a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; in response to the video source device transmitting the video source to a terminal through a transmission channel, determining a second quality score corresponding to the process of the transmission channel transmitting the video source, where the second quality score is used to characterize the quality evaluation of the video source transmission process; in response to the terminal decoding and displaying the video source output by the transmission channel, determining a third quality score corresponding to the decoding and display of the video source; and determining the quality evaluation of the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score.

Description

Technical Field

[0001] The present application relates to the field of computer technology. Specifically, the present application relates to a video quality evaluation method, device, equipment, storage medium and program product. Background Art

[0002] The prior art often only evaluates the quality of the generated video source. The prior art includes, for example, PSNR (Peak Signal to Noise Ratio), SSIM (Structural SIMilarity index), VMAF (Video Multimethod Assessment Fusion), etc. based on full-reference video images, as well as BRISQUE (Blind / Referenceless Image Spatial QUality Evaluator), RankIQA (Learning from Rankings for No-reference Image Quality Assessment), DIQA (Deep CNN-Based Blind Image Quality Predictor), etc. Since only the quality of the generated video source is considered alone, the quality of the video source throughout the processes of generation, transmission, and display cannot be evaluated. Summary of the Invention

[0003] In view of the disadvantages of the existing methods, the present application provides a video quality evaluation method, device, equipment, computer-readable storage medium and computer program product to solve the problem of how to evaluate the quality of the video source throughout the processes of generation, transmission, and display.

[0004] In a first aspect, the present application provides a video quality evaluation method, which is executed by a server and includes:

[0005] Determine a first quality score corresponding to the video source generated by the video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source;

[0006] In response to the video source device transmitting the video source to the terminal through the transmission channel, determine a second quality score corresponding to the process of the transmission channel transmitting the video source, and the second quality score is used to characterize the quality evaluation of the video source transmission process;

[0007] In response to the terminal decoding and displaying the video source output on the transmission channel, determine the third quality score corresponding to the decoding and display of the video source. The third quality score is used to characterize the terminal's decoding ability and display ability for the video source;

[0008] Based on the first quality score, the second quality score, and the third quality score, determine the quality evaluation of the generation, transmission, and display process for the video source.

[0009] In one embodiment, determining the second quality score corresponding to the process of the transmission channel transmitting the video source includes:

[0010] Determine the first packet loss video quality factor corresponding to the packet loss in the process of the transmission channel transmitting the video source, the second packet loss video quality factor corresponding to the frame loss in the process of the transmission channel transmitting the video source, and the third packet loss video quality factor corresponding to the delay and freeze in the process of the transmission channel transmitting the video source;

[0011] Based on the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor, determine the second quality score corresponding to the process of the transmission channel transmitting the video source.

[0012] In one embodiment, determining the first packet loss video quality factor corresponding to the packet loss in the process of the transmission channel transmitting the video source includes:

[0013] Obtain the total number of first data packets corresponding to the video source generated by the video source device. The total number of first data packets includes the total number of redundant packets;

[0014] Based on the total number of first data packets and the total number of redundant packets, determine the redundant packet ratio coefficient;

[0015] Obtain the total number of second data packets that the terminal successfully decodes the video source;

[0016] Based on the total number of first data packets, the total number of second data packets, the redundant packet ratio coefficient, and a preset first parameter, through a preset loss function, determine the first packet loss video quality factor. The preset first parameter is used to control the loss degree of the preset loss function.

[0017] In one embodiment, determining the second packet loss video quality factor corresponding to the frame loss in the process of the transmission channel transmitting the video source includes:

[0018] Obtain the total number of first frames corresponding to the video source generated by the video source device;

[0019] Obtain the total number of second frames that the terminal successfully decodes the video source;

[0020] Determine a second packet loss video quality factor through a preset loss function based on a first total number of frames, a second total number of frames, and a preset second parameter, where the preset second parameter is used to control the loss degree of the preset loss function.

[0021] In one embodiment, determining a third packet loss video quality factor corresponding to delay jitter during the transmission of the video source by the transmission channel includes:

[0022] Obtain a first time interval between two adjacent frames of the video source generated by the video source device;

[0023] Obtain a second time interval between the two adjacent frames that the terminal successfully decodes the video source;

[0024] Based on the first time interval and the second time interval, determine a delay jitter parameter, where the delay jitter parameter is used to characterize whether delay jitter occurs during the transmission of the video source by the transmission channel;

[0025] Obtain a second total number of frames that the terminal successfully decodes the video source;

[0026] Based on the delay jitter parameter, the second total number of frames, and a preset third parameter, determine a third packet loss video quality factor through a preset loss function, where the preset third parameter is used to control the loss degree of the preset loss function.

[0027] In one embodiment, determining a third quality score corresponding to the decoding and display of the video source includes:

[0028] Determine a decoding compatibility ability factor of the terminal and a frame rate presentation quality factor of the terminal, where the decoding compatibility ability factor is used to characterize the decoding ability of the terminal for the video source, and the frame rate presentation quality factor is used to characterize the display ability of the terminal for the video source;

[0029] Based on the decoding compatibility ability factor and the frame rate presentation quality factor, determine a third quality score corresponding to the decoding and display of the video source.

[0030] In one embodiment, determining the decoding compatibility ability factor of the terminal includes:

[0031] Obtain the video format type corresponding to the video source generated by the video source device, and the set of video format types supported by the terminal;

[0032] Based on the video format type corresponding to the video source and the set of video format types, determine the decoding compatibility ability factor of the terminal.

[0033] In one embodiment, determining the frame rate presentation quality factor of the terminal includes:

[0034] Obtain the received frame rate of the source video reported by the terminal and the playback frame rate of the source video;

[0035] Determine the frame rate presentation quality factor of the terminal based on the received frame rate and the playback frame rate.

[0036] In one embodiment, determining the frame rate presentation quality factor of the terminal based on the received frame rate and the playback frame rate includes:

[0037] If the received frame rate is greater than the playback frame rate, determine the frame rate presentation quality factor of the terminal through a preset loss function based on the received frame rate, the playback frame rate, and a preset fourth parameter, where the preset fourth parameter is used to control the loss degree of the preset loss function.

[0038] In a second aspect, the present application provides a video quality evaluation device, which is applied to a server and includes:

[0039] A first processing module, configured to determine a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source;

[0040] A second processing module, configured to determine a second quality score corresponding to the process of the transmission channel transmitting the video source in response to the video source device transmitting the video source to the terminal through the transmission channel, where the second quality score is used to characterize the quality evaluation of the video source transmission process;

[0041] A third processing module, configured to determine a third quality score corresponding to the decoding and display of the video source in response to the terminal decoding and displaying the video source output by the transmission channel, where the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source;

[0042] A fourth processing module, configured to determine the quality evaluation of the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score.

[0043] In a third aspect, the present application provides an electronic device, including: a processor, a memory, and a bus;

[0044] The bus is used to connect the processor and the memory;

[0045] The memory is used to store operation instructions;

[0046] The processor is configured to execute the video quality evaluation method of the first aspect of the present application by calling the operation instructions.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, and the computer program is used to execute the video quality evaluation method of the first aspect of the present application.

[0048] In a fifth aspect, the present application provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the video quality evaluation method in the first aspect of the present application.

[0049] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0050] The server determines a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; in response to the video source device transmitting the video source to the terminal through a transmission channel, the server determines a second quality score corresponding to the process of the transmission channel transmitting the video source, and the second quality score is used to characterize the quality evaluation of the video source transmission process; in response to the terminal decoding and displaying the video source output by the transmission channel, the server determines a third quality score corresponding to the decoding and displaying of the video source, and the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source; the server determines the quality evaluation for the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score; thus, through the first quality score, the second quality score, and the third quality score, it is possible to evaluate the quality of the video source throughout the entire process of generation, transmission, and display. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description in the embodiments of the present application.

[0052] Figure 1 It is a schematic diagram of the architecture of the video quality evaluation system provided by the embodiments of the present application;

[0053] Figure 2 It is a schematic flowchart of a video quality evaluation method provided by the embodiments of the present application;

[0054] Figure 3 It is a schematic diagram of video quality evaluation provided by the embodiments of the present application;

[0055] Figure 4 It is a schematic flowchart of another video quality evaluation method provided by the embodiments of the present application;

[0056] Figure 5 It is a schematic diagram of the structure of a video quality evaluation device provided by the embodiments of the present application;

[0057] Figure 6 It is a schematic diagram of the structure of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The embodiments of the present application will be described below with reference to the accompanying drawings in the present application. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0059] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the", and "said" used herein may also include the plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude being implemented as other features, information, data, steps, operations, elements, components, and / or their combinations supported by the art of the present technology. It should be understood that when we say an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein can include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates being implemented as "A", or being implemented as "B", or being implemented as "A and B".

[0060] It can be understood that in the specific implementation of the present application, when it comes to data related to video quality evaluation, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0061] To make the purpose, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] The embodiment of the present application is a video quality evaluation method provided by a video quality evaluation system. This video quality evaluation method involves fields such as video, maps, and traffic. Exemplarily, in the embodiment of the present application, for scenarios such as the evaluation of the smoothness of video image quality involved, the quality of the video source generated by video technology evaluation can be evaluated.

[0063] Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable machines to have the functions of perception, reasoning, and decision-making.

[0064] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, autonomous driving, and intelligent transportation.

[0065] Intelligent Traffic System (ITS), also known as Intelligent Transportation System, is the effective and comprehensive application of advanced scientific and technological (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) in transportation, service control, and vehicle manufacturing, strengthening the connection between vehicles, roads, and users, thus forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0066] To better understand and illustrate the solutions of the embodiments of this application, some technical terms involved in the embodiments of this application are briefly described below.

[0067] Parameters of the video: resolution, color space, bit depth representation, brightness range, frame rate, bitstream format, dynamic range, etc.; among them, resolution: the number of pixels of the width and height in the video image source, generally expressed as W*H, such as 1920*1080; color space: the color space of the video image source, such as BT601, BT70, BT2020, etc.; bit depth representation: the data representation range of a single pixel in the video image source, generally 8bit, 10bit, 12bit, etc.; frame rate: the number of video frames per second in the video image source. Bitstream format: the bitstream format of the video image source, such as H.264, HEVC, AV1, AVS, AVS2, VVC, etc.; dynamic range: such as SDR, HDR, etc.

[0068] Parameters of the transmission channel: packet loss rate, transmission delay, etc.; among them, the packet loss rate: when transmitting video packets, due to channel quality problems, the proportion of lost data packets; the transmission delay: refers to the time required for a station to send data, that is, the time required for a data block to enter the transmission medium from the station, that is, the total time required for a station to start sending a data frame until the data frame is sent (or the total time for the receiving station to receive all of a data frame).

[0069] Parameters of the display device: resolution, color space, brightness range, decoding ability, frame rate, etc.; among them, the resolution: the number of pixels of the width and height of the screen occupied by the video played by the display device, generally expressed as W*H, for example, 1920*1080; the color space: supported by the display device, such as BT601, BT709, BT2020, etc.; the decoding ability: supports decoding video image bitstream formats such as H.264, HEVC, AV1, AVS, AVS2, VVC, etc.; the frame rate: the number of video frames refreshed per second supported by the display device; the brightness range: the unit of light intensity of the display screen: cd / m2 or nit (nit, 1 nit = 1 cd / m2); post-processing rendering enhancement algorithms: white balance, sharpness enhancement, deblocking, denoising, de-flickering, de-moire, etc.

[0070] PSNR: PSNR (Peak Signal to Noise Ratio) is a purely objective evaluation method with a wide range of applications. The value range is generally 20 - 40; PSNR is proportional to the video quality.

[0071] SSIM: SSIM (Structural SIMilarity index) is a relatively subjective objective evaluation method with complex calculations; compared with PSNR, SSIM can better reflect the subjective quality perception of the human eye, and the value range is 0 - 1; the larger the SSIM, the better the quality.

[0072] VMAF: VMAF (Video Multimethod Assessment Fusion) is a video evaluation standard that is more subjective.

[0073] BRISQUE: BRISQUE (Blind / Referenceless Image Spatial QUality Evaluator) is a no-reference image quality assessment algorithm in the spatial domain; BRISQUE represents an image (various distortions and degrees) as a feature vector designed by humans, and then uses a support vector machine (SVM) for classification.

[0074] RankIQA: The idea of RankIQA (Learning from Rankings for No-reference Image Quality Assessment) includes the first step of RankIQA, where a siamese network is trained with pairwise data (original images and distorted images). The distorted images can be infinitely generated by deforming the original images through various distortion methods without manual annotation. The second step is fine-tuning, where quality scores are learned through IQA (Image Quality Assessment) data.

[0075] DIQA: Since the number of datasets with manual scores is too small, DIQA (Deep CNN-Based Blind Image Quality Predictor) also undergoes two-stage training like RankIQA. In the first stage, it is only necessary to determine that the relevant information of a pair of images is distorted to different degrees, and then the second-stage training is carried out based on the first stage.

[0076] The solution provided in the embodiments of this application relates to video technology. The technical solution of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0077] To better understand the solution provided in the embodiments of this application, the solution will be described below with a specific application scenario.

[0078] In one embodiment, Figure 1 shows a schematic architecture diagram of a video quality evaluation system applicable to the embodiments of this application. It can be understood that the video quality evaluation method provided in the embodiments of this application can be applicable to but not limited to application scenarios such as Figure 1 shown.

[0079] In this example, as Figure 1As shown, the architecture of the video quality evaluation system in this example may include, but is not limited to, a server 10, a video source device 20, a terminal 30, and a database 40. The server 10, the video source device 20, the terminal 30, and the database 40 may interact through a network 50. The video source device 20 may be a server, a terminal, etc. The server 10 determines a first quality score corresponding to the video source generated by the video source device 20, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; in response to the video source device 20 transmitting the video source to the terminal 30 through a transmission channel, the server 10 determines a second quality score corresponding to the process of the transmission channel transmitting the video source, and the second quality score is used to characterize the quality evaluation of the video source transmission process; in response to the terminal 30 decoding and displaying the video source output by the transmission channel, the server 10 determines a third quality score corresponding to the decoding and display of the video source, and the third quality score is used to characterize the decoding ability and display ability of the terminal 30 for the video source; the server 10 determines the quality evaluation of the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score. The server 10 stores the quality evaluation of the generation, transmission, and display process of the video source in the database 40, and the data in the database 40 can be called by an operation system, a recommendation system, a query system, etc.

[0080] It can be understood that the above is only an example, and this embodiment is not limited herein.

[0081] Among them, the terminal includes, but is not limited to, a smart phone (such as an Android phone, an iOS phone, etc.), a mobile phone emulator, a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (Personal Digital Assistant), a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc.

[0082] The server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server or a server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0083] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, allowing various application systems to obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0084] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0085] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0086] The so-called artificial intelligence cloud service is generally also called AIaaS (AIas a Service, Chinese for "AI as a Service"). This is the current mainstream service mode of artificial intelligence platforms. Specifically, the AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall: all developers can access and use one or more artificial intelligence services provided by the platform through API interfaces. Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0087] The above network may include but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, Wi-Fi and other networks that realize wireless communication. The specific can also be determined based on the actual application scenario requirements and is not limited here.

[0088] See also Figure 2 ,Figure 2 FIG. 1 shows a schematic flow chart of a video quality evaluation method provided by an embodiment of the present application. Among them, this method can be executed by any electronic device, such as a server. As an alternative embodiment, this method can be executed by a server. For the convenience of description, in the description of some alternative embodiments below, the server will be used as an example of the execution subject of this method. As Figure 2 shown, the video quality evaluation method provided by the embodiment of the present application includes the following steps:

[0089] S201, determine a first quality score corresponding to the video source generated by the video source device. The video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source.

[0090] Specifically, the video source device can be a server, a terminal, etc.; for example, in a video call between user A and user B, the video source device that generates the video source can be a terminal; for another example, when user C watches a video number through a video APP, the video source device that generates the video source can be a server.

[0091] The parameters of the video include resolution, color space, bit depth representation, brightness range, frame rate, bitstream format, dynamic range, etc. The video source device reports the parameters of the video to the server. Based on the parameters of the video, the server calculates the quality of the video source generated by the video source device through at least one of PSNR, SSIM, VMAF, BRISQUE, RankIQA, and DIQA to obtain a quality score Q, that is, the first quality score. The quality score Q can be normalized to 0 to Q_max. The higher the quality score Q, the better the quality of the video source; among them, Q_max can be 1, 100, etc. For example, the quality score Q can be normalized to 0 to 1, and the quality score Q can also be normalized to 0 to 100.

[0092] S202, in response to the video source device transmitting the video source to the terminal through the transmission channel, determine a second quality score corresponding to the process of the transmission channel transmitting the video source. The second quality score is used to characterize the quality evaluation of the video source transmission process.

[0093] Specifically, the type of the transmission channel can be a wireless channel, a wired channel, etc.; the same video source can be transmitted to multiple terminals through multiple transmission channels respectively, and different videos can be transmitted to the same terminal through multiple transmission channels respectively.

[0094] S203, in response to the terminal decoding and displaying the video source output by the transmission channel, determine a third quality score corresponding to the decoding and display of the video source. The third quality score is used to characterize the decoding ability and display ability of the terminal for the video source.

[0095] Specifically, the terminal receives the video source output by the transmission channel; the terminal decodes the video source to obtain the decoded video; the terminal renders and enhances the display of the decoded video on the screen. The third quality score can be used to characterize the presentation quality of the source video by the terminal, that is, the decoding ability and display ability of the terminal for the video source.

[0096] S204. Based on the first quality score, the second quality score, and the third quality score, determine the quality evaluation of the generation-transmission-display process for the video source.

[0097] Specifically, the first quality score is used to characterize the quality evaluation of the video source, the second quality score is used to characterize the quality evaluation of the video source transmission process, and the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source; multiply the first quality score, the second quality score, and the third quality score to obtain the quality evaluation of the generation-transmission-display process for the video source.

[0098] In an embodiment of the present application, the server determines the first quality score corresponding to the video source generated by the video source device. The video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; in response to the video source device transmitting the video source to the terminal through the transmission channel, the server determines the second quality score corresponding to the process of the transmission channel transmitting the video source, and the second quality score is used to characterize the quality evaluation of the video source transmission process; in response to the terminal decoding and displaying the video source output by the transmission channel, the server determines the third quality score corresponding to the decoding and display of the video source, and the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source; the server determines the quality evaluation of the generation-transmission-display process for the video source based on the first quality score, the second quality score, and the third quality score; thus, through the first quality score, the second quality score, and the third quality score, it is possible to evaluate the quality of the video source throughout the generation, transmission, and display processes.

[0099] In one embodiment, determining the second quality score corresponding to the process of the transmission channel transmitting the video source includes steps A1 - A2:

[0100] Step A1, determine the first packet loss video quality factor corresponding to the packet loss in the process of the transmission channel transmitting the video source, the second packet loss video quality factor corresponding to the frame loss in the process of the transmission channel transmitting the video source, and the third packet loss video quality factor corresponding to the delay jitter in the process of the transmission channel transmitting the video source.

[0101] Specifically, the video source device, as the encoding end, encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device packets the video source, and can calculate the total number F1 of data packets and redundant packets of the video source. The total number F1 is the total number of the first data packets. Among them, the data packets of the video source include source data packets and retransmission data packets. The source data packets are the assembled data packets after encoding, and the retransmission data packets refer to when the source data packets are lost in the channel, and the encoding end is requested to resend at least one corresponding data packet. The video source device reports the total number F1 to the server, such as the background server.

[0102] In view of the unstable channel, in order to resist packet loss, redundant packets are added during the channel transmission process, and multiple redundant packets need to be sent. The video source device calculates the proportion p% of the total number of redundant packets in the total number F1. The p% is the redundant packet proportion coefficient. The video source device reports the redundant packet proportion coefficient p% to the server.

[0103] In view of the impact brought by actual packet loss, the total number F2 of data packets that are actually successfully decoded is calculated on the decoder of the terminal. The total number of data packets F2 is the total number of the second data packets. The terminal reports the total number of data packets F2 to the server.

[0104] The server receives the total number F1 and the redundant packet proportion coefficient p% reported by the video source device, and the total number of data packets F2 reported by the terminal. The server calculates the first packet loss video quality factor Q_d through the F function based on the total number F1, the redundant packet proportion coefficient p%, the total number of data packets F2, and the preset first parameter a_d. The formula (1) for calculating the first packet loss video quality factor Q_d is as follows:

[0105] Q_d = F_d(F2, F1*(1 - p%), a_d) Formula (1)

[0106] Among them, F_d is the F function, the F function is a loss function, and the value range of the F function is 0 - 1.

[0107] Specifically, the video source device, as the encoding end, encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device packets the video source, and can calculate the total number of frames Frame1 of the video source. The total number of frames Frame1 is the first total number of frames. The video source device reports the total number of frames Frame1 to the server.

[0108] In view of the impact brought by actual frame loss, the total number of frames Frame2 that are actually successfully decoded is calculated on the decoder of the terminal. The total number of frames Frame2 is the second total number of frames. The terminal reports the total number of frames Frame2 to the server.

[0109] The server receives the total number of frames Frame1 reported by the video source device and the total number of frames Frame2 reported by the terminal; based on the total number of frames Frame1, the total number of frames Frame2, and the preset second parameter a_frame, the server calculates the second packet loss video quality factor Q_frame through the F function. The formula (2) for calculating the second packet loss video quality factor Q_frame is as follows:

[0110] Q_frame = F_frame(Frame2, Frame1, a_frame) Formula (2)

[0111] Among them, F_frame is the F function, the F function is a loss function, and the value range of the F function is 0 - 1.

[0112] Specifically, the video source device, as the encoding end, encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device can calculate the ideal display refresh time interval Tsend_i from the (i - 1)-th frame to the i-th frame of the source video, and the time interval Tsend_i is the first time interval. The video source device reports the time interval Tsend_i to the server.

[0113] The terminal obtains the frame reception jitter situation of the source video (video frame) and the data reception time interval Ti from the (i - 1)-th frame to the i-th frame received by the terminal, and the data reception time interval Ti is the second time interval. The terminal reports the data reception time interval Ti to the server.

[0114] Regarding the impact of actual frame loss, the total number of frames Frame2 that are actually successfully decoded is calculated on the decoder of the terminal, and the total number of frames Frame2 is the second total number of frames. The terminal reports the total number of frames Frame2 to the server.

[0115] The server receives the time interval Tsend_i reported by the video source device, the total number of frames Frame2 reported by the terminal, and the data reception time interval Ti; based on the time interval Tsend_i and the data reception time interval Ti, the server calculates the delay jitter parameter Si. The formulas (3) and (4) for calculating the delay jitter parameter Si are as follows:

[0116] If Ti > K times Tsend_i, then Si = 1 Formula (3)

[0117] If Ti <= K times Tsend_i, then Si = 0 Formula (4)

[0118] Among them, Si = 1 indicates delay jitter, and Si = 0 indicates normal.

[0119] For the factors of time delay and lag, the server calculates the third packet loss video quality factor Q_s through the F function based on the time delay and lag parameter Si, the total number of frames Frame2, and the preset third parameter a_s. The formula (5) for calculating the third packet loss video quality factor Q_s is as follows:

[0120] Q_s = F_s(Frame2 - S, Frame2, a_s) Formula (5)

[0121] Among them, F_s is the F function, the F function is a loss function, and the value range of the F function is 0 - 1; the server sums Si from i = 1 to Frame2 to obtain S; for example, if Frame2 is 3, the server sums Si from i = 1 to 3, that is, S = S1 + S2 + S3. If S1 = 1, S2 = 0, and S3 = 0, then S = 1 + 0 + 0 = 1.

[0122] Step A2, based on the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor, determine the second quality score corresponding to the process of the transmission channel transmitting the video source.

[0123] Specifically, multiply the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor to obtain the second quality score corresponding to the process of the transmission channel transmitting the video source. For example, the second quality score = Q_d * Q_frame * Q_s.

[0124] In one embodiment, determining the first packet loss video quality factor corresponding to the packet loss in the process of the transmission channel transmitting the video source includes:

[0125] Obtain the total number of the first data packets corresponding to the video source generated by the video source device, and the total number of the first data packets includes the total number of redundant packets;

[0126] Based on the total number of the first data packets and the total number of redundant packets, determine the redundant packet ratio coefficient;

[0127] Obtain the total number of the second data packets successfully decoded by the terminal for the video source;

[0128] Based on the total number of the first data packets, the total number of the second data packets, the redundant packet ratio coefficient, and the preset first parameter, determine the first packet loss video quality factor through the preset loss function, and the preset first parameter is used to control the loss degree of the preset loss function.

[0129] Specifically, the video source device, as the encoding end, encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device packets the video source, and can calculate the total number F1 of data packets and redundant packets of the video source. The total number F1 is the total number of the first data packets. Among them, the data packets of the video source include source data packets and retransmission data packets. The source data packets are the assembled data packets after encoding, and the retransmission data packets refer to when the source data packets are lost in the channel, and the encoding end is requested to retransmit at least one corresponding data packet. The video source device reports the total number F1 to the server, such as the background server.

[0130] In view of the unstable channel, in order to resist packet loss, redundant packets are added during the channel transmission process, and multiple redundant packets need to be sent. The video source device calculates the proportion p% of the total number of redundant packets in the total number F1. The p% is the redundant packet proportion coefficient. The video source device reports the redundant packet proportion coefficient p% to the server.

[0131] In view of the impact brought by actual packet loss, the total number F2 of data packets that are actually successfully decoded is calculated on the decoder of the terminal. The total number F2 of data packets is the total number of the second data packets. The terminal reports the total number F2 of data packets to the server.

[0132] The server receives the total number F1 and the redundant packet proportion coefficient p% reported by the video source device, and the total number F2 of data packets reported by the terminal. The server calculates the first packet loss video quality factor Q_d through the F function based on the total number F1, the redundant packet proportion coefficient p%, the total number F2 of data packets, and the preset first parameter a_d. The formula (1) for calculating the first packet loss video quality factor Q_d is as follows:

[0133] Q_d = F_d(F2, F1*(1 - p%), a_d) Formula (1)

[0134] Among them, F_d is the F function, the F function is a loss function, and the value range of the F function is 0 - 1. The preset first parameter a_d is used to control the loss degree of F_d.

[0135] In one embodiment, determining the second packet loss video quality factor corresponding to the lost frames during the transmission of the video source by the transmission channel includes:

[0136] Obtain the first total number of frames corresponding to the video source generated by the video source device;

[0137] Obtain the second total number of frames when the terminal successfully decodes the video source;

[0138] Based on the first total number of frames, the second total number of frames, and the preset second parameter, determine the second packet loss video quality factor through the preset loss function. The preset second parameter is used to control the loss degree of the preset loss function.

[0139] Specifically, as the encoding end, the video source device encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device packets the video source, and can calculate the total number of frames Frame1 of the video source. The total number of frames Frame1 is the first total number of frames. The video source device reports the total number of frames Frame1 to the server.

[0140] Regarding the impact of actual frame loss, on the decoder of the terminal, calculate the total number of frames Frame2 that are actually successfully decoded. The total number of frames Frame2 is the second total number of frames. The terminal reports the total number of frames Frame2 to the server.

[0141] The server receives the total number of frames Frame1 reported by the video source device and the total number of frames Frame2 reported by the terminal; based on the total number of frames Frame1, the total number of frames Frame2, and the preset second parameter a_frame, through the F function, calculate the second packet loss video quality factor Q_frame. The formula (2) for calculating the second packet loss video quality factor Q_frame is as follows:

[0142] Q_frame = F_frame(Frame2, Frame1, a_frame) Formula (2)

[0143] Wherein, F_frame is the F function, the F function is a loss function, and the value range of the F function is 0 - 1; the preset second parameter a_frame is used to control the loss degree of F_frame.

[0144] In one embodiment, determining the third packet loss video quality factor corresponding to the delay and freeze during the transmission of the video source through the transmission channel includes:

[0145] Obtain the first time interval between two adjacent frames of the video source generated by the video source device;

[0146] Obtain the second time interval between the two adjacent frames that the terminal successfully decodes the video source;

[0147] Based on the first time interval and the second time interval, determine the delay and freeze parameter, and the delay and freeze parameter is used to characterize whether there is delay and freeze during the transmission of the video source through the transmission channel;

[0148] Obtain the second total number of frames that the terminal successfully decodes the video source;

[0149] Based on the delay and freeze parameter, the second total number of frames, and the preset third parameter, through the preset loss function, determine the third packet loss video quality factor. The preset third parameter is used to control the loss degree of the preset loss function.

[0150] Specifically, as the encoding end, the video source device encodes the video to obtain the encoded video, and the encoded video is the video source. The video source device can calculate the time interval Tsend_i for the ideal display refresh from the (i-1)-th frame to the i-th frame of the source video, and the time interval Tsend_i is the first time interval. The video source device reports the time interval Tsend_i to the server.

[0151] The terminal obtains the frame reception jitter of the source video (video frame), and the data reception time interval Ti for the terminal to receive the data from the (i-1)-th frame to the i-th frame. The data reception time interval Ti is the second time interval. The terminal reports the data reception time interval Ti to the server.

[0152] Regarding the impact of actual frame loss, the total number of frames Frame2 that are actually successfully decoded is calculated on the decoder of the terminal, and the total number of frames Frame2 is the second total number of frames. The terminal reports the total number of frames Frame2 to the server.

[0153] The server receives the time interval Tsend_i reported by the video source device, the total number of frames Frame2 and the data reception time interval Ti reported by the terminal; the server calculates the delay jitter parameter Si based on the time interval Tsend_i and the data reception time interval Ti. The formulas (3) and (4) for calculating the delay jitter parameter Si are as follows:

[0154] If Ti > K times Tsend_i, then Si = 1 Formula (3)

[0155] If Ti <= K times Tsend_i, then Si = 0 Formula (4)

[0156] Among them, Si = 1 indicates delay jitter, and Si = 0 indicates normal.

[0157] Regarding the factors of delay jitter, the server calculates the third packet loss video quality factor Q_s through the F function based on the delay jitter parameter Si, the total number of frames Frame2 and the preset third parameter a_s. The formula (5) for calculating the third packet loss video quality factor Q_s is as follows:

[0158] Q_s = F_s(Frame2 - S, Frame2, a_s) Formula (5)

[0159] Among them, \(F_s\) is the F function, the F function is the loss function, and the value range of the F function is 0 - 1; the server sums \(S_i\) from \(i = 1\) to Frame2 to obtain \(S\); for example, if Frame2 is 3, the server sums \(S_i\) from \(i = 1\) to 3, that is, \(S = S1 + S2 + S3\). If \(S1 = 1\), \(S2 = 0\), and \(S3 = 0\), then \(S = 1+0+0 = 1\); the preset third parameter \(a_s\) is used to control the loss degree of \(F_s\).

[0160] In one embodiment, determining the third quality score corresponding to the decoding and display of the video source includes steps B1 - B2:

[0161] Step B1, determining the decoding compatibility ability factor of the terminal and the frame rate presentation quality factor of the terminal. The decoding compatibility ability factor is used to characterize the decoding ability of the terminal for the video source, and the frame rate presentation quality factor is used to characterize the display ability of the terminal for the video source.

[0162] Specifically, the decoding format of the source video is H1, and H1 is a video format, that is, the video format type. The video format can be h.264, H.265, H.266, AVS1, AVS2, AVS3, AV1, VP9, etc. The video source device reports the video format type H1 to the server.

[0163] The decoding ability on the playback device of the terminal is H2, and H2 is a set of video formats, that is, a set of video format types. For example, H2 includes h.264, H.265, H.266, AVS1, AVS2, AVS3, AV1, VP9, etc. The terminal reports the set of video format types H2 to the server.

[0164] The server receives the video format type H1 reported by the video source device and the set of video format types H2 reported by the terminal; based on the video format type H1 and the set of video format types H2, the server calculates the decoding compatibility ability factor (decoder compatibility ability quality) \(Q_{dec}\) of the terminal. The formulas (6) and (7) for calculating the decoding compatibility ability factor \(Q_{dec}\) of the terminal are as follows:

[0165] If \(H2\geq H1\), that is, H1 belongs to H2, then \(Q_{dec}=1\) Formula (6)

[0166] If \(H2 < H1\), that is, H1 does not belong to H2, then \(Q_{dec}=0\) Formula (7)

[0167] Specifically, the receiving frame rate of the source video is fps1, and the actual refresh frame rate (the playing frame rate of the source video) on the playback device of the terminal is fps2. Among them, the receiving frame rate is the ideal display frame rate of the decoded video. Due to the influence of the refresh frame rate and calculation speed of the playback device, the actual refresh frame rate is the actual playing frame rate. For example, a 120fps video is displayed on a monitor with a refresh frame rate of 60fps. The terminal reports the receiving frame rate fps1 of the source video and the playing frame rate fps2 of the source video to the server.

[0168] The server receives the receiving frame rate fps1 of the source video and the playing frame rate fps2 of the source video reported by the terminal. Based on the receiving frame rate fps1 and the playing frame rate fps2, the server calculates the frame rate presentation quality factor (frame rate presentation quality) Q_fps of the terminal. The formulas (8) and (9) for calculating the frame rate presentation quality factor Q_fps of the terminal are as follows:

[0169] If fps2 >= fps1, then Q_fps = 1 Formula (8)

[0170] If fps2 < fps1, then Q_fps = F_fps(fps2, fps1, a_fps) Formula (9)

[0171] Among them, F_fps is the F function, the F function is a loss function, and the value range of the F function is 0 - 1. The preset fourth parameter a_fps is used to control the loss degree of F_fps.

[0172] Step B2: Determine the third quality score corresponding to the decoded display of the video source based on the decoding compatibility ability factor and the frame rate presentation quality factor.

[0173] Specifically, multiply the decoding compatibility ability factor and the frame rate presentation quality factor to obtain the third quality score corresponding to the decoded display of the video source. For example, the third quality score = Q_dec * Q_fps.

[0174] In one embodiment, determining the decoding compatibility ability factor of the terminal includes:

[0175] Obtain the video format type corresponding to the video source generated by the video source device and the set of video format types supported by the terminal;

[0176] Based on the video format type corresponding to the video source and the set of video format types, determine the decoding compatibility ability factor of the terminal.

[0177] Specifically, the decoding format of the source video is H1. H1 is a video format, that is, a video format type. The video format can be h.264, H.265, H.266, AVS1, AVS2, AVS3, AV1, VP9, etc. The video source device reports the video format type H1 to the server.

[0178] The decoding capability on the playback device of the terminal is H2. H2 is a set of video formats, that is, a set of video format types. For example, H2 includes h.264, H.265, H.266, AVS1, AVS2, AVS3, AV1, VP9, etc. The terminal reports the set of video format types H2 to the server.

[0179] The server receives the video format type H1 reported by the video source device and the set of video format types H2 reported by the terminal; the server calculates the decoding compatibility capability factor (decoder compatibility capability quality) Q_dec of the terminal based on the video format type H1 and the set of video format types H2. The formulas (6) and (7) for calculating the decoding compatibility capability factor Q_dec of the terminal are as follows:

[0180] If H2 >= H1, that is, H1 belongs to H2, then Q_dec = 1 Formula (6)

[0181] If H2 < H1, that is, H1 does not belong to H2, then Q_dec = 0 Formula (7)

[0182] In one embodiment, determining the frame rate presentation quality factor of the terminal includes:

[0183] Obtain the received frame rate of the source video and the playback frame rate of the source video reported by the terminal;

[0184] Determine the frame rate presentation quality factor of the terminal based on the received frame rate and the playback frame rate.

[0185] Specifically, the received frame rate of the source video is fps1, and the actual refresh frame rate (playback frame rate of the source video) on the playback device of the terminal is fps2; among them, the received frame rate is the ideal display frame rate of the decoded video, and the actual refresh frame rate is affected by the refresh frame rate and calculation speed of the playback device, and the actual refresh frame rate is the actual playback frame rate; for example, a 120fps video is displayed on a monitor with a refresh frame rate of 60fps. The terminal reports the received frame rate fps1 of the source video and the playback frame rate fps2 of the source video to the server.

[0186] The server receives the reception frame rate fps1 of the source video reported by the terminal and the playback frame rate fps2 of the source video; based on the reception frame rate fps1 and the playback frame rate fps2, the server calculates the frame rate presentation quality factor (frame rate presentation quality) Q_fps of the terminal. The formulas (8) and (9) for calculating the frame rate presentation quality factor Q_fps of the terminal are as follows:

[0187] If fps2 >= fps1, then Q_fps = 1 Formula (8)

[0188] If fps2 < fps1, then Q_fps = F_fps(fps2, fps1, a_fps) Formula (9)

[0189] Among them, F_fps is the F function, the F function is a loss function, and the value range of the F function is 0 - 1; the preset fourth parameter a_fps is used to control the loss degree of F_fps.

[0190] In one embodiment, determining the frame rate presentation quality factor of the terminal based on the reception frame rate and the playback frame rate includes:

[0191] If the reception frame rate is greater than the playback frame rate, then based on the reception frame rate, the playback frame rate, and the preset fourth parameter, through the preset loss function, determine the frame rate presentation quality factor of the terminal, and the preset fourth parameter is used to control the loss degree of the preset loss function.

[0192] Specifically, as shown in Formula (9), if the reception frame rate fps1 is greater than the playback frame rate fps2, then based on the reception frame rate fps1, the playback frame rate fps2, and the preset fourth parameter a_fps, through the preset loss function F_fps, determine the frame rate presentation quality factor Q_fps of the terminal.

[0193] In one embodiment, determining the quality evaluation of the generation, transmission, and display process for the video source based on the first quality score, the second quality score, and the third quality score includes:

[0194] Multiply the first quality score, the second quality score, and the third quality score to obtain the quality evaluation of the generation, transmission, and display process for the video source.

[0195] Specifically, the quality evaluation Q_final of the generation, transmission, and display process for the video source = Q * Q_d * Q_frame * Q_s * Q_dec * Q_fps; among them, the first quality score is Q, the second quality score is Q_d * Q_frame * Q_s, and the third quality score is Q_dec * Q_fps; the larger the value of Q_final, the higher the quality of the generation, transmission, and display process for the video source.

[0196] In one embodiment, the F function is a loss function, i.e., a quality function; the F function is shown in formula (10):

[0197]

[0198] For example, F_d(F2, F1*(1 - p%), a_d) in formula (1) is F(A, A max , α i ), where F2 is A, F1*(1 - p%) is A max , and a_d is α i .

[0199] Applying the embodiments of the present application has at least the following beneficial effects:

[0200] Through the first quality score, the second quality score, and the third quality score, it is possible to evaluate the quality of the video source throughout the processes of generation, transmission, and display.

[0201] To better understand the method provided by the embodiments of the present application, the solutions of the embodiments of the present application will be further described below with examples of specific application scenarios.

[0202] The method provided by the embodiments of the present application can be applied to any quality evaluation scenarios for video file transmission and display and video calls. For example, for scenarios such as video calls, video accounts, video file transmission, and videos, the evaluation of the user's online video viewing experience.

[0203] In one embodiment, as Figure 3 shown, the video source device generates a video source, the video source device sends the video source, the video source is transmitted through a transmission channel, the terminal receives the video source transmitted through the transmission channel, the terminal decodes the video source, and the terminal renders and enhances the display of the decoded video. The background server evaluates the quality of the video source throughout the processes of generation, transmission, and display through the method provided by the embodiments of the present application. For example, the background server calculates the video quality of a single user's single video experience, and the video quality of a single user's single video experience can be the quality evaluation Q_final for the generation, transmission, and display processes of the video source.

[0204] Various video image quality scores include video image quality scores based on a single video source, video image quality scores based on a single user, video image quality scores based on comprehensive experience, etc.; among them, the video image quality score based on a single video source: the average value of the video image quality of the same video source viewed by multiple users; the video image quality score based on a single user: the average value of the video image quality of multiple video sources viewed by a single user; the video image quality score based on comprehensive experience: the average value of the video image quality of multiple video sources viewed by multiple users.

[0205] The server determines various video image quality scores based on the video quality of multiple single-user single-time video experiences. The background server provides various video image quality scores to the operation system, recommendation system, query system, etc. Various video image quality scores can be used to detect operation quality, assist in video image recommendation, and assist in optimizing the quality improvement strategy of video images, etc.

[0206] In a specific application scenario embodiment, for example, in the video image quality evaluation scenario, refer to Figure 4 , which shows the processing flow of a video quality evaluation method. As Figure 4 shown, the processing flow of the video quality evaluation method provided by the embodiments of the present application includes the following steps:

[0207] S301, the video source device generates multiple video sources and reports the parameters of the video of each video source in the multiple video sources to the background server.

[0208] S302, the background server calculates the quality of each video source based on the parameters of the video of each video source, and obtains the first quality score of each video source.

[0209] S303, the video source device transmits the multiple video sources to multiple terminals (users) through different transmission channels respectively.

[0210] S304, the server determines the second quality score corresponding to the process of transmitting each video source through different transmission channels.

[0211] S305, each terminal receives each video source transmitted through different transmission channels, each terminal decodes each video source, and each terminal renders and enhances the display of each decoded video.

[0212] S306, the background server determines the third quality score corresponding to the decoding and display of each video source.

[0213] S307, the background server determines the quality evaluation of the generation, transmission, and display process of each video source for each terminal based on the first quality score, the second quality score, and the third quality score.

[0214] Specifically, for one terminal (user), the quality evaluation of the generation, transmission, and display process of one video source is the video quality of a single-user single-time video experience, and the quality evaluation of the generation, transmission, and display process of multiple video sources is the video quality of multiple single-user single-time video experiences.

[0215] S308, the server determines various video image quality scores based on the video quality of multiple single-user single-time video experiences.

[0216] Specifically, various video image quality scores include video image quality scores based on a single video source, video image quality scores based on a single user, video image quality scores based on comprehensive experience, etc.

[0217] For example, the same video source A is sent to multiple users (terminals), and the multiple users (terminals) are user 1 (terminal 1), user 2 (terminal 2), and user 3 (terminal 3) respectively. The server determines the quality evaluation Q_final-1 of the generation, transmission, and display process of video source A for user 1, the quality evaluation Q_final-2 of the generation, transmission, and display process of video source A for user 2, and the quality evaluation Q_final-3 of the generation, transmission, and display process of video source A for user 3. The server calculates the average value B among Q_final-1, Q_final-2, and Q_final-3, that is, (Q_final-1 + Q_final-2 + Q_final-3) / 3 = average value B, and the average value B is the video image quality score based on a single video source.

[0218] For example, the same user C (terminal) receives multiple video sources, and the multiple video sources are video source 1, video source 2, video source 3, and video source 4 respectively; for user C, the server determines the quality evaluation Q_final-4 of the generation, transmission, and display process of video source 1, the quality evaluation Q_final-5 of the generation, transmission, and display process of video source 2, the quality evaluation Q_final-6 of the generation, transmission, and display process of video source 3, and the quality evaluation Q_final-7 of the generation, transmission, and display process of video source 4. The server calculates the average value D among Q_final-4, Q_final-5, Q_final-6, and Q_final-7, that is, (Q_final-4 + Q_final-5 + Q_final-6 + Q_final-7) / 4 = average value D, and the average value D is the video image quality score based on a single user.

[0219] For example, user E (terminal) receives video source 5 and video source 6, and user F (terminal) receives video source 7, video source 8, and video 9. The server determines the quality evaluation Q_final-8 of the generation, transmission, and display process of video source 5 for user E, the quality evaluation Q_final-9 of the generation, transmission, and display process of video source 6 for user E, the quality evaluation Q_final-10 of the generation, transmission, and display process of video source 7 for user F, the quality evaluation Q_final-11 of the generation, transmission, and display process of video source 8 for user F, and the quality evaluation Q_final-12 of the generation, transmission, and display process of video source 9 for user F. The server calculates the average value G among Q_final-8, Q_final-9, Q_final-10, Q_final-11, and Q_final-12, that is, (Q_final-8 + Q_final-9 + Q_final-10 + Q_final-11 + Q_final-12) / 5 = average value G. The average value G is the video image quality score based on the comprehensive experience.

[0220] S309. The background server provides various video image quality scores to the operation system, recommendation system, query system, etc.

[0221] Applying the embodiments of the present application has at least the following beneficial effects:

[0222] The server realizes the quality evaluation of each video source in the entire process of generation, transmission, and display, that is, the video quality of a single user's single video experience, through the first quality score, the second quality score, and the third quality score; the server determines various video image quality scores based on the video quality of multiple single user's single video experiences; various video image quality scores can be used to detect operation quality, assist in video image recommendation, and assist in optimizing the quality improvement strategy of video images, etc.

[0223] The embodiments of the present application also provide a video quality evaluation device, which is applied to the server. The structural schematic diagram of the video quality evaluation device is as Figure 5 shown. The video quality evaluation device 90 includes a first processing module 901, a second processing module 902, a third processing module 903, and a fourth processing module 904.

[0224] The first processing module 901 is used to determine the first quality score corresponding to the video source generated by the video source device. The video source is the encoded video, and the first quality score is used to represent the quality evaluation of the video source.

[0225] The second processing module 902 is configured to determine a second quality score corresponding to the process of transmitting the video source by the transmission channel in response to the video source device transmitting the video source to the terminal through the transmission channel, where the second quality score is used to characterize the quality evaluation of the video source transmission process;

[0226] The third processing module 903 is configured to determine a third quality score corresponding to the decoding and display of the video source output by the transmission channel in response to the terminal decoding and displaying the video source, where the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source;

[0227] The fourth processing module 904 is configured to determine the quality evaluation of the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score.

[0228] In one embodiment, the second processing module 902 is specifically configured to:

[0229] Determine a first packet loss video quality factor corresponding to the packet loss in the process of the transmission channel transmitting the video source, a second packet loss video quality factor corresponding to the frame loss in the process of the transmission channel transmitting the video source, and a third packet loss video quality factor corresponding to the delay and jitter in the process of the transmission channel transmitting the video source;

[0230] Determine the second quality score corresponding to the process of the transmission channel transmitting the video source based on the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor.

[0231] In one embodiment, the second processing module 902 is specifically configured to:

[0232] Obtain the total number of first data packets corresponding to the video source generated by the video source device, where the total number of first data packets includes the total number of redundant packets;

[0233] Determine the redundant packet proportion coefficient based on the total number of first data packets and the total number of redundant packets;

[0234] Obtain the total number of second data packets that the terminal successfully decodes the video source;

[0235] Determine the first packet loss video quality factor through a preset loss function based on the total number of first data packets, the total number of second data packets, the redundant packet proportion coefficient, and a preset first parameter, where the preset first parameter is used to control the loss degree of the preset loss function.

[0236] In one embodiment, the second processing module 902 is specifically configured to:

[0237] Obtain the total number of first frames corresponding to the video source generated by the video source device;

[0238] Obtain the total number of second frames that the terminal successfully decodes the video source;

[0239] Determine a second packet loss video quality factor through a preset loss function based on a first total number of frames, a second total number of frames, and a preset second parameter, where the preset second parameter is used to control the loss degree of the preset loss function.

[0240] In one embodiment, the second processing module 902 is specifically configured to:

[0241] Obtain a first time interval between two adjacent frames of a video source generated by a video source device;

[0242] Obtain a second time interval between the two adjacent frames that the terminal successfully decodes the video source;

[0243] Determine a delay and freeze parameter based on the first time interval and the second time interval, where the delay and freeze parameter is used to characterize whether there is delay and freeze during the process of the transmission channel transmitting the video source;

[0244] Obtain a second total number of frames that the terminal successfully decodes the video source;

[0245] Determine a third packet loss video quality factor through a preset loss function based on the delay and freeze parameter, the second total number of frames, and a preset third parameter, where the preset third parameter is used to control the loss degree of the preset loss function.

[0246] In one embodiment, the third processing module 903 is specifically configured to:

[0247] Determine a decoding compatibility ability factor of the terminal and a frame rate presentation quality factor of the terminal, where the decoding compatibility ability factor is used to characterize the decoding ability of the terminal for the video source, and the frame rate presentation quality factor is used to characterize the display ability of the terminal for the video source;

[0248] Determine a third quality score corresponding to the decoding and display of the video source based on the decoding compatibility ability factor and the frame rate presentation quality factor.

[0249] In one embodiment, the third processing module 903 is specifically configured to:

[0250] Obtain the video format type corresponding to the video source generated by the video source device and the set of video format types supported by the terminal;

[0251] Determine the decoding compatibility ability factor of the terminal based on the video format type corresponding to the video source and the set of video format types.

[0252] In one embodiment, the third processing module 903 is specifically configured to:

[0253] Obtain the received frame rate of the source video reported by the terminal and the played frame rate of the source video;

[0254] Determine the frame rate presentation quality factor of the terminal based on the reception frame rate and the playback frame rate.

[0255] In one embodiment, the third processing module 903 is specifically configured to:

[0256] If the reception frame rate is greater than the playback frame rate, determine the frame rate presentation quality factor of the terminal based on the reception frame rate, the playback frame rate, and a preset fourth parameter through a preset loss function, where the preset fourth parameter is used to control the loss degree of the preset loss function.

[0257] Applying the embodiments of the present application has at least the following beneficial effects:

[0258] The server determines a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; the server determines a second quality score corresponding to the process of the transmission channel transmitting the video source in response to the video source device transmitting the video source to the terminal through the transmission channel, and the second quality score is used to characterize the quality evaluation of the video source transmission process; the server determines a third quality score corresponding to the decoding and display of the video source in response to the terminal decoding and displaying the video source output by the transmission channel, and the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source; the server determines the quality evaluation for the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score; thus, through the first quality score, the second quality score, and the third quality score, it is possible to evaluate the quality of the video source throughout the entire process of generation, transmission, and display.

[0259] The embodiments of the present application further provide an electronic device, and the structural schematic diagram of the electronic device is as Figure 6 shown Figure 6 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0260] The processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0261] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 6 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0262] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited here.

[0263] The memory 4003 is used to store the computer program for implementing the embodiments of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0264] Among them, the electronic device includes but is not limited to: servers, etc.

[0265] Applying the embodiments of the present application has at least the following beneficial effects:

[0266] The server determines a first quality score corresponding to the video source generated by the video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; in response to the video source device transmitting the video source to the terminal through the transmission channel, the server determines a second quality score corresponding to the process of the transmission channel transmitting the video source, and the second quality score is used to characterize the quality evaluation of the video source transmission process; in response to the terminal decoding and displaying the video source output by the transmission channel, the server determines a third quality score corresponding to the decoding and display of the video source, and the third quality score is used to characterize the decoding ability and display ability of the terminal for the video source; the server determines the quality evaluation of the generation, transmission, and display process of the video source based on the first quality score, the second quality score, and the third quality score; thus, through the first quality score, the second quality score, and the third quality score, it is possible to evaluate the quality of the video source throughout the entire process of generation, transmission, and display.

[0267] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0268] The embodiments of the present application further provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0269] Based on the same principle as the method provided in the embodiments of the present application, the embodiments of the present application further provide a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any optional embodiment of the present application above.

[0270] It should be understood that although the flowchart of the embodiments of the present application indicates various operation steps by arrows, the execution order of these steps is not limited to the order indicated by the arrows. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.

[0271] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. A video quality evaluation method, executed by a server, characterized in that, Including: Determine a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; In response to the video source device transmitting the video source to a terminal through a transmission channel, determine a second quality score corresponding to the process of the transmission channel transmitting the video source, where the second quality score is used to characterize the quality evaluation of the video source transmission process; In response to the terminal decoding and displaying the video source output by the transmission channel, determine a decoding compatibility ability factor of the terminal and a frame rate presentation quality factor of the terminal. Based on the decoding compatibility ability factor and the frame rate presentation quality factor, determine a third quality score corresponding to the decoding and display of the video source. The decoding compatibility ability factor is used to characterize the decoding ability of the terminal for the video source, and the frame rate presentation quality factor is used to characterize the display ability of the terminal for the video source; Based on the first quality score, the second quality score, and the third quality score, determine the quality evaluation of the generation, transmission, and display process of the video source. The quality evaluation of the generation, transmission, and display process of the video source is used to determine various video image quality scores, and the various video image quality scores include video image quality scores based on a single video source, video image quality scores based on a single user, and video image quality scores based on a comprehensive experience.

2. The method according to claim 1, wherein The determining of the second quality score corresponding to the process of the transmission channel transmitting the video source includes: Determine a first packet loss video quality factor corresponding to packet loss in the process of the transmission channel transmitting the video source, a second packet loss video quality factor corresponding to frame loss in the process of the transmission channel transmitting the video source, and a third packet loss video quality factor corresponding to delay and jitter in the process of the transmission channel transmitting the video source; Based on the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor, determine the second quality score corresponding to the process of the transmission channel transmitting the video source.

3. The method according to claim 2, characterized in that, The determining of the first packet loss video quality factor corresponding to packet loss in the process of the transmission channel transmitting the video source includes: Obtain a first total number of data packets corresponding to the video source generated by the video source device, where the first total number of data packets includes the total number of redundant packets; Based on the first total number of data packets and the total number of redundant packets, determine a redundant packet proportion coefficient; Obtain a second total number of data packets that the terminal successfully decodes the video source; Based on the first total number of data packets, the second total number of data packets, the redundant packet proportion coefficient, and a preset first parameter, through a preset loss function, determine the first packet loss video quality factor, where the preset first parameter is used to control the loss degree of the preset loss function.

4. The method according to claim 2, characterized in that, The determining of the second packet loss video quality factor corresponding to frame loss in the process of the transmission channel transmitting the video source includes: Obtain a first total number of frames corresponding to the video source generated by the video source device; Obtain a second total number of frames that the terminal successfully decodes the video source; Based on the first total number of frames, the second total number of frames, and a preset second parameter, determine a second packet-loss video quality factor through a preset loss function, where the preset second parameter is used to control the loss degree of the preset loss function.

5. The method according to claim 2, wherein Determining the third packet-loss video quality factor corresponding to the delay and freeze during the transmission of the video source by the transmission channel includes: Obtain a first time interval between two adjacent frames of the video source generated by the video source device; Obtain a second time interval between two adjacent frames that the terminal successfully decodes the video source; Based on the first time interval and the second time interval, determine a delay and freeze parameter, where the delay and freeze parameter is used to characterize whether there is delay and freeze during the transmission of the video source by the transmission channel; Obtain the second total number of frames that the terminal successfully decodes the video source; Based on the delay and freeze parameter, the second total number of frames, and a preset third parameter, determine a third packet-loss video quality factor through a preset loss function, where the preset third parameter is used to control the loss degree of the preset loss function.

6. The method according to claim 1, wherein Determining the decoding compatibility ability factor of the terminal includes: Obtain the video format type corresponding to the video source generated by the video source device, and the set of video format types supported by the terminal; Based on the video format type corresponding to the video source and the set of video format types, determine the decoding compatibility ability factor of the terminal.

7. The method according to claim 1, characterized in that, Determining the frame rate presentation quality factor of the terminal includes: Obtain the received frame rate and the playback frame rate of the video source reported by the terminal; Based on the received frame rate and the playback frame rate, determine the frame rate presentation quality factor of the terminal.

8. The method according to claim 7, wherein The determining the frame rate presentation quality factor of the terminal based on the received frame rate and the playback frame rate includes: If the received frame rate is greater than the playback frame rate, then based on the received frame rate, the playback frame rate, and a preset fourth parameter, determine the frame rate presentation quality factor of the terminal through a preset loss function, where the preset fourth parameter is used to control the loss degree of the preset loss function.

9. A video quality evaluation device, applied to a server, characterized in that, Includes: A first processing module, configured to determine a first quality score corresponding to a video source generated by a video source device, where the video source is an encoded video, and the first quality score is used to characterize the quality evaluation of the video source; A second processing module, configured to, in response to the video source device transmitting the video source to a terminal through a transmission channel, determine a second quality score corresponding to the process of the transmission channel transmitting the video source, where the second quality score is used to characterize the quality evaluation of the video source transmission process; A third processing module, configured to, in response to the terminal decoding and displaying the video source output on the transmission channel, determine a decoding compatibility capability factor of the terminal and a frame rate presentation quality factor of the terminal, and determine a third quality score corresponding to decoding and displaying the video source based on the decoding compatibility capability factor and the frame rate presentation quality factor, where the decoding compatibility capability factor is used to characterize the decoding ability of the terminal for the video source, and the frame rate presentation quality factor is used to characterize the display ability of the terminal for the video source; A fourth processing module, configured to determine a quality evaluation of the generation, transmission, and display process for the video source based on the first quality score, the second quality score, and the third quality score, where the quality evaluation of the generation, transmission, and display process for the video source is used to determine various types of video image quality scores, and the various types of video image quality scores include video image quality scores based on a single video source, video image quality scores based on a single user, and video image quality scores based on comprehensive experience.

10. The device according to claim 9, characterized in that, The second processing module is specifically configured to: Determine a first packet loss video quality factor corresponding to packet loss in the process of the transmission channel transmitting the video source, a second packet loss video quality factor corresponding to frame loss in the process of the transmission channel transmitting the video source, and a third packet loss video quality factor corresponding to delay and freezing in the process of the transmission channel transmitting the video source; Determine a second quality score corresponding to the process of the transmission channel transmitting the video source based on the first packet loss video quality factor, the second packet loss video quality factor, and the third packet loss video quality factor.

11. The device according to claim 10, characterized in that, When the second processing module determines the first packet loss video quality factor corresponding to packet loss in the process of the transmission channel transmitting the video source, it is specifically configured to: Obtain a first total number of data packets corresponding to the video source generated by the video source device, where the first total number of data packets includes the total number of redundant packets; Determine a redundant packet ratio coefficient based on the first total number of data packets and the total number of redundant packets; Obtain a second total number of data packets that the terminal successfully decodes the video source; Determine the first packet loss video quality factor through a preset loss function based on the first total number of data packets, the second total number of data packets, the redundant packet ratio coefficient, and a preset first parameter, where the preset first parameter is used to control the loss degree of the preset loss function.

12. The device according to claim 10, wherein, When the second processing module determines the second packet loss video quality factor corresponding to frame loss in the process of the transmission channel transmitting the video source, it is specifically configured to: Obtain a first total number of frames corresponding to the video source generated by the video source device; Obtain a second total number of frames that the terminal successfully decodes the video source; Determine the second packet loss video quality factor through a preset loss function based on the first total number of frames, the second total number of frames, and a preset second parameter, where the preset second parameter is used to control the loss degree of the preset loss function.

13. The device according to claim 10, characterized in that, When the second processing module determines the third packet loss video quality factor corresponding to delay and freezing in the process of the transmission channel transmitting the video source, it is specifically configured to: Obtain a first time interval between two adjacent frames of the video source generated by the video source device; Obtain a second time interval between two adjacent frames that the terminal successfully decodes the video source; Based on the first time interval and the second time interval, determine a delay and freeze parameter, where the delay and freeze parameter is used to characterize whether there is delay and freeze during the process of the transmission channel transmitting the video source; Obtain the total number of the second frames that the terminal successfully decodes the video source; Based on the delay and freeze parameter, the total number of the second frames, and a preset third parameter, determine a third packet loss video quality factor through a preset loss function, where the preset third parameter is used to control the loss degree of the preset loss function.

14. The device according to claim 9, characterized in that, When the third processing module determines the decoding compatibility ability factor of the terminal, it specifically is used for: Obtain the video format type corresponding to the video source generated by the video source device, and the set of video format types supported by the terminal; Based on the video format type corresponding to the video source and the set of video format types, determine the decoding compatibility ability factor of the terminal.

15. The device according to claim 9, characterized in that When the third processing module determines the frame rate presentation quality factor of the terminal, it specifically is used for: Obtain the received frame rate and the played frame rate of the video source reported by the terminal; Based on the received frame rate and the played frame rate, determine the frame rate presentation quality factor of the terminal.

16. The device according to claim 15, characterized in that, When the third processing module determines the frame rate presentation quality factor of the terminal based on the received frame rate and the played frame rate, it specifically is used for: If the received frame rate is greater than the played frame rate, then based on the received frame rate, the played frame rate, and a preset fourth parameter, determine the frame rate presentation quality factor of the terminal through a preset loss function, where the preset fourth parameter is used to control the loss degree of the preset loss function.

17. An electronic device, including a memory, a processor, and a computer program stored on the memory, is characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-8.

Images