A method and device for evaluating the smoothness of cloud game images

Abstract

The present invention discloses a method and device for evaluating the screen smoothness of cloud games, which relate to the field of computer technology and solve the technical problem that the evaluation of the screen smoothness of cloud games is not accurate and comprehensive enough. The key points of the technical solution are to evaluate the screen smoothness of cloud games from three dimensions: receiving frame grouping, data decoding and screen rendering, so as to reflect the screen smoothness more comprehensively and accurately; at the same time, through the average time consumption statistics of each dimension, the problem is further analyzed to improve the overall fluency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for evaluating the smoothness of cloud game images. Background Art

[0002] Cloud gaming refers to a game that runs based on the cloud. The rendered game screen is compressed and encoded, and then transmitted to the terminal through the network in the form of audio and video streams. The terminal side decodes and plays the audio and video, and uploads the control commands to the cloud to form a game interaction process.

[0003] The elements of cloud gaming experience are generally consistent with local gaming. The difference is that cloud gaming introduces codecs, network transmission and other links, and its service experience is affected by more factors such as cloud, pipe and end. The higher the requirements for cloud gaming image quality experience, the greater the requirements for resolution, frame rate, bit rate, etc., and the higher the network throughput requirements. Fluency is one of the most direct and important factors affecting users' evaluation of cloud gaming audio-visual experience. Whether the gaming experience is smooth has become an important indicator for players to measure their performance.

[0004] When players operate cloud gaming terminal devices, the screen is composed of multiple frames of static images. When the screen is displayed smoothly, the user's eyes will feel more comfortable and the user experience will be better. If the screen is not displayed smoothly, there will be freezes and jitters, which will cause the player to feel freezes or sluggish operation. Whether the image changes of mobile terminals are smooth directly affects the user experience. Therefore, the evaluation of fluency performance is of great practical significance for optimizing and improving terminal applications and improving user experience.

[0005] In the prior art, the smoothness of image changes in mobile terminals is usually reflected by detecting the FPS (Frames Per Second) of the picture to reflect the smoothness of the picture operation. However, only reflecting the smoothness of the picture operation by detecting the FPS value of the picture cannot accurately and comprehensively make an overall assessment of the smoothness of the picture. Summary of the invention

[0006] The present application provides a method and device for evaluating the smoothness of cloud game pictures, the technical purpose of which is to accurately and comprehensively evaluate the smoothness of cloud game pictures to improve the user's audio-visual experience of cloud games.

[0007] The above technical objectives of this application are achieved through the following technical solutions:

[0008] A method for evaluating the smoothness of a cloud game screen, comprising:

[0009] The received average frame rate is judged. If the received average frame rate is within the first interval, the evaluation result is "screen freeze", and if the received average frame rate is within the second interval, the third interval, or the fourth interval, the decoded average frame rate is judged; if the decoded average frame rate is within the first interval, the evaluation result is "screen freeze", and if the decoded average frame rate is within the second interval, the third interval, or the fourth interval, the rendered average frame rate is judged; if the rendered average frame rate is within the first interval, the evaluation result is "screen freeze", and if the rendered average frame rate is within the second interval, the evaluation result is "generally smooth", if the rendered average frame rate is within the third interval, the evaluation result is "relatively smooth", and if the rendered average frame rate is within the fourth interval, the evaluation result is "relatively smooth";

[0010] The received average frame rate is expressed as FSP1=F1 / T, T represents the statistical cycle time, and F1 represents the total number of frames of the frame obtained within the statistical cycle time T when the frame is received; the total reception time is expressed as T1, and the average time consumed per frame of the frame reception is AVG1=T1 / F1;

[0011] The average decoding frame rate is expressed as FSP2=F2 / T, where F2 represents the total number of frames decoded within the statistical cycle time T when decoding the data; the total decoding time is T2, and the average time per frame of data decoding is AVG2=T2 / F2;

[0012] The rendering average frame rate is expressed as FSP3=F3 / T, where F3 represents the total number of rendering frames obtained within the statistical cycle time T when rendering the picture; the total rendering time is T3, and the average rendering time per frame is AVG3=T3 / F3.

[0013] A cloud game screen smoothness evaluation device, comprising:

[0014] The receiving frame statistical unit, when receiving the frame, obtains the total number of frames F1 and the total receiving time T1 of the frame within the statistical cycle time T, then the average frame rate of the frame reception is expressed as FSP1=F1 / T, and the average time per frame of the frame reception is AVG1=T1 / F1;

[0015] The data decoding statistics unit, when decoding the data, obtains the total number of decoded frames F2 and the total decoding time T2 within the statistical cycle time T, then the average frame rate of data decoding is expressed as FSP2 = F2 / T, and the average time per frame of data decoding is AVG2 = T2 / F2;

[0016] The picture rendering statistics unit, when rendering the picture, obtains the total number of rendered frames F3 and the total rendering time T3 within the statistical cycle time T, then the average frame rate of the picture rendering is expressed as FSP3=F3 / T, and the average time per frame of the picture rendering is AVG3=T3 / F3;

[0017] A fluency evaluation unit evaluates the fluency of the picture, including: judging the received average frame rate, if the received average frame rate is within the first interval, the evaluation result is "picture freeze", if the received average frame rate is within the second interval, the third interval or the fourth interval, judging the decoding average frame rate; if the decoding average frame rate is within the first interval, the evaluation result is "picture freeze", if the decoding average frame rate is within the second interval, the third interval or the fourth interval, judging the rendering average frame rate; if the rendering average frame rate is within the first interval, the evaluation result is "picture freeze", if the rendering average frame rate is within the second interval, the evaluation result is "generally smooth", if the rendering average frame rate is within the third interval, the evaluation result is "relatively smooth", if the rendering average frame rate is within the fourth interval, the evaluation result is "relatively smooth".

[0018] The beneficial effects of this application are: this application evaluates the smoothness of cloud game screens from three dimensions: receiving frame grouping, data decoding, and screen rendering, reflecting the smoothness of the screen more comprehensively and accurately; at the same time, through the average time statistics of each dimension, further analyze the problem to improve the overall smoothness. The evaluation of screen smoothness is of great practical significance for optimizing and improving cloud game terminal applications and improving the audio-visual experience of game players. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of the evaluation method described in this application;

[0020] Figure 2 This is a judgment flow chart of the evaluation method described in this application;

[0021] Figure 3 This is a structural diagram of the evaluation device described in this application. DETAILED DESCRIPTION

[0022] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0023] like Figure 1 As shown, when receiving the framing, the total number of frames F1 and the total receiving time T1 of the framing are obtained within the statistical cycle time T. The total receiving time T1 is the total time consumed by accumulating the time consumed from the beginning of receiving each frame of data to the completion of the final package, in ms. The average frame rate of the framing reception is calculated as FSP1=F1 / T, and the average time consumed per frame of the framing reception is AVG1=T1 / F1.

[0024] When decoding data, the total number of decoded frames F2 and the total decoding time T2 are obtained within the statistical cycle time T. The total decoding time T2 is the total time consumed by each frame of data from the time it is sent to the data decoding statistical unit to the time it is finally decoded and output, and the total time consumed is accumulated. The average frame rate of data decoding is calculated as FSP2 = F2 / T, and the average time consumed per frame of data decoding is AVG2 = T2 / F2;

[0025] The picture rendering statistics unit, when rendering the picture, obtains the total number of rendered frames F3 and the total rendering time T3 within the statistical cycle time T. The total rendering time T3 is the total time consumed by accumulating the time consumed by each frame data from the time it is sent to the picture rendering statistics unit to the time it is finally output, in ms. The average frame rate of the picture rendering is calculated as FSP3 = F3 / T, and the average time consumed by each frame of the picture rendering is AVG3 = T3 / F3.

[0026] The judgment process of fluency assessment is as follows: Figure 2 As shown, the received average frame rate is first judged. If the received average frame rate is within the first interval, the evaluation result is "screen freeze"; if the received average frame rate is within the second interval, the third interval, or the fourth interval, the decoding average frame rate is judged; if the decoding average frame rate is within the first interval, the evaluation result is "screen freeze"; if the decoding average frame rate is within the second interval, the third interval, or the fourth interval, the rendering average frame rate is judged; if the rendering average frame rate is within the first interval, the evaluation result is "screen freeze"; if the rendering average frame rate is within the second interval, the evaluation result is "generally smooth"; if the rendering average frame rate is within the third interval, the evaluation result is "relatively smooth"; if the rendering average frame rate is within the fourth interval, the evaluation result is "relatively smooth".

[0027] Specifically, the smoothness is evaluated according to the frame rate threshold interval table 1 and the frame rate calculated in each dimension. First, the average frame rate of the received framing is compared. If it is in interval 0, it means that the data is insufficient from the source, which will inevitably cause the subsequent screen to freeze. The evaluation result of "screen freeze" is directly given. The evaluation result format is {[fps=20, screen freeze], [receive framing, 50ms], [data decoding, 48ms], [screen rendering, 47ms]}. The evaluation result consists of four parts. The first part is the screen smoothness judgment, and the following three parts represent the average time consumption of each frame processing in the three dimensions. If the received average frame rate distribution is greater than interval 0, the decoding average frame rate is judged. If the above conditions are met, the rendering average frame rate is finally judged to give the final evaluation result, such as {[fps=40, relatively smooth], [receive framing, 20ms], [data decoding, 23ms], [screen rendering, 24ms]}.

[0028] When the received average frame rate distribution is greater than interval 0, that is, distributed in interval 1 (corresponding to the second interval), interval 2 (corresponding to the third interval) and interval 3 (corresponding to the fourth interval), as shown in Table 1, there are multiple situations for subsequent evaluation.

[0029] Table 1

[0030] Interval Name Interval range (in fps) Fluency 0 (0,25) Screen freezes 1 [25,30) Generally smooth 2 [30,50] Relatively smooth 3 >50 Relatively smooth

[0031] When the received average frame rate is within the second interval, the decoded average frame rate is judged, and the decoded average frame rate is within the first interval or the second interval. If the decoded average frame rate is within the second interval, the rendered average frame rate is judged, and the rendered average frame rate is within the first interval or the second interval.

[0032] When the received average frame rate is within the third interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, or the third interval. If the decoding average frame rate is within the second interval or the third interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, or the third interval.

[0033] When the received average frame rate is within the fourth interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, the third interval, or the fourth interval. If the decoding average frame rate is within the second interval, the third interval, or the fourth interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, the third interval, or the fourth interval.

[0034] Specifically, when the received average frame rate is in interval 2, that is, the received average frame rate is in the interval [30,50], it is necessary to continue to judge the decoding average frame rate. If the decoding average frame rate is in (0,25), that is, in interval 0, the evaluation result is directly "screen freeze", and there is no need to judge the rendering average frame rate. When the decoding average frame rate is not in interval 0, further judgment is required. If the decoding average frame rate is in interval 1, it is necessary to continue to judge the rendering average frame rate. If the rendering average frame rate is in interval 0, the evaluation result is also "screen freeze". If the rendering average frame rate is in interval 1, the evaluation result is "generally smooth". Since the decoding average frame rate is in interval 1, the rendering average frame rate is at most in interval 1 and cannot exceed interval 1. The received average frame rate is in interval 2, so the decoding average frame rate is also at most in interval 2. When the decoding average frame rate is in interval 2, the rendering average frame rate is judged as above, and no further explanation is given.

[0035] Figure 3 This is a structural diagram of the cloud gaming screen smoothness evaluation device described in the present application, which includes a smoothness evaluation unit, a frame receiving and grouping statistics unit, a data decoding statistics unit and a screen rendering statistics unit.

[0036] The receiving frame statistics unit is used to obtain the total number of frames F1 and the total receiving time T1 of the frame within the statistical cycle time T when receiving the frame. The average frame rate of the frame reception is expressed as FSP1=F1 / T, and the average time per frame of the frame reception is AVG1=T1 / F1.

[0037] The receiving frame statistics unit receives data and combines the sub-packetized data into a complete frame of data to be decoded; on the other hand, it counts the time taken for each frame of data from the beginning of reception to the final framing completion, and accumulates the total time taken and the total number of frames processed within the statistical cycle.

[0038] The data decoding statistics unit is used to obtain the total number of decoded frames F2 and the total decoding time T2 within the statistical cycle time T when decoding the data. The average frame rate of data decoding is expressed as FSP2=F2 / T, and the average time per frame of data decoding is AVG2=T2 / F2.

[0039] The data decoding statistics unit counts the total number of frames decoded within a cycle time, calculates the time taken for each frame of data from being sent to the decoding unit to being decoded, and accumulates the time.

[0040] The screen rendering statistics unit is used to obtain the total number of rendered frames F3 and the total rendering time T3 within the statistical cycle time T when rendering the screen. The average frame rate of the screen rendering is expressed as FSP3=F3 / T, and the average time per frame of the screen rendering is AVG3=T3 / F3.

[0041] The image rendering statistics unit counts the total number of frames rendered within a cycle time, calculates the time taken for each frame from being sent to the rendering unit to being rendered, and accumulates the time.

[0042] A fluency evaluation unit is used to evaluate the fluency of a picture, including: judging the received average frame rate; if the received average frame rate is within a first interval, the evaluation result is "picture freeze"; if the received average frame rate is within a second interval, a third interval, or a fourth interval, judging the decoded average frame rate; if the decoded average frame rate is within the first interval, the evaluation result is "picture freeze"; if the decoded average frame rate is within the second interval, a third interval, or a fourth interval, judging the rendered average frame rate; if the rendered average frame rate is within the first interval, the evaluation result is "picture freeze"; if the rendered average frame rate is within the second interval, the evaluation result is "generally smooth"; if the rendered average frame rate is within the third interval, the evaluation result is "relatively smooth"; if the rendered average frame rate is within the fourth interval, the evaluation result is "relatively smooth".

[0043] The smoothness evaluation unit evaluates the picture smoothness further including: when the received average frame rate is within the second interval, judging the decoding average frame rate, the decoding average frame rate is within the first interval or the second interval, and if the decoding average frame rate is within the second interval, judging the rendering average frame rate, the rendering average frame rate is within the first interval or the second interval;

[0044] When the received average frame rate is within the third interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, or the third interval; if the decoding average frame rate is within the second interval or the third interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, or the third interval;

[0045] When the received average frame rate is within the fourth interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, the third interval, or the fourth interval. If the decoding average frame rate is within the second interval, the third interval, or the fourth interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, the third interval, or the fourth interval.

[0046] The above are exemplary embodiments of the present application, and the protection scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for evaluating the smoothness of a cloud game screen, characterized in that: include: The received average frame rate is judged. If the received average frame rate is within the first interval, the evaluation result is "screen freeze", if the received average frame rate is within the second interval, the third interval, or the fourth interval, the decoding average frame rate is judged; if the decoding average frame rate is within the first interval, the evaluation result is "screen freeze", if the decoding average frame rate is within the second interval, the third interval, or the fourth interval, the rendering average frame rate is judged; if the rendering average frame rate is within the first interval, the evaluation result is "screen freeze", if the rendering average frame rate is within the second interval, the evaluation result is "generally smooth", if the rendering average frame rate is within the third interval, the evaluation result is "relatively smooth", if the rendering average frame rate is within the fourth interval, the evaluation result is "relatively smooth"; The received average frame rate is expressed as FSP1=F1 / T, T represents the statistical cycle time, and F1 represents the total number of frames of the frame obtained within the statistical cycle time T when the frame is received; the total reception time is expressed as T1, and the average time consumed per frame of the frame reception is AVG1=T1 / F1; The average decoding frame rate is expressed as FSP2=F2 / T, where F2 represents the total number of frames decoded within the statistical cycle time T when decoding the data; the total decoding time is T2, and the average time per frame of data decoding is AVG2=T2 / F2; The average rendering frame rate is expressed as FSP3=F3 / T, where F3 represents the total number of frames rendered within the statistical cycle time T when rendering the picture; the total rendering time is T3, and the average rendering time per frame is AVG3=T3 / F3; Wherein, when the received average frame rate is within the second interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval or the second interval; if the decoding average frame rate is within the second interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval or the second interval; When the received average frame rate is within the third interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, or the third interval; if the decoding average frame rate is within the second interval or the third interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, or the third interval; When the received average frame rate is within the fourth interval, the decoding average frame rate is judged, and the decoding average frame rate is within the first interval, the second interval, the third interval, or the fourth interval. If the decoding average frame rate is within the second interval, the third interval, or the fourth interval, the rendering average frame rate is judged, and the rendering average frame rate is within the first interval, the second interval, the third interval, or the fourth interval.

2. The evaluation method according to claim 1, characterized in that: The first interval is (0, 25), the second interval is [25, 30), the third interval is [30, 50], and the fourth interval is greater than 50.

3. A cloud game screen smoothness evaluation device, the evaluation device is used for the cloud game screen smoothness evaluation method described in any one of claims 1-2, characterized in that: include: The receiving frame statistical unit, when receiving the frame, obtains the total number of frames F1 and the total receiving time T1 of the frame within the statistical cycle time T, then the average frame rate of the frame reception is expressed as FSP1=F1 / T, and the average time per frame of the frame reception is AVG1=T1 / F1; The data decoding statistics unit, when decoding the data, obtains the total number of decoded frames F2 and the total decoding time T2 within the statistical cycle time T, then the average frame rate of data decoding is expressed as FSP2 = F2 / T, and the average time per frame of data decoding is AVG2 = T2 / F2; The picture rendering statistics unit, when rendering the picture, obtains the total number of rendered frames F3 and the total rendering time T3 within the statistical cycle time T, then the average frame rate of the picture rendering is expressed as FSP3=F3 / T, and the average time per frame of the picture rendering is AVG3=T3 / F3; The smoothness evaluation unit evaluates the smoothness of the picture, including: judging the received average frame rate, if the received average frame rate is within the first interval, the evaluation result is "picture freeze", if the received average frame rate is within the second interval, the third interval or the fourth interval, the decoding average frame rate is judged; if the decoding average frame rate is within the first interval, the evaluation result is "picture freeze", if the decoding average frame rate is within the second interval, the third interval or the fourth interval, the rendering average frame rate is judged; if the rendering average frame rate is within the first interval, the evaluation result is "picture freeze", if the rendering average frame rate is within the second interval, the evaluation result is "generally smooth", if the rendering average frame rate is within the third interval, the evaluation result is "relatively smooth", if the rendering average frame rate is within the fourth interval, the evaluation result is "relatively smooth".

Images