A method and system for evaluating cloud gaming adaptation performance

By integrating a performance acquisition SDK and Flink service system into cloud gaming, various game experience metrics are collected and calculated in real time, solving the problem of inaccurate evaluation by a single metric and achieving accurate evaluation of cloud gaming adaptation performance.

CN117427328BActive Publication Date: 2026-11-17WELLINK TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311428743.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-31
Publication Date
2026-11-17
Estimated Expiration
2043-10-31

AI Technical Summary

Technical Problem

In existing technologies, the evaluation of user experience in cloud gaming relies on a single technical indicator, leading to inaccurate evaluation results.

Method used

By integrating a performance acquisition SDK into the client, various game experience metrics data are collected in real time. The distributed messaging system and Flink service system are used for calculation, and combined with the adapted performance scoring model, detailed evaluation results are generated.

Benefits of technology

It improves the accuracy of cloud gaming adaptation performance evaluation, quantifies user experience through multi-dimensional indicators, and provides more accurate evaluation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117427328B_ABST
    Figure CN117427328B_ABST
Patent Text Reader

Abstract

This invention provides a method and system for evaluating cloud gaming adaptation performance. The method includes: a client-integrated performance acquisition SDK collecting game experience metrics data every second during a user's cloud gaming session, including multiple basic metrics; the performance acquisition SDK sending the multiple game experience metrics data to a distributed messaging system; the distributed messaging system receiving the multiple game experience metrics data and sending it to a Flink service system; the Flink service system calculating secondary metrics data corresponding to each basic metric data according to a preset algorithm; and outputting the evaluation result of cloud gaming adaptation performance according to a preset adaptation performance scoring model. This invention can intuitively reflect the detailed data of user experience cloud gaming adaptation performance evaluation, solving the technical problem in related technologies where using a single technical indicator to quantify user experience leads to inaccurate user experience evaluation results, and improving the accuracy of cloud gaming adaptation performance evaluation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer data processing technology, and more particularly to a method and system for evaluating cloud gaming adaptation performance. Background Technology

[0002] Cloud gaming is a gaming model based on cloud computing. Game visuals are calculated and rendered in the cloud, compressed, and transmitted to the client via the network. The client then decodes and plays the streaming media to control the game.

[0003] Cloud gaming is a complex technological system involving multiple stages, including control signaling transmission, image rendering, audio and video encoding and decoding, and data transmission. The quality of any one of these stages will affect the player's gaming experience. To optimize cloud gaming, user experience can be statistically analyzed. User experience can be quantified using objective metrics. For example, corresponding technical indicators can be used to evaluate user experience. However, using a single technical indicator (such as stuttering rate) to quantify user experience in related technologies leads to inaccurate evaluation results due to the lack of correlation between different indicators. Summary of the Invention

[0004] The present invention aims to solve the above-mentioned problems.

[0005] The main objective of this invention is to provide a method for evaluating cloud gaming adaptation performance.

[0006] Another objective of this invention is to provide a cloud gaming adaptation performance evaluation system.

[0007] To achieve the above objectives, the technical solution of the present invention is specifically implemented as follows:

[0008] This invention provides a method for evaluating cloud gaming adaptation performance, the method comprising:

[0009] The client-integrated performance acquisition software development kit (SDK) collects game experience metric data once per second during a user's cloud gaming session, obtaining multiple game experience metric data, which include multiple basic metric data.

[0010] The performance acquisition SDK integrated into the client sends multiple game experience metric data to the distributed messaging system. The distributed messaging system receives the multiple game experience metric data and sends them to the Flink service system.

[0011] The Flink service system receives multiple game experience metric data and calculates the secondary metric data corresponding to each basic metric data according to a preset algorithm.

[0012] The Flink service system outputs an evaluation result of the cloud game's adaptation performance based on the secondary indicator data corresponding to each of the basic indicator data, according to a preset adaptation performance scoring model.

[0013] Optionally, the basic metrics data include at least one of the following: bitrate, network latency, server-side frame rate, client-side frame rate, rendering frame rate, decoding frame rate, resolution, total number of audio and video packets, number of audio packet losses, number of video packet losses, packet loss rate, total number of packet losses, audio stuttering time, number of stutters per second, and stuttering duration per second.

[0014] Optionally, the performance acquisition SDK integrated into the client sends multiple game experience metric data points to a distributed messaging system, including:

[0015] The performance acquisition SDK integrated in the client reports multiple game experience metric data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system.

[0016] The performance acquisition SDK integrated in the client writes multiple game experience metric data into a local data reporting SDK. The data reporting SDK reports multiple game experience metric data to the cloud server by calling an API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes it to the PaaS log collection service system. The PaaS log collection service system sends the multiple game experience metric data to the distributed messaging system.

[0017] The performance acquisition SDK integrated into the client reports multiple game experience metric data to the distributed messaging system by calling the API.

[0018] Optionally, the Flink service system receives multiple game experience metric data points and calculates secondary metric data corresponding to each basic metric data point according to a preset algorithm, including:

[0019] The Flink service system obtains multiple game experience metric data for each basic metric data from multiple sets of game experience metric data;

[0020] When the basic indicator data includes at least one of the following: bitrate, server frame rate, client frame rate, rendering frame rate, decoding frame rate, and packet loss rate, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the values ​​of multiple game experience indicator data of the basic indicator data are arranged in descending order from high to low, and the value at the 90th position is taken as the secondary indicator data of the basic indicator data.

[0021] When the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the average value of multiple game experience indicator data of the basic indicator data is taken as the secondary indicator data of the basic indicator data.

[0022] When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicator data of the basic indicator data during the user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

[0023] Optionally, the Flink service system outputs an evaluation result of the cloud game's adaptation performance for each of the basic metric data and its corresponding secondary metric data according to a preset adaptation performance scoring model, including:

[0024] The Flink service system obtains the preset adaptation performance scoring model, which includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high.

[0025] The Flink service system inputs the secondary indicator data corresponding to each of the basic indicator data into the preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each of the basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each of the basic indicator data.

[0026] Another aspect of the present invention provides a cloud gaming adaptation performance evaluation system, comprising: a client, a distributed messaging system, and a Flink service system, wherein:

[0027] The performance data acquisition software development kit (SDK) integrated into the client is used to collect game experience metric data once per second during a user's cloud gaming session, and obtain multiple game experience metric data, wherein the game experience metric data includes multiple basic metric data.

[0028] The performance acquisition SDK integrated into the client is also used to send multiple game experience metric data to a distributed messaging system;

[0029] The distributed messaging system is used to receive multiple game experience metric data and send the multiple game experience metric data to the Flink service system;

[0030] The Flink service system is used to receive multiple game experience indicator data and calculate the secondary indicator data corresponding to each basic indicator data according to a preset algorithm.

[0031] The Flink service system is also used to output an evaluation result of the adaptation performance of the cloud game for each secondary indicator data corresponding to the basic indicator data according to a preset adaptation performance scoring model.

[0032] Optionally, the basic metrics data include at least one of the following: bitrate, network latency, server-side frame rate, client-side frame rate, rendering frame rate, decoding frame rate, resolution, total number of audio and video packets, number of audio packet losses, number of video packet losses, packet loss rate, total number of packet losses, audio stuttering time, number of stutters per second, and stuttering duration per second.

[0033] Optionally, the performance data collection SDK integrated into the client sends multiple game experience metric data points to the distributed messaging system in the following manner:

[0034] The performance acquisition SDK integrated in the client reports multiple game experience metric data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system.

[0035] The performance acquisition SDK integrated in the client writes multiple game experience metric data into a local data reporting SDK. The data reporting SDK reports multiple game experience metric data to the cloud server by calling an API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes it to the PaaS log collection service system. The PaaS log collection service system sends the multiple game experience metric data to the distributed messaging system.

[0036] The performance acquisition SDK integrated into the client reports multiple game experience metric data to the distributed messaging system by calling the API.

[0037] Optionally, the Flink service system receives multiple game experience metric data in the following manner, and calculates the secondary metric data corresponding to each basic metric data according to a preset algorithm:

[0038] The Flink service system obtains multiple game experience metric data for each basic metric data from multiple sets of game experience metric data;

[0039] When the basic indicator data includes at least one of the following: bitrate, server frame rate, client frame rate, rendering frame rate, decoding frame rate, and packet loss rate, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the values ​​of multiple game experience indicator data of the basic indicator data are arranged in descending order from high to low, and the value at the 90th position is taken as the secondary indicator data of the basic indicator data.

[0040] When the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the average value of multiple game experience indicator data of the basic indicator data is taken as the secondary indicator data of the basic indicator data.

[0041] When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicator data of the basic indicator data during the user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

[0042] Optionally, the Flink service system outputs an evaluation result of the cloud game's adaptation performance based on a preset adaptation performance scoring model for each of the basic metric data's corresponding secondary metric data:

[0043] The Flink service system obtains the preset adaptation performance scoring model, which includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high.

[0044] The Flink service system inputs the secondary indicator data corresponding to each of the basic indicator data into the preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each of the basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each of the basic indicator data.

[0045] As can be seen from the technical solution provided by the present invention, the present invention provides a cloud gaming adaptation performance evaluation method and system, which can intuitively reflect the detailed data of user experience cloud gaming adaptation performance evaluation. It solves the technical problem in related technologies that use a single technical indicator to quantify user experience, resulting in inaccurate user experience evaluation results. By combining technologies such as game client APP, Kafka message system, and Flink streaming data computing framework, the present invention collects game experience indicator data related to cloud game operation per second in real time, calculates the operation status of cloud game in real time, and evaluates the operation status of cloud game through a series of quantitative indicators, thereby improving the accuracy of cloud game adaptation performance evaluation. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the structure of a cloud gaming adaptation performance evaluation system provided in an embodiment of the present invention;

[0048] Figure 2 A flowchart of a cloud gaming adaptation performance evaluation method provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of a computing device for cloud gaming adaptation performance evaluation provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0051] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0052] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0053] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0054] The embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0055] This embodiment provides a cloud gaming adaptation performance evaluation system. Figure 1 This is a schematic diagram of the structure of a cloud gaming adaptation performance evaluation system.

[0056] like Figure 1 As shown, the cloud gaming adaptation performance evaluation system provided in this embodiment includes: a client, a distributed messaging system, and a Flink service system. The client integrates a performance acquisition SDK (Software Development Kit) to collect game experience metric data every second and ultimately report it to the distributed messaging system via the network. The distributed messaging system is a distributed streaming media platform whose main functions include: publishing and subscribing to message streams; recording message streams in a fault-tolerant manner and storing them in files; and processing messages during publication. The distributed messaging system is primarily used to report the game experience metric data reported by the client's performance acquisition SDK to the Flink service system. Optionally, the distributed messaging system can be a Kafka messaging system; the above is merely an example, and this embodiment does not impose any limitations on it. The Flink service system is used to calculate the received reported data, input the calculation results into the adaptation performance scoring model, and obtain the evaluation results.

[0057] Based on the cloud gaming adaptation performance evaluation system provided in this embodiment, this embodiment also provides a cloud gaming adaptation performance evaluation method. Figure 2 This is a flowchart of the cloud gaming adaptation performance evaluation method provided in this embodiment. Figure 2 As shown, the method includes the following steps (S101-S104):

[0058] S101, the client-integrated performance data acquisition software development kit (SDK) collects game experience metric data once per second during a user's cloud gaming session, obtaining multiple game experience metric data, which include multiple basic metric data.

[0059] In this embodiment, the user plays cloud games using a terminal device, which is configured with a client corresponding to the cloud game. The terminal device may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Device), PAD, desktop computer, etc. The above is merely an example, and this embodiment does not impose any limitations on it.

[0060] In specific implementation, data collection is performed on a session of cloud gaming played by a user. During this session, for example, within 10 minutes, one game experience metric is collected per second. Each metric includes multiple basic metrics. As an optional implementation in this embodiment, the basic metrics include at least one of the following: bitrate, network latency, server-side frame rate, client-side frame rate, rendering frame rate, decoding frame rate, resolution, total audio / video packets, audio packet loss, video packet loss, packet loss rate, total number of packets lost, audio stuttering time, number of stutters per second, and stuttering duration per second. The above is merely an example, and this embodiment does not impose any limitations. In this embodiment, by collecting the above-mentioned different types of basic metrics, the detailed data for cloud gaming adaptation performance evaluation of user experience can be intuitively reflected. This solves the technical problem in related technologies where using a single technical indicator to quantify user experience leads to inaccurate evaluation results, thus improving the accuracy of cloud gaming adaptation performance evaluation.

[0061] S102. The performance collection SDK integrated on the client sends multiple game experience metric data to the distributed message system. The distributed message system receives the multiple game experience metric data and sends them to the Flink service system.

[0062] As an optional implementation method in this embodiment, the performance collection SDK integrated on the client sends multiple game experience metric data to the distributed messaging system. The distributed messaging system receives the multiple game experience metric data and sends them to the Flink service system, including one of the following three methods:

[0063] Method 1: The client-integrated performance collection SDK reports multiple game experience metric data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system.

[0064] Method 2: The client-integrated performance collection SDK writes multiple game experience metric data to the local data reporting SDK. The data reporting SDK then reports multiple game experience metric data to the cloud server via API calls. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and distributes it to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system.

[0065] Specifically, in both of the above methods, the PaaS log collection service system is implemented using Java programs and deployed on a cloud server. After the client performance acquisition SDK collects data, it sends it to the PaaS log collection service system via API calls. The PaaS log collection service system writes the data to disk as file logs, and then uses the Filebeat tool to send the written logs line by line to the distributed messaging system.

[0066] In this embodiment, the client can integrate not only the performance acquisition SDK, but also the data reporting SDK. The performance acquisition SDK writes the collected game experience metrics data into the local data reporting SDK storage, and the data reporting SDK connects to the PAAS log collection service system through the file service.

[0067] Method 3: The performance collection SDK integrated into the client reports multiple game experience metrics to the distributed messaging system by calling the API.

[0068] Specifically, the performance acquisition SDK can be the Kafka Producer SDK, which directly connects to the Kafka messaging system.

[0069] This embodiment provides multiple methods for clients to report data to a distributed messaging system, featuring multiple off-site expansion or disaster recovery capabilities. The client-integrated data reporting SDK stores multiple game experience metric data locally before reporting them, without affecting the performance of the performance collection SDK, and ensures data integrity and uniqueness.

[0070] S103 and the Flink service system receive multiple game experience indicator data and calculate the secondary indicator data corresponding to each basic indicator data according to the preset algorithm.

[0071] As an optional implementation method in this embodiment, the Flink service system receives multiple game experience indicator data and calculates the secondary indicator data corresponding to each basic indicator data according to a preset algorithm, including:

[0072] The Flink service system obtains multiple game experience metric data for each basic metric from multiple game experience metric data;

[0073] When the basic performance metrics include at least one of the following: bitrate, server-side frame rate, client-side frame rate, rendered frame rate, decoded frame rate, and packet loss rate, the secondary performance metrics corresponding to the basic performance metrics are calculated as follows: The values ​​of multiple game experience metrics from the basic performance metrics are sorted in descending order from highest to lowest. The value at the 90th percentile is taken as the secondary performance metrics. For example, taking rendered frame rate as an example, all received rendered frame rate values ​​up to the current time are sorted in descending order from highest to lowest. The rendered frame rate at the 90th percentile is recorded as the 90th percentile rendered frame rate, which is the secondary performance metrics for rendered frame rate. In this embodiment, using the 90th percentile calculation method can effectively eliminate some low-frame-rate scenes in the actual game visuals, thus more accurately representing the user's real experience.

[0074] As an optional implementation of this embodiment, when the basic indicator data includes at least one of the following: server frame count and client frame count, the difference in average frames per second between the client and the server (avg(abs(client frame count - server frame count))) can also be calculated as secondary indicator data.

[0075] If the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data shall be calculated as follows: the average value of multiple game experience indicator data of the basic indicator data shall be used as the secondary indicator data of the basic indicator data; for example, taking the number of stutters per second as an example, the average value of all the stuttering times per second received up to the present shall be calculated as the secondary indicator data of the number of stutters per second.

[0076] When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicators in a user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

[0077] For example, Table 1 shows the calculation methods for each basic indicator data and the corresponding secondary indicator data.

[0078]

[0079]

[0080] Table 1 shows the data for each basic indicator and its corresponding secondary indicator.

[0081] The basic indicator data and corresponding secondary indicator data in Table 1 above are merely examples, and this embodiment does not impose any limitations on them. In this embodiment, secondary indicator data can be obtained by performing secondary calculations on different basic indicator data. The secondary indicator data defines the main indicators that need to be focused on from three aspects that directly affect the user's perception: video features, audio features, and continuity and integrity features. This intuitively reflects the detailed data of cloud game adaptation performance evaluation of user experience, ensuring that user experience is quantified from multiple dimensions of technical indicators and improving the accuracy of cloud game adaptation performance evaluation.

[0082] S104 and the Flink service system output the evaluation results of the adaptation performance of cloud gaming based on the secondary indicator data corresponding to each basic indicator data according to the preset adaptation performance scoring model.

[0083] In one optional implementation of this embodiment, the Flink service system outputs the evaluation results of the adaptation performance of cloud gaming for each basic indicator data and the corresponding secondary indicator data according to a preset adaptation performance scoring model, including (steps S201-S202):

[0084] S201. The Flink service system obtains the preset adaptation performance scoring model. The preset adaptation performance scoring model includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high.

[0085] Specifically, several adaptation performance scoring models are given below as examples. These are just examples, and this embodiment does not limit them in any way.

[0086] Taking screen resolution as the basic performance indicator, the adaptation performance scoring model is as follows:

[0087]

[0088]

[0089] Taking frame rate (server-side frame rate, client-side frame rate, rendering frame rate, decoding frame rate) as the basic metric data, the adapted performance scoring model is as follows:

[0090]

[0091] Taking packet loss rate as the basic metric, the adaptation performance scoring model is as follows:

[0092]

[0093] Taking network latency as the basic metric, the adapted performance scoring model is as follows:

[0094]

[0095] Taking bitrate as an example, the adaptation performance scoring model is as follows:

[0096]

[0097]

[0098] Taking audio stuttering time as the basic metric, the performance scoring model is as follows:

[0099]

[0100] Taking the basic metric of the number of stutters per second as an example, the adapted performance scoring model is as follows:

[0101]

[0102] Taking the basic metric of lag duration within one second as an example, the corresponding performance scoring model is as follows:

[0103]

[0104] S202. The Flink service system inputs the secondary indicator data corresponding to each basic indicator data into the preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each basic indicator data.

[0105] Specifically, inputting the secondary indicator data corresponding to the basic indicator data into the aforementioned adaptation performance scoring model will yield the corresponding evaluation score. Optionally, a comprehensive score can be obtained by combining the scores of the secondary indicator data corresponding to each basic indicator data. For example, the total score can be obtained by summing the scores of each item, which serves as the evaluation result. Alternatively, each evaluation score can be multiplied by a weighting coefficient and then summed to obtain a weighted total score, which also serves as the evaluation result.

[0106] In practice, the Flink service system is essentially a real-time data warehouse computing framework. Through the Flink service system and the Kafka messaging system, an architecture similar to an offline data warehouse is formed, utilizing the message data storage capabilities of the Kafka messaging system to replace hierarchical storage solutions. The Flink service system acts as the internal computing tool, while the Kafka messaging system acts as the storage.

[0107] As an optional implementation method in this embodiment, the method outputs the evaluation results of cloud gaming adaptation performance based on the secondary indicator data corresponding to each basic indicator data according to a preset adaptation performance scoring model. The method also includes displaying the evaluation results. Specifically, after real-time data warehouse calculations are completed through the Flink service system, the data is written to Doris (a high-performance, real-time analytical database with an MPP architecture) to form a database table. The adaptation performance scoring model is then input, the evaluation results are output, and the results are displayed on the BI platform using a self-developed web UI.

[0108] The cloud gaming adaptation performance evaluation method provided in this embodiment can intuitively reflect the detailed data of user experience cloud gaming adaptation performance evaluation. It solves the technical problem in related technologies that use a single technical indicator to quantify user experience, resulting in inaccurate user experience evaluation results. By combining technologies such as game client APP, Kafka message system, and Flink streaming data computing framework, it collects relevant game experience indicator data per second of cloud game operation in real time, calculates the operation status of cloud game in real time, and evaluates the operation status of cloud game through a series of quantitative indicators, thereby improving the accuracy of cloud game adaptation performance evaluation.

[0109] This embodiment also provides a cloud gaming adaptation performance evaluation system. Figure 2 A cloud gaming adaptation performance evaluation system is presented. This system employs the aforementioned cloud gaming adaptation performance evaluation method. The following is only a brief description of this system; for other matters not covered herein, please refer to the relevant descriptions in the aforementioned cloud gaming adaptation performance evaluation method. Further details will not be elaborated upon here. See [link to relevant documentation]. Figure 2 The cloud gaming adaptation performance evaluation system provided in this embodiment of the invention includes: the system comprises a client, a distributed messaging system, and a Flink service system, wherein:

[0110] The client-integrated performance data acquisition software development kit (SDK) is used to collect game experience metric data once per second during a user's cloud gaming session, resulting in multiple game experience metric data, which include several basic metric data.

[0111] The performance data collection SDK integrated into the client is also used to send multiple game experience metrics to a distributed messaging system;

[0112] A distributed messaging system is used to receive multiple game experience metric data and send them to the Flink service system.

[0113] The Flink service system is used to receive multiple game experience metric data and calculate the secondary metric data corresponding to each basic metric data according to a preset algorithm.

[0114] The Flink service system is also used to output the evaluation results of the adaptation performance of cloud gaming based on the secondary indicator data corresponding to each basic indicator data according to the preset adaptation performance scoring model.

[0115] In this embodiment, the user plays cloud games using a terminal device, which is configured with a client corresponding to the cloud game. The terminal device may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Device), PAD, desktop computer, etc. The above is merely an example, and this embodiment does not impose any limitations on it.

[0116] In this embodiment, the client integrates a performance acquisition SDK to collect game experience metric data every second and ultimately reports it to a distributed messaging system via the network. The distributed messaging system is a distributed streaming media platform whose main functions include: publishing and subscribing to message streams; recording message streams in a fault-tolerant manner and storing them in files; and processing messages during publication. The distributed messaging system is primarily used to report the game experience metric data reported by the client's performance acquisition SDK to the Flink service system. Optionally, the distributed messaging system can be a Kafka messaging system; the above is merely an example, and this embodiment does not impose any limitations on it. The Flink service system is used to calculate the received reported data, input the calculation results into the adaptation performance scoring model, and obtain the evaluation results.

[0117] As an optional implementation of this invention, the basic indicator data includes at least one of the following: bitrate, network latency, server-side frame rate, client-side frame rate, rendered frame rate, decoded frame rate, resolution, total number of audio and video packets, number of audio packet losses, number of video packet losses, packet loss rate, total number of packet losses, audio stuttering time, number of stutters per second, and stuttering duration per second. The above is merely an example, and this embodiment does not impose any limitations. In this embodiment, by collecting the above-mentioned different types of basic indicator data, the detailed data for cloud gaming adaptation performance evaluation of user experience can be intuitively reflected, solving the technical problem in related technologies where using a single technical indicator to quantify user experience leads to inaccurate user experience evaluation results, and improving the accuracy of cloud gaming adaptation performance evaluation.

[0118] As an optional implementation of this invention, the performance collection SDK integrated into the client sends multiple game experience metric data to the distributed messaging system in the following manner:

[0119] The client-integrated performance collection SDK reports multiple game experience metrics data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing operations on the multiple game experience metrics data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metrics data to the distributed messaging system.

[0120] The client-integrated performance acquisition SDK writes multiple game experience metric data to the local data reporting SDK. The data reporting SDK then reports multiple game experience metric data to the cloud server via API calls. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and distributes it to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system.

[0121] The client-integrated performance data collection SDK reports multiple game experience metrics to a distributed messaging system by calling APIs.

[0122] This embodiment provides multiple methods for clients to report data to a distributed messaging system, featuring multiple off-site expansion or disaster recovery capabilities. The client-integrated data reporting SDK stores multiple game experience metric data locally before reporting them, without affecting the performance of the performance collection SDK, and ensures data integrity and uniqueness.

[0123] As an optional implementation of this invention, the Flink service system receives multiple game experience indicator data in the following manner, and calculates the secondary indicator data corresponding to each basic indicator data according to a preset algorithm:

[0124] The Flink service system obtains multiple game experience metric data for each basic metric from multiple game experience metric data;

[0125] If the basic indicator data includes at least one of the following: bitrate, server frame rate, client frame rate, rendering frame rate, decoding frame rate, and packet loss rate, the secondary indicator data corresponding to the basic indicator data shall be calculated in the following way: sort the values ​​of multiple game experience indicator data of the basic indicator data in descending order from high to low, and take the value at the 90th position as the secondary indicator data of the basic indicator data.

[0126] If the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data shall be calculated in the following way: take the average value of multiple game experience indicator data of the basic indicator data as the secondary indicator data of the basic indicator data.

[0127] When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicators in a user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

[0128] Refer to the basic indicator data and corresponding secondary indicator data in Table 1 above. Of course, Table 1 is just an example, and this embodiment does not impose any limitations on it. In this embodiment, secondary indicator data can be obtained by performing secondary calculations on different basic indicator data. The secondary indicator data defines the main indicators that need attention from three aspects that directly affect the user's perception: video features, audio features, and continuity and integrity features. This intuitively reflects the detailed data for cloud game adaptation performance evaluation of user experience, ensuring that user experience is quantified from multiple dimensions of technical indicators, and improving the accuracy of cloud game adaptation performance evaluation.

[0129] As an optional implementation of this invention, the Flink service system outputs the evaluation results of the cloud gaming adaptation performance based on a preset adaptation performance scoring model for the secondary indicator data corresponding to each basic indicator data in the following manner:

[0130] The Flink service system obtains a preset adaptation performance scoring model, which includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high.

[0131] The Flink service system inputs the secondary indicator data corresponding to each basic indicator data into a preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each basic indicator data.

[0132] Refer to the several adaptation performance scoring models given above as examples. Of course, these adaptation performance scoring models are just examples, and this embodiment does not limit them in any way.

[0133] Specifically, inputting the secondary indicator data corresponding to the basic indicator data into the aforementioned adaptation performance scoring model will yield the corresponding evaluation score. Optionally, a comprehensive score can be obtained by combining the scores of the secondary indicator data corresponding to each basic indicator data. For example, the total score can be obtained by summing the scores of each item, which serves as the evaluation result. Alternatively, each evaluation score can be multiplied by a weighting coefficient and then summed to obtain a weighted total score, which also serves as the evaluation result.

[0134] In practice, the Flink service system is essentially a real-time data warehouse computing framework. Through the Flink service system and the Kafka messaging system, an architecture similar to an offline data warehouse is formed, utilizing the message data storage capabilities of the Kafka messaging system to replace hierarchical storage solutions. The Flink service system acts as the internal computing tool, while the Kafka messaging system acts as the storage.

[0135] As an optional implementation in this embodiment, in addition to outputting the evaluation results of cloud gaming adaptation performance for the secondary indicator data corresponding to each basic indicator data according to the preset adaptation performance scoring model, the Flink service system is also used to display the evaluation results. Specifically, after the real-time data warehouse calculation is completed by the Flink service system, the data is written to Doris (a high-performance, real-time analytical database with MPP architecture) to form a database table, input to the adaptation performance scoring model, output the evaluation results, and display them on the BI platform through a self-developed web UI.

[0136] The cloud gaming adaptation performance evaluation system provided in this embodiment can intuitively reflect the detailed data of user experience cloud gaming adaptation performance evaluation. It solves the technical problem in related technologies that use a single technical indicator to quantify user experience, resulting in inaccurate user experience evaluation results. By combining technologies such as game client APP, Kafka message system, and Flink streaming data computing framework, it collects relevant game experience indicator data per second of cloud game operation in real time, calculates the operation status of cloud games in real time, and evaluates the operation status of cloud games through a series of quantitative indicators, thereby improving the accuracy of cloud game adaptation performance evaluation.

[0137] Furthermore, a portion of this application can be applied as a computer program product, such as computer program instructions. When executed by a computer, these instructions, through the operation of the computer, can invoke or provide the methods and / or technical solutions according to this application. The program instructions invoking the methods of this application may be stored in a fixed or removable recording medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the working memory of a computer device operating according to the program instructions. Here, embodiments of this application also provide a computing device for cloud gaming adaptation performance evaluation, such as... Figure 3 As shown, the computing device includes one or more memories for storing machine-readable instructions and a processor for executing the machine-readable instructions, wherein when the machine-readable instructions are executed by the processor, the device is triggered to execute the methods and / or technical solutions based on the foregoing embodiments of this application.

[0138] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0139] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A cloud game adaptation performance evaluation method, characterized in that, The method includes: The client-integrated performance acquisition software development kit (SDK) collects game experience metric data once per second during a user's cloud gaming session, obtaining multiple game experience metric data, which include multiple basic metric data. The performance acquisition SDK integrated into the client sends multiple game experience metric data to the distributed messaging system. The distributed messaging system receives the multiple game experience metric data and sends them to the Flink service system. The Flink service system receives multiple game experience metric data and calculates the secondary metric data corresponding to each basic metric data according to a preset algorithm. The Flink service system outputs an evaluation result of the cloud game's adaptation performance based on a preset adaptation performance scoring model for each of the basic indicator data and the corresponding secondary indicator data. The performance data collection SDK integrated into the client sends multiple game experience metric data points to the distributed messaging system, including: The performance acquisition SDK integrated in the client reports multiple game experience metric data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system. The performance acquisition SDK integrated in the client writes multiple game experience metric data into a local data reporting SDK. The data reporting SDK reports multiple game experience metric data to the cloud server by calling an API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes it to the PaaS log collection service system. The PaaS log collection service system sends the multiple game experience metric data to the distributed messaging system. The performance acquisition SDK integrated into the client reports multiple game experience metric data to the distributed messaging system by calling the API. The Flink service system receives multiple game experience metric data points and calculates the secondary metric data corresponding to each basic metric data point according to a preset algorithm, including: The Flink service system obtains multiple game experience indicator data for each basic indicator data from multiple game experience indicator data; When the basic indicator data includes at least one of the following: bitrate, server frame rate, client frame rate, rendering frame rate, decoding frame rate, and packet loss rate, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the values ​​of multiple game experience indicator data of the basic indicator data are arranged in descending order from high to low, and the value at the 90th position is taken as the secondary indicator data of the basic indicator data. When the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the average value of multiple game experience indicator data of the basic indicator data is taken as the secondary indicator data of the basic indicator data. When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicator data of the basic indicator data during the user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

2. The method according to claim 1, characterized in that, The Flink service system outputs an evaluation result of the cloud game's adaptation performance based on the secondary indicator data corresponding to each basic indicator data, according to a preset adaptation performance scoring model. This evaluation includes: The Flink service system obtains the preset adaptation performance scoring model, which includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high. The Flink service system inputs the secondary indicator data corresponding to each of the basic indicator data into the preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each of the basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each of the basic indicator data.

3. A cloud game adaptation performance evaluation system, characterized in that, The system is used to implement the method of claim 1 or 2, and the system includes: a client, a distributed messaging system, and a Flink service system, wherein: The performance data acquisition software development kit (SDK) integrated into the client is used to collect game experience metric data once per second during a user's cloud gaming session, and obtain multiple game experience metric data, wherein the game experience metric data includes multiple basic metric data. The performance acquisition SDK integrated into the client is also used to send multiple game experience metric data to a distributed messaging system; The distributed messaging system is used to receive multiple game experience metric data and send the multiple game experience metric data to the Flink service system; The Flink service system is used to receive multiple game experience indicator data and calculate the secondary indicator data corresponding to each basic indicator data according to a preset algorithm. The Flink service system is also used to output an evaluation result of the adaptation performance of the cloud game for each secondary indicator data corresponding to the basic indicator data according to a preset adaptation performance scoring model. The performance data collection SDK integrated into the client sends multiple game experience metric data points to the distributed messaging system in the following ways: The performance acquisition SDK integrated in the client reports multiple game experience metric data to the cloud server by calling the API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes them to the PaaS log collection service system. The PaaS log collection service system then sends the multiple game experience metric data to the distributed messaging system. The performance acquisition SDK integrated in the client writes multiple game experience metric data into a local data reporting SDK. The data reporting SDK reports multiple game experience metric data to the cloud server by calling an API. The cloud server performs domain name resolution and load balancing on the multiple game experience metric data and then distributes it to the PaaS log collection service system. The PaaS log collection service system sends the multiple game experience metric data to the distributed messaging system. The performance acquisition SDK integrated into the client reports multiple game experience metric data to the distributed messaging system by calling the API. The Flink service system receives multiple game experience metric data points in the following manner, and calculates the secondary metric data corresponding to each basic metric data point according to a preset algorithm: The Flink service system obtains multiple game experience indicator data for each basic indicator data from multiple game experience indicator data; When the basic indicator data includes at least one of the following: bitrate, server frame rate, client frame rate, rendering frame rate, decoding frame rate, and packet loss rate, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the values ​​of multiple game experience indicator data of the basic indicator data are arranged in descending order from high to low, and the value at the 90th position is taken as the secondary indicator data of the basic indicator data. When the basic indicator data includes at least one of the following: network latency, audio stuttering time, number of stutters per second, and stuttering duration per second, the secondary indicator data corresponding to the basic indicator data is calculated in the following way: the average value of multiple game experience indicator data of the basic indicator data is taken as the secondary indicator data of the basic indicator data. When the basic indicator data includes resolution, the secondary indicator data corresponding to the basic indicator data is calculated as follows: the resolution with the highest proportion of multiple game experience indicator data of the basic indicator data during the user's cloud gaming session is taken as the secondary indicator data of the basic indicator data.

4. The system according to claim 3, characterized in that, The Flink service system outputs the evaluation results of the cloud game's adaptation performance based on a preset adaptation performance scoring model for each of the basic indicator data's corresponding secondary indicator data: The Flink service system obtains the preset adaptation performance scoring model, which includes sorting each basic indicator data from low to high and then dividing it into corresponding evaluation scores from low to high. The Flink service system inputs the secondary indicator data corresponding to each of the basic indicator data into the preset adaptation performance scoring model to obtain the evaluation score corresponding to the secondary indicator data corresponding to each of the basic indicator data, and / or the comprehensive score of the secondary indicator data corresponding to each of the basic indicator data.

Citation Information

Patent Citations

  • Cloud game detection method and device, storage medium and electronic device

    CN110812835A

  • Game resource allocation method based on cloud computing, and cloud game service platform

    CN112190927A