Information processing device and method

Abstract

The present technology relates to an information processing device and method capable of improving the quality of user experience. This information processing device is provided with: a media generation unit for generating media data that includes frames to which is added meta information for inducing the transmission of controller signals and that is for playing back media comprising at least any of video and audio of a plurality of frames in a test game for evaluating the quality of user experience; and a communication unit for transmitting the media data to a gaming terminal. If controller signals transmitted by the gaming terminal are received by the communication unit, the media generation unit generates media data in which operations instructed by the controller signals are depicted. The present technology can be applied to cloud gaming systems.

Description

Information processing device and method

[0001] The present technology relates to an information processing device and method, and more particularly to an information processing device and method that can improve the quality of experience of a user.

[0002] In contrast to traditional console-style gaming experiences, cloud gaming systems, which do not require high specs from users' devices, are attracting attention.

[0003] A cloud gaming system is a thin client system in which the main rendering functions are stored on the cloud, and the user's device is only responsible for receiving and displaying video and sending controller signals, so advanced hardware specifications are not required for the user's device.

[0004] On the other hand, since cloud gaming systems are based on communication via a network, it is essential to evaluate the communication environment in order to stabilize the end-to-end (E2E) quality of the game experience, i.e., the quality of the user experience.

[0005] In particular, a cloud gaming system (cloud gaming) is a system in which signals are transmitted bidirectionally, with video and audio streaming data being transmitted via a downlink and controller signals being transmitted via an uplink. In other words, the cloud gaming system is an interactive system in which video and audio streaming data are generated based on controller signals.

[0006] For this reason, the usual method used in video distribution systems, such as providing a large buffer on the user's terminal (receiving side) to absorb network quality fluctuations at the receiving side, is no longer effective, and evaluation of the communication environment is becoming even more important in order to provide precise control over such quality fluctuations.

[0007] There are a variety of factors that affect the quality of information transmission between the server that distributes the game and the client (user's terminal), and there are also a variety of metrics for measuring the level of quality, etc. For example, known metrics include the quality of the transmitted game screen, encoding quality factors, server quality factors, network quality factors, and terminal quality factors.

[0008] A method for comprehensively evaluating these metrics from the perspective of user experience quality is required.

[0009] Furthermore, as a technology relating to game control, a technology for controlling game play for a user based on a default game play style has been proposed (see, for example, Patent Document 1). Furthermore, as a technology relating to game quality verification, a technology for detecting bugs occurring during the execution of multiple automatic sessions of a game application has also been proposed (see, for example, Patent Document 2).

[0010] Japanese Patent Application Publication No. 2021-41227 Special Publication No. 2022-510600

[0011] In cloud-based games provided by cloud gaming systems, the game screen transitions interactively with the user's controller input, and the game scene and controller input change from moment to moment.

[0012] Therefore, when evaluating the user's quality of experience in cloud-based games, especially when evaluating the communication environment, it is necessary to take into account the user's controller input.

[0013] However, because the above-mentioned technology does not take controller input into consideration, it cannot be said that sufficient evaluation is possible, and as a result, it has been difficult to improve the quality of the user's experience.

[0014] The present technology has been made in view of such circumstances, and is intended to improve the quality of experience for users.

[0015] An information processing device according to a first aspect of the present technology includes a media generation unit that generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, the frames including frames to which meta information for inducing the transmission of a controller signal is attached, and a communication unit that transmits the media data to a gaming terminal, wherein when the controller signal transmitted by the gaming terminal is received by the communication unit, the media generation unit generates the media data that reflects the operation indicated by the controller signal.

[0016] An information processing method according to a first aspect of the present technology generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, the frames including frames to which meta information for inducing the transmission of a controller signal is attached, transmits the media data to a gaming terminal, and, when the controller signal transmitted by the gaming terminal is received, generates the media data that reflects the operation indicated by the controller signal.

[0017] In a first aspect of the present technology, media data is generated for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating the quality of a user's experience, including frames to which meta information for inducing the transmission of a controller signal is attached, the media data is transmitted to a gaming terminal, and when the controller signal transmitted by the gaming terminal is received, the media data is generated in which the operation indicated by the controller signal is reflected.

[0018] An information processing device according to a second aspect of the present technology includes a communication unit that receives media data from a server for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal is attached, and a quality test processing unit that generates evaluation data indicating the evaluation results of the user's quality of experience based on an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment, and the media data received from the server, and when the communication unit receives the media data of the frame to which the meta information is attached, it transmits the controller signal to the server.

[0019] An information processing method according to a second aspect of the present technology receives media data from a server for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal is attached; generates evaluation data indicating the evaluation results of the user's quality of experience based on an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment, and the media data received from the server; and transmits the controller signal to the server when the media data of the frames to which the meta information is attached is received.

[0020] In a second aspect of the present technology, media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal is attached, is received from a server, evaluation data indicating the evaluation result of the user's quality of experience is generated based on an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment, and the media data received from the server, and when the media data of the frame to which the meta information is attached is received, the controller signal is transmitted to the server.

[0021] FIG. 1 is a diagram illustrating an example of the configuration of a cloud gaming system. FIG. 2 is a diagram illustrating evaluation of user experience quality. FIG. 3 is a diagram illustrating cloud gaming play. FIG. 4 is a diagram illustrating an example of an ideal reference. FIG. 5 is a diagram illustrating an example of delay and loss in the uplink. FIG. 6 is a diagram illustrating an example of delay and loss in the downlink. FIG. 7 is a diagram illustrating an example of the configuration of a cloud gaming server. FIG. 8 is a diagram illustrating an example of the configuration of a test game distribution server. FIG. 9 is a diagram illustrating an example of the configuration of a gaming terminal. FIG. 10 is a flowchart illustrating test game distribution processing. FIG. 11 is a flowchart illustrating test game execution processing. FIG. 12 is a flowchart illustrating game distribution processing. FIG. 13 is a flowchart illustrating game execution processing. FIG. 14 is a diagram illustrating an example of the configuration of a computer.

[0022] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0023] First Embodiment Example of Configuration of Cloud Gaming System The present technology relates to a system evaluation technology required to improve the quality of experience of users in a cloud gaming system.

[0024] With this technology, prior to the delivery of a cloud-based game, i.e., prior to gameplay via cloud gaming, an evaluation is conducted on the user's quality of experience, which is determined by a combination of factors such as the content of the game being played and the user's communication environment.

[0025] In particular, the cloud gaming system of the present technology is provided with a test game distribution server that distributes a dedicated test game for evaluating the quality of user experience.

[0026] For example, the quality of the user experience is evaluated by sending and receiving specified data between the test game distribution server and the user's gaming device, i.e., via E2E, and by aggregating logs of information including jitter in the game video.

[0027] Evaluating the reception status of streaming data for each game genre and scene, given the user's communication environment, can provide useful information for individual optimization of each game streaming session. This technology provides a specific evaluation method prior to this individual optimization.

[0028] FIG. 1 is a diagram illustrating an example of the configuration of an embodiment of a cloud gaming system to which the present technology is applied.

[0029] The cloud gaming system shown in FIG. 1 includes a cloud gaming server 11 , a test game distribution server 12 , and a gaming terminal 13 .

[0030] The cloud gaming server 11 is a server that distributes cloud-based games, i.e., distributes cloud games. The cloud gaming server 11 may be configured from one server (information processing device) or multiple servers.

[0031] The test game distribution server 12 is a server that distributes a test game prior to distribution of a game by the cloud gaming server 11. The test game is a game dedicated to evaluating the experience quality of the game provided (distributed) by the cloud gaming server 11.

[0032] The test game distribution server 12 may be configured by one server (information processing device), or may be configured by multiple servers.

[0033] In this example, the cloud gaming server 11 and the test game distribution server 12 are illustrated as different entities. However, this is not limiting, and the cloud gaming server 11 and the test game distribution server 12 may be realized by the same device, such as a single server (information processing device).

[0034] The gaming terminal 13 is, for example, an information processing device owned by the user, such as a smartphone, tablet terminal, personal computer, gaming device, or head-mounted display.

[0035] The gaming terminal 13 is connected to a network such as the Internet via an entry point. The entry point is a point for accessing the Internet, and may be, for example, an access point for a mobile network or a wireless LAN (Local Area Network).

[0036] The gaming terminal 13 connects to the cloud gaming server 11 and the test game distribution server 12 via an entry point or a network, receives the distribution of the game (cloud game) or the test game, and sends a controller signal indicating the user's operation.

[0037] <About this technology> This section explains the evaluation of user experience quality in a cloud gaming system and cloud gaming play.

[0038] In a cloud gaming system, prior to a user's actual cloud gaming play, a test game is run to measure the user's experience quality for each given item, such as the game content (genre) or scene.

[0039] With regard to the test game, video and audio data that would be reproduced if the test game were played in an ideal environment is shared as an ideal reference between the test game distribution server 12 and the gaming terminal 13 by some means.

[0040] At the gaming terminal 13, the user's quality of experience is evaluated according to this shared ideal reference.

[0041] Specifically, a sequence of expected received data (data sequence), which is streaming data of video and audio received in an ideal environment, is compared with a sequence of received data, which is streaming data of a test game in an actual environment, and the user's quality of experience is evaluated. At this time, the evaluation can be performed based on an appropriate error function, etc.

[0042] The evaluation result of the user's quality of experience may be presented to the user on the gaming terminal 13 side as a degree of comfort of play, for example.

[0043] Furthermore, for example, evaluation data indicating the evaluation results of the user's quality of experience can be supplied to and shared with the cloud gaming server 11 via the test game distribution server 12. By using this evaluation data, the cloud gaming server 11 can perform adaptive streaming control when distributing an actual game, thereby improving the quality of experience when the actual user plays the game.

[0044] The test game is executed in the background without actually being displayed on the display screen of the gaming terminal 13. In other words, the user does not need to actually operate the controller for the test game, and does not see the image display of the test game.

[0045] Furthermore, if the test game is played for too long, there is a possibility that the discrepancy between the expected received data in an ideal environment and the received data in a real environment may become too large, so the test game play time is limited to a maximum of about 10 seconds (for example, about 600 frames at 60 fps).

[0046] Games that are actually played by users vary in genre (content), scenes within a single game, etc. Therefore, one or more types of test games are run at times when users are not playing the game, such as when the console (gaming terminal 13) is turned on or in sleep mode, or during dark time or dead time, when processing other than gameplay is being performed.

[0047] The test game is played by exchanging data between the test game distribution server 12 and the game terminal 13, as shown in FIG.

[0048] In FIG. 2, the vertical direction indicates time, with the bottom side indicating the future.

[0049] In the example of FIG. 2, a test game session is first established during the first period W11, and then during the period W12 following that period W11, the test game is played (executed) and the user's quality of experience is evaluated.

[0050] In particular, when playing the test game, controller inputs, that is, interactions of transmitting controller signals, are appropriately performed, triggered by the reception of specific image frames of the test game.

[0051] For example, when a session is established in period W11, a test game list, which is list information of a plurality of different test games, is shared.

[0052] The test game list may be shared, for example, by transmitting the test game list from the test game distribution server 12 to the gaming terminal 13 when a session is established. Alternatively, the test game list may be shared by the gaming terminal 13 acquiring (receiving) the test game list from the test game distribution server 12 or another device in advance of the period W11.

[0053] The test game distribution server 12 prepares test games for multiple game-related items, such as for each genre (content) of the game provided by the cloud gaming server 11 or for each scene in the game. The test game list is a list of multiple different test games that are prepared in advance, and includes, for example, information such as a test game ID that can uniquely identify each test game.

[0054] During period W11, for example, the gaming terminal 13 selects a game, more specifically, a test game corresponding to the game genre, scene, etc., from the test game list in accordance with the genre, scene, etc. of the game that is about to be played. At least one test game is selected here.

[0055] As indicated by an arrow A11, the gaming terminal 13 transmits a test game start request indicating the selected test game to the test game distribution server 12, requesting the start of the test game. For example, the test game start request includes the selected test game ID.

[0056] During a period W11, after the test game start request is transmitted, the ideal reference and response pattern are shared as indicated by an arrow A12. That is, the ideal reference and response pattern are transmitted from the test game distribution server 12 to the gaming terminal 13 at an appropriate timing.

[0057] The ideal reference is data that includes streaming data of the test game that is expected to be received by the gaming terminal 13 when the test game is played in an ideal environment, more specifically, a data string of media data for reproducing the video and audio of the test game.

[0058] In particular, the media data of the test game here consists of video data for playing the video (video frames) of the test game and audio data for playing the audio (audio frames) of the test game. Note that the media data of the test data may consist of at least either video data or audio data.

[0059] The ideal reference may include at least one of video data and audio data as media data. In the following description, it is assumed that the ideal reference includes at least video data as media data.

[0060] The response pattern is data indicating the response pattern during test game play, such as the method by which the gaming terminal 13 generates and transmits controller signals, that is, the content (type) of the generated controller signals and the transmission timing.

[0061] In other words, the response pattern is data indicating what operation content of the controller signal is to be transmitted and at what timing when a video frame that triggers a response is received.

[0062] When the gaming terminal 13 receives and records the ideal reference and response pattern for the test game that is about to be played (executed), it appropriately sends an acknowledgement (ACK) to the test game distribution server 12 indicating that sharing has been completed.

[0063] In this way, a session is established between the gaming terminal 13 and the test game distribution server 12 during the period W11, and then distribution of the test game begins, and the test game is played (executed) during the period W12.

[0064] During period W12, the test game is distributed at regular time intervals from the test game distribution server 12 to the gaming terminal 13. That is, frames (media data) of the test game are transmitted to the gaming terminal 13 as streaming data.

[0065] In this example, the streaming data of the test game includes at least the video data (video frames) of the test game.

[0066] Furthermore, some of the video data (video frames) of the test game are provided with meta-information for inducing (inducing) the generation and transmission of controller signals based on controller input, i.e., response patterns. Note that although the meta-information may be provided to audio frames instead of video frames, the following description will be given assuming that the meta-information is provided to video frames.

[0067] The meta information is information that indicates a response point (video frame) that triggers the generation and transmission of a controller signal.

[0068] For example, the meta information may be information indicating which of the predetermined response points in the test game is the timing at which a controller input should be made, i.e., the timing at which a controller signal should be generated. Specifically, the meta information may be, for example, ID information that identifies the response point.

[0069] By referring to the response pattern, the gaming terminal 13 can determine what operation content of the controller signal to generate and transmit at what timing, such as after how many seconds, for the response point indicated by the meta information.

[0070] Specifically, for example, the response pattern is data in which the content of the controller signal (operation content) and the transmission timing, i.e., generation timing, of the controller signal are associated with information that identifies the response point (ID information, etc.).

[0071] The gaming terminal 13 transmits a controller signal when meta information is attached to the video data as streaming data of the test game received from the test game distribution server 12, i.e., when meta information is included in the streaming data (video data).

[0072] That is, the gaming terminal 13 generates a controller signal in accordance with the response pattern and transmits it to the test game distribution server 12. The generation of the controller signal is performed automatically without requiring any input operation by the user.

[0073] When the streaming data of the test game is received, the media (game content), i.e., the video and audio of the test game, can be processed based on the streaming data, such as decoding, etc., as required to play the media, but the video and audio are not displayed or output.

[0074] When the test game distribution server 12 receives a controller signal from the game terminal 13 , it generates streaming data of video frames and audio frames corresponding to the controller signal and transmits it to the game terminal 13 .

[0075] The gaming terminal 13 does not generate or transmit a controller signal if meta-information is not attached to the video data as streaming data of the test game received from the test game distribution server 12. Note that even in this case, processing such as decoding for media playback can be performed.

[0076] During the period W12, streaming data of the test game and controller signals are exchanged (transmitted and received) between the gaming terminal 13 and the test game distribution server 12.

[0077] In the following, video frames with meta information attached, i.e., frames of the test game that serve as response points, will be referred to as trigger frames, and frames that are not trigger frames, i.e., frames that do not have meta information attached, will be referred to as non-trigger frames.

[0078] During period W12, when the play (distribution) of the test game is completed, the gaming terminal 13 generates (calculates) evaluation data indicating the evaluation result of the user's quality of experience based on the ideal reference (expected received data) shared in advance and the streaming data (received data) received during the test game play. For example, the evaluation data is generated by calculating a predetermined appropriate error function.

[0079] As indicated by arrow A13, the gaming terminal 13 transmits the generated evaluation data to the test game distribution server 12. Furthermore, as indicated by arrow A14, the test game distribution server 12 records the evaluation data received from the gaming terminal 13 and transmits the evaluation data to the cloud gaming server 11 as feedback.

[0080] The cloud gaming server 11 uses the evaluation data received from the test game distribution server 12 for actual cloud gaming, i.e., for streaming control of game distribution.

[0081] The evaluation data used for streaming control in the cloud gaming server 11 is an E2E evaluation that takes into account controller input. In other words, the evaluation data represents the results of an integrated evaluation of factors (metrics) that affect the user's quality of experience while taking into account the interaction of controller input. Therefore, by using such evaluation data, streaming can be controlled more appropriately, thereby improving the user's quality of experience.

[0082] Note that the test games may be conducted for each genre or scene of a game that the user intends to play or is expected to play in the future. In other words, multiple different test games may be conducted in sequence during period W12.

[0083] In this case, the user's quality of experience is evaluated for each genre and scene, and evaluation data is obtained for each genre and scene. This allows for appropriate streaming control for each game genre and scene, further improving the user's quality of experience.

[0084] Cloud gaming play will be described with reference to Fig. 3. In Fig. 3, the vertical direction represents time, with the bottom side representing the future.

[0085] In this example, as indicated by arrow A21, when a game start request is sent from the gaming terminal 13 to the cloud gaming server 11, the game is distributed from the cloud gaming server 11 to the gaming terminal 13 during the subsequent period W21.

[0086] At this time, the cloud gaming server 11 controls streaming based on the evaluation data previously received from the test game distribution server 12. Also, during period W21, when the user performs a controller input (operation), a controller signal corresponding to the user's controller input is transmitted from the gaming terminal 13 to the cloud gaming server 11.

[0087] For example, during game play, at an appropriate timing, such as during a gap in the game, the cloud gaming server 11 may request the test game distribution server 12 to execute a test game, as shown by arrow A22.

[0088] In this case, as indicated by arrow A23, the test game distribution server 12 distributes the test game in response to a request from the cloud gaming server 11. That is, as described with reference to FIG. 2 , streaming data and controller signals of the test game are transmitted and received between the test game distribution server 12 and the gaming terminal 13.

[0089] When the test game play is completed, the test game distribution server 12 transmits evaluation data about the executed test game to the cloud gaming server 11, and the cloud gaming server 11 uses the received evaluation data for subsequent game distribution.

[0090] For example, suppose that the cloud gaming server 11 requests the test game distribution server 12 to run a test game immediately before a predetermined scene of the game that the cloud gaming server 11 distributes to the gaming terminal 13. In such a case, the cloud gaming server 11 can use the evaluation data received in response to the request to run the test game to control streaming when distributing the predetermined scene of the game. In this way, by using the evaluation data indicating the results of the user's evaluation of the quality of experience performed immediately before distribution, more appropriate streaming control becomes possible.

[0091] Furthermore, the game is distributed from the cloud gaming server 11 to the gaming terminal 13, and game play ends when the streaming data of the final frame of the game is transmitted as indicated by arrow 24. The final frame is generated when the game is played to the end or when the user instructs the game to end midway, i.e., when a trigger is generated by the user. In the following, even if not explicitly stated, the final frame of the game is assumed to be generated when the game is played to the end or when a trigger is generated by the user.

[0092] The calculation of the ideal reference, response pattern, and evaluation data mentioned above will now be described.

[0093] (Ideal Reference) When conducting a test game, it is possible to calculate in advance how the video and audio of the test game will progress when ideal communication is carried out with no delays (transmission delays) or losses (frame losses) when transmitting streaming data such as video frames.

[0094] Therefore, in the cloud gaming system of the present technology, the data sequence of the media data of the test game received by the gaming terminal 13 in such an ideal communication environment is used as an ideal reference, and this ideal reference is used to calculate the evaluation data.

[0095] The ideal reference may consist of only a sequence of video frames (images), or may consist of only a sequence of audio frames, or may consist of a sequence of video and audio frames.

[0096] For example, when calculating evaluation data, it is assumed that an ideal sequence of video and audio frames identical to the ideal reference is received, and the difference (error) from this ideal sequence is evaluated using an error function, etc.

[0097] Here, an example of the ideal reference and the occurrence of an error from the ideal reference will be described with reference to Figures 4 to 6. Note that in Figures 4 to 6, the vertical direction represents time, with the bottom side representing the future. In Figures 4 to 6, corresponding parts are assigned the same reference numerals, and their description will be omitted where appropriate.

[0098] In an ideal communication environment, for example, as shown in FIG. 4, streaming data of the test game, that is, frames of media (video and audio) of the test game, and controller signals are exchanged.

[0099] In this example, dashed (dotted) arrows, such as arrow A31 and arrow A34, pointing from the test game distribution server 12 to the gaming terminal 13 (from left to right in the drawing) represent the transmission of non-trigger frames. In other words, this indicates that the streaming data of the test game distributed (transmitted) to the gaming terminal 13 is media data for playing non-trigger frames.

[0100] In contrast, a solid arrow, such as arrow A32, pointing from the test game distribution server 12 to the gaming terminal 13 represents the transmission of a trigger frame. In other words, this indicates that the streaming data of the test game distributed (transmitted) to the gaming terminal 13 is media data for playing back the trigger frame.

[0101] The media data for playing back the trigger frame includes meta information for inducing the generation and transmission of the controller signal described above, i.e., meta information is attached to the trigger frame.

[0102] Therefore, when the trigger frame is received on the gaming terminal 13 side, a controller signal corresponding to the meta information attached to the trigger frame is generated and transmitted to the test game distribution server 12 .

[0103] In this example, for example, a controller signal indicated by an arrow A33 is generated and transmitted in response to a trigger frame indicated by an arrow A32. Furthermore, based on the controller signal indicated by an arrow A33, a non-trigger frame indicated by an arrow A34 is generated as the next frame and transmitted (transmitted) to the gaming terminal 13.

[0104] In the test game distribution server 12, a sequence of trigger frames and non-trigger frames transmitted at regular time intervals as shown in FIG. 4, that is, time-series media data, is prepared as an ideal reference.

[0105] It is to be noted that the way of responding to the trigger frame at each timing, such as at a predetermined timing T11, i.e., the timing of the response (generation of a controller signal) and the number of controller signals generated in response, may differ (change) for each user.

[0106] For this reason, although an ideal reference common to all players (users) may be used regardless of the player's level, etc., multiple ideal references that differ in how they respond to trigger frames may be prepared for the same test game, for example. In this case, it may be possible to prepare ideal references for each player level (level of skill), such as beginner or advanced, for one test game.

[0107] As with the ideal reference, a common response pattern may be used for all players (users), or different response patterns may be prepared for each player's level (level of skill). Furthermore, ideal references and response patterns may be prepared for each user or game-related item, such as for each type of character operated by a player (user) in the game.

[0108] Furthermore, when playing a test game corresponding to the ideal reference shown in Figure 4, if a transmission delay or transmission loss occurs in the uplink (uplink), i.e., in the transmission from the gaming terminal 13 to the test game distribution server 12, the sequence of media data on the gaming terminal 13 side will be, for example, as shown in Figure 5.

[0109] In the example shown in FIG. 5, for example, a controller signal indicated by an arrow A33 is generated and sent (transmitted) in response to a trigger frame indicated by an arrow A32, but a delay occurs in the transmission of this controller signal.

[0110] As a result, at timing T21, the timing of receiving the controller signal, indicated by arrow A33, and the timing of sending the non-trigger frame, indicated by arrow A41, which follows the trigger frame, indicated by arrow A32, are swapped. This swap of orders (before and after) would not have occurred if there had been no transmission delay.

[0111] The non-trigger frame indicated by arrow A41 is normally generated based on the controller signal indicated by arrow A33. However, in this example, a transmission delay causes a delay in the timing of reception of the controller signal indicated by arrow A33, so the non-trigger frame indicated by arrow A41 is generated and transmitted as if the controller signal had not been received.

[0112] As a result, a difference occurs between the non-trigger frame indicated by arrow A41 and the corresponding frame in the ideal reference, and this difference is accumulated as an error when the evaluation data is calculated.

[0113] Furthermore, in this example, the controller signal indicated by arrow A33, which should have been reflected in the generation of the non-trigger frame indicated by arrow A41, is instead reflected in the generation of the next frame, resulting in a further difference from the ideal reference.

[0114] In the example shown in FIG. 5, a controller signal indicated by an arrow A43 is generated and sent (transmitted) in response to a trigger frame indicated by an arrow A42, but this controller signal is lost.

[0115] Therefore, the non-trigger frame indicated by arrow A44, which would have been generated based on the controller signal indicated by arrow A43, is generated as if the controller signal had not been received. As a result, the non-trigger frame indicated by arrow A44 observed (received) on the gaming terminal 13 side is a video frame that is different from the video frame observed (received) when the controller signal indicated by arrow A43 has been correctly transmitted.

[0116] As a result, even though no transmission delay occurs, a difference occurs between the received frame and the corresponding frame in the ideal reference, and this difference is accumulated as an error when calculating the evaluation data.

[0117] Furthermore, when playing a test game corresponding to the ideal reference shown in Figure 4, transmission delays and transmission losses (frame losses) may occur on the downlink (downlink) rather than the uplink (uplink), as shown in Figure 6, for example.

[0118] For example, in the example shown in FIG. 6, a controller signal indicated by an arrow A52 is generated and sent (transmitted) in response to a trigger frame indicated by an arrow A51.

[0119] However, a delay occurs in the transmission of the trigger frame indicated by the arrow A51, and as a result, the timing at which the test game distribution server 12 receives the controller signal indicated by the arrow A52 is delayed.

[0120] As a result, at timing T31, the timing of receiving the controller signal, indicated by arrow A52, and the timing of sending the non-trigger frame, indicated by arrow A53, which follows the trigger frame, indicated by arrow A51, are swapped in order, resulting in the same swapping (before and after) as occurred at timing T21 in Figure 5.

[0121] Therefore, the non-trigger frame indicated by arrow A53 differs from the corresponding frame in the ideal reference. In addition, a delay in the reception timing of the non-trigger frame indicated by arrow A53 causes a shift in the presentation timing of the video frames, etc., i.e., jitter in the video frames, etc.

[0122] As a result, a difference occurs between the actually received frame sequence and the ideal reference, and this difference is accumulated as an error when calculating the evaluation data. In other words, a difference (error) from the ideal reference occurs due to transmission delays and jitter in the downlink.

[0123] 6, a non-trigger frame is lost as indicated by arrow A54. Therefore, at the subsequent timing T32, the non-trigger frame indicated by arrow A54 is not received, and the next trigger frame indicated by arrow A55 is received. In other words, a frame loss occurs.

[0124] Such frame loss also causes a difference between the actually received frame sequence and the ideal reference, and this difference is accumulated as an error when calculating the evaluation data.

[0125] (Response Pattern) The response pattern can be considered to be advance information for inducing the generation and transmission of a controller signal that causes a specified controller input to be performed at the timing when a specified frame, i.e., a video frame with meta information attached, is received on the gaming terminal 13. In other words, the response pattern is information that indicates the pattern of the response when a frame with meta information attached (trigger frame) is received, and the response pattern is used to generate a controller signal on the gaming terminal 13 side.

[0126] For example, several response patterns are determined in advance for each test game.

[0127] As with the ideal reference, a response pattern may be prepared for each player level, and it is predetermined which response pattern is used for which ideal reference.

[0128] As an example, in a scene where no input from the user (player) is expected, information indicating a response of not inputting anything is used as the response pattern.

[0129] Furthermore, for example, if the test game is a racing game, the response pattern is information indicating continuous inputs that allow the player to properly complete turns in a curved area.

[0130] (Calculation of Evaluation Data) The ideal reference is an ideal sequence of video and audio frames.

[0131] For ease of explanation, the calculation of evaluation data will be described using an example in which the ideal reference is a sequence of video frames arranged in time series.

[0132] For example, let the ideal reference for one test game be (X t ) T t=1 Here, t is a parameter indicating the number of frames, and T is the total number of frames in the video that makes up the test game. t indicates the image (image frame) at frame t.

[0133] In response to this, the sequence of video frames received by the gaming terminal 13 during test game play in an actual environment through communication with the test game distribution server 12 is expressed as (Y t ) T t=1 Here, Y t is the image (image frame) at frame t.

[0134] In general streaming, a frame number is assigned to each video frame, so a sequence of video frames like this (Y t ) T t=1 can be obtained.

[0135] In the gaming terminal 13, for example, these ideal references (X t ) T t=1 and the sequence of received video frames (Y t ) T t=1 Based on this, the error, that is, the error function loss, is calculated by calculating the following equation (1).

[0136]

[0137] In addition, in formula (1), w t indicates the weight at frame t, that is, the weight determined for each frame. This weight w t may be different for each test game.

[0138] Also, in equation (1), f(Xt ,Y t ) is the video frame X t and video frame Y t is an appropriate function that takes as input the error between frames. For example, the function f(X t ,Y t ) is a function that calculates the squared error or absolute error per bit, a function that calculates the PSNR (Peak Signal to Noise Ratio), or a function that combines these functions.

[0139] It should be noted that there may be cases where the game terminal 13 is unable to receive a video frame due to frame loss.

[0140] However, in such a case, in equation (1), w t f(X t ,Y t ) is replaced with a special value that is predetermined for frame loss, or the same as the previous frame (t-1) is reproduced. t f(X t ,Y t-1 ) is replaced by

[0141] Evaluation data is calculated based on the error function loss obtained as described above at the gaming terminal 13. Note that the value of the error function loss may be used as the evaluation data as it is.

[0142] In this way, by calculating the evaluation data using equation (1) or the like, it becomes possible to numerically evaluate the user's quality of experience based on the received video frames.

[0143] In addition, when controlling streaming during game distribution by the cloud gaming server 11, evaluation data obtained by running a test game is basically used.

[0144] However, without being limited to this, quality evaluation metrics (indicators) such as terminal information, jitter related to media playback, and the degree of quality degradation of received media data may be sent (provided) to the cloud gaming server 11 along with the evaluation data, and these quality evaluation metrics may also be used to calculate the evaluation data.

[0145] The terminal information is information relating to the gaming terminal 13. Specifically, for example, the terminal information may be location information indicating the location of the gaming terminal 13, speed information indicating the movement speed of the gaming terminal 13, information indicating the communication state (communication state) such as congestion of 5G or Wi-Fi (registered trademark), etc. Information indicating the communication state is, for example, information indicating the radio wave strength at the gaming terminal 13, delays in the network used for communication, etc.

[0146] Furthermore, jitter related to media playback is a deviation (variation) in the reception timing of video frames and audio frames, and can be calculated from the video frames and audio frames of the test game received by the gaming terminal 13, i.e., the frame reception timing, etc. Furthermore, quality degradation of media data refers to, for example, degradation in the image quality of received video frames. With this technology, an ideal reference is prepared in advance, so it is possible to evaluate image quality degradation of video frames, such as PSNR evaluation of image quality, based on the ideal reference and the received video frames.

[0147] In addition, methods for sharing ideal references and response patterns in advance from the test game distribution server 12 to the gaming terminal 13 include distributing them on demand (default) and sharing all ideal references and response patterns at once.

[0148] Alternatively, for example, the gaming terminal 13 may be provided with a cache, and some ideal references and response patterns, such as ideal references and response patterns of frequently played test games, may be stored in the cache according to the test game play history, etc. In other words, the gaming terminal 13 may store ideal references and response patterns that are frequently used.

[0149] <Configuration Example of Cloud Gaming Server> A configuration example of the cloud gaming server 11, the test game distribution server 12, and the gaming terminal 13 that make up the cloud gaming system shown in FIG. 1 will be described.

[0150] FIG. 7 is a diagram illustrating an example of the functional configuration of the cloud gaming server 11.

[0151] The cloud gaming server 11 includes a media generation unit 41 , a streaming processing unit 42 , a user information management unit 43 , a game processing unit 44 , and a communication unit 45 .

[0152] When a controller signal is received from the gaming terminal 13, the media generation unit 41 generates video data (video frames) and audio data (audio frames) as game media data in response to the controller signal.

[0153] The streaming processing unit 42 performs necessary processing such as encoding on the generated game media data to convert the media data into streaming data in a format that can be transmitted to the game terminal 13 .

[0154] In the streaming processing unit 42, when forming the streaming data, that is, when generating the streaming data, adaptive streaming control is performed based on the evaluation data of the user's quality of experience.

[0155] In other words, the streaming processing unit 42 controls streaming when streaming game data (media data) is delivered to the gaming terminal 13 based on the evaluation data.

[0156] Specifically, for example, streaming control involves dynamically (adaptively) changing (controlling) communication redundancy such as FEC (Forward Error Correction), and setting (controlling) the transmission rate (bit rate) of video frames, i.e., streaming image quality (image quality of video frames). In particular, communication redundancy and the bit rate of video frames are intelligently controlled (selected) based on evaluation data, which is the evaluation result for each environment in which users play the game, obtained by a test game.

[0157] The user information management unit 43 has, for example, a non-volatile memory, and manages evaluation data of the user's quality of experience obtained by the test game.

[0158] For example, a user ID that uniquely identifies a user and evaluation data obtained for the user identified by the user ID are linked and managed in the user information management unit 43. Note that the function (mechanism) for managing user evaluation data realized by the user information management unit 43 may be provided in the test game distribution server 12.

[0159] The game processing unit 44 performs processing such as decoding on the controller signal received from the game terminal 13 as appropriate, and supplies the controller signal to the media generation unit 41 .

[0160] The communication unit 45 communicates with the gaming terminal 13 and the test game distribution server 12. For example, the communication unit 45 receives controller signals transmitted by the gaming terminal 13 and transmits game streaming data to the gaming terminal 13.

[0161] <Configuration Example of Test Game Distribution Server> FIG. 8 is a diagram showing an example of the functional configuration of the test game distribution server 12. As shown in FIG.

[0162] The test game distribution server 12 includes a media generation unit 71 , a streaming processing unit 72 , a test game management unit 73 , a game processing unit 74 , a communication unit 75 , and a user information management unit 76 .

[0163] When the media generation unit 71 receives a controller signal from the gaming terminal 13, it generates video data (video frames) and audio data (audio frames) as media data for the test game in response to the controller signal.

[0164] The streaming processing unit 72 performs necessary processing such as encoding on the generated media data of the test game, thereby converting the media data into streaming data in a format that can be transmitted to the gaming terminal 13 .

[0165] The test game management unit 73 has, for example, a non-volatile memory, and manages a plurality of test patterns, that is, test games generated for each item related to the game, such as genre and scene.

[0166] For example, the test game management unit 73 loads data related to an appropriate test game in response to a request from the gaming terminal 13, supplies the data to the media generation unit 71 and the game processing unit 74 as appropriate, and instructs them to generate media data for the test game, etc.

[0167] The test game management unit 73 also manages (holds) and shares ideal references and response patterns according to the test game.

[0168] The game processing unit 74 performs processing such as decoding on the controller signal received from the game terminal 13 as appropriate, and supplies the controller signal to the media generation unit 71 .

[0169] The communication unit 75 communicates with the gaming terminal 13 and the cloud gaming server 11. For example, the communication unit 75 transmits an ideal reference or a response pattern to the gaming terminal 13, receives evaluation data from the gaming terminal 13, and transmits the evaluation data to the cloud gaming server 11.

[0170] The user information management unit 76 has, for example, non-volatile memory, and stores evaluation data received from the gaming terminal 13 for each test game, and transmits the stored evaluation data via the communication unit 75 and shares it with the cloud gaming server 11.

[0171] The cloud gaming server 11 may also function as the test game distribution server 12 .

[0172] In such a case, for example, the media generation unit 41, streaming processing unit 42, game processing unit 44, and communication unit 45 of the cloud gaming server 11 also function as the media generation unit 71, streaming processing unit 72, game processing unit 74, and communication unit 75 of the test game distribution server 12.

[0173] <Configuration Example of Gaming Terminal> FIG. 9 is a diagram showing an example of the functional configuration of the gaming terminal 13. As shown in FIG.

[0174] The gaming terminal 13 includes a communication unit 91 , a media playback unit 92 , a streaming processing unit 93 , a quality test processing unit 94 , and a game input processing unit 95 .

[0175] The communication unit 91 communicates with the test game distribution server 12 and the cloud gaming server 11. The media playback unit 92 is composed of, for example, a display and a speaker, and plays game media, i.e., video (video frames) and audio (audio frames), based on the game media data.

[0176] The streaming processing unit 93 performs necessary processing such as decoding on the received streaming data of the test game or game, thereby converting the streaming data into media data in a format that allows media playback.

[0177] The quality test processing unit 94 realizes functions related to the test game.

[0178] For example, the quality test processing unit 94 realizes a function of requesting the test game distribution server 12 to distribute (transmit) a test game, i.e., a function of transmitting a test game start request. In addition, for example, the quality test processing unit 94 also realizes a function of receiving (acquiring) and storing an ideal reference and a response pattern as advance information required for playing the test game.

[0179] For example, the quality test processing unit 94 compares the media data of the test game that has been made playable by the media playback unit 92 with an ideal reference, and also realizes the function of calculating evaluation data by calculating error functions, etc., and the function of converting the evaluation data into a format that can be transmitted.

[0180] The game input processing unit 95 generates a controller signal in response to an input operation (controller input) from the user or an operation specified by the quality test processing unit 94, and converts the controller signal into a data format that can be transmitted to an external device by processing such as encoding.

[0181] <Description of Test Game Distribution Process> The process performed in the cloud gaming system shown in FIG. 1 will be described.

[0182] First, the test game distribution process performed by the test game distribution server 12 will be described with reference to the flowchart in Fig. 10. This test game distribution process is performed, for example, when the test game distribution server 12 is started up.

[0183] After the test game distribution process has started, when a request to send a test game list sent from the gaming terminal 13 is received by the communication unit 75 at any timing, the test game management unit 73 reads the test game list from a memory not shown and supplies it to the communication unit 75.

[0184] In step S11, the communication unit 75 transmits the test game list provided by the test game management unit 73 to the gaming terminal 13. Note that the test game list may be shared (transmitted) at any timing.

[0185] In step S12, the test game management unit 73 controls the communication unit 75 to establish a session with the game terminal 13 and enters a standby state for a test game start request.

[0186] A test game is selected from the test game list at any timing in the gaming terminal 13. Then, a test game start request indicating the selected test game is transmitted from the gaming terminal 13 to the test game distribution server 12. The communication unit 75 receives the transmitted test game start request and supplies it to the test game management unit 73. For example, the test game start request includes the user ID and the test game ID of the test game to be executed.

[0187] The test game management unit 73 reads out the ideal reference and response pattern of the test game specified by the test game start request from a memory (not shown) and supplies them to the communication unit 75 .

[0188] In this case, if ideal references and response patterns are prepared for the test game for each player level, the test game management unit 73 reads out the ideal references and response patterns prepared for the level of the user of the gaming terminal 13 and supplies them to the communication unit 75.

[0189] For example, information indicating the level of the user (player) may be included in the test game start request. Alternatively, for example, the user information management unit 76 may associate and manage information indicating the user's level with a user ID, and the test game start request may include the user ID. In such a case, the test game management unit 73 obtains, from the user information management unit 76, information indicating the user's level associated with the user ID, based on the user ID included in the test game start request.

[0190] Alternatively, the user's level as a player may be determined by the user information management unit 76 or the gaming terminal 13 based on the user's game playing record (history) or the like.

[0191] In step S13 , the communication unit 75 transmits the ideal reference and response pattern provided by the test game management unit 73 to the gaming terminal 13 .

[0192] This results in the sharing of the ideal reference and response pattern. The ideal reference and response pattern may be transmitted for each test game immediately before the execution of that test game, or the ideal references and response patterns for multiple test games may be transmitted together at the same time. The ideal reference and response pattern may also be shared in advance before the session is established.

[0193] The test game distribution server 12 can more appropriately evaluate the quality of the user's experience by preparing test games, ideal references, and response patterns for each subdivided item, such as preparing test games for each game genre or scene, or preparing ideal references and response patterns for each player level.

[0194] After the ideal reference and response pattern are shared and the communication unit 75 receives an acknowledgment (ACK) or the like from the gaming terminal 13 indicating that the sharing has been completed, the process proceeds to step S14.

[0195] In step S14, the test game management unit 73 starts distribution of the test game.

[0196] Specifically, the test game management unit 73 loads data related to the test game from a memory not shown, supplies the data to the media generation unit 71 and the game processing unit 74, and instructs the media generation unit 71 and the game processing unit 74 to generate media data, etc.

[0197] In response to instructions from the test game management unit 73, the media generation unit 71 generates video data and audio data for each frame of the test game as media data for the test game based on the data supplied from the test game management unit 73.

[0198] When generating the media data, the media generating unit 71 adds meta information to the video frame of the trigger frame in response to an instruction from the test game managing unit 73 .

[0199] The streaming processing unit 72 performs necessary processing such as encoding on the media data of the test game generated by the media generation unit 71, and supplies the resulting streaming data to the communication unit 75. The communication unit 75 transmits the streaming data of the test game supplied from the streaming processing unit 72 to the gaming terminal 13.

[0200] As a result, prior to playing the game on the gaming terminal 13, streaming distribution of a test game corresponding to the game, more specifically, corresponding to the genre and scene of the game, is initiated.

[0201] When streaming distribution of the test game, i.e., transmission (distribution) of streaming data (media data) of the test game, begins, the media generation unit 71 generates media data for frames of the test game at regular time intervals. The media data for each of the multiple frames used to play back the media (video and audio) of the test game includes media data for trigger frames to which meta information is attached and media data for non-trigger frames to which meta information is not attached.

[0202] Furthermore, once the distribution (play) of the test game has started, controller signals are subsequently transmitted from the gaming terminals 13 as appropriate in response to the trigger frames.

[0203] In step S15, the communication unit 75 determines whether or not a controller signal has been transmitted.

[0204] If it is determined in step S15 that a controller signal has not been transmitted, the media generation unit 71 generates streaming data of the frame to be transmitted of the test game in step S16, assuming that there is no controller signal.

[0205] More specifically, the media generation unit 71 generates media data assuming that there is no controller signal, and the streaming processing unit 72 shapes the media data generated by the media generation unit 71 and supplies the resulting streaming data to the communication unit 75.

[0206] On the other hand, if it is determined in step S15 that a controller signal has been transmitted, the communication unit 75 receives the controller signal transmitted from the gaming terminal 13 and supplies it to the game processing unit 74 in step S17.

[0207] The game processing unit 74 performs processing such as decoding on the controller signal supplied from the communication unit 75 as appropriate, and supplies the controller signal to the media generation unit 71 .

[0208] In step S18 , the media generation unit 71 generates streaming data in accordance with the controller signal supplied from the game processing unit 74 .

[0209] More specifically, the media generation unit 71 dynamically generates media data that reflects the input operation indicated by the controller signal, based on the controller signal. The streaming processing unit 72 shapes the media data generated by the media generation unit 71 and supplies the resulting streaming data to the communication unit 75.

[0210] In this way, when a controller signal transmitted by the gaming terminal 13 is received during streaming distribution of the test game, media data is generated that reflects the operation content indicated by the controller signal.

[0211] In steps S18 and S16, meta-information is added to the video data (video frames) generated as media data for trigger frames, but meta-information is not added to non-trigger frames. Alternatively, meta-information indicating that the non-trigger frames are not response points may be added to the video data.

[0212] After the process of step S16 or step S18 is performed, the process of step S19 is performed.

[0213] In step S19 , the communication unit 75 transmits the streaming data of the test game supplied from the streaming processing unit 72 to the gaming terminal 13 .

[0214] In step S20, the test game management unit 73 determines whether or not to end the test game.

[0215] For example, the test game management unit 73 determines that the test game is to end when the media generation unit 71 generates media data for the last frame of the test game, i.e., the end frame, and transmits streaming data corresponding to that media data.

[0216] If it is determined in step S20 that the test game is not yet finished, the process then returns to step S15, and the above-described process is repeated.

[0217] On the other hand, if it is determined in step S20 that the test game is to be ended, the process then proceeds to step S21.

[0218] When the test game is finished, the gaming terminal 13 transmits evaluation data on the user's quality of experience obtained by playing the test game to the test game distribution server 12 .

[0219] In step S21, the communication unit 75 receives the evaluation data transmitted from the gaming terminal 13 and supplies it to the user information management unit 76.

[0220] In step S22, the user information management unit 76 supplies the evaluation data supplied from the communication unit 75 to a memory (not shown), and records the evaluation data in association with the user ID and the test game ID.

[0221] In addition, the user information management unit 76 supplies the evaluation data and the user ID obtained by executing (playing) the test game to the communication unit 75 and instructs it to transmit them to the cloud gaming server 11 .

[0222] In step S23, the communication unit 75 transmits the evaluation data and the user ID provided by the user information management unit 76 to the cloud gaming server 11, and the test game distribution process ends.

[0223] In the cloud gaming server 11, the communication unit 45 receives the evaluation data and user ID transmitted from the test game distribution server 12. The user information management unit 43 associates the evaluation data and user ID received by the communication unit 45 and records them in a memory (not shown). In this case, the evaluation data is recorded so that it is possible to identify which game genre or scene the evaluation data corresponds to. That is, for example, information indicating the game genre or scene is recorded in association with the evaluation data and the user ID.

[0224] In addition, together with the evaluation data, information regarding quality evaluation (quality evaluation metrics) such as the user's terminal information, jitter related to media playback, and the degree of quality degradation of the media data may be sent from the gaming terminal 13 to the test game distribution server 12.

[0225] In such a case, in step S21, information related to the quality evaluation, such as terminal information, is received along with the evaluation data, and in step S22, not only the evaluation data but also the received information related to the quality evaluation is recorded. Furthermore, in step S23, in addition to the evaluation data and the user ID, information related to the quality evaluation, such as terminal information, is also transmitted to the cloud gaming server 11.

[0226] Alternatively, after a session with the gaming terminal 13 is established, only one test game may be executed, or multiple test games may be executed.

[0227] For example, when multiple different test games, such as test games corresponding to multiple scenes or multiple genres, are played together in one session, the processing of steps S14 to S22 described above is performed for each test game.

[0228] In this way, the test game distribution server 12 runs a test game that takes controller input into consideration, receives the evaluation data obtained as a result, records the evaluation data, and transmits it to the cloud gaming server 11.

[0229] By conducting such test games, it is possible to obtain evaluation data that is E2E and takes controller input into account, thereby improving the quality of experience when actual users play the game.

[0230] <Explanation of Test Game Execution Process> When the process of step S11 is performed in the test game distribution process described with reference to FIG. 10, the test game execution process shown in FIG. 11 is started in the gaming terminal 13.

[0231] The test game execution process performed by the gaming terminal 13 will be described below with reference to the flowchart of FIG.

[0232] 10 from the test game distribution server 12, and supplies it to the quality test processing unit 94. The quality test processing unit 94 supplies the test game list supplied from the communication unit 91 to a memory (not shown) for recording.

[0233] In addition, the quality test processing unit 94 selects a test game to be requested to be executed from the test game list, generates a test game start request including the test game ID of the selected test game and a user ID indicating the user, and supplies the request to the communication unit 91.

[0234] Here, for example, a test game defined for the genre or scene of the game that the user is about to play is selected from among the games distributed by the cloud gaming server 11. Furthermore, in cases where the user's level as a player can be identified, for example, by the user's input operation or a setting made in advance by the user, information indicating the user's level may also be stored in the test game start request.

[0235] In step S52 , the communication unit 91 transmits the test game start request supplied from the quality test processing unit 94 to the test game distribution server 12 .

[0236] When the test game start request is transmitted, an ideal reference and a response pattern are transmitted in response to the test game start request from the test game distribution server 12. That is, the test game distribution server 12 performs the process of step S13 in FIG.

[0237] In step S53 , the communication unit 91 receives the ideal reference and response pattern transmitted from the test game distribution server 12 , and supplies them to the quality test processing unit 94 .

[0238] The quality test processing unit 94 supplies the ideal reference and response pattern supplied from the communication unit 91 to a memory (not shown) for recording. The quality test processing unit 94 also generates an acknowledgement (ACK) indicating that the sharing of the ideal reference and response pattern has been completed, and transmits the acknowledgement to the test game distribution server 12 via the communication unit 91.

[0239] Then, the test game distribution server 12 performs the process of step S14 in FIG. 10, and transmits the streaming data of the test game.

[0240] In step S54 , the communication unit 91 receives the streaming data transmitted from the test game distribution server 12 and supplies it to the streaming processing unit 93 .

[0241] The streaming processing unit 93 performs necessary processing such as decoding on the streaming data of the test game supplied from the communication unit 91 , and supplies the resulting media data to the quality test processing unit 94 .

[0242] The quality test processing unit 94 temporarily records the media data obtained by the streaming processing unit 93 as data constituting a received data string. The media data for one frame recorded in this way, particularly the video data for one frame (video frame), is converted into the video frame Y t Corresponds to.

[0243] In step S55, the quality test processing unit 94 determines whether the media data obtained in step S54 is trigger frame data, i.e., whether media data of a frame (trigger frame) to which meta information is attached has been received.

[0244] For example, if meta-information is added to the video data of the frame to be processed that is supplied as media data from the streaming processing unit 93, the quality test processing unit 94 determines that the frame to be processed is a trigger frame.

[0245] If it is determined in step S55 that the data is not a trigger frame data, that is, if it is determined that the data is a non-trigger frame data, the process of step S56 is not performed, and the process then proceeds to step S57.

[0246] On the other hand, if it is determined in step S55 that the data is a trigger frame, the communication unit 91 transmits a controller signal in step S56.

[0247] Specifically, for example, the quality test processing unit 94 identifies the response point by referring to meta information added to the video data (video frame) as media data.

[0248] The quality test processing unit 94 also refers to the test game ID of the test game being performed and the response pattern defined for the user's level as a player, and identifies the operation content and transmission timing defined for the response points.

[0249] The quality test processing unit 94 then instructs the game input processing unit 95 to generate a controller signal according to the identified operation content and transmission timing.

[0250] The game input processing unit 95 generates a controller signal of the specified operation content in accordance with an instruction from the quality test processing unit 94, and supplies the controller signal to the communication unit 91 at a specified transmission timing. In other words, the game input processing unit 95 generates a controller signal based on the meta information and the response pattern in accordance with an instruction from the quality test processing unit 94. At this time, the controller signal is subjected to processing such as encoding as appropriate.

[0251] The communication unit 91 transmits the controller signal supplied from the game input processing unit 95 to the test game distribution server 12. When the controller signal is transmitted in this manner, if no transmission loss occurs, the test game distribution server 12 determines in step S15 of FIG.

[0252] When the test game is executed (played), the media (video and audio) of the test game is not played by the media playback unit 92, i.e., no video is displayed or audio is output, and controller signals are generated without user input. In other words, the test game is executed in the background.

[0253] Once the process of step S56 has been performed, or if it is determined in step S55 that the data is not a trigger frame data, the process then proceeds to step S57.

[0254] In step S57, the quality test processing unit 94 determines whether or not to end the test game.

[0255] For example, when the quality test processing unit 94 receives media data of the final frame from the streaming processing unit 93 and temporarily records the media data as data constituting the received data string, it determines that the test game should be ended.

[0256] If it is determined in step S57 that the test game is not yet finished, the process returns to step S54, and the above-described process is repeated.

[0257] On the other hand, if it is determined in step S57 that the test game is to be ended, the process then proceeds to step S58.

[0258] In step S58, the quality test processing unit 94 compares the temporarily recorded data sequence of the media data (the received data sequence) with an ideal reference recorded in a memory (not shown) to generate evaluation data of the user's quality of experience. In other words, the quality of experience is evaluated.

[0259] For example, the quality test processing unit 94 reads the test game ID of the test game that was played and an ideal reference determined for the user's level as a player. Then, based on the read ideal reference and the data string of the temporarily recorded media data (the received data string), the quality test processing unit 94 calculates the error function "loss" by, for example, calculating the above-mentioned formula (1), and calculates the evaluation data from the error function "loss."

[0260] Furthermore, the quality test processing unit 94 appropriately formats the obtained evaluation data into a format that can be transmitted to the test game distribution server 12 and supplies the format to the communication unit 91, or formats the evaluation data into a format that can be presented to the user and supplies the format to the display serving as the media playback unit 92. When the display serving as the media playback unit 92 receives the evaluation data from the quality test processing unit 94, it displays the evaluation data and presents it to the user. In this case, for example, the evaluation data may be appropriately converted into a score or the like and displayed. This allows the user to understand the environment related to their game play and take measures such as changing settings to improve the communication environment.

[0261] In addition to calculating the evaluation data, the quality test processing unit 94 may also obtain or calculate information regarding quality evaluation (quality evaluation metrics) such as user terminal information, jitter related to media playback, and the degree of quality degradation of the media data.

[0262] For example, jitter related to media playback can be obtained from the reception status of the streaming data of the test game, and the degree of quality degradation of the media data can be obtained from the received streaming data (media data) of the test game and an ideal reference.

[0263] The quality test processing unit 94 may supply not only the evaluation data but also information relating to the quality evaluation, such as the terminal information thus obtained, to the communication unit 91 .

[0264] Alternatively, for example, in addition to the error function loss, information on quality evaluation such as terminal information may also be used to calculate the evaluation data.

[0265] In step S59, the communication unit 91 transmits the evaluation data supplied from the quality test processing unit 94 to the test game distribution server 12, and the test game execution process ends.

[0266] When the communication unit 91 is also supplied with information relating to quality evaluation such as terminal information from the quality test processing unit 94 , the communication unit 91 also transmits the information relating to quality evaluation to the test game distribution server 12 .

[0267] Furthermore, after a session is established, if multiple test games are played for each genre, scene, etc., the processes of steps S52 to S59 described above are performed for each test game. In this case, for example, the processes of receiving the ideal reference and response pattern in steps S52 and S53 and the process of transmitting the evaluation data in step S59 may be performed collectively for multiple test games rather than being performed for each test game.

[0268] In this way, the gaming terminal 13 receives the streaming data of the test game from the test game distribution server 12, and generates a controller signal in response to the trigger frame as appropriate, and transmits it to the test game distribution server 12. The gaming terminal 13 also calculates evaluation data based on the data string of the received media data and the ideal reference, and transmits it to the test game distribution server 12.

[0269] This allows us to obtain evaluation data that is E2E and takes controller input into account, improving the quality of experience that actual users have when playing the game.

[0270] The above-described test game distribution process and test game execution process are performed prior to the play of the actual game, i.e., the establishment of a game session, between the cloud gaming server 11 and the gaming terminal 13. This allows the user's quality of experience to be estimated (evaluated) prior to the play of the actual game, and the evaluation data resulting from this estimation can be used during game play to ensure an appropriate user quality of experience.

[0271] <Description of Game Distribution Processing> After the evaluation data is supplied to and recorded by the cloud gaming server 11, the cloud gaming server 11 then performs a game distribution processing for distributing the game.

[0272] Hereinafter, the game distribution process performed by the cloud gaming server 11 will be described with reference to the flowchart of FIG.

[0273] In step S91, the media generation unit 41 enters a standby state until a game start request is received from the gaming terminal 13, that is, enters a standby state for a game start request.

[0274] A game is selected by the user at any time on the gaming terminal 13. Then, a game start request requesting distribution of the selected game is sent from the gaming terminal 13 to the cloud gaming server 11. For example, the game start request includes a user ID and a game ID indicating the game to be played.

[0275] When a game start request is sent from the gaming terminal 13 , the communication unit 45 receives the game start request and supplies it to the media generation unit 41 .

[0276] In step S92, the media generation unit 41 starts distributing the game to the gaming terminal 13 in response to the game start request.

[0277] Specifically, the media generation unit 41 loads data relating to the game indicated by the game start request from a memory (not shown), and generates video data and audio data for each frame of the game as media data for the game based on that data.

[0278] The streaming processing unit 42 performs necessary processing such as encoding on the game media data generated by the media generation unit 41 , and supplies the resulting streaming data to the communication unit 45 .

[0279] At this time, the streaming processing unit 42 acquires user evaluation data and the like from the user information management unit 43 and controls streaming.

[0280] That is, the user information management unit 43 reads from a memory (not shown) information indicating the genre and scenes of the game to be distributed, as well as the evaluation data recorded in association with the user ID included in the game start request, and supplies this to the streaming processing unit 42.

[0281] At this time, if information regarding quality evaluation such as terminal information (quality evaluation metrics) is also recorded in association with the evaluation data, the user information management unit 43 also reads out the information regarding quality evaluation and supplies it to the streaming processing unit 42.

[0282] The streaming processing unit 42 sets the error correction code, i.e., the FEC communication redundancy and transmission rate, based on the evaluation data supplied from the user information management unit 43 and information (quality evaluation metrics) related to quality evaluation such as terminal information.

[0283] The streaming processing unit 42 then generates streaming data based on the media data of the game so that the transmission rate and FEC of the streaming data match the set transmission rate and FEC of the communication redundancy, and supplies the data to the communication unit 45 .

[0284] The communication unit 45 transmits the game streaming data supplied from the streaming processing unit 42 to the gaming terminal 13. This starts the streaming distribution of the game.

[0285] When the distribution of the game starts, the media generation unit 41 generates media data for game frames at regular time intervals. After the distribution (play) of the game starts, controller signals are transmitted from the gaming terminal 13 as appropriate as the user's controller inputs (responses) in the game.

[0286] In step S93, the communication unit 45 determines whether or not a controller signal has been transmitted.

[0287] If it is determined in step S93 that a controller signal has not been transmitted, then in step S94 the media generation unit 41 generates streaming data of the frame that the game is to transmit.

[0288] More specifically, the media generation unit 41 generates media data assuming that there is no controller signal. The streaming processing unit 42 generates streaming data based on the media data generated by the media generation unit 41 while performing streaming control based on the above-mentioned evaluation data, etc., and supplies the streaming data to the communication unit 45.

[0289] On the other hand, if it is determined in step S93 that a controller signal has been transmitted, the communication unit 45 receives the controller signal transmitted from the gaming terminal 13 and supplies it to the game processing unit 44 in step S95.

[0290] The game processing unit 44 performs processing such as decoding on the controller signal supplied from the communication unit 45 as appropriate, and supplies the controller signal to the media generation unit 41 .

[0291] In step S96, the media generation unit 41 generates streaming data in accordance with the controller signal supplied from the game processing unit 44.

[0292] More specifically, the media generation unit 41 generates media data that reflects the input operation indicated by the controller signal, based on the controller signal. The streaming processing unit 42 generates streaming data based on the media data generated by the media generation unit 41 while performing streaming control based on the above-mentioned evaluation data, etc., and supplies the streaming data to the communication unit 45.

[0293] After the processing of step S94 or step S96 is performed, the processing of step S97 is performed.

[0294] In step S97 , the communication unit 45 transmits the game streaming data supplied from the streaming processing unit 42 to the gaming terminal 13 .

[0295] In step S98, the media generation unit 41 determines whether or not to end the game.

[0296] For example, the media generation unit 41 determines that the game should end when media data for the last frame of the game, i.e., the termination frame, is generated and streaming data corresponding to that media data is transmitted. More specifically, the termination frame is generated when the game is played to the end and ends, or when the user instructs to end play during play, etc.

[0297] If it is determined in step S98 that the game is not yet finished, the process then returns to step S93, and the above-described process is repeated.

[0298] In addition, during the distribution of the game, the user information management unit 43 may send a request to run the test game to the test game distribution server 12 via the communication unit 45, for example, immediately before playing a specific scene of the game.

[0299] In such a case, for example, the test game distribution server 12 performs a process similar to the test game distribution process described with reference to Fig. 10, and the gaming terminal 13 performs a process similar to the test game execution process described with reference to Fig. 11. That is, in the middle of game play, the test game is streamed (the test game is played) and the user's experience quality is evaluated.

[0300] Then, evaluation data obtained by executing the test game is sent from the test game distribution server 12 to the cloud gaming server 11, and the streaming processing unit 42 controls the streaming of the game based on the evaluation data received by the communication unit 45.

[0301] In this way, by running a test game during game play, or more specifically, at a time when the user is not performing any operations while the game is being streamed, it is possible to use evaluation data and the like that show evaluation results closer to the current time for streaming control, which results in more appropriate streaming control and further improves the quality of the user experience.

[0302] On the other hand, if it is determined in step S98 that the game is to be ended, each unit of the cloud gaming server 11 stops the operation it is currently performing, and the game distribution process ends.

[0303] In this way, the cloud gaming server 11 generates game streaming data and transmits it to the gaming terminal 13 while controlling streaming using the evaluation data that takes into account controller inputs in an E2E manner. This makes it possible to improve the quality of the user's experience when playing a game.

[0304] <Description of Game Execution Process> When the game distribution process described with reference to FIG. 12 is performed, the game terminal 13 performs a game execution process shown in FIG.

[0305] 13, a description will be given of the game execution process performed by the gaming terminal 13. For example, the game execution process is started when a user specifies a game and issues a command to start playing the specified game.

[0306] In step S131 , the communication unit 91 transmits a game start request to the cloud gaming server 11 .

[0307] For example, the game input processing unit 95 generates a game start request requesting the execution (start of distribution) of a game specified by the user, and supplies the request to the communication unit 91. For example, the game start request includes a user ID and a game ID indicating the game to be executed.

[0308] The communication unit 91 transmits the game start request supplied from the game input processing unit 95 to the cloud gaming server 11 .

[0309] When the game start request is transmitted, the cloud gaming server 11 performs the process of step S92 in FIG. 12 in response to the game start request, and transmits game streaming data.

[0310] In step S132 , the communication unit 91 receives the streaming data transmitted from the cloud gaming server 11 and supplies it to the streaming processing unit 93 .

[0311] The streaming processing unit 93 performs necessary processing such as decoding on the game streaming data supplied from the communication unit 91 , and supplies the resulting media data to the media playback unit 92 .

[0312] In step S133 , the media playback unit 92 plays back the game media, that is, the video (video frames) and audio (audio frames), based on the media data supplied from the streaming processing unit 93 .

[0313] That is, the media playback unit 92 displays game images based on the video data as media data, and outputs game audio based on the audio data as media data. While watching the played video and listening to the audio, the user operates buttons (not shown) as needed to perform controller input (controller operation), i.e., input operations. The results of the input operations performed by the user as needed are supplied to the game input processing unit 95.

[0314] In step S134, the game input processing section 95 determines whether or not to transmit a controller signal.

[0315] For example, in step S134, if the user performs a controller input and the result of the controller input (result of the input operation) is supplied to the game input processing unit 95, it is determined that a controller signal is to be transmitted.

[0316] If it is determined in step S134 that the controller signal is to be transmitted, the communication unit 91 transmits the controller signal in step S135.

[0317] That is, the game input processing unit 95 generates a controller signal indicating the operation content of the controller input based on the result of the user's controller input, and supplies the controller signal to the communication unit 91. Then, the communication unit 91 transmits the controller signal supplied from the game input processing unit 95 to the cloud gaming server 11.

[0318] On the other hand, if it is determined in step S134 that the controller signal is not to be transmitted, the process of step S135 is not performed, and the process then proceeds to step S136.

[0319] If the process of step S135 is performed or if it is determined in step S134 that the controller signal will not be transmitted, then the process of step S136 is performed.

[0320] In step S136, the streaming processing unit 93 determines whether or not to end the game. For example, the streaming processing unit 93 determines to end the game when media (video and audio) is played based on the media data of the final frame of the game.

[0321] If it is determined in step S136 that the game is not yet finished, the process returns to step S132, and the above-described process is repeated.

[0322] On the other hand, if it is determined in step S136 that the game is to be ended, the respective units of the gaming terminal 13 stop the processing they are currently performing, and the game execution processing ends.

[0323] In this way, the gaming terminal 13 receives the streaming data sent (distributed) by the cloud gaming server 11 and plays the game media, and also generates a controller signal in response to the user's input operation as appropriate and sends it to the cloud gaming server 11.

[0324] In this case, the cloud gaming server 11 performs streaming control based on E2E evaluation data that takes controller input into account, thereby improving the quality of experience experienced by actual users when playing games on the gaming terminal 13 side.

[0325] In the cloud gaming system described above, when playing a cloud-based game delivered by the cloud gaming server 11, the quality of the user's experience can be evaluated using at least a single indicator, the evaluation data.

[0326] Generally, in cloud gaming, there are many factors that affect the user experience quality, making it difficult to evaluate the user experience quality using a single indicator.

[0327] In contrast, the cloud gaming system of the present technology uses E2E evaluation data that takes controller input into account, enabling quality control, i.e., streaming control, to be performed according to one or more combinations of the user (player) level, game genre, and game scene.

[0328] In other words, the streaming processing unit 42 of the cloud gaming server 11 can estimate the necessary conditions for a user to play comfortably, for example, for a particular game, and set the transmission rate and FEC communication redundancy.

[0329] As a result, more appropriate streaming control can be achieved, improving the quality of experience for users.

[0330] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, that can execute various functions by installing various programs.

[0331] FIG. 14 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0332] In the computer, a CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 , and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

[0333] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0334] The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, etc. The output unit 507 includes a display, a speaker, etc. The recording unit 508 includes a hard disk, a non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0335] In a computer configured as described above, the CPU 501 loads a program recorded in the recording unit 508, for example, into the RAM 503 via the input / output interface 505 and the bus 504, and executes the program, thereby performing the above-described series of processes.

[0336] The program executed by the computer (CPU 501) can be provided by being recorded on a removable recording medium 511 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0337] In a computer, a program can be installed in the recording unit 508 via the input / output interface 505 by inserting a removable recording medium 511 into the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.

[0338] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0339] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0340] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0341] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0342] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0343] Furthermore, the present technology can also be configured as follows.

[0344] (1) An information processing device comprising: a media generation unit that generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, the test game including frames to which meta information for inducing transmission of a controller signal is attached; and a communication unit that transmits the media data to a gaming terminal, wherein when the controller signal transmitted by the gaming terminal is received by the communication unit, the media generation unit generates the media data reflecting an operation indicated by the controller signal. (2) The information processing device described in (1), wherein the communication unit streams the media data of the test game corresponding to the game to the gaming terminal prior to play of the game on the gaming terminal or during play of the game. (3) The information processing device described in (2), wherein the test games are prepared for each of a plurality of items related to the game. (4) The information processing device described in (3), wherein the items include at least one of a genre and a scene of the game. (5) The information processing device according to any one of (2) to (4), wherein the communication unit transmits to the gaming terminal an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment. (6) The information processing device according to (5), wherein the communication unit transmits to the gaming terminal a response pattern that is used for generating the controller signal at the gaming terminal and indicates a response pattern when a frame to which the meta information is attached is received. (7) The information processing device according to (6), wherein the ideal reference or the response pattern is prepared for each level of the user or each type of character controlled in the game. (8) The information processing device according to any one of (5) to (7), wherein the communication unit receives from the gaming terminal evaluation data that indicates an evaluation result of the user's quality of experience, the evaluation result being generated based on the media data of the test game received by the gaming terminal and the ideal reference.(9) The information processing device according to (8), further comprising a streaming processing unit that performs streaming control when streaming media data of the game to the gaming terminal based on the evaluation data. (10) The information processing device according to (9), wherein the streaming processing unit controls a transmission rate or communication redundancy as the streaming control. (11) An information processing method, wherein an information processing device generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing transmission of a controller signal is attached, transmits the media data to the gaming terminal, and, when the controller signal transmitted by the gaming terminal is received, generates the media data reflecting an operation indicated by the controller signal. (12) An information processing device comprising: a communication unit that receives, from a server, media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing transmission of a controller signal is attached; and a quality test processing unit that generates evaluation data indicating an evaluation result of the user's quality of experience based on an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment, and the media data received from the server, wherein the communication unit transmits the controller signal to the server when it receives the media data of the frame to which the meta information is attached. (13) The information processing device described in (12), wherein the communication unit receives the media data of the test game corresponding to the game, streamed by the server, prior to play of the game on the information processing device or during play of the game. (14) The information processing device described in (13), wherein the test game is prepared for each of a plurality of items related to the game. (15) The information processing device described in (14), wherein the items include at least one of the genre and scene of the game.(16) The information processing device according to any one of (13) to (15), further comprising a game input processing unit that generates the controller signal based on the meta information and a response pattern that indicates a pattern of a response when a frame to which the meta information is attached is received. (17) The information processing device according to (16), wherein the ideal reference or the response pattern is prepared for each level of the user or each type of character controlled in the game. (18) The information processing device according to (16) or (17), wherein the communication unit receives the ideal reference and the response pattern from the server. (19) The information processing device according to any one of (12) to (18), wherein the quality test processing unit generates the evaluation data by calculating an error function based on the ideal reference and the media data received from the server. (20) An information processing method in which an information processing device receives media data from a server for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing transmission of a controller signal is attached; generates evaluation data indicating the evaluation result of the user's quality of experience based on an ideal reference, which is the media data of the media that would be played if the test game were played in an ideal environment, and the media data received from the server; and transmits the controller signal to the server when the media data of the frame to which the meta information is attached is received.(21) An information processing system having a server and a gaming terminal, wherein the server comprises: a media generation unit that generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing transmission of a controller signal is attached; and a first communication unit that transmits the media data to the gaming terminal, wherein the gaming terminal comprises: a second communication unit that receives the media data transmitted by the first communication unit of the server; and a quality test processing unit that generates evaluation data indicating an evaluation result of the user's quality of experience based on an ideal reference, which is the media data of the media that will be played when the test game is played in an ideal environment, and the media data received from the server, wherein the second communication unit of the gaming terminal transmits the controller signal to the server when it receives the media data of the frame to which the meta information is attached, An information processing system in which, when the controller signal transmitted by the second communication unit of the gaming terminal is received by the first communication unit, the media generation unit of the server generates the media data that reflects the operation indicated by the controller signal.

[0345] REFERENCE SIGNS LIST 11 Cloud gaming server, 12 Test game distribution server, 13 Gaming terminal, 71 Media generation unit, 72 Streaming processing unit, 73 Test game management unit, 74 Game processing unit, 75 Communication unit, 76 User information management unit, 91 Communication unit, 92 Media playback unit, 93 Streaming processing unit, 94 Quality test processing unit, 95 Game input processing unit

Claims

1. An information processing device comprising: a media generation unit that generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, the frames including frames to which meta information for inducing the transmission of a controller signal is attached; and a communication unit that transmits the media data to a gaming terminal, wherein when the controller signal transmitted by the gaming terminal is received by the communication unit, the media generation unit generates the media data reflecting an operation indicated by the controller signal.

2. The information processing device according to claim 1, wherein the communication unit streams the media data of the test game corresponding to the game to the gaming terminal prior to or during play of the game on the gaming terminal.

3. The information processing device according to claim 2, wherein the test games are prepared for each of a plurality of items related to the game.

4. The information processing device according to claim 3, wherein the items include at least one of the genre and scene of the game.

5. The information processing device according to claim 2, wherein the communication unit transmits to the gaming terminal an ideal reference, which is the media data of the media that is reproduced when the test game is played in an ideal environment.

6. The information processing device according to claim 5, wherein the communication unit is used to generate the controller signal in the gaming terminal, and transmits to the gaming terminal a response pattern indicating a response pattern when a frame to which the meta information is attached is received.

7. The information processing device according to claim 6, wherein the ideal reference or the response pattern is prepared for each level of the user or for each type of character operated in the game.

8. An information processing device as described in claim 5, wherein the communication unit receives evaluation data from the gaming terminal indicating an evaluation result of the user's quality of experience generated based on the media data of the test game received by the gaming terminal and the ideal reference.

9. The information processing device according to claim 8, further comprising a streaming processing unit that performs streaming control when streaming media data of the game to the gaming terminal based on the evaluation data.

10. The information processing device according to claim 9, wherein the streaming processing unit controls a transmission rate or a communication redundancy as the streaming control.

11. An information processing method, in which an information processing device generates media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal is added, transmits the media data to a gaming terminal, and, when the controller signal transmitted by the gaming terminal is received, generates the media data reflecting the operation indicated by the controller signal.

12. An information processing device comprising: a communication unit that receives from a server media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal; and a quality test processing unit that generates evaluation data indicating the evaluation result of the user's quality of experience based on an ideal reference, which is the media data of the media that is played when the test game is played in an ideal environment, and the media data received from the server, wherein the communication unit transmits the controller signal to the server when it receives the media data of a frame to which the meta information is added.

13. An information processing device as described in claim 12, wherein the communication unit receives the media data of the test game corresponding to the game, which is streamed by the server prior to or during play of the game on the information processing device.

14. The information processing device according to claim 13, wherein the test games are prepared for each of a plurality of items related to the game.

15. The information processing device according to claim 14, wherein the items include at least one of the genre and scene of the game.

16. The information processing device according to claim 13, further comprising a game input processing unit that generates the controller signal based on the meta information and a response pattern that indicates a response pattern when a frame to which the meta information is attached is received.

17. The information processing device according to claim 16, wherein the ideal reference or the response pattern is prepared for each level of the user or for each type of character operated in the game.

18. The information processing device according to claim 16, wherein the communication unit receives the ideal reference and the response pattern from the server.

19. The information processing device according to claim 12, wherein the quality test processing unit generates the evaluation data by calculating an error function based on the ideal reference and the media data received from the server.

20. An information processing method in which an information processing device receives from a server media data for playing media consisting of at least one of video and audio for each of a plurality of frames of a test game for evaluating a user's quality of experience, including frames to which meta information for inducing the transmission of a controller signal is added; generates evaluation data indicating the evaluation result of the user's quality of experience based on an ideal reference, which is the media data of the media that is played when the test game is played in an ideal environment, and the media data received from the server; and transmits the controller signal to the server when the media data of the frame to which the meta information is added is received.

Images