Video transmission system and video transmission method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- NIPPON TELEGRAPH & TELEPHONE CORP
- Filing Date
- 2022-10-17
- Publication Date
- 2026-08-03
Smart Images
Figure 0007898688000001 
Figure 0007898688000002 
Figure 0007898688000003
Abstract
Description
Technical Field
[0001] This disclosure relates to cloud gaming technology.
Background Art
[0002] A system has been proposed in which a server sequentially transmits corresponding game videos according to a user's operation on a client. The provision of game videos by sequential transmission enables even a terminal with low processing power to play high-definition games because the client does not need to perform game processing. On the other hand, there is a problem that an increase in network delay due to video transmission occurs, resulting in a deterioration of the perceived quality.
[0003] Speculative video transmission "CloudHide" has been proposed, in which all videos corresponding to all inputs that a user can take are transmitted in advance (see, for example, Non-Patent Document 1). However, in Non-Patent Document 1, while reducing response delay, there is a problem that a large amount of traffic occurs according to the number of possible inputs.
[0004] In order to prevent the large amount of traffic required for pre-transmission of videos, "Outatime" has been proposed, in which likely inputs among the inputs that a user can take are estimated, and a plurality of candidate game videos are generated and pre-transmitted (see, for example, Non-Patent Document 2). In Non-Patent Document 2, when the estimation is accurate, a game video can be immediately output on the client side for an input, but when the estimation is inaccurate, correction is performed from the received image to generate an appropriate game video.
Prior Art Documents
Non-Patent Documents
[0005]
Non-Patent Document 1
[0006] Non-patent document 2 suggests that traffic can be reduced by limiting the game video transmitted by the server. However, this presents a problem of increased response delay if the server's prediction is incorrect. Additionally, it requires advanced image processing on the user's client side.
[0007] This disclosure aims to reduce traffic while preventing increased response delays caused by server prediction failures. [Means for solving the problem]
[0008] The system of this disclosure is a system in which a server sequentially transmits corresponding game images in response to user operations on a client, and the server and client execute the method of this disclosure. The program of this disclosure includes a program for realizing a computer as a function of the server or client of this disclosure, and a program for causing the computer to execute each step of the method executed by the server or client of this disclosure.
[0009] The server in this disclosure is A server that transmits corresponding game video in response to user actions on the client, For any given game state at a given point in time, all game screens to be displayed are generated based on input from the client. The generated game screen is divided into predetermined tiles, Compare tile images displayed in the same position, For predefined, standardizable tile images, a common tile image is sent to the client. For tile images that cannot be standardized and are predetermined, individual tile images will be sent to the client.
[0010] The clients of this disclosure are: A client that receives game video from a server and displays the game screen in response to user input, For any given game state, the server receives tile images that can generate all the game screens to be displayed based on client input, before the client input is sent. When input indicating a change in the game state is received, the system combines it with previously received tile images to generate and display a game screen corresponding to the user's input.
[0011] This disclosure demonstrates that by adopting the above configuration, it is possible to reduce traffic while preventing an increase in response delays caused by server prediction failures. This makes it possible to suppress traffic while preventing a decline in perceived quality.
[0012] The predefined tile images that can be shared are, for example, the same tile images. In this case, the server sends the common tile images to the client for the same tile images and sends individual tile images to the client for tile images that are not the same.
[0013] The predefined tile images that can be shared are, for example, similar tile images. In this case, the server compares the similarity between the tile images displayed at the same position, sends the common tile images to the client for the similar tile images, and sends individual tile images to the client for the dissimilar tile images. Here, the comparison of similarity may be performed by calculating the hash values of the tile images and comparing the distances of the calculated hash values.
[0014] In the present disclosure, one of the game screens to be displayed based on the input from the client may be defined as a reference screen, the tile images different from the reference screen may be extracted, and the extracted tile images may be sent as the individual tile images.
[0015] In addition, the above disclosures can be combined as much as possible.
Effect of the Invention
[0016] According to the present disclosure, it is possible to reduce traffic while preventing an increase in response delay caused by prediction failure of the arbiter.
Brief Description of the Drawings
[0017] [Figure 1] Shows a configuration example of a sequential transmission system. [Figure 2] Shows an overview of the present disclosure. [Figure 3] Shows a system configuration example of the present embodiment. [Figure 4] It is an explanatory diagram showing an example of the operation of the server. [Figure 5] It is an explanatory diagram showing an example of the operation of the client. [Figure 6]It is an explanatory diagram showing an example of a tile image included in a game screen. [Figure 7] It is an explanatory diagram showing an example of one tile image for calculating a hash value.
Embodiments for Carrying out the Invention
[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These examples are merely illustrative, and the present disclosure can be implemented in various modified forms based on the knowledge of those skilled in the art. In the present specification and drawings, components with the same reference numerals indicate the same components.
[0019] (Sequential Transmission System) FIG. 1 shows a configuration example of a sequential transmission system. In the sequential transmission system, the server 20 transmits a game video of the game state according to the instruction of the client 10 each time an instruction from the client 10 is received.
[0020] The client 10 includes an input unit 11 to which an instruction from a user is input, an instruction transmission unit 12 that transmits a game instruction, a video reception unit 13 that receives video data from the server 20, a game screen decoding unit 14 that decodes the received video data and restores it to the game screen, a display unit 15 that displays the game screen, and is provided with.
[0021] The server 20 includes an instruction reception unit 21 that receives an instruction from the client 10, a game state update unit 22 that updates the game state according to the instruction received by the instruction reception unit 21, a game screen generation unit 23 that generates a game screen according to the updated game state, a game screen encoding unit 24 that encodes the game screen, A video transmission unit 25 that transmits encoded video data, It is equipped with.
[0022] (Summary of this disclosure) Figure 2 shows an overview of the sequential transmission system of this disclosure. When the server 20 of this disclosure receives an instruction from the client 10 sent at time t0, it sends to the client 10 before time t3 image data capable of generating game screens for all possible game states that may occur at any predetermined time t3 after the instruction. When the instruction is input to the client 10 at time t3, it uses the previously received image data to generate and display a game screen corresponding to the instruction.
[0023] In games, character movements change rapidly, but the background does not change as much. Therefore, in this disclosure, the game screen is divided into a predetermined number of tiles, and for predetermined tile images that can be shared, a common tile image is sent to the client, while for predetermined tile images that cannot be shared, individual tile images are sent to the client.
[0024] Here, identical tile images can be standardized. Therefore, in this disclosure, a common tile image is sent to the client for identical tile images, and individual tile images are sent to the client for non-identical tile images.
[0025] Even if tile images are not completely identical, there are cases where tile images can be shared. In this disclosure, the similarity of tile images displayed in the same position may be compared, and a common tile image may be sent to the client for similar tile images, while individual tile images may be sent to the client for dissimilar tile images.
[0026] (First embodiment) Figure 3 shows an example of the system configuration of this embodiment. The server 20 of this embodiment includes an instruction receiving unit 21, a game state updating unit 22, a game screen generation unit 31, a tile image comparison unit 32, a tile image extraction unit 33, a tile image encoding unit 34, and a video transmission unit 35. The client 10 of this embodiment includes an input unit 11, an instruction transmission unit 12, a video receiving unit 41, a tile image decoding unit 42, a game screen generation unit 43, and a display unit 15.
[0027] The server 20 and client 10 of this disclosure can also be implemented by a computer and a program, and the program can be recorded on a recording medium or provided over a network.
[0028] The operation of server 20 will be explained with reference to Figure 4. On server 20, when the instruction receiving unit 21 receives an instruction from client 10, the game state update unit 22 updates the game state according to the instruction. The game screen generation unit 31 generates game screens for all inputs that the client 10 can input in the latest game state. The tile image comparison unit 32 sets one of these screens as the reference screen.
[0029] The tile image comparison unit 32 divides all game screens into multiple tiles, compares each game screen with the reference screen tile by tile, and the tile image extraction unit 33 extracts tile images with different images.
[0030] The tile image encoding unit 34 encodes the following: • All tile images included in the reference screen • Tile images that are not the reference screen but have different visuals from the reference screen.
[0031] The video transmission unit 35 sequentially transmits the video data of the tile images encoded by the tile image encoding unit 34 to the client 10.
[0032] The operation of client 10 will be explained with reference to Figure 5. In client 10, when the video receiving unit 41 receives video data, the tile image decoding unit 42 decodes the video data into tile images and stores each tile image. When an operation is input from the user to the input unit 11, the game screen generation unit 43 acquires the input instruction, combines the stored tile images to generate a game screen corresponding to the instruction, and displays it on the display unit 15.
[0033] In this way, the server 20 extracts screen data to be transmitted speculatively for each tile, enabling the generation of a game screen for all inputs. Therefore, this embodiment reduces the amount of video data transmitted speculatively and prevents increased delay due to prediction failures because it does not perform input prediction.
[0034] (Second embodiment) In this embodiment, when the tile image comparison unit 32 compares tile images, it calculates the similarity of the hash values of the tile images and determines which tile image to extract according to the similarity threshold.
[0035] Specifically, as shown in Figure 6, frame0(t)~frame N When a game screen (t) is generated, and each game screen is divided into 16 tile images in a 4x4 grid, and frame0(t) is selected as the reference image, the tile image comparison unit 32 compares the hash values of each tile image of the reference screen frame0(t) with all other game screens frame1(t) to frame N Calculate the hash value for each tile image of (t).
[0036] For example, the tile image (0,0) of frame0(t) is converted to a grayscale image with a predetermined number of pixels, such as 8x8 pixels, as shown in Figure 7. The average brightness h(i,j) of each pixel is calculated, and the image is binarized to "0" or "1" depending on whether the average value h(i,j) is higher or lower than a predetermined threshold. This generates eight lines of 8-bit binary data. By concatenating these eight lines of binary data, 64-bit binary data is generated. This allows us to calculate the ahash of the tile image (0,0) of frame0(t).
[0037] The hash value calculation method described herein is not limited to ahash; for example, a low-frequency DCT coefficient may be selected and used to calculate the hash value, such as phash. do.
[0038] The tile image comparison unit 32 calculates the similarity of hash values between tile images placed at the same position. For example, it calculates the similarity between the tile image (0,0) of frame0(t) and the tile image (0,0) of frame1(t). Here, the similarity is high when the distance between hash values, such as the Hamming distance, is small, and the similarity is low when the distance is large. Therefore, only when the distance between hash values is greater than a predetermined similarity threshold is it determined that the reference screen and the video are different tile images.
[0039] This embodiment reduces the delay required for similarity determination by using a hash function for comparing tile images. Furthermore, by reducing the number of pixels used when calculating the hash value, the similarity determination processing speed can be further increased.
[0040] The tile image extraction unit 33, the tile image encoding unit 34, and the video transmission unit 35 are the same as in the first embodiment. However, the tile image encoding unit 34 may encode each tile image using the hash value of the tile image. In this case, the video transmission unit 35 may transmit video data using tile-wise delta encoding. The operation of the client 10 used by the user is the same as in the first embodiment, displaying the game screen on the display unit 15 according to the actual input and sequentially transmitting its own input to the server 20.
[0041] This embodiment has the following advantages: • Normally, video encoding involves encoding between screens on different time axes. In contrast, this embodiment introduces encoding for game screens on the same time axis. • This embodiment allows for easy determination of the similarity between tile images using a hash function. • The client 10 used by the user does not require computing resources beyond what is needed for screen display. • The client 10 used by the user can display the game screen directly in response to input using the received tile images.
[0042] As explained above, this disclosure can minimize response delay by pre-transmitting game screens corresponding to user inputs. Furthermore, this disclosure can reduce traffic by not transmitting regions with high similarity among the generated game screens. In addition, in the second embodiment, the similarity between game screens can be easily determined using a hash function. [Explanation of Symbols]
[0043] 10: Client 11: Input section 12: Instruction transmission unit 13: Video receiving unit 14: Game screen decoding unit 15: Display section 20: Server 21: Instruction receiving unit 22: Game Status Update Section 23: Game screen generation unit 24: Game screen encoding section 25: Video transmission unit 31: Game screen generation unit 32: Tile Image Comparison Section 33: Tile Image Extraction Unit 34: Tile Image Encoding Unit 35: Video transmission unit 41: Video receiving unit 42: Tile Image Decoding Unit 43: Game screen generation unit
Claims
1. A server that transmits corresponding game video in response to user actions on the client, For any given game state at a given point in time, all game screens to be displayed are generated based on input from the client. The generated game screen is divided into predetermined tiles, The similarity between tile images is compared by calculating the hash value of the tile images displayed in the same position and comparing the distance between the calculated hash values. For similar tile images, send a common tile image to the client. For tile images that are not similar, send individual tile images to the client. server.
2. The server according to claim 1, A client that receives game video from the server and displays the game screen in response to user input, For any given game state, the server receives tile images from the client before the client input is received, which are capable of generating all the game screens to be displayed based on the client input. When input indicating a change in the game state is received, the system combines pre-received tile images to generate and display a game screen corresponding to the user's input. With the client, A system equipped with these features.
3. A method executed by a server that transmits corresponding game footage in response to user actions on the client, For any given game state at a given point in time, all game screens to be displayed are generated based on input from the client. The generated game screen is divided into predetermined tiles, The similarity between tile images is compared by calculating the hash value of the tile images displayed in the same position and comparing the distance between the calculated hash values. For similar tile images, send a common tile image to the client. For tile images that are not similar, send individual tile images to the client. method.
4. A method executed by a server that transmits corresponding game footage in response to user actions on the client, For any given game state at a given point in time, all game screens to be displayed are generated based on input from the client. The generated game screen is divided into predetermined tiles, The similarity between tile images is compared by calculating the hash value of the tile images displayed in the same position and comparing the distance between the calculated hash values. For similar tile images, send a common tile image to the client. For tile images that are not similar, send individual tile images to the client. Methods and, A method executed by a client that receives game video from the server and displays the game screen in response to user input, For any given game state, the server receives tile images from the client before the client input is received, which are capable of generating all the game screens to be displayed based on the client input. When input indicating a change in the game state is received, the system combines pre-received tile images to generate and display a game screen corresponding to the user's input. Methods and, A method for providing this.
5. A program for realizing a computer as a server according to claim 1.