A cloud game audio traffic saving and sound quality improving method and device

Abstract

This application provides a method and apparatus for saving audio traffic and improving sound quality in cloud gaming. The method includes: a client acquiring current game information and sending it to a server; the server determining the appropriate audio type and effect to be played based on the current control information, and generating first instruction information; the server concatenating the first instruction information and the current game information to form a control instruction, and sending it to the client; the client reading the current game information and the first instruction information from the control instruction, reading the corresponding audio type and effect based on the first instruction information, and adding the audio type and effect to the game playback based on the current game information. This application, by replacing audio transmission with control information transmission, greatly reduces the amount of data that needs to be transmitted, improves the timeliness of data transmission, enhances the gaming experience, and saves traffic.

Description

Technical Field

[0001] This application claims protection for a cloud gaming sound processing technology, and more particularly relates to a method for saving audio bandwidth and improving sound quality in cloud gaming. This application also relates to an apparatus for saving audio bandwidth and improving sound quality in cloud gaming. Background Technology

[0002] Cloud gaming is a gaming technology that displays and operates on a client side while running the game on a cloud server. This technology has high requirements for network quality. Therefore, due to the influence of network quality, the development and final actual effect of cloud gaming may differ from the initial concept.

[0003] In existing technologies, all audio data for cloud gaming is sent from the cloud gaming server. Unlike video data, audio data only has a limited number of sound effects and music tracks per game. Repeatedly transmitting unchanging data wastes bandwidth, and due to the smoothness requirements of streaming, the audio quality cannot be provided at its best over the network, failing to give cloud gaming users an immersive audio experience. Furthermore, network quality significantly impacts the timeliness of sound effects. Summary of the Invention

[0004] To address one or more of the technical problems mentioned in the prior art, this application provides a method for saving audio bandwidth and improving sound quality in cloud gaming. This application also relates to an apparatus for saving audio bandwidth and improving sound quality in cloud gaming.

[0005] This application provides a method for saving audio bandwidth and improving sound quality in cloud gaming, including:

[0006] Based on the connection relationship between the client and the server, the client obtains the current game information, generates the current control information based on the current game information, and sends it to the server.

[0007] The server determines the audio type and audio effect that should be played based on the current control information, and generates first instruction information based on the audio type and audio effect;

[0008] The server combines the first instruction information and the current game information to form a control instruction, and then sends it to the client.

[0009] The client reads the current game information and the first instruction information from the control instructions, reads the music corresponding to the local audio type and the audio effect transmitted from the server according to the first instruction information, and adds the audio type and audio effect to the game playback according to the current game information.

[0010] Optionally, the audio type includes: sound effect encoding.

[0011] Optionally, the audio effects include: ambient sound effects and surround sound effects.

[0012] Optionally, the music corresponding to the audio type is stored in the client's memory.

[0013] Optionally, the client can be a mobile phone or a computer.

[0014] This application also provides a device for saving audio bandwidth and improving sound quality in cloud gaming, including:

[0015] The first transmission module is used to, based on the connection relationship between the client and the server, have the client obtain current game information, generate current control information based on the current game information, and send it to the server.

[0016] The instruction generation module is used by the server to determine the audio type and audio effect that should be played based on the current control information, and to generate first instruction information based on the audio type and audio effect.

[0017] The second transmission module is used by the server to combine the first instruction information and the current game information to form a control instruction, and then send it to the client.

[0018] The sound effects processing module is used by the client to read the current game information and the first instruction information from the control instructions, read the music corresponding to the local audio type and the audio effects transmitted from the server according to the first instruction information, and add the audio type and audio effects to the game playback according to the current game information.

[0019] Optionally, the audio type includes: sound effect encoding.

[0020] Optionally, the audio effects include: ambient sound effects and surround sound effects.

[0021] Optionally, the music corresponding to the audio type is stored in the client's memory.

[0022] Optionally, the client can be a mobile phone or a computer.

[0023] The advantages of this application compared to the prior art are:

[0024] This application provides a method for saving audio traffic and improving sound quality in cloud gaming, comprising: based on the connection relationship between the client and the server, the client obtains current game information, generates current control information based on the current game information, and sends it to the server; the server determines the audio type and audio effect that should be played based on the current control information, and generates first instruction information based on the audio type and audio effect; the server concatenates the first instruction information and the current game information to form a control instruction, and sends it to the client; the client reads the current game information and the first instruction information from the control instruction, reads the corresponding audio type and audio effect based on the first instruction information, and adds the audio type and audio effect to the game playback based on the current game information. This application, by replacing audio transmission with control information transmission, greatly reduces the amount of data that needs to be transmitted, improves the timeliness of data transmission, enhances the gaming experience, and saves traffic. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the cloud gaming audio traffic saving and sound quality improvement in this application.

[0026] Figure 2 This is a flowchart of the instruction assembly process in this application.

[0027] Figure 3 This is a schematic diagram of the cloud gaming audio traffic saving and sound quality improvement device in this application. Detailed Implementation

[0028] The following are examples of specific implementation processes provided to illustrate the technical solutions to be protected in this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can implement this application by different technical means under the guidance of the concept of this application. Therefore, this application is not limited to the specific embodiments below.

[0029] This application provides a method for saving audio traffic and improving sound quality in cloud gaming, comprising: based on the connection relationship between the client and the server, the client obtains current game information, generates current control information based on the current game information, and sends it to the server; the server determines the audio type and audio effect that should be played based on the current control information, and generates first instruction information based on the audio type and audio effect; the server concatenates the first instruction information and the current game information to form a control instruction, and sends it to the client; the client reads the current game information and the first instruction information from the control instruction, reads the music corresponding to the corresponding audio type from the local storage and the audio effect transmitted from the server according to the first instruction information, and adds the audio type and audio effect to the game playback according to the current game information. This application, by replacing audio transmission with control information transmission, greatly reduces the amount of data that needs to be transmitted, improves the timeliness of data transmission, enhances the gaming experience, and saves traffic.

[0030] Figure 1 This is a flowchart illustrating the cloud gaming audio traffic saving and sound quality improvement in this application.

[0031] Please refer to Figure 1 As shown, in step S101, based on the connection relationship between the client and the server, the client obtains the current game information, generates current control information based on the current game information, and sends it to the server.

[0032] The cloud gaming audio bandwidth saving and sound quality improvement method described in this application is mainly applied in the cloud gaming field. The cloud server is used for game operation, and the client is used for controlling the display of the cloud game. The control is provided by a game controller connected to the client, which enhances the input for game control.

[0033] The client and server are connected via a network. The client dynamically and in real-time collects current game information, including input control signals and other signals such as music and sound effects, from the game it is currently running. Based on this game information, it generates current control information. Specifically, the current control information refers to control information regarding changes to game content. This control information is then sent to the cloud server. In this application, the current game information includes the current game screen, received commands, and the corresponding sound effects information for those screens and commands.

[0034] In this application, a virtual machine is set up in the cloud server corresponding to the client. The virtual machine runs the game currently displayed on the client, but the virtual machine does not play the sound effects of the game, but sends the sound effect information to the client.

[0035] In this application, the client has pre-downloaded and stored all the sound effect content data of the current game in its local storage, which is available for the client to retrieve according to the instructions of the cloud server.

[0036] On the other hand, when the client does not store the sound effect content data, it will not play the sound effect when it receives the instruction from the server.

[0037] On the other hand, after the cloud server starts the cloud game, it sends a verification message, instructing the client to verify the sound effect content data to determine if there is any undownloaded and unstored sound effect content data. If not, a command to update the data content is issued when the game starts.

[0038] Please refer to Figure 1 As shown in S102, the server determines the audio type and audio effect that should be played based on the current control information, and generates first instruction information based on the audio type and audio effect.

[0039] After receiving the current control information, the server first reads the control instructions contained in the current control information, and determines the audio type and audio effects required for the current game content based on the control instructions.

[0040] Specifically, the audio type and audio effect refer to the music and sound effects played in the game, respectively. Further, the audio type refers to a specific piece of music within the background music or other music, and the audio effect is the sound effect, such as the feedback sound when a key is pressed, or the sound effects accompanying changes in game content. Examples include ambient sound effects and surround sound effects.

[0041] The cloud server can determine the audio type and audio effect based on the pre-set audio type and audio effect corresponding to the control command.

[0042] In the cloud server, each cloud game has a corresponding table storing a mapping between control commands and sound effects. When a cloud game starts in the virtual machine, this mapping table is added to the game's running process. The server reads control commands received from the cloud server (specifically, the virtual machine within the server) in real time and determines the corresponding sound effects. The sound effects include the audio types and audio effects described above.

[0043] Specifically, determining the corresponding sound effect refers to determining the encoding of the sound effect. The encoding of the sound effect is uniquely represented in each cloud game.

[0044] Once the audio type and audio effect are determined in the cloud server, the first instruction information can be generated based on the audio type and audio effect.

[0045] Specifically, the first instruction information includes at least the encoding of the sound effect.

[0046] Please refer to Figure 1 As shown, in S103, the server concatenates the first instruction information and the current game information to form a control instruction, and sends it to the client.

[0047] The first instruction information contains an audio effect encoding, which uniquely represents the type of audio or audio effect to be played.

[0048] The first instruction information and the current game information are spliced ​​together. In fact, it is necessary to match the sound effect with the current game information to achieve accurate timing of sound effect playback.

[0049] Figure 2 This is a flowchart of the instruction assembly process in this application.

[0050] Please refer to Figure 2 As shown, S201 calculates the splicing time.

[0051] The specific splicing time can be represented by the following expression:

[0052]

[0053] Among them, the This indicates the starting point when the sound effect is played, where 's' is a network quality parameter. Indicates the starting point of the sound effect, the The network latency indicating the reception of information, the This represents the network latency for sending information, where n represents the preset number of historical transmissions. The number of historical transmissions refers to the number of data transmissions within a set time period.

[0054] Please refer to Figure 2 As shown, S202 begins data splicing.

[0055] Specifically, the splicing includes at least encoding the sound effect, These are encoded into the first instruction information. The encoding can be formed by directly concatenating multiple character codes.

[0056] Please refer to Figure 1 As shown in S104, the client reads the current game information and the first instruction information from the control instructions, reads the music corresponding to the local audio type and the audio effect transmitted from the server according to the first instruction information, and adds the audio type and audio effect to the game playback according to the current game information.

[0057] The client receives the first instruction information sent by the server and reads the encoding of the sound effect from the first instruction information. Then determine the audio type and audio effect.

[0058] Furthermore, the client queries and retrieves the sound effect from the pre-stored sound effects according to the sound effect's encoding, and then executes playback.

[0059] Specifically, the sound effect playback process has already started. When the sound effect is selected, its playback is directly incorporated into the sound effect process, so the start time of the sound effect can be ignored.

[0060] On the other hand, after the client reads the sound effects of the current game, it activates the sound effects and, according to the... Select the starting point for playing the sound effect.

[0061] Finally, the sound effects are incorporated into the running game.

[0062] Preferably, the client is a mobile phone or a computer.

[0063] This application also provides a cloud gaming audio traffic saving and sound quality improvement device, which includes: a first transmission module 301, an instruction generation module 302, a second transmission module 303, and a sound effect processing module 304.

[0064] Figure 3 This is a schematic diagram of the cloud gaming audio traffic saving and sound quality improvement device in this application.

[0065] Please refer to Figure 3 As shown, the first transmission module 301 is used to obtain current game information from the client based on the connection relationship between the client and the server, generate current control information based on the current game information, and send it to the server.

[0066] The cloud gaming audio bandwidth saving and sound quality improvement method described in this application is mainly applied in the cloud gaming field. The cloud server is used for game operation, and the client is used for controlling the display of the cloud game. The control is provided by a game controller connected to the client, which enhances the input for game control.

[0067] The client and server are connected via a network. The client dynamically and in real-time collects current game information, including input control signals and other signals such as music and sound effects, and then generates current control information based on this information. Specifically, the current control information refers to control information for changes in game content. This control information is then sent to the cloud server.

[0068] In this application, a virtual machine is set up in the cloud server corresponding to the client. The virtual machine runs the game currently displayed on the client, but the virtual machine does not play the sound effects of the game, but sends the sound effect information to the client.

[0069] In this application, the client has pre-downloaded and stored all the sound effect content data of the current game in its local storage, which is available for the client to retrieve according to the instructions of the cloud server.

[0070] On the other hand, when the client does not store the sound effect content data, it will not play the sound effect when it receives the instruction from the server.

[0071] On the other hand, after the cloud server starts the cloud game, it sends a verification message, instructing the client to verify the sound effect content data to determine if there is any undownloaded and unstored sound effect content data. If not, a command to update the data content is issued when the game starts.

[0072] Please refer to Figure 3 As shown, the instruction generation module 302 is used by the server to determine the audio type and audio effect that should be played at the moment based on the current control information, and to generate first instruction information based on the audio type and audio effect.

[0073] After receiving the current control information, the server first reads the control instructions contained in the current control information, and determines the audio type and audio effects required for the current game content based on the control instructions.

[0074] Specifically, the audio type and audio effect refer to the music and sound effects played in the game, respectively. Further, the audio type refers to a specific piece of music within the background music or other music, and the audio effect is the sound effect, such as the feedback sound when a key is pressed, or the sound effects accompanying changes in game content. Examples include ambient sound effects and surround sound effects.

[0075] The cloud server can determine the audio type and audio effect based on the pre-set audio type and audio effect corresponding to the control command.

[0076] In the cloud server, each cloud game has a corresponding table storing a mapping between control commands and sound effects. When a cloud game starts in the virtual machine, this mapping table is added to the game's running process. The server reads control commands received from the cloud server (specifically, the virtual machine within the server) in real time and determines the corresponding sound effects. The sound effects include the audio types and audio effects described above.

[0077] Specifically, determining the corresponding sound effect refers to determining the encoding of the sound effect. The encoding of the sound effect is uniquely represented in each cloud game.

[0078] Once the audio type and audio effect are determined in the cloud server, the first instruction information can be generated based on the audio type and audio effect.

[0079] Specifically, the first instruction information includes at least the encoding of the sound effect.

[0080] Please refer to Figure 3 As shown, the second transmission module 303 is used by the server to combine the first instruction information and the current game information to form a control instruction, and then send it to the client.

[0081] The first instruction information contains an audio effect encoding, which uniquely represents the type of audio or audio effect to be played.

[0082] The first instruction information and the current game information are spliced ​​together. In fact, it is necessary to match the sound effect with the current game information to achieve accurate timing of sound effect playback.

[0083] Figure 2 This is a flowchart of the instruction assembly process in this application.

[0084] Please refer to Figure 2 As shown, S201 calculates the splicing time.

[0085] The specific splicing time can be represented by the following expression:

[0086]

[0087] Among them, the This indicates the starting point when the sound effect is played, where 's' is a network quality parameter. Indicates the starting point of the sound effect, the The network latency indicating the reception of information, the This represents the network latency for sending information, where n represents the preset number of historical transmissions. The number of historical transmissions refers to the number of data transmissions within a set time period.

[0088] Please refer to Figure 2 As shown, S202 begins data splicing.

[0089] Specifically, the splicing includes at least encoding the sound effect, These are encoded into the first instruction information. The encoding can be formed by directly concatenating multiple character codes.

[0090] Please refer to Figure 3As shown, the sound effects processing module 304 is used by the client to read the current game information and the first instruction information from the control instructions, read the music corresponding to the local audio type and the audio effects transmitted from the server according to the first instruction information, and add the audio type and audio effects to the game playback according to the current game information.

[0091] The client receives the first instruction information sent by the server and reads the encoding of the sound effect from the first instruction information. Then determine the audio type and audio effect.

[0092] Furthermore, the client queries and retrieves the sound effect from the pre-stored sound effects according to the sound effect's encoding, and then executes playback.

[0093] Specifically, the sound effect playback process has already started. When the sound effect is selected, its playback is directly incorporated into the sound effect process, so the start time of the sound effect can be ignored.

[0094] On the other hand, after the client reads the sound effects of the current game, it activates the sound effects and, according to the... Select the starting point for playing the sound effect.

[0095] Finally, the sound effects are incorporated into the running game.

[0096] Preferably, the client is a mobile phone or a computer.

[0097] The above description is merely an example of the specific implementation process provided to illustrate the technical solution to be protected in this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can implement this application by different technical means under the guidance of the concept of this application. Therefore, this application is not limited to the specific embodiments below.

Claims

1. A method for saving audio bandwidth and improving sound quality in cloud gaming, characterized in that, include: Based on the connection relationship between the client and the server, the client obtains the current game information, generates the current control information based on the current game information, and sends it to the server. The server determines the audio type and audio effect that should be played based on the current control information, and generates first instruction information based on the audio type and audio effect; the audio file corresponding to the audio type is pre-stored completely in the client's local storage; The server combines the first instruction information and the current game information to form a control instruction, and then sends it to the client. The client reads the current game information and the first instruction information from the control commands, reads the audio file corresponding to the local audio type and the audio effect transmitted from the server according to the first instruction information, and dynamically adjusts the audio playback according to the audio effect, while also based on the start time point. Add the audio file and adjusted audio effects to the game playback; The splicing control instruction includes calculating the splicing time and encoding the audio effect and the splicing time into the first instruction information respectively; The expression for the splicing time is: ； in, This indicates the starting point for sound effect playback; 's' is a network quality parameter. Indicates the starting point of the sound effect. Indicates the network latency for receiving information. This represents the network latency for sending information, and n represents the preset number of historical transmissions.

2. The method for saving audio traffic and improving sound quality in cloud gaming according to claim 1, characterized in that, The audio type includes: the sound effect number.

3. The method for saving audio traffic and improving sound quality in cloud gaming according to claim 1, characterized in that, The audio effects include: ambient sound effects and surround sound effects.

4. The method for saving game audio bandwidth and improving sound quality according to any one of claims 1 to 3, characterized in that, The music corresponding to the audio type is stored in the client's memory.

5. The method for saving audio traffic and improving sound quality in cloud gaming according to claim 1, characterized in that, The client can be a mobile phone or a computer.

6. A device for saving audio bandwidth and improving sound quality in cloud gaming, characterized in that, include: The first transmission module is used to, based on the connection relationship between the client and the server, have the client obtain current game information, generate current control information based on the current game information, and send it to the server. The instruction generation module is used by the server to determine the audio type and audio effect that should be played based on the current control information, and to generate first instruction information based on the audio type and audio effect; the audio file corresponding to the audio type is pre-stored completely in the local storage of the client; The second transmission module is used by the server to combine the first instruction information and the current game information to form a control instruction, and then send it to the client. The sound effects processing module is used by the client to read the current game information and the first instruction information from the control commands, read the music corresponding to the local audio type and the audio effects transmitted from the server according to the first instruction information, dynamically adjust the audio playback according to the audio effects, and simultaneously adjust the playback based on the start time. Add the audio file and adjusted audio effects to the game playback; The splicing control instruction includes calculating the splicing time and encoding the audio effect and the splicing time into the first instruction information respectively; The expression for the splicing time is: ； in, This indicates the starting point for sound effect playback; 's' is a network quality parameter. Indicates the starting point of the sound effect. Indicates the network latency for receiving information. This represents the network latency for sending information, and n represents the preset number of historical transmissions.

7. The cloud gaming audio traffic saving and sound quality improvement device according to claim 6, characterized in that, The audio type includes: the sound effect number.

8. The cloud gaming audio traffic saving and sound quality improvement device according to claim 7, characterized in that, The audio effects include: ambient sound effects and surround sound effects.

9. The game audio data saving and sound quality improvement device according to any one of claims 6 to 8, characterized in that, The music corresponding to the audio type is stored in the client's memory.

10. The cloud gaming audio traffic saving and sound quality improvement device according to claim 6, characterized in that, The client can be a mobile phone or a computer.

Images