Data processing methods and apparatus, computer-readable storage media, terminals

Abstract

A data processing method, apparatus, computer-readable storage medium, and terminal are disclosed. The data processing method, executed by an edge server, includes: upon receiving a game launch request from a game client, creating a corresponding process for the game client; injecting a dynamic link library into the process and starting the dynamic link library to inject hook functions into the process; using the hook functions to perform hook operations on all system APIs used by the target game to take over the system APIs, including screen APIs and / or audio APIs; after taking over the screen APIs, collecting screen data and encoding the screen data to obtain encoded video data; after taking over the audio APIs, collecting audio data and encoding the audio data to obtain encoded audio data; and distributing the encoded video data and audio data to the game client. This solution can increase the number of concurrent paths in the game and reduce the cost per path.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more particularly to a data processing method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] As mobile games become increasingly complex, the performance of many users' devices (phones, TVs, etc.) is insufficient to meet the demands, resulting in stuttering and lag during gameplay. Serving games in the cloud is essentially the future trend.

[0003] However, current cloud gaming primarily utilizes virtual machines or Docker-like solutions, with Docker being an application container engine. However, current cloud gaming implementations using virtual machines or Docker solutions support a limited number of concurrent connections, and each connection incurs high costs. Summary of the Invention

[0004] The technical problem solved by the embodiments of the present invention is how to increase the number of concurrent paths in a game and reduce the cost per path.

[0005] To address the aforementioned technical problems, this invention provides a data processing method executed by an edge server. The method includes: upon receiving a game launch request from a game client, creating a corresponding process for the game client; injecting a dynamic link library into the process and starting the dynamic link library to inject hook functions into the process; using the hook functions to perform hook operations on all system APIs used by the target game to take over the system APIs, including a video API and / or an audio API; after taking over the video API, collecting video data and encoding the video data to obtain encoded video data; after taking over the audio API, collecting audio data and encoding the audio data to obtain encoded audio data; and distributing the encoded video data and the encoded audio data to the game client, so that the game client renders and presents the received encoded video data and encoded audio data.

[0006] Optionally, the system API also includes a control method API, and the method further includes: after taking over the control method API, obtaining the control instructions generated by the game client by triggering each control method, wherein the control method includes at least one of the following: keyboard, gamepad, mouse; controlling a pre-constructed virtual control method to perform corresponding operations according to the control instructions, and obtaining the screen data and the audio data in response to the operations performed by the virtual control method.

[0007] Optionally, the step of controlling the pre-constructed underlying virtual control mode to perform corresponding operations according to the control instructions includes: when the operating system used by the game client is different from the operating system of the edge server, mapping the key value corresponding to the system used by the game client to the key value corresponding to the operating system of the edge server according to the preset key value mapping information; and combining the key value mapped to the operating system of the edge server, controlling the virtual control mode to perform corresponding operations according to the control instructions.

[0008] Optionally, the data processing method further includes: after creating a corresponding process for the game client, creating a corresponding number and type of virtual control methods for the process according to the pre-configured control method of the target game; obtaining the user identifier of the game client, and binding the user identifier with the created virtual control methods.

[0009] Optionally, the data processing method further includes: when the target game supports multiplayer team play based on the game type of the target game, multiple game clients participating in the target game share the same process, and within the process, create a corresponding number and type of virtual control methods for each game client; for each game client, bind each user identifier to the corresponding virtual control method according to the user identifier corresponding to each game client.

[0010] Optionally, the data processing method further includes: according to the game type of the target game, when the target game supports watching live streams, distributing the encoded video data and the encoded audio data to game clients participating in watching the live stream.

[0011] Optionally, the system API also includes an archive API, and the method further includes: after taking over the archive API, isolating the archive path and locating the archive path to a specified location.

[0012] Optionally, locating the archive path to a specified location includes: locating the archive path to a NAS.

[0013] Optionally, encoding the image data to obtain encoded video data includes: encoding the acquired image data for each frame to obtain encoded video data.

[0014] Optionally, encoding the image data to obtain encoded video data includes: sharing GPU memory through a preset image API system library to convert multiple different types of image APIs into a specified type of image API; and encoding the image data using the rendering method corresponding to the specified type of image API to obtain the encoded video data.

[0015] Optionally, the step of audio encoding the audio data to obtain encoded audio data includes: converting the sound rendering type of the target game into a specified sound rendering type; and, based on the created sound cache channel environment, using the specified sound rendering type to encode the audio data to obtain the encoded audio data.

[0016] Optionally, distributing the encoded video data and the encoded audio data to the game client includes: obtaining the video decoding type supported by the game client, packaging and encapsulating the encoded video data using the video decoding type supported by the game client, and sending the encapsulated video data to the game client using the transmission protocol supported by the game client; obtaining the audio decoding type supported by the game client, packaging and encapsulating the encoded audio data using the audio decoding type supported by the game client, and sending the encapsulated audio data to the game client using the transmission protocol supported by the game client.

[0017] Optionally, the edge server is determined as follows: when the game client opens the target game, each candidate edge server sends test data to the game client and records the time when the test data arrives at the game client; based on the time when the test data arrives at the game client, the data transmission time of each candidate edge server in transmitting the test data to the game client is determined, and the candidate edge server with the shortest data transmission time is selected as the edge server corresponding to the game client.

[0018] Optionally, the data processing method further includes: periodically receiving data consumption messages from the game client; comparing the currently encoded frame number with the consumed frame number reported in the data consumption message; and determining the distribution of the encoded video data based on the comparison result.

[0019] Optionally, determining the distribution status of the video data based on the difference ratio result includes: when the difference ratio is greater than a set first threshold, pausing the distribution of the encoded video data to the game client until the difference ratio does not exceed the set first threshold, and then resuming the distribution of the encoded video data to the game client.

[0020] Optionally, the data processing method further includes: dynamically adjusting at least one of the encoding method, bit rate, and resolution of the video data when encoding the video data, based on the relationship between the difference ratio and a set second threshold.

[0021] This invention also provides a data processing apparatus, comprising: a process creation unit, configured to create a corresponding process for the game client when a game launch request is received from the game client; a process virtualization unit, configured to inject a dynamic link library into the process and start the dynamic link library to inject hook functions into the process, and to perform hook operations on all system APIs used by the target game through the hook functions to take over the system APIs, the system APIs including: a screen API and / or an audio API; a video encoding unit, configured to take over the screen API, collect screen data, and encode the screen data to obtain encoded video data; an audio encoding unit, configured to take over the audio API, collect audio data, and encode the audio data to obtain encoded audio data; and a distribution unit, configured to distribute the encoded video data and the encoded audio data to the game client, so that the game client renders and presents the received encoded video data and encoded audio data.

[0022] This invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of any of the above-described data processing methods.

[0023] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the above-described data processing methods when running the computer program.

[0024] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0025] Upon receiving a game launch request from a game client, a corresponding process is created for the game client. Simultaneously, a dynamic link library (DLL) is injected into this process and started, injecting hook functions into the process. These hook functions then perform hook operations on all system APIs used by the target game, effectively taking over the system APIs. After taking over the screen API, screen data is collected and encoded to obtain encoded video data. Similarly, after taking over the audio API, audio data is collected and encoded to obtain encoded audio data. The encoded video and audio data are then distributed to the game client, allowing the game client to render and present the received encoded video and audio data. On the edge server, by injecting hook functions into the process, system APIs can be hooked, making the target game believe it is running on a normal operating system (such as Windows OS). This allows the game to collect audio and / or screen data through system API hook operations, encode the collected audio and screen data, and transmit it to the game client. This effectively reduces performance overhead, with virtually no additional performance loss, thereby increasing the number of concurrent streams and reducing the cost per stream. Furthermore, the game client only needs to render and present the received encoded video and audio data; all game-related data processing is performed on the edge server. The game client essentially acts as a media player, further reducing the performance requirements for the game client's configuration. Attached Figure Description

[0026] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of the structure of a data processing system for implementing a data processing method according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a data processing device according to an embodiment of the present invention. Detailed Implementation

[0029] As mentioned above, current implementations using virtual machines or similar Docker solutions support a limited number of concurrent paths and have a high cost per path.

[0030] To address the aforementioned issues, in this embodiment of the invention, hook functions are injected into the process on an edge server. This allows for hooking operations on system APIs, making the target game believe it is running on a normal operating system (such as Windows OS). By hooking the system APIs, audio and / or video data from the game can be collected, encoded, and transmitted to the game client. This effectively reduces performance overhead, with virtually no additional performance loss, thereby increasing the game's concurrency and reducing the cost per thread. Furthermore, the game client only needs to render and present the received encoded video and audio data; all game-related data processing is performed on the edge server. The game client essentially acts as a media player, further reducing the performance requirements for the game client's configuration.

[0031] To make the above-mentioned objectives, features and beneficial effects of the embodiments of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0032] This invention provides a data processing method that can be executed by an edge server. In some non-limiting embodiments, the edge server can be an edge x86 server, which can also be called a CISC (Complex Instruction Set Computing) architecture server, commonly known as a PC server. It is a server based on the PC architecture, using Intel or other x86-compatible processor chips and the Windows operating system.

[0033] Reference Figure 1 The present invention provides a flowchart of a data processing method according to an embodiment of the invention, which may specifically include the following steps:

[0034] Step S11: When a game launch request is received from the game client, a corresponding process is created for the game client.

[0035] In practice, the game client can be a mobile phone, computer (PC), tablet, etc. Users can access the target game through the game's webpage URL, the game icon on the game client, or through a mini-program on the game client. After a user enters the target game through the game client, the game launch is triggered, and the game client sends a game launch request. Upon receiving the game launch request from the game client, a corresponding process is created for the game client. After creating the corresponding process, the game client can be bound to that process. Specifically, in some non-limiting embodiments, the game launch request may include a user identifier, and a binding between the user identifier and a thread can be established.

[0036] To further improve the timeliness of data transmission during gameplay and reduce data transmission latency, in some non-limiting embodiments, the edge server closest to the game client is selected as the edge server corresponding to the game client.

[0037] In some embodiments, the edge server corresponding to the game client can be determined as follows: When the game client is detected to have opened the target game, each candidate edge server can send test data to the game client, and the data transmission time of the test data sent by each candidate edge server to the game client is recorded. Based on the time it takes for the test data sent by each candidate edge server to reach the game client, the data transmission time of each candidate edge server in transmitting the test data to the game client is determined, and the candidate edge server with the shortest data transmission time is selected as the edge server corresponding to the game client. By using the data transmission time, the edge server closest to the game client can be selected, and the edge server closest to the game client can be used as the game client server. In this way, not only can data transmission latency be reduced and data transmission timeliness improved, but the user's (also known as the player's) gaming experience can also be enhanced.

[0038] To further improve the efficiency of server location for the game client and reduce overhead, in some non-limiting embodiments of this invention, the test data can be 1-byte text data to reduce network occupancy and network utilization during test data transmission. It is understood that other sizes or types of data can also be used as test data; examples will not be provided here.

[0039] To further improve the efficiency of determining the server corresponding to the game client, candidate edge servers for data transmission time testing can be selected based on the game client's location. For example, edge servers within a defined area of ​​the game client can be selected as candidate edge servers. That is, edge servers within the defined area of ​​the game client are selected as candidate edge servers, test data is sent to the game client, and the data transmission time is recorded. From the edge servers within the defined area of ​​the game client, the candidate edge server with the shortest data transmission time is selected as the edge server corresponding to the game client.

[0040] Step S12: Inject the dynamic link library into the process and start the dynamic link library to inject hook functions into the process. Through the hook functions, perform hook operations on all system APIs used by the target game to take over the system APIs.

[0041] In practical implementation, when creating a corresponding process for the game client, a Dynamic Link Library (DLL) can be injected into that process simultaneously, and the DLL can be started to inject hook functions into the process. By injecting the core code DLL into the process, the core functionality and the target game can run in the same process. The hook functions can then be used to hook and manipulate all the system application programming interfaces (APIs) used by the target game, thereby taking over the system APIs. After taking over the system APIs, it is possible to collect the game's audio and video data.

[0042] In some non-limiting embodiments, the Easyhook open-source library can be used to inject dynamic link libraries into the process.

[0043] In specific implementations, the system API may include a video API and / or an audio API.

[0044] Specifically, taking the Windows system as an example, hook functions utilize the event capture mechanism provided by the Windows system. After hooking the system API, the function address corresponding to the system API can be obtained, and then the corresponding data can be obtained through the function address corresponding to the system API.

[0045] Step S13: After taking over the screen API, screen data is collected and encoded to obtain encoded video data.

[0046] Since different game clients support different types of graphics APIs, in order to ensure compatibility with game clients on different operating systems, in some non-limiting embodiments, a preset graphics API system library is used to share GPU memory to convert various types of graphics APIs into a specified type of graphics API; the graphics data is then encoded using the rendering method corresponding to the specified type of graphics API to obtain the encoded video data.

[0047] For example, by sharing GPU memory through the DirectX 3D system library, the conversion between DirectX 3D, DirectX 3D 10, DirectX 3D 11 and DirectX 3D 12 can be achieved, converting various different types of graphics APIs into a specified type of graphics API. DirectX 3D 12 is used to construct the encoding context data, fully utilizing the efficient DirectX 3D 12 rendering method for video encoding, resulting in encoded video data.

[0048] In some non-limiting embodiments, low-latency encoding can be employed to improve the gaming experience. Specifically, for each frame of screen data acquired, the acquired screen data is encoded to obtain encoded video data. Specifically, for each frame of screen data input to the video encoder, the input screen data is encoded to obtain encoded video data.

[0049] Step S14: After taking over the audio API, collect audio data and encode the audio data to obtain encoded audio data.

[0050] Different games use different sound rendering types. Some games use multi-channel sound rendering, some use 64-bit floating-point (Float) type, some use integer (INT) type, and some use short integer (Short) type. Different types of game clients support different sound rendering types. If a game client does not support a sound rendering type, it may cause sound playback failure. For example, some older versions of Android are not compatible with 64-bit Float type sound playback.

[0051] To ensure compatibility with different types of game clients, in some non-limiting embodiments, the sound rendering type of the target game is converted to a specified sound rendering type; based on the created sound cache channel environment, the audio data is encoded using the specified sound rendering type to obtain the encoded audio data.

[0052] For example, if the sound rendering type is specified as Short, and the target game's sound rendering type is not Short, it will be converted to Short. The created game sound channel caching environment can be a 16-bit channel cache.

[0053] Step S15: Distribute the encoded video data and the encoded audio data to the game client.

[0054] In practice, when distributing the encoded video and audio data to the game client, frame synchronization can be used to ensure synchronization between the video and audio data. The game client renders and presents the received encoded video and audio data. The edge server provides a start marker for each frame during transmission, and the process of detecting and obtaining this marker at the game client receiving end is called frame synchronization.

[0055] In some non-limiting embodiments, step S15 above can be implemented as follows: obtaining the video decoding type supported by the game client, packaging and encapsulating the encoded video data using the video decoding type supported by the game client, and sending the encapsulated video data to the game client using the transmission protocol supported by the game client. Similarly, obtaining the audio decoding type supported by the game client, packaging and encapsulating the encoded audio data using the audio decoding type supported by the game client, and sending the encapsulated audio data to the game client using the transmission protocol supported by the game client.

[0056] In practice, a connection can be established with the game client when the game client is detected entering the target game. During or after the connection is established, the edge server can transmit data with the game client and obtain information such as the video decoding type, audio decoding type, and transmission protocol supported by the game client.

[0057] In practical applications, when using low-latency encoding and transmission, network jitter and other factors may cause game clients to be unable to consume the distributed video data in time, resulting in latency. To address this issue, in a non-limiting embodiment of the present invention, data consumption messages from game clients can be received periodically, and the number of currently encoded frames can be compared with the number of consumed frames reported in the data consumption messages; the distribution status of the encoded video data can be determined based on the comparison result.

[0058] Specifically, if the difference ratio is greater than a set first threshold, the distribution of the encoded video data to the game client is paused. The system continues to receive data consumption messages from the game client in the next cycle, and calculates the difference ratio based on the number of frames consumed indicated in the data consumption message from the game client in the next cycle, until the difference ratio does not exceed the set first threshold. Then, the distribution of the encoded video data to the game client resumes. If the difference ratio is less than the set first threshold, the distribution of the encoded video data to the game client continues.

[0059] For example, after each preset frame of screen display, that is, after consuming the video and audio data of each preset frame, the game client sends back a data consumption message. The data consumption message provides feedback on the game client's consumption of the video and audio data of the preset frames. Based on the data consumption message, it can be determined which frame the game client has consumed. The currently encoded frame number is then compared with the frame number consumed by the game client as reported in the data consumption message. This comparison involves calculating the difference between the currently encoded frame number and the frame number consumed by the game client as reported in the data consumption message. The distribution of the encoded video data is determined based on the comparison result.

[0060] The difference ratio can be selected as 3 frames, 4 frames, or other data, which can be configured according to the acceptable level of latency.

[0061] In some non-limiting embodiments, at least one of the encoding method, bitrate, and resolution of the video data can be dynamically adjusted during encoding based on the relationship between the difference ratio and a set second threshold. If the difference ratio is greater than the second threshold, it is determined that the network status of the game client is poor. The bitrate and resolution can be reduced, and an encoding type that consumes fewer resources during decoding can be used to improve the game playback smoothness and enhance the gaming experience. This helps to achieve the lowest transmission latency under the current network environment, resulting in high-definition and smooth gameplay.

[0062] Using the above scheme, when a game launch request is received from a game client, a corresponding process is created for the game client. Simultaneously, a dynamic link library is injected into this process and started, injecting hook functions into the process. These hook functions then perform hook operations on all system APIs used by the target game, thereby taking over the system APIs. After taking over the screen API, screen data is collected and encoded to obtain encoded video data. After taking over the audio API, audio data is collected and encoded to obtain encoded audio data. The encoded video data and encoded audio data are then distributed to the game client, allowing the game client to render and present the game based on the received encoded video and audio data. On the edge server, by injecting hook functions into the process, system APIs can be hooked, making the target game believe it is running on a normal operating system (such as Windows). By hooking the system APIs, audio and / or video data from the game can be collected, encoded, and transmitted to the game client. This effectively reduces performance overhead, with virtually no additional performance loss, thereby increasing the game's concurrency and reducing the cost per thread. Furthermore, the game client only needs to render and present the received encoded video and audio data; all game-related data processing is implemented on the edge server. The game client essentially acts as a media player, requiring only the ability to play videos and control the game, thus further reducing the performance requirements for the game client.

[0063] Furthermore, on the edge server, by injecting hook functions into the process, hook operations can be performed on the system API, making the target game believe that it is running on a normal operating system (such as Windows OS). By performing hook operations on the system API, the game's audio data and / or screen data can be collected without the need for GPU and CPU overhead beyond virtualization. With the same configuration as the virtual machine solution, the number of concurrent paths is higher and the cost per path is lower.

[0064] Furthermore, on the edge server, by injecting hook functions into the process, hook operations can be performed on the system API, which can realize API virtualization. Since the API virtualization and the game are in the same process, the underlying rendering logic of the game GPU can be optimized for different game engines, thereby reducing the GPU resources consumed per path.

[0065] Furthermore, the system API also includes an archive API. The archive API provides an archive service to save user game progress and related data. By taking over the archive API, the archive path can be isolated and its location can be specified.

[0066] Furthermore, the archive path is located to Network Attached Storage (NAS). NAS refers to a device connected to a network that has data storage capabilities; it is also called network storage and is a type of dedicated data storage server. NAS is data-centric, completely separating storage devices from servers, centrally managing data, thereby freeing up bandwidth, improving performance, and reducing total cost of ownership. The archive API can include APIs that save files (CreateFile) and APIs that write to the registry (RegCreate). This facilitates the deployment of a game package, enabling cross-platform, cross-system, and cross-terminal operation.

[0067] Furthermore, the system API also includes a control method API. The control method API provides interface services for control methods such as keyboard, gamepad, and mouse. After taking over the control method API, the system obtains the control commands generated by the game client through triggering various control methods, including at least one of the following: keyboard, gamepad, and mouse. Based on the control commands, the system controls a pre-constructed virtual control method to execute corresponding operations. In response to the operations executed by the virtual control method, the system obtains the screen data and the audio data. Hooking the control method API facilitates the customization of game control methods for different terminals.

[0068] Furthermore, the step of controlling the pre-constructed underlying virtual control mode to perform corresponding operations according to the control instructions can be implemented in the following way: when the operating system used by the game client is different from the operating system of the edge server, the key value corresponding to the system used by the game client is mapped to the key value corresponding to the operating system of the edge server according to the preset key value mapping information; combined with the key value mapped to the operating system of the edge server, the virtual control mode is controlled to perform corresponding operations according to the control instructions.

[0069] In practice, after creating a corresponding process for the game client, a corresponding number and type of virtual control methods can be created for the process based on the pre-configured control methods of the target game. The user identifier of the game client is obtained, and the user identifier is bound to the created virtual control methods. This prevents cross-contamination and mismatch of control methods between different processes or different games, ensuring the normal operation of the target game.

[0070] In specific implementation, when the target game supports multiplayer team play based on its game type, multiple game clients participating in the target game share the same process. Within this process, a corresponding number and type of virtual control methods are created for each game client. For each game client, each user identifier is bound to its corresponding virtual control method based on its user identifier. Video and audio data generated within the same process are distributed to the game clients within that process. Alternatively, depending on the actual scenario requirements, corresponding video and audio data are generated in response to different game clients, and then distributed to the corresponding game clients.

[0071] In practice, depending on the game type of the target game, when the target game supports live streaming, the encoded video data and encoded audio data are distributed to the game clients participating in the live stream viewing. Specifically, the user identifier corresponding to the game client watching the live stream can be bound to an established thread to facilitate the subsequent sending of video and audio data to the game client watching the live stream.

[0072] To facilitate a better understanding of the embodiments of the present invention by those skilled in the art, please refer to Figure 2 A schematic diagram of the structure of a data processing system for implementing data processing methods is given below. Figure 2 The data processing methods are explained.

[0073] In practical implementation, the data processing system may include several edge servers 100 and several game clients 500. Figure 2 The illustration uses one edge server 100 and one game client 500 as an example. Those skilled in the art will understand that the number of edge servers 100 and game clients 500 can also be other values. Figure 2 The number of edge servers 100 and game clients 500 illustrated in the diagram do not affect the scope of protection of this invention. The edge servers 100 can serve as the execution entity for the data processing method provided in the above embodiments and are used to implement the data processing method provided in the above embodiments.

[0074] The edge server 100 may include: a game storage unit 10, an environment initialization unit 20, a game process virtualization service unit 30, and a distribution unit 40.

[0075] The game storage unit 10 is used to store game-related data, which may include the game package. The game storage unit 10 may include: a package file quantity and size calculation module 11, a network environment intelligent verification module 12, a game packaging and transmission module 13, and an edge server speed testing module 14. Specifically, the game developer (CP) packages the game into an .exe format (Windows version) and uploads it to the edge server through the game storage unit. The edge server can be an edge x86 server.

[0076] The file quantity and size calculation module 11 is used to verify the integrity and security of the game package. Typically, the game package needs to be encrypted when uploaded to the edge server 100. The file quantity and size calculation module 11 can decrypt the encrypted game package (e.g., by performing MD5 hashes) to verify its integrity and security. In this embodiment of the invention, the game package can be verified by the number and size of files, ensuring necessary verification while achieving minimal performance in verifying the integrity of the game package.

[0077] The network environment intelligent verification module 12 establishes a transmission link test with the environment initialization unit 20 to ensure a normal connection. This is also a prerequisite for ensuring the integrity of the game package. If a poor network is detected, the user is reminded to check the network.

[0078] The game package transmission 13 can securely transmit the game package to the environment initialization unit 20 via the FTP protocol (or other supported transmission protocols). After waiting for the return of a success notification, the game can be tested and started.

[0079] The edge server speed testing module 14 can be used to find the edge server closest to the user (i.e., the game client). For example, the game client obtains 1 byte of text data from candidate edge servers, records the arrival time of each edge server, and selects the edge server with the shortest arrival time as the game server for the game client. This minimizes the latency of the user's gaming experience and effectively reduces bandwidth utilization. This edge server speed testing module 14 can be used in the above data processing method to allocate the corresponding edge server to the game client.

[0080] The environment initialization unit 20 is used for initializing, managing, and statistically analyzing resources such as the central processing unit (CPU) and graphics processing unit (GPU) of the edge server, and for dynamically receiving game uploads and uninstalls. It is an essential initialization task for the game process virtualization service unit 30. The environment initialization unit 20 may include a game package file decompression and verification module 21, a server resource statistics module 22, a DLL injection game module 23, a process virtualization basic library initialization module 24, and a game virtualization configuration generation module 25.

[0081] The game package file decompression and verification module 21 is used to ensure that the game is complete and runnable by calculating and verifying the number and size of the game package when it receives the instruction of the game package from the game storage unit 10.

[0082] The server resource statistics module 22 is used to count the current proportion of CPU, GPU and memory resources of the server before the game is started. After the game is started, if a performance test can be performed using a smart testing tool like Monkey, it will record the highest value of CPU, GPU or memory proportion after running for a certain period of time. The highest value of proportion is subtracted from the value counted before the game was started to obtain the resource consumption result of the uploaded game. Then the resource consumption result is saved to the game virtualization configuration generation module 25.

[0083] DLL injection into the game module 23 can utilize the Easyhook open-source library to inject the core code DLL into the thread, so that our core functions and the game run in the same process. Through the hook function injected into the thread, we can collect data on game sound and screen, change the game control method, etc.

[0084] The process virtualization base library initialization module 24 is mainly a library outside of HOOK, used to ensure the operation of subsequent programs, such as ensuring that services are always alive and checking whether each library has been initialized.

[0085] The game virtualization configuration generation module 25 notifies testers of the type of graphics rendering used by a new game, specifically the graphics API used, which can include Direct3D9, DirectX3D10, DirectX3D11, DirectX3D12, Vulkan, and OpenGL. It can also determine the keyboard and gamepad connection method (DirectInput, XInput, RawInput, or HID USB devices). Furthermore, it can determine whether the game is stored via files or the registry. Finally, the obtained graphics API type, keyboard and gamepad connection method, and game storage method are written into the game virtualization configuration for service updates by the game process virtualization service unit 30.

[0086] The game process virtualization service unit 30 is the core module of the entire edge server. It is the operating system (OS) that runs the game. All aspects of the game, including screen capture, sound capture, save data, and input control, are completed through the game process virtualization service unit 30. The game process virtualization service unit 30 may include a Windows underlying API virtualization isolation subunit 31, a game screen sharing video encoding subunit 32, an audio and video encoding subunit 33, and a game controller and keyboard virtualization subunit 34.

[0087] The Windows underlying API virtualization isolation subunit 31 primarily utilizes the Easyhook open-source library to hook all system APIs used by the target game, thereby taking over all system APIs used by the target game and making the game believe it is running on a normal Windows OS. The advantage of this approach is minimal performance overhead, with virtually no additional overhead. The Windows underlying API virtualization isolation subunit 31 may include a game control virtualization module 311, a game graphics rendering virtualization module 312, a game sound virtualization module 313, a game save virtualization module 314, and a game process multi-instance restriction virtualization module 315.

[0088] The game control virtualization module 311 is used to virtualize and implement game control methods. Game control methods can include gamepads, keyboards, mice, etc. Corresponding control method API (direct API) types can include: DirectInput API, XInput API, RawInput API, Human Interface Device (HID) API, and JoyStick API, etc. After taking over these control method APIs, the module simulates their implementations.

[0089] The game rendering virtualization module 312 can be used to hook the graphics API and, after taking over the graphics API, obtain graphics data. Specifically, by hooking the graphics API, GPU function addresses can be obtained, and graphics data can be obtained based on the GPU function addresses. Graphics APIs (also known as image APIs) mainly include the following types: DirectX 3D9, DirectX 3D10, DirectX 3D11, DirectX 3D12, Vulkan, and OpenGL, etc. After taking over these graphics APIs, the graphics can be redirected to video encoding, such as redirecting to a video encoder for video encoding, instead of being output to the screen. The game rendering virtualization module 312 can be used to implement some of the steps in step S13 above.

[0090] The game sound virtualization module 313 is used to hook the audio API. After taking over the audio API, audio data can be captured. Audio can be output simply by taking over the Windows Audio Session API. The audio data can then be encoded, and the audio encoding type can include Advanced Audio Coding (AAC), etc.

[0091] The game save virtualization module 314 is used to virtualize the game save API and isolate the save path. The save API can include APIs that save files (CreateFile) and APIs that write to the registry (RegCreate). By hooking and taking over the save API, the save path can be isolated, and shared storage can be used to store these save files and data. Typically, a game requires tens of gigabytes (GB) of storage; the save virtualization can be located on a NAS to share these files. By hooking the save API, data sharing can be achieved, allowing users to quickly access the latest save data when their location changes. Furthermore, when some games support team play, because a game server shares a single Windows system and shared storage, the game save system does not need to be migrated, facilitating data updates.

[0092] The game process multi-instance restriction virtualization module 315 can be used to determine whether the corresponding process already exists to prevent duplicate openings. APIs affecting multi-instance operations include CreateMutant and GetProcessImageFileNameA, etc., and the multi-instance restriction can be selected and configured according to the type of game that restricts multi-instance operations.

[0093] The game screen shared video encoding subunit 32 can be used for video encoding of screen data. Since different game screen rendering methods are used, the video encoding method can be converted to a unified video encoding type. In specific implementation, the process of encoding the game GPU screen into hardware varies depending on the type of screen API. Specifically:

[0094] The first method: Map the GPU memory from the graphics APIs such as DirectX3D9, DirectX3D10, DirectX3D11, and DirectX12 directly to the hardware encoding input data memory of the control panel through the DirectX3D system library. The control panel can be the NVIDIA control panel.

[0095] The second method: When the control panel encoding does not support the game's extension of graphics APIs such as DirectX3D11 and DirectX12, it is necessary to convert the GPU memory of DirectX3D11 and DirectX12 graphics APIs to the GPU memory of DirectX3D10 through shared memory, and then map the DirectX3D10 GPU to the control panel hardware encoding input data memory.

[0096] The third method: The OpenGL graphics API is mapped from GPU memory to the control panel hardware encoding input data memory.

[0097] The fourth method: Vulkan screen API maps from GPU memory to control panel hardware encoding.

[0098] The game screen sharing video encoding subunit 32 may include a game rendering screen capture shared GPU memory module 321, an encoding thread parameter environment construction module 322, an encoding sharing environment creation module 323, and a game shared GPU memory conversion and synchronous encoding module 324.

[0099] Among them, the game rendering screen capture shared GPU memory module 321 obtains the screen GPU memory environment context from screen APIs of various types such as DirectX3D9, DirectX3D10, DirectX3D11, DirectX3D12, OpenGL and Vulkan, and copies it to the encoding thread parameter environment construction module 322.

[0100] The encoding thread parameter environment construction module 322 initializes the game screen GPU storage context for graphics APIs such as DirectX3D9, DirectX3D10, DirectX3D11, DirectX3D12, OpenGL, and Vulkan, and creates the relevant API GPU memory according to the size and color format specified in the configuration file.

[0101] When the control panel encoding does not support extending the DirectX3D11 and DirectX12 APIs for the game part, the DirectX3D11 and DirectX12 GPU memory obtained from the encoding thread parameter environment construction module 322 is converted into DirectX3D10 GPU memory through shared memory and mapped to the control panel hardware encoding input data memory.

[0102] Shared memory is also created according to the size and color format specified in the configuration file for the relevant API GPU shared memory.

[0103] The game shares GPU memory for conversion and synchronous encoding module 324. The final video encoding data is transmitted from the encoding thread parameter environment construction module 322 and the encoding shared environment creation module 323. This synchronization method adopts a multi-threaded synchronization method, that is, the acquisition and generation of GPU memory and the consumption of GPU memory to encoding are two threads running in parallel. This parallel collaboration method has lower latency, that is, video encoding will not block the game's operation, and the game will not affect video encoding. Finally, the control panel API hardware encoding is called to perform H264 or H265 encoding.

[0104] The audio / video encoding subunit 33 may include a game sound normalization encoding module 331 and a video low-latency encoding module 332. Different games typically use different sound rendering types; some games use multi-channel sound rendering, some use 64-bit Float, some use INT, and some use Short. However, different game clients support different sound rendering types, which may affect sound playback. For example, some older Android versions are incompatible with Float sound playback. To ensure compatibility with different game clients, the game sound normalization encoding module 331 can convert various sound rendering types to a specified sound rendering type, such as converting to Short sound rendering, so that various types of game clients can support it and play sound correctly. Specifically, the game sound normalization encoding module 331 can create a game sound channel buffer environment, such as a 16-bit sound channel buffer environment. In some non-limiting embodiments, the transcoding algorithm used is audio=audio. 32768, where audio is audio data.

[0105] The low-latency video encoding module 332 is used to implement low-latency video encoding to improve user experience. For example, if the video stream uses H.264 encoding, primarily 720P / 30fps and 1080P / 60fps, a container format can be defined to encapsulate the raw H.264 and AAC streams with control flow data to form low-latency transmission data. Low-latency encoding uses B-frames and sets the GOP to -1 to ensure that only the first few frames are I-frames, thus reducing the amount of data transmitted. This ensures that the encoder immediately outputs the encoded data for each frame, eliminating the need for encoder buffered frame data. Bitrate control is also implemented, as games often have periods of static screen time with very low bitrates; the bitrate can be adaptively adjusted. The encoded video and audio data are then transmitted to the distribution unit 40 via the audio / video encoding subunit 33. A GOP (Group of Pictures) is a series of consecutive frames consisting of one I-frame and several B / P frames. It is the basic unit accessed by video image encoders and decoders, and its order will be repeated until the end of the video. I-frames are internally coded frames (also known as keyframes), P-frames are forward-predicted frames (forward reference frames), and B-frames are bidirectional interpolated frames (bidirectional reference frames).

[0106] The gamepad and keyboard virtualization subunit 34 can be used to construct the underlying virtual driver for gamepads, solving the problem of different mouse and gamepad implementations for each game, which necessitates adapting to many different gamepads. Simulating the gamepad from the underlying driver fundamentally ensures the uniformity of gamepads. The gamepad and keyboard virtualization subunit 34 may include a specific gamepad and keyboard driver data module 341, a virtualized gamepad and keyboard Win32 underlying interface module 342, and a cross-platform control key mapping module 343.

[0107] The specific gamepad and keyboard driver data module 341 can register a corresponding number of gamepads and bind user identifiers and game IDs when the game starts, so as to realize the binding between the user and the gamepad, thus preventing the gamepad from being cross-connected with other games.

[0108] The Win32 low-level interface module 342 for virtualized gamepads and keyboards can implement a standard Win32 gamepad driver, including registration, creation, enumeration, and acquisition of gamepad data. Here, Win32 refers to the 32-bit environment of the Microsoft Windows operating system; this explanation uses Win32 as an example, but it can also be understood as Win64, etc.

[0109] Cross-platform control key value mapping module 343 This module receives key value information from different operating systems passed by the user and converts it into the corresponding key value for Windows, thus realizing the adaptation of input control methods to multiple platforms and different operating systems.

[0110] Through the cooperation of the environment initialization unit 20 and the game process virtualization service unit 30, the above steps S11 to S13 can be achieved.

[0111] The distribution unit 40 is used for distributing video and audio data, game control events, and game save data. The distribution unit 40 may include a game control event distribution module 41, a game video distribution module 42, a game audio distribution module 43, a game save data distribution module 44, and a game multi-channel transmission service management module 45. To improve latency, this embodiment of the invention creates a separate service to handle the transmission of various data, divided into a receiving service thread and a sending service thread. Specifically, when a game control event command is received, the game control event distribution module 41 is triggered. This module receives data, parses out keyboard or gamepad messages, and sends the parsed keyboard or gamepad message data to the game controller and keyboard virtualization subunit 34. When audio data is received from the audio / video encoding subunit 33, the game audio distribution module 43 is triggered. This module selects the container type supported by the game client 500 (such as mp3, AAC, etc.) according to the decoding capability of the game client 500, packages and encapsulates the audio data, and sends it to the game client 500. When video data is received from the audio / video encoding subunit 33, the game video distribution module 42 is triggered. This module, based on the network protocols supported by the game client 500, including Transmission Control Protocol (TCP), User Datagram Protocol (UDP), UDP-based Data Transfer Protocol (UTT), WebSocket (a protocol for full-duplex communication over a single TCP connection), and Web Real-Time Communication (WebRTC), sends the video data to the game client 500 using the network protocols supported by the game client 500. The game control event distribution module 41, in conjunction with the game control virtualization module 311, can take over the control method API in the above data control method, and obtain the control commands generated by the game client through triggering various control methods. The control methods include at least one of the following: keyboard, gamepad, and mouse; and control the pre-constructed virtual control method to perform corresponding operations according to the control commands.

[0112] In some embodiments, since zero-latency transmission is used when transmitting audio and video data, when network jitter is caused by factors such as the distribution unit 40 sending data (video or audio data) too quickly, the game client 500 may not be able to consume it in time, resulting in a large delay and affecting the user experience.

[0113] In some non-limiting embodiments, a video CS synchronization algorithm submodule 421 can be designed to solve the synchronization problem between the edge server 100 and the game client 500, where C stands for Client, referring to the game client 500; and S stands for Server, referring to the edge server 100. Specifically, after the game client 500 completes the display of a preset frame, it sends a data consumption message to the distribution unit 40. The distribution unit 40 can determine which frame the game client 500 has consumed based on the data consumption message, and compares the number of frames currently encoded by the distribution unit 40 with the number of frames consumed by the game client 500 as indicated in the data consumption message. If the difference does not exceed a set first threshold, the next frame is transmitted to the game client 500. Otherwise, if the difference exceeds the set first threshold, the distribution unit waits for the data consumption message from the game client 500 until the number of frames consumed by the game client 500 catches up. Therefore, when network jitter occurs, the distribution unit 40 can promptly detect and stop distributing audio and video data to the game client 500, and resume sending audio and video data after the network jitter is eliminated, so as to improve the user's gaming experience.

[0114] The game multi-channel transmission service management module 45 manages multi-channel audio and video transmission. Each audio and video channel is transmitted in an independent thread, without interference. The audio and video data comes from the game video distribution module 42 and the game audio distribution module 43. In some embodiments, when the target game supports multiplayer team play based on its game type, since multiple game clients participating in the target game share the same process, audio and video data are sent to multiple game clients within the same thread. In other embodiments, when the target game supports live streaming, the encoded video data and encoded audio data are distributed to game clients participating in the live streaming within the same process.

[0115] The aforementioned distribution unit 40 can be used to implement the aforementioned step S15.

[0116] The game client 500 provides users with the ability to operate the game and connect to the edge server 100, as well as render and display the corresponding screen and sound based on the audio and video data output by the edge server 100. The game client 500 may include an audio and video decoding module 51, an audio and video rendering module 52, a QoS module 53, and a control module 54.

[0117] When audio and video data (audio data and video data) distributed by the edge server 100 through the distribution unit 40 is received, the audio and video decoding module 51 is triggered. In some embodiments, the game client 500 uses low-latency decoding. The decoding of the game client 500 can be adapted to both software and hardware decoding depending on the device model of the game client 500.

[0118] Hardware decoding adaptation refers to interfacing with chip manufacturers. Since chip manufacturers provide dedicated low-latency interfaces, the buffering process can be eliminated to directly output the image. Low-latency decoding of audio and video is achieved through the low-latency interface built into the chip in the game client 500.

[0119] Software decoding refers to using software decoding to support low-latency output, such as using GPU for acceleration.

[0120] The audio / video rendering module 52 performs YUV to OpenGL GPU rendering based on the data decoded by the audio / video decoding module 51. Compared to the traditional YUV to RGB rendering, this RGB to GPU rendering method simplifies the rendering process by eliminating the YUV to RGB conversion process. OpenGL is a GPU function calling specification that defines a series of APIs for manipulating graphics and images. GPU hardware developers need to provide implementations that comply with the OpenGL specification. These implementations are usually called "drivers," and they are responsible for translating the API commands defined by OpenGL into GPU instructions. YUV is a color encoding method. Y represents luminance (or luma), which is the grayscale value, while U and V represent chrominance (or chroma), which describes the color and saturation of an image and is used to specify the color of a pixel. The RGB color mode is an industry color standard that obtains various colors by varying the red (R), green (G), and blue (B) color channels and superimposing them. RGB represents the colors of the red, green, and blue channels.

[0121] The Quality of Service (QOS) module 53 is used to dynamically adjust the bit rate, resolution, and bitrate when the network deteriorates or improves, in order to increase the smoothness for players.

[0122] The control module 54 is used to receive operation data from the user's controller, mouse, or remote control. The control module 54 can send data such as received game control event instructions to the game control event distribution module 41 in the distribution unit 40.

[0123] Among them, the above Figure 2 The data processing system illustrated is merely one implementation method for the data processing method. Based on this, those skilled in the art can make corresponding modifications, such as splitting or merging units (or modules), and changing the names of units or modules. It should be understood that as long as the functions and effects described in the above embodiments are achieved, they all fall within the protection scope of this invention. In specific implementations, the specific workflow and principles of the data processing system can be found in the description of the data processing method in the above embodiments. Correspondingly, the specific workflow and principles of the above data processing method can also be found in the description of the data processing system terminal in the above embodiments.

[0124] This invention also provides a data processing apparatus, with reference to... Figure 3 The provided schematic diagram illustrates the structure of a data processing apparatus according to an embodiment of the present invention. The data processing apparatus 600 may include:

[0125] The process creation unit 61 is used to create a corresponding process for the game client when a game start request is received from the game client.

[0126] The process virtualization unit 62 is used to inject a dynamic link library into the process and start the dynamic link library to inject hook functions into the process. The hook functions are used to perform hook operations on all system APIs used by the target game to take over the system APIs. The system APIs include: screen APIs and / or audio APIs.

[0127] The video encoding unit 63 is used to take over the screen API, collect screen data, and encode the screen data to obtain encoded video data.

[0128] The audio encoding unit 64 is used to take over the audio API, collect audio data, and encode the audio data to obtain encoded audio data.

[0129] The distribution unit 65 is used to distribute the encoded video data and the encoded audio data to the game client, so that the game client renders and presents the received encoded video data and encoded audio data.

[0130] In specific implementation, the specific working principle and workflow of the data processing device 600 can be found in the description of the data processing method or data processing system provided in the above embodiments, and will not be repeated here.

[0131] This invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of any of the above-described data processing methods.

[0132] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the above-described data processing methods when running the computer program.

[0133] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in any computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0134] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A data processing method, characterized in that, The data processing method, executed by an edge server, includes: When a game launch request is received from a game client, a corresponding process is created for the game client; The dynamic link library is injected into the process and started to inject hook functions into the process. The hook functions perform hook operations on all system APIs used by the target game to take over the system APIs, so that the target game thinks it is running on a normal operating system. The system APIs include: screen API and / or audio API. After taking over the video API, video data is collected and encoded to obtain encoded video data. After taking over the audio API, audio data is collected and the audio data is encoded to obtain encoded audio data; The encoded video data and the encoded audio data are distributed to the game client, so that the game client renders and presents the received encoded video data and the encoded audio data. The system API also includes a control method API, and the method further includes: After taking over the control method API, the control commands generated by the game client by triggering each control method are obtained. The control methods include at least one of the following: keyboard, gamepad, and mouse. The control commands control a pre-constructed virtual control mode to perform corresponding operations, and in response to the operations performed by the virtual control mode, the image data and the audio data are obtained. The step of controlling the pre-constructed underlying virtual control method to perform corresponding operations according to the control command includes: When the operating system used by the game client is different from the operating system of the edge server, the key value corresponding to the system used by the game client is mapped to the key value corresponding to the operating system of the edge server according to the preset key value mapping information. By combining the key value corresponding to the operating system mapped to the edge server, the virtual control method is controlled to perform corresponding operations according to the control instructions; It also includes: after creating a corresponding process for the game client, creating a corresponding number and type of virtual control methods for the process according to the pre-configured control method of the target game; Obtain the user identifier of the game client and bind the user identifier to the created virtual control method; It also includes: when the target game supports multiplayer team play based on its game type, multiple game clients participating in the target game share the same process; within the process, a corresponding number and type of virtual control methods are created for each game client; for each game client, each user identifier is bound to the corresponding virtual control method based on the user identifier corresponding to each game client; video and audio data generated by the same process are distributed to the game clients of that process respectively; or, according to actual scenario requirements, corresponding video and audio data are generated in response to different game clients, and the generated video and audio data are distributed to the corresponding game clients.

2. The data processing method as described in claim 1, characterized in that, Also includes: Depending on the game type of the target game, when the target game supports live streaming, the encoded video data and the encoded audio data are distributed to the game clients participating in watching the live stream.

3. The data processing method as described in claim 1, characterized in that, The system API also includes an archive API, and the method further includes: After taking over the archive API, the archive path is isolated and the archive path is located to a specified location.

4. The data processing method as described in claim 3, characterized in that, The step of locating the archive path to a specified location includes: locating the archive path to a NAS.

5. The data processing method as described in claim 1, characterized in that, The process of encoding the image data to obtain encoded video data includes: Each frame of video data is acquired and then encoded to obtain encoded video data.

6. The data processing method as described in claim 1, characterized in that, After taking over the video API, the system collects video data and encodes the video data to obtain encoded video data, including: By sharing GPU memory through a pre-defined graphics API system library, various types of graphics APIs can be converted into a specified type of graphics API. The image data is encoded using the rendering method corresponding to the specified image API type to obtain the encoded video data.

7. The data processing method as described in claim 1, characterized in that, The step of encoding the audio data to obtain encoded audio data includes: Convert the target game's sound rendering type to the specified sound rendering type; Based on the created sound cache channel environment, the audio data is encoded using the specified sound rendering type to obtain the encoded audio data.

8. The data processing method as described in claim 1, characterized in that, The step of distributing the encoded video data and the encoded audio data to the game client includes: Obtain the video decoding type supported by the game client, package and encapsulate the encoded video data using the video decoding type supported by the game client, and send the encapsulated video data to the game client using the transmission protocol supported by the game client; Obtain the audio decoding type supported by the game client, package and encapsulate the encoded audio data using the audio decoding type supported by the game client, and send the encapsulated audio data to the game client using the transmission protocol supported by the game client.

9. The data processing method according to any one of claims 1 to 8, characterized in that, The edge server is determined in the following manner: When the game client is detected to have opened the target game, each candidate edge server sends test data to the game client and records the time when the test data sent by each candidate edge server reaches the game client; Based on the time it takes for the test data sent by each candidate edge server to reach the game client, the data transmission time of each candidate edge server to transmit the test data to the game client is determined, and the candidate edge server with the shortest data transmission time is selected as the edge server corresponding to the game client.

10. The data processing method according to any one of claims 1 to 8, characterized in that, Also includes: Periodically receive data consumption messages from the game client; Compare the number of frames currently encoded with the number of frames consumed as reported in the data consumption message; The distribution of the encoded video data is determined based on the difference ratio result.

11. The data processing method as described in claim 10, characterized in that, Determining the distribution of the video data based on the difference ratio result includes: When the difference ratio is greater than a set first threshold, the distribution of the encoded video data to the game client is paused until the difference ratio does not exceed the set first threshold, and then the distribution of the encoded video data to the game client is resumed.

12. The data processing method as described in claim 10, characterized in that, Also includes: Based on the relationship between the difference ratio and a set second threshold, when encoding the video data, at least one of the encoding method, bit rate, and resolution of the video data is dynamically adjusted.

13. A data processing apparatus, characterized in that, include: The process creation unit is used to create a corresponding process for the game client when a game launch request is received from the game client. A process virtualization unit is used to inject a dynamic link library into the process and start the dynamic link library to inject hook functions into the process. The hook functions perform hook operations on all system APIs used by the target game to take over the system APIs, so that the target game believes that it is running on a normal operating system. The system APIs include: screen APIs and / or audio APIs. The video encoding unit is used to take over the screen API, collect screen data, and encode the screen data to obtain encoded video data. The audio encoding unit is used to take over the audio API, collect audio data, and encode the audio data to obtain encoded audio data. A distribution unit is configured to distribute the encoded video data and the encoded audio data to the game client, so that the game client renders and presents the received encoded video data and encoded audio data. The data processing device is further configured to, after taking over the control method API, obtain control commands generated by the game client through triggering various control methods, wherein the control methods include at least one of the following: keyboard, gamepad, and mouse; control a pre-constructed virtual control method to perform corresponding operations according to the control commands, and obtain the screen data and the audio data in response to the operations performed by the virtual control method; the step of controlling the pre-constructed underlying virtual control method to perform corresponding operations according to the control commands includes: when the operating system used by the game client is different from the operating system of the edge server, mapping the key value corresponding to the system used by the game client to the key value corresponding to the operating system of the edge server according to the preset key value mapping information; and combining the key value mapped to the operating system of the edge server, controlling the virtual control method to perform corresponding operations according to the control commands. The data processing device is also used to create a corresponding process for the game client, and then, according to the pre-configured control method of the target game, create a corresponding number and type of virtual control methods for the process; obtain the user identifier of the game client, and bind the user identifier to the created virtual control methods; The data processing device is further configured to: determine, based on the game type of the target game, that when the target game supports multiplayer team play, multiple game clients participating in the target game share the same process; within the process, create a corresponding number and type of virtual control methods for each game client; bind each user identifier to the corresponding virtual control method according to the user identifier corresponding to each game client; distribute video and audio data generated by the same process to the game clients of that process; or, according to actual scenario requirements, generate corresponding video and audio data in response to different game clients, and distribute the generated video and audio data to the corresponding game clients.

14. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the data processing method according to any one of claims 1 to 12.

15. A terminal comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the data processing method according to any one of claims 1 to 12.

Images