Data processing method and system

Abstract

Embodiments of the present specification provide a data processing method and system, wherein the data processing method comprises: receiving an operation request submitted by a target application, wherein the target application runs on a client; determining image data corresponding to the target application in response to the operation request, and writing the image data into a frame buffer; determining preset function identification information, and calling a data copy function corresponding to the function identification information; copying the image data in the frame buffer by using the data copy function, and sending the image data to the client.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to data processing methods and systems. Background Technology

[0002] With the development of cloud computing and network technologies, remote applications have become a trend. By performing the application's computation in the cloud and the display and interaction processes locally, the local computing pressure can be greatly reduced, while ensuring a better application operating environment. This is particularly prominent for rendering-intensive applications, such as AR, VR, and 3D games. In existing technologies, most remote application transmission solutions are based on screenshot transmission, such as using VNC software; however, this solution is costly and wastes too many cloud resources, resulting in low resource utilization. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing system, a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program, to address technical deficiencies in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, applied to a server, comprising:

[0005] Receive an operation request submitted by a target application, wherein the target application runs on a client.

[0006] In response to the operation request, determine the image data corresponding to the target application and write the image data into the frame buffer;

[0007] Determine the preset function identifier information and call the data copy function corresponding to the function identifier information;

[0008] The image data is copied from the frame buffer using the data copy function and then sent to the client.

[0009] According to a second aspect of the embodiments of this specification, a data processing method is provided, applied to a client, comprising:

[0010] Receive operation instructions submitted by the user through the target application;

[0011] Create an operation request according to the operation instructions and upload it to the server;

[0012] Receive image data from the server in response to the operation request;

[0013] The target screen is rendered based on the image data and displayed to the user through the target application.

[0014] According to a third aspect of the embodiments of this specification, a data processing system is provided, comprising:

[0015] The client is configured to receive operation instructions submitted by the user through an extended reality application, create an operation request based on the operation instructions, and upload it to the server.

[0016] The server is configured to, in response to the operation request, determine image data associated with the virtual scene in the extended reality application and write the image data into a frame buffer; determine preset function identification information and call the data copy function corresponding to the function identification information; use the data copy function to copy the image data in the frame buffer and send the image data to the client;

[0017] The client is configured to update the virtual scene based on the image data, obtain the target image corresponding to the updated virtual scene, and display it to the user through the extended reality application.

[0018] According to a fourth aspect of the embodiments of this specification, another data processing system is provided, comprising:

[0019] The client is configured to receive operation instructions submitted by the user through the target application, create an operation request based on the operation instructions, and upload it to the server.

[0020] The server is configured to, in response to the operation request, determine the image data corresponding to the target application and write the image data into a frame buffer; determine preset function identification information and call the data copy function corresponding to the function identification information; use the data copy function to copy the image data in the frame buffer and send the image data to the client;

[0021] The client is configured to render a target screen based on the image data and display it to the user through the target application.

[0022] According to a fifth aspect of the embodiments of this specification, a data processing apparatus is provided, applied to a server, comprising:

[0023] The receiving module is configured to receive operation requests submitted by a target application, wherein the target application runs on a client.

[0024] The determination module is configured to determine the image data corresponding to the target application in response to the operation request, and write the image data into the frame buffer;

[0025] The calling module is configured to determine preset function identifier information and call the data copy function corresponding to the function identifier information;

[0026] The sending module is configured to copy the image data in the frame buffer using the data copy function and send the image data to the client.

[0027] According to a sixth aspect of the embodiments of this specification, a data processing apparatus is provided, applied to a client, comprising:

[0028] The instruction receiving module is configured to receive operation instructions submitted by the user through the target application;

[0029] The upload request module is configured to create an operation request based on the operation instruction and upload it to the server.

[0030] The data receiving module is configured to receive image data fed back by the server in response to the operation request;

[0031] The rendering module is configured to render a target screen based on the image data and display it to the user through the target application.

[0032] According to a seventh aspect of the embodiments of this specification, a computing device is provided, comprising:

[0033] Memory and processor;

[0034] The memory is used to store computer-executable instructions, and the processor is used to implement the steps of any of the above-described data processing methods when executing the computer-executable instructions.

[0035] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data processing method described above.

[0036] According to a ninth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data processing method.

[0037] The data processing method provided in this specification allows the server to respond to an operation request submitted by the target application running on the client side. The server determines the image data corresponding to the target application and writes it to the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier is first determined, followed by calling a data copy function according to the identifier. Finally, the data copy function copies the image data from the frame buffer and sends it to the client. This enables the direct reading of image data from video memory and its transmission to the client when the target application is running via cloud resources. By saving on cloud resource rendering operations, this process effectively reduces rendering overhead, avoids resource waste, and lowers the deployment cost of remote applications. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a data processing method provided in one embodiment of this specification;

[0039] Figure 2 This is a schematic diagram of the architecture of a data processing method provided in one embodiment of this specification;

[0040] Figure 3 This is a function call diagram in a data processing method provided in one embodiment of this specification;

[0041] Figure 4 This is a flowchart illustrating the processing procedure of a data processing method provided in one embodiment of this specification.

[0042] Figure 5 This is a flowchart of another data processing method provided in one embodiment of this specification;

[0043] Figure 6 This is a schematic diagram of the structure of a data processing system provided in one embodiment of this specification;

[0044] Figure 7 This is a schematic diagram of the structure of another data processing system provided in one embodiment of this specification;

[0045] Figure 8 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification;

[0046] Figure 9 This is a schematic diagram of the structure of another data processing device provided in one embodiment of this specification;

[0047] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0048] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0049] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0051] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0052] VNC (Virtual Network Console) basically consists of two parts: a client application (vncviewer) and a server application (vncserver). The basic operating principle of VNC is similar to some remote control software under Windows.

[0053] GPU (graphics processing unit), also known as display core, visual processor, display chip, or graphics card, is a microprocessor specifically designed for performing image and graphics-related calculations on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0054] GPU memory: This is the memory of the GPU, located inside the graphics card. GPU memory is used to store the results of GPU rendering.

[0055] OpenGL: OpenGL is a cross-platform, cross-language, device-independent 2D and 3D graphics rendering programming interface. When writing graphics applications, in order to achieve cross-device and cross-platform operation, people often do not directly use the GPU's native commands for programming, but use a set of standard interfaces to operate, and OpenGL is the de facto standard for such interfaces.

[0056] Remote applications refer to applications whose main service logic, rendering, and other computational tasks occur on a remote server (such as a cloud server), while local implementation only involves display and interaction.

[0057] XR: Extended Reality (XR) is an interactive environment that combines real and virtual elements, created through computer technology and wearable devices. XR includes various forms such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). In other words, XR is a general term encompassing AR, VR, and MR. XR can be divided into multiple levels, ranging from virtual worlds with limited sensor input to fully immersive virtual worlds.

[0058] Rendering: In computer graphics, this refers to the process of generating an image from a model using software. A model is a strictly defined language or data structure describing a three-dimensional object, including geometry, viewpoint, texture, and lighting information. Rendering involves projecting a model from a three-dimensional scene into a digital image according to pre-defined environment, lighting, materials, and rendering parameters.

[0059] AR (Augmented Reality) is a technology that cleverly integrates virtual information with the real world. It widely uses various technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0060] VR (Virtual Reality) Virtual reality technology encompasses computer, electronic information, and simulation technology. Its basic implementation method is to use computers to simulate a virtual environment, thereby giving people a sense of immersion in the environment.

[0061] MR (Mixed Reality) creates virtual scenes that can enter real life and recognize you. For example, through our devices, you can see a scene in your eyes and measure the scale and location of objects in real life. Its biggest feature is that the virtual world and the real world can interact.

[0062] Cloud XR: Cloud-based XR is a brand-new model: intelligent terminals, wide bandwidth, and cloud applications. Benefiting from 5G networks, Cloud XR migrates physical hardware to the cloud / edge, allowing "thin clients" to retain only the most basic XR functions, improving their mobility and flexibility, significantly reducing terminal costs, and opening up a broader market for AR / VR / MR.

[0063] PaaS: PaaS essentially refers to delivering a software development platform (service infrastructure platform) as a service to users in a SaaS model. Therefore, PaaS is also an application of the SaaS model. The reason PaaS can drive the development of SaaS is primarily because it provides enterprises with a middleware platform for customized development, encompassing databases and application servers. PaaS can increase the amount of resources utilized on the web platform. For example, it allows for the use of Data-as-a-Service (Data-as-a-Service) via remote web services and the use of visual APIs. Users or vendors can quickly develop the applications and products they need based on the PaaS platform. Furthermore, applications developed on the PaaS platform can better build enterprise applications based on SOA architecture.

[0064] This specification provides a data processing method, and also relates to a data processing system, a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0065] Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown, applied to a server, and specifically includes the following steps.

[0066] The data processing methods described in this manual are applied to the Cloud XR PaaS platform. By configuring three components in the cloud—platform services, application services, and media services—it supports the implementation of remote applications. The platform services are primarily responsible for application management and resource management and scheduling. The application services abstract cloud resources such as ECS and OSS to the application side, providing a virtualized XR runtime platform for the application and handling data transmission, video streams, audio streams, and interactive streams downstream. The media services are responsible for processing the transmission of video streams, audio streams, and interactive streams. During this process, the network is primarily responsible for meeting the platform's high bandwidth, low latency, and high concurrency requirements during transmission.

[0067] The client-side component, which connects to cloud resources, is primarily responsible for transmitting media from the cloud, including video, audio, and interactive streams. The client-side data processing unit handles data from on-device sensors such as cameras, microphones, and controllers. The conversion into corresponding media services focuses on processing video, audio, and interactive streams, while also playing audio and video streams from the cloud.

[0068] In other words, the data processing method applied to the above-mentioned platform completes the application operation and processing through cloud resources, without requiring the cloud to consume additional resources for rendering or transcoding. It directly reads the data that needs to be rendered or transcoded from the video memory or memory and sends it to the client, where the client uses local resources to render or transcode it. Finally, the data is displayed or played to the user through the client, thereby saving cloud resources and reducing overhead costs.

[0069] See Figure 2 The diagram illustrates that the client provides XR services to the user through a compatible display terminal. When the user inputs corresponding commands through the application using external devices such as a camera, microphone, IMU, controller, or gloves, the application will respond to the user's commands by performing streaming data processing (gesture recognition, speech recognition, SLAM, or audio / video playback, etc.). To achieve fast and efficient streaming data processing while minimizing local resource consumption, the data is uploaded to the cloud via a network with guaranteed bandwidth, latency, and concurrency. The cloud runs application management and resource scheduling tasks to obtain requests related to video streams, audio streams, window mode adaptation, application management, and command responses via the network. Then, the cloud's GPU resources respond to these requests, running the application and generating corresponding results. The cloud resources perform audio / video encoding, streaming media routing, command consolidation, and media transmission to directly send data to the client. This allows the client to receive the results and complete the final processing using limited local resources to provide the user with the corresponding service.

[0070] In summary, after the server receives an operation request from the target application running on the client, it can respond to the request by determining the image data corresponding to the target application and writing the image data into the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier can be determined first, followed by calling the data copy function according to the function identifier. Finally, the data copy function copies the image data from the frame buffer and sends it to the client. This allows the target application to directly read image data from video memory and send it to the client for rendering when running through cloud resources. This process effectively saves rendering overhead by reducing the cloud resource rendering process, thus avoiding resource waste and reducing the deployment cost of remote applications.

[0071] Step S102: Receive an operation request submitted by the target application, wherein the target application runs on the client.

[0072] The data processing method provided in this embodiment is applied to a scenario where the client interacts with the cloud. That is, the target application provides the corresponding application service to the user through the client. The implementation of the application service requires cloud resources to be completed, so as to realize the data stream transmission in the way of client-cloud interaction. While making full use of cloud resources, client resources are saved, and the normal operation of application services is guaranteed.

[0073] For example, in a gaming scenario, a user uses game application A on the client. Game application A responds to the user's action by sending an operation request to the cloud. The cloud then uses its resources to complete the operation of game application A and returns the result to the client. Finally, the client renders the game screen based on the result and displays it to the user. Similarly, in a virtual reality scenario, a user uses virtual reality application B on the client. Virtual reality application B responds to instructions collected by motion capture equipment by sending an operation request to the cloud. The cloud then uses its resources to complete the operation of virtual reality application B and returns the result to the client. Finally, the client renders the virtual reality screen based on the result and displays it to the user.

[0074] Specifically, the server refers to the server providing cloud resources, which supports high concurrency and fast response cloud services by deploying a large number of computing resources. Correspondingly, the client refers to the user's terminal device, including but not limited to mobile phones, computers, and smart wearable devices. The target application refers to the application running on the client, which can provide users with services such as games, virtual reality, and browsing. This application relies on cloud resources to provide its services. The operation request refers to the request corresponding to the operation command submitted by the user holding the client to the target application, and the subsequent application service needs to be provided to the user in response to this request.

[0075] Therefore, when a user holding the client interacts with the target application on the client, the client will create an operation request based on the user's instructions and send the operation request to the server. At this point, the server will receive the operation request submitted by the target application running on the client. In order to support the target application in providing corresponding services to the user, the server will subsequently respond to the operation request by calling cloud resources for use.

[0076] It should be noted that, considering that the server's cloud resources will provide services to a large number of clients, and the target applications running on different clients may be different, in order to ensure full utilization of cloud resources, after receiving an operation request, the target application running on the client can be determined based on the application identifier carried in the operation request. Then, resources can be selected specifically to run the target application, thereby satisfying subsequent data processing operations.

[0077] This embodiment uses the application of data processing methods in a game scenario as an example for ease of description. Descriptions of other application scenarios can refer to the same or corresponding descriptions in this embodiment, and will not be elaborated on here.

[0078] Step S104: In response to the operation request, determine the image data corresponding to the target application and write the image data into the frame buffer.

[0079] Specifically, after receiving the operation request submitted by the target application running on the client, in order to reduce the consumption of client computing resources by the target application running on the client, the image data corresponding to the target application can be determined in response to the operation request, and then the image resources can be written into the GPU's video memory, i.e., the frame buffer, so that the data can be directly read from the video memory and sent to the client for use during subsequent display.

[0080] Image data specifically refers to the data obtained after rendering the screen to be displayed by the target application in response to an operation request. This includes, but is not limited to, the RGB values ​​of pixels, the number of pixels, and the pixel order. Correspondingly, the frame buffer is a direct mapping of the image displayed on the screen, also known as a bit map or raster. Each storage unit in the frame buffer corresponds to a pixel on the screen, and the entire frame buffer corresponds to one frame of image.

[0081] In other words, after the server uses cloud resources to render the image data of the next screen corresponding to the target application in response to the operation request, the image data can be directly written to the frame buffer in the video memory. This makes it convenient to read the image data directly from the video memory and send it to the client for rendering and display, thereby saving the operation of rendering with cloud resources.

[0082] Furthermore, during the process of determining image data in response to an operation request, the server considers that the image data needs to be displayed on the client, and the screen to be displayed needs to be connected to the previous screen. Therefore, it needs to determine the attribute data and projection position before determining the image data. In this embodiment, the specific implementation method is as follows:

[0083] Read the initial attribute data corresponding to the target application; update the initial attribute data according to the operation request to obtain the target attribute data and determine the projection position; create the image data based on the projection position and the target attribute data.

[0084] Specifically, initial attribute data refers to the attribute data of objects contained in the frames already displayed by the target application, including but not limited to the object's position, shape, and color. Correspondingly, target attribute data refers to the attribute data of objects contained in the frames to be displayed, and the frames to be displayed are the next frames after the already displayed frames; that is, they are consecutive frames. Correspondingly, projection position refers to the position where the frame to be displayed is projected, i.e., the camera position during rendering.

[0085] Based on this, when the server receives the operation request corresponding to the target application, in order to write the image data corresponding to the screen to be displayed into the GPU memory, it can read the initial attribute data of the objects in the screen already displayed by the target application. Then, according to the operation request corresponding to the operation instruction submitted by the user in the client, the initial attribute data is updated to obtain the target attribute data that satisfies the user's operation instruction, and the projection position corresponding to the screen to be displayed is determined according to the operation request. Finally, the image data is determined based on the projection position and the target attribute data, and written into the frame buffer for easy copying via functions later.

[0086] In summary, in order to write the image data of the corresponding target application into the frame buffer, it will be created by combining the projection position and target attribute data, which can ensure that the image data fed back to the client in the end better meets the operation request.

[0087] Furthermore, when creating image data based on the projection position and target attribute data, cloud resources are actually used to render the image data of the screen to be displayed and write it into the frame buffer to await copying and display. During this process, it is necessary to render the target attribute data in response to operation requests. In this embodiment, the specific implementation method is as follows:

[0088] In response to the operation request, a rendering instruction corresponding to the rendering unit is determined; the rendering unit performs rendering processing on the target attribute data according to the rendering instruction and the projection position to obtain the image data.

[0089] Specifically, rendering instructions refer to commands that render target attribute data, used to render image data that meets the client's display requirements. Correspondingly, the rendering unit refers to OpenGL, which executes rendering instructions to perform rendering processing operations.

[0090] Based on this, after obtaining the target attribute data and projection position, the server can respond to the operation request to determine the rendering instruction corresponding to the rendering unit. At this time, the rendering unit corresponding to the cloud resource can perform rendering processing on the target attribute data according to the rendering instruction and projection position, thereby obtaining the image attribute based on the rendering processing result and writing it into the frame buffer for easy copying via function later.

[0091] In other words, during the execution of rendering instructions, the rendering unit will use the projection position as the view position to render the target attribute data, thereby making the obtained image data continuous with the display screen of the previous frame and improving the user's participation experience.

[0092] For example, a user uses game application A through a client. When the user controls the game character to open a treasure chest in the game, the current screen displayed on the client shows the game character in front of the treasure chest, inserting the key into the keyhole. The user inputs the operation command to open the treasure chest via the keyboard, and the client creates an operation request based on the operation command and uploads it to the server. After receiving the operation request, the server determines that the next scene to be rendered is the scene of the treasure chest keyhole being rotated by the key. At this time, it can read the position data, shape data, color data, etc. of the treasure chest and the game character from the previous frame. Then, according to the operation request, it updates the position data, shape data, and color data respectively, to obtain the target attribute data of the corresponding treasure chest and the target attribute data of the corresponding game character, and uses the camera position corresponding to the previous frame as the camera position of the next frame.

[0093] Furthermore, OpenGL rendering instructions can be used to render the target attribute data corresponding to the treasure chest and game character at the camera position, respectively, to obtain the image data corresponding to the next frame. The image data is then saved to the GPU's video memory, i.e., the frame buffer, so that the image data can be directly copied from the frame buffer for use during the subsequent display stage.

[0094] In summary, by triggering the rendering instructions corresponding to the rendering unit and rendering the target attribute data based on the projection position, it is possible to ensure that the rendered image data is continuous with the previous frame, thereby improving the user's viewing experience.

[0095] Step S106: Determine the preset function identifier information and call the data copy function corresponding to the function identifier information.

[0096] Specifically, after writing the image data corresponding to the target application into the frame buffer as described above, further details can be found in [link to documentation]. Figure 3 The schematic diagram shown in Figure (a) illustrates that, in practical applications, graphics rendering pipelines often employ double buffering to avoid synchronization issues between image display and rendering. This means that the displayed image and the image being rendered are stored in different frame buffers. Once a frame is rendered, the display and rendering buffers are swapped, thus achieving buffer swapping. Specifically, the frame buffer corresponding to the displayed image is swapped with the frame buffer that has been rendered, so that the next frame's image can be displayed, while the previous frame's image is released.

[0097] During this process, because the buffer swapping technology performs display processing after completion, and the cloud display processing is not viewed, displaying the image after processing consumes additional computing resources. Therefore, in order to directly copy image data from the GPU memory and save display resources, after the image data is written to the frame buffer, a suitable insertion point needs to be selected to copy the image data from the frame buffer to the client for display.

[0098] In order to read complete image data from the frame buffer, the insertion point needs to be selected after the next frame's image data has been rendered and written to the frame buffer, but before buffer swapping is performed. Therefore, the insertion point needs to be set after the image data is written to the frame buffer and before any display processing is performed on the image data in the frame buffer. The image data is then read from the frame buffer and sent to the client for display using this insertion point as the node.

[0099] That is, after the image data has been written to the frame buffer, a preset function identifier can be determined, and the subsequent copy operation can be performed by calling the data copy function corresponding to the function identifier. In other words, the data copy function will be executed to copy the image data before buffer swapping, thus eliminating the need to run buffer swapping again.

[0100] Specifically, the preset function identifier information refers to the function identifier corresponding to the data copy function to be called, which includes, but is not limited to, the function name and ID. Correspondingly, the data copy function specifically refers to a function capable of directly copying image data from the frame buffer.

[0101] In practical applications, buffer swapping is actually handled by a dedicated function in the graphics rendering pipeline. In OpenGL and Linux, this function is `glXSwapBuffers`, which swaps the pointers corresponding to the frame buffers to support subsequent display operations. However, considering that cloud-based XR applications are existing programs—that is, already compiled and whose source code cannot be modified—further processing operations can only be supported with zero intrusion and zero modification to the original program. Therefore, see [link to relevant documentation]. Figure 3 As shown in the schematic diagram in Figure (b), the solution provided in this embodiment is to set a data copy function with the same function name as the function glXSwapBuffers, and set the function address of the data copy function before the function glXSwapBuffers. This allows the data copy function with the address preceding the function name to be called first when it is called again, so as to perform subsequent image data copy processing operations.

[0102] Furthermore, before calling the data copy function to copy image data, the function must be triggered only if an exchange occurs in the frame buffer. Therefore, it is necessary to first determine the identifiers corresponding to the display frame buffer and the drawing frame buffer. If the identifiers change, step S106 is executed. In this embodiment, the specific implementation is as follows:

[0103] Determine the display identifier corresponding to the display frame buffer and the rendering identifier corresponding to the frame buffer; if the display identifier and the rendering identifier are transformed, execute the steps of determining the preset function identifier information and calling the data copy function corresponding to the function identifier information.

[0104] Specifically, the display frame buffer refers to the frame buffer that stores the image data corresponding to the currently displayed screen; correspondingly, the display identifier refers to the pointer corresponding to the display frame buffer; and correspondingly, the rendering identifier refers to the pointer corresponding to the frame buffer that stores the image data corresponding to the currently drawn screen.

[0105] Based on this, since the insertion point is set after the image data rendering is completed and before the buffer exchange, the cloud resource will determine the display identifier corresponding to the display frame buffer and the rendering identifier corresponding to the frame buffer before displaying. If the display identifier and the rendering identifier change, step S106 will be executed, that is, the preset function identifier information will be determined and the data copy function corresponding to the function identifier information will be called.

[0106] In summary, by setting the insertion point before the buffer swap, the data copy function can be triggered in time before display, thus saving cloud display resources and reducing overhead costs.

[0107] Furthermore, when the data copy function is called, it actually calls a default function with the same name as the original function. To ensure successful call execution, see [link to relevant documentation]. Figure 3 The schematic diagram shown in (b) will trigger the call to the data copy function via a Hook. In this embodiment, the specific implementation method is as follows:

[0108] The function identification information is determined by executing a preset application function; the function address is determined based on the function identification information; and the data copy function is called according to the function address.

[0109] Specifically, the application function refers to the function that can locate the target function through function identification information; correspondingly, the function address refers to the entry address of the data copy function. Based on this, after the image data corresponding to the frame buffer is saved, the function identification information that can directly copy the image data from the frame buffer can be determined by executing the preset application function, and the address of the function with the highest priority can be determined according to the function identification information. Then, the data copy function is called according to the function address for subsequent processing.

[0110] In other words, since the function `glXSwapBuffers` is a standard function and cannot be modified, the operating system usually provides a way to replace the library function, which is a hook. That is, when an application attempts to call the function `glXSwapBuffers`, a hook can be used to call a pre-embedded data copy function with the same name as the function `glXSwapBuffers`. During the call, the Linux system can use the commonly used `dlsym` function to find the entry address corresponding to the function identifier information. By preserving the original function `glXSwapBuffers`, after the image data is written to the frame buffer, the original function can still be used to implement the functionality required by the application. At the same time, the new data copy function can directly read image data from the GPU's video memory.

[0111] Following the previous example, after the image data is written to the frame buffer, the dlsym function (Hook) can be determined when the pointers of the display frame buffer and the drawing frame buffer change. Then, the dlsym function calls the data copy function with the highest function address priority, glXSwapBuffers, to replace the original function. After that, the data copy function can be used to directly copy the data in the video memory for sending.

[0112] In summary, by using a preset function identifier to call the data copy function, the application function can call the data copy function before the original function is called, thereby avoiding obtaining image data before display processing and saving resources.

[0113] Step S108: Use the data copy function to copy the image data in the frame buffer and send the image data to the client.

[0114] Specifically, after obtaining the data copy function mentioned above, the image data can be directly copied from the frame buffer and sent to the client. This allows the server to save on the operation of displaying the image data to be shown, and only a portion of the client resources are needed to achieve the display, thus making full use of resources and saving cloud resources.

[0115] In practical applications, considering that GPUs store image data in their video memory using a frame buffer, and that the frame buffer stores image data in units of memory, writing to the frame buffer will be done pixel by pixel. Similarly, copying image data will also be done pixel by pixel. In this embodiment, the specific implementation is as follows:

[0116] A storage unit sequence is determined in the frame buffer, and pixel data from the image data is sequentially written into each storage unit in the storage unit sequence; the storage unit sequence is determined in the frame buffer, and each storage unit in the storage unit sequence is read using the data copy function; the pixel data is obtained based on the read result, and the image data is generated based on the pixel data.

[0117] Specifically, the storage unit sequence refers to the set of all storage units corresponding to the frame buffer. Each storage unit is used to store the data corresponding to a pixel, i.e., pixel data. Correspondingly, pixel data refers to the data corresponding to each pixel, which is used to represent the RGB value, position, etc. of that pixel.

[0118] Based on this, after obtaining the image data, the rendering frame buffer can be determined in the GPU's video memory, followed by the determination of its corresponding storage unit sequence. Then, the image data is parsed to obtain the pixel data corresponding to each pixel point. Finally, using a one-to-one correspondence, each pixel data is sequentially written to each storage unit. When the data copy function is called, the storage unit sequence corresponding to the frame buffer can be determined first. Then, all pixel data is determined by reading each storage unit. Finally, image data is generated based on the pixel data and sent to the client for display processing.

[0119] In summary, writing and copying at the pixel level ensures the global integrity of image data and avoids problems such as bad pixels.

[0120] Furthermore, when using the data copy function to copy image data, the image data is actually determined in the frame buffer first, and then copied. In this embodiment, the specific implementation is as follows:

[0121] The image data is determined in the frame buffer using the determination function in the data copy function; the image data is copied in the frame buffer using the copy function in the data copy function.

[0122] Specifically, the "determine" function locates the image position in the frame buffer of video memory; correspondingly, the "copy" function copies the image data from the frame buffer. Therefore, once the data copy function is determined, the "determine" function within it can be used to locate the image data, and then the "copy" function can be used to copy the image data from the frame buffer. This data can then be sent to the client.

[0123] After receiving the image data, the client can determine the response data corresponding to the screen to be displayed, and finally render the screen through the display module to complete the display operation.

[0124] Following the previous example, once the data copy function is determined, it can be used to first determine the sequence of storage units in the frame buffer that store image data. Then, the glReadPixels function in OpenGL reads the pixel data from each storage unit. Finally, the image data is integrated based on the read results and sent to the client. After receiving the image data, the client can render the scene of the keyhole of the treasure chest being rotated by the key.

[0125] The data processing method provided in this specification allows the server to respond to an operation request submitted by the target application running on the client side. The server determines the image data corresponding to the target application and writes it to the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier is first determined, followed by calling a data copy function according to the identifier. Finally, the data copy function copies the image data from the frame buffer and sends it to the client. This enables the direct reading of image data from video memory and its transmission to the client when the target application is running via cloud resources. By saving on cloud resource rendering operations, this process effectively reduces rendering overhead, avoids resource waste, and lowers the deployment cost of remote applications.

[0126] The following is in conjunction with the appendix Figure 4 Taking the application of the data processing method provided in this specification in a cloud-based XR scenario as an example, the data processing method will be further explained. Figure 4 A flowchart illustrating the processing procedure of a data processing method according to an embodiment of this specification is shown, specifically including the following steps.

[0127] Step S402: Receive the operation request submitted by the target application, wherein the target application runs on the client.

[0128] Step S404: Read the initial attribute data corresponding to the target application.

[0129] Step S406: Update the initial attribute data according to the operation request, obtain the target attribute data, and determine the projection position.

[0130] Step S408: In response to the operation request, determine the rendering instruction corresponding to the rendering unit.

[0131] In step S410, the rendering unit performs rendering processing on the target attribute data according to the rendering instructions and projection position to obtain image data.

[0132] Step S412: Determine the display identifier corresponding to the display frame buffer and the rendering identifier corresponding to the frame buffer.

[0133] Step S414: When the display identifier and the rendering identifier undergo identifier transformation, the function identifier information is determined by executing a preset application function.

[0134] Step S416: Determine the function address based on the function identifier information, and call the data copy function according to the function address.

[0135] Step S418: Determine image data in the frame buffer using the determination function in the data copy function, and copy image data in the frame buffer using the copy function in the data copy function.

[0136] In step S420, the image data is sent to the client, and the client displays the target image based on the image data.

[0137] In summary, after the server receives an operation request from the target application running on the client, it can respond to the request by determining the image data corresponding to the target application and writing the image data into the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier can be determined first, followed by calling the data copy function according to the function identifier. Finally, the data copy function copies the image data from the frame buffer and sends it to the client. This allows the target application to directly read image data from video memory and send it to the client for rendering when running through cloud resources. This process effectively saves rendering overhead by reducing the cloud resource rendering process, thus avoiding resource waste and reducing the deployment cost of remote applications.

[0138] Corresponding to the above-described method embodiments, this specification also provides another data processing method. Figure 5 A flowchart of another data processing method provided in one embodiment of this specification is shown, such as... Figure 5 As shown, another data processing method is applied to the client side, specifically including the following steps:

[0139] Step S502: Receive the operation command submitted by the user through the target application.

[0140] Step S504: Create an operation request according to the operation instruction and upload it to the server.

[0141] Step S506: Receive image data from the server in response to the operation request.

[0142] Step S508: Render the target screen based on the image data and display it to the user through the target application.

[0143] It should be noted that the client in this embodiment specifically refers to a mobile terminal held by a user that can run cloud-based applications. The relevant descriptions can be found in the above embodiments, and will not be elaborated further in this embodiment.

[0144] Corresponding to the above method embodiments, this specification also provides a data processing system. Figure 6 This specification shows a schematic diagram of the structure of a data processing system according to one embodiment. Figure 6As shown, the data processing system 600 includes a client 610 and a server 620; the specific implementation is as follows:

[0145] Client 610 is configured to receive operation instructions submitted by users through extended reality applications, create operation requests based on the operation instructions, and upload them to the server.

[0146] The server 620 is configured to, in response to the operation request, determine image data associated with the virtual scene in the extended reality application and write the image data into a frame buffer; determine preset function identification information and call the data copy function corresponding to the function identification information; use the data copy function to copy the image data in the frame buffer and send the image data to the client;

[0147] The client 610 is configured to update the virtual scene based on the image data, obtain the target image corresponding to the updated virtual scene, and display it to the user through the extended reality application.

[0148] The data processing system provided in this embodiment can refer to the same or corresponding content in the data processing method described above, and will not be elaborated further here.

[0149] Specifically, extended reality applications refer to applications that provide extended reality services to users, including but not limited to augmented reality (AR), virtual reality (VR), or mixed reality (MR) applications. Furthermore, extended reality applications are cloud-based, meaning their operation relies on cloud resources, with the client only providing the display. Correspondingly, virtual scenes refer to the scenes rendered for users to view within extended reality applications, and all user actions within the extended reality application are completed within these virtual scenes; in other words, the virtual scenes provided by extended reality applications serve as the environment for human-computer interaction. For example, in a game scenario, when a user controls an augmented reality character to release a skill, this skill release operation is completed within the virtual scene provided by the augmented reality application. This virtual scene can be preset by the augmented reality application or constructed by the augmented reality application through data collection from the actual environment. Alternatively, after capturing user motion data using motion capture devices, the motion data can be used to drive the movement of a virtual character in a virtual reality application, and the virtual character's movement can be displayed within the virtual scene provided by the virtual reality application. In practical applications, virtual scenes can be constructed according to the settings of the extended reality application; this embodiment does not impose any limitations on this.

[0150] Based on this, when using extended reality applications on a terminal device, users can submit operation commands through extended reality (XR) applications. These operation commands can be commands to control characters in the augmented reality scene, or commands to drive virtual characters in the virtual reality scene by determining motion data collected by motion capture devices. These commands will cause changes to the display screen on the client. At this time, an operation request can be created based on the command to complete the rendering of the screen to be displayed using the resources provided by the server, thereby saving client resources.

[0151] Furthermore, upon receiving an operation request, the server can respond by determining the image data of the virtual scene associated with the extended real-world application, thus identifying the data corresponding to the screen to be displayed on the client. During this process, considering that the server typically processes the image data for display after receiving it, and that the resulting screen is not viewed on the server, to conserve display resources, a preset function identifier can be determined after writing the image data to the frame buffer. Based on this identifier, a data copy function can be called, allowing the image data to be read from the frame buffer before display processing and directly sent to the client, thereby saving resources consumed by the server's display processing operations.

[0152] Furthermore, after receiving the image data from the server in response to the operation request, the client determines that the image data is data that updates the screen corresponding to the currently displayed virtual scene. Therefore, the client can update the currently displayed virtual scene based on the image data, that is, re-render the new screen, so as to obtain the target screen corresponding to the virtual scene based on the update result. Finally, the target screen can be displayed to the user through extended reality applications.

[0153] For example, a user uses motion capture equipment and VR glasses to make a virtual character dance in a virtual reality application. Once the motion capture equipment collects the user's motion data, it sends motion commands carrying that data to the VR application. The application then creates a request based on these commands and sends it to the server. The server responds to this request, determines the virtual scene in which the virtual character moves, and then uses the motion data to determine the final image of the character within that scene. The server then renders the corresponding image data using the graphics card and writes it to the graphics card's frame buffer. When the VR application needs to convert the current image to the final image, it can define a preset function identifier and use that identifier to call a data copy function. This function copies the image data from the frame buffer and sends it directly to the VR glasses. Upon receiving the image data, the VR glasses use local rendering resources to process it, generating the final image of the virtual character based on the motion data, which is then displayed to the user wearing the VR glasses.

[0154] Furthermore, during the process of determining image data in response to an operation request, the server considers that the image data needs to be displayed on the client, and the screen to be displayed needs to be connected to the previous screen. Therefore, it is necessary to determine the attribute data and projection position before determining the image data. In this embodiment, the specific implementation is as follows: The server is further configured to read the initial attribute data corresponding to the virtual scene in the extended reality application; update the initial attribute data according to the operation request to obtain target attribute data and determine the projection position; and create the image data of the virtual scene based on the projection position and the target attribute data. The image data determination process provided in this embodiment can be found in the same or corresponding descriptions in the above embodiments, and will not be elaborated further here.

[0155] In summary, after the server receives an operation request from the extended reality application running on the client, it can respond to the request by determining the corresponding image data for the extended reality application and writing the image data to the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier can be determined first, followed by calling the data copy function according to the function identifier. Finally, the image data is copied from the frame buffer using the data copy function and sent to the client. This allows the extended reality application running via cloud resources to directly read image data from video memory using the data copy function and send it to the client for rendering and display. This process effectively saves rendering overhead by reducing the cloud resource rendering process, thus avoiding resource waste and reducing the deployment cost of remote applications.

[0156] Corresponding to the above method embodiments, this specification also provides data processing system embodiments. Figure 7 A schematic diagram of another data processing system provided in one embodiment of this specification is shown. For example... Figure 7 As shown, the data processing system 700 includes a client 710 and a server 720;

[0157] Client 710 is configured to receive operation instructions submitted by the user through the target application, create an operation request based on the operation instructions, and upload it to the server.

[0158] The server 720 is configured to, in response to the operation request, determine the image data corresponding to the target application and write the image data into a frame buffer; determine preset function identification information and call the data copy function corresponding to the function identification information; use the data copy function to copy the image data in the frame buffer and send the image data to the client;

[0159] The client 710 is configured to render a target screen based on the image data and display it to the user through the target application.

[0160] In an optional embodiment, the client 710 is further configured to:

[0161] Read the initial attribute data corresponding to the target application; update the initial attribute data according to the operation request to obtain the target attribute data and determine the projection position; create the image data based on the projection position and the target attribute data.

[0162] In an optional embodiment, the client 710 is further configured to:

[0163] In response to the operation request, a rendering instruction corresponding to the rendering unit is determined; the rendering unit performs rendering processing on the target attribute data according to the rendering instruction and the projection position to obtain the image data.

[0164] In an optional embodiment, the client 710 is further configured to:

[0165] Determine the display identifier corresponding to the display frame buffer and the rendering identifier corresponding to the frame buffer; if the display identifier and the rendering identifier are transformed, execute the steps of determining the preset function identifier information and calling the data copy function corresponding to the function identifier information.

[0166] In an optional embodiment, the client 710 is further configured to:

[0167] The function identification information is determined by executing a preset application function; the function address is determined based on the function identification information; and the data copy function is called according to the function address.

[0168] In an optional embodiment, the client 710 is further configured to:

[0169] In the frame buffer, a sequence of storage units is determined, and pixel data from the image data is sequentially written into each storage unit in the sequence. Correspondingly, copying the image data in the frame buffer using the data copy function includes: determining the sequence of storage units in the frame buffer and reading each storage unit in the sequence using the data copy function; obtaining the pixel data based on the reading result and generating the image data based on the pixel data.

[0170] In an optional embodiment, the client 710 is further configured to:

[0171] The image data is determined in the frame buffer using the determination function in the data copy function; the image data is copied in the frame buffer using the copy function in the data copy function.

[0172] The above is an illustrative scheme of another data processing system according to this embodiment. It should be noted that the technical solution of this data processing system and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the data processing method described above.

[0173] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 8 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 8 As shown, the device, applied to the server side, includes:

[0174] The receiving module 802 is configured to receive an operation request submitted by a target application, wherein the target application runs on a client.

[0175] The determination module 804 is configured to determine the image data corresponding to the target application in response to the operation request, and write the image data into the frame buffer;

[0176] The calling module 806 is configured to determine preset function identifier information and call the data copy function corresponding to the function identifier information;

[0177] The sending module 808 is configured to copy the image data in the frame buffer using the data copy function and send the image data to the client.

[0178] In an optional embodiment, the determining module 804 is further configured to:

[0179] Read the initial attribute data corresponding to the target application; update the initial attribute data according to the operation request to obtain the target attribute data and determine the projection position; create the image data based on the projection position and the target attribute data.

[0180] In an optional embodiment, the determining module 804 is further configured to:

[0181] In response to the operation request, a rendering instruction corresponding to the rendering unit is determined; the rendering unit performs rendering processing on the target attribute data according to the rendering instruction and the projection position to obtain the image data.

[0182] In an optional embodiment, the data processing apparatus further includes:

[0183] The identifier determination module is configured to determine the display identifier corresponding to the display frame buffer and the rendering identifier corresponding to the frame buffer;

[0184] When the display identifier and the rendering identifier undergo identifier transformation, the determination module 804 is executed.

[0185] In an optional embodiment, the calling module 806 is further configured to:

[0186] The function identification information is determined by executing a preset application function; the function address is determined based on the function identification information; and the data copy function is called according to the function address.

[0187] In an optional embodiment, the calling module 806 is further configured to:

[0188] A sequence of storage units is determined in the frame buffer, and pixel data from the image data is sequentially written into each storage unit in the sequence of storage units;

[0189] Accordingly, the sending module 808 is further configured as follows:

[0190] The sequence of storage units is determined in the frame buffer, and each storage unit in the sequence is read using the data copy function; the pixel data is obtained based on the reading result, and the image data is generated based on the pixel data.

[0191] In an optional embodiment, the sending module 808 is further configured to:

[0192] The image data is determined in the frame buffer using the determination function in the data copy function; the image data is copied in the frame buffer using the copy function in the data copy function.

[0193] In summary, after the server receives an operation request from the target application running on the client, it can respond to the request by determining the image data corresponding to the target application and writing the image data into the frame buffer of the cloud resource. To conserve cloud resources, a preset function identifier can be determined first, followed by calling the data copy function according to the function identifier. Finally, the data copy function copies the image data from the frame buffer and sends it to the client. This allows the target application to directly read image data from video memory and send it to the client for rendering when running through cloud resources. This process effectively saves rendering overhead by reducing the cloud resource rendering process, thus avoiding resource waste and reducing the deployment cost of remote applications.

[0194] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0195] Corresponding to the above method embodiments, this specification also provides another data processing apparatus embodiment. Figure 9 A schematic diagram of another data processing apparatus provided in one embodiment of this specification is shown. Figure 9 As shown, the device, applied to a client, includes:

[0196] The instruction receiving module 902 is configured to receive operation instructions submitted by the user through the target application;

[0197] Upload request module 904 is configured to create an operation request based on the operation instruction and upload it to the server.

[0198] The data receiving module 906 is configured to receive image data fed back by the server in response to the operation request;

[0199] The rendering module 908 is configured to render a target screen based on the image data and display it to the user through the target application.

[0200] The above is an illustrative scheme of another data processing device according to this embodiment. It should be noted that the technical solution of this data processing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the data processing method described above.

[0201] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0202] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0203] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0204] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1000 can also be a mobile or stationary server.

[0205] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0206] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0207] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0208] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0209] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data processing method.

[0210] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the data processing method described above belong to the same concept. Details not described in detail in the technical solution of the computer program can be found in the description of the technical solution of the data processing method described above.

[0211] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0212] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0213] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0214] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0215] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied on a server side, comprising: Receive an operation request submitted by a target application, wherein the target application runs on a client. In response to the operation request, determine the image data corresponding to the target application and write the image data into the frame buffer; After the image data is rendered and written to the frame buffer, but before the frame buffer and the display frame buffer are swapped, a preset function identification information is determined and the data copy function corresponding to the function identification information is called. The display frame buffer refers to the frame buffer that stores the image data corresponding to the screen being displayed, and the frame buffer refers to the frame buffer that stores the image data corresponding to the screen being drawn. The image data is copied from the frame buffer using the data copy function and then sent to the client.

2. The data processing method according to claim 1, wherein determining the image data corresponding to the target application in response to the operation request includes: Read the initial attribute data corresponding to the target application; The initial attribute data is updated according to the operation request to obtain the target attribute data and determine the projection position; The image data is created based on the projection position and the target attribute data.

3. The data processing method according to claim 2, wherein creating the image data based on the projection position and the target attribute data comprises: In response to the operation request, determine the rendering instruction corresponding to the rendering unit; The rendering unit performs rendering processing on the target attribute data according to the rendering instructions and the projection position to obtain the image data.

4. The data processing method according to claim 1, before the step of determining the preset function identifier information and calling the data copy function corresponding to the function identifier information, further includes: Determine the display identifier corresponding to the display frame buffer, and the rendering identifier corresponding to the frame buffer; When the display identifier and the rendering identifier undergo identifier transformation, the steps of determining the preset function identifier information and calling the data copy function corresponding to the function identifier information are performed.

5. The data processing method according to any one of claims 1-4, wherein determining the preset function identifier information and calling the data copy function corresponding to the function identifier information comprises: The function identification information is determined by executing a preset application function; The function address is determined based on the function identifier information, and the data copy function is called according to the function address.

6. The data processing method according to claim 1, wherein writing the image data into the frame buffer comprises: A sequence of storage units is determined in the frame buffer, and pixel data from the image data is sequentially written into each storage unit in the sequence of storage units; Accordingly, copying the image data in the frame buffer using the data copy function includes: The sequence of storage cells is determined in the frame buffer, and each storage cell in the sequence of storage cells is read using the data copy function; The pixel data is obtained based on the reading results, and the image data is generated based on the pixel data.

7. The data processing method according to claim 1, wherein copying the image data in the frame buffer using the data copy function comprises: The image data is determined in the frame buffer using the determination function in the data copy function; The image data is copied in the frame buffer using the copy function in the data copy function.

8. A data processing method, applied to a client, comprising: Receive operation instructions submitted by the user through the target application; Create an operation request according to the operation instructions and upload it to the server; The server receives image data in response to the operation request. The image data is determined by the server in response to the operation request, corresponding to the target application, and written into the frame buffer. After the image data is rendered and written into the frame buffer, but before the frame buffer and the display frame buffer are swapped, a preset function identifier is determined, and the data copy function corresponding to the function identifier is called. The image data is copied into the frame buffer using the data copy function. The display frame buffer refers to the frame buffer that stores the image data corresponding to the currently displayed screen, and the frame buffer refers to the frame buffer that stores the image data corresponding to the currently drawn screen. The target screen is rendered based on the image data and displayed to the user through the target application.

9. A data processing system, comprising: The client is configured to receive operation instructions submitted by the user through an extended reality application, create an operation request based on the operation instructions, and upload it to the server. The server is configured to determine, in response to the operation request, the image data associated with the virtual scene in the extended reality application, and write the image data into the frame buffer; After the image data is rendered and written to the frame buffer, but before the frame buffer and the display frame buffer are swapped, a preset function identifier is determined, and the data copy function corresponding to the function identifier is called; the image data is copied from the frame buffer using the data copy function, and the image data is sent to the client. Here, the display frame buffer refers to the frame buffer that stores the image data corresponding to the currently displayed screen, and the frame buffer refers to the frame buffer that stores the image data corresponding to the currently drawn screen. The client is configured to update the virtual scene based on the image data, obtain the target image corresponding to the updated virtual scene, and display it to the user through the extended reality application.

10. The data processing system according to claim 9, wherein the client is a terminal device running the extended reality application, and the extended reality application includes augmented reality applications, virtual reality applications, or mixed reality applications.

11. The data processing system according to claim 9, wherein the server is further configured to read initial attribute data corresponding to the virtual scene in the extended reality application; update the initial attribute data according to the operation request to obtain target attribute data and determine the projection position; and create image data of the virtual scene based on the projection position and the target attribute data.

12. A data processing system, comprising: The client is configured to receive operation instructions submitted by the user through the target application, create an operation request based on the operation instructions, and upload it to the server. The server is configured to determine the image data corresponding to the target application in response to the operation request, and write the image data into the frame buffer; After the image data is rendered and written to the frame buffer, but before the frame buffer and the display frame buffer are swapped, a preset function identifier is determined, and the data copy function corresponding to the function identifier is called; the image data is copied from the frame buffer using the data copy function, and the image data is sent to the client. Here, the display frame buffer refers to the frame buffer that stores the image data corresponding to the currently displayed screen, and the frame buffer refers to the frame buffer that stores the image data corresponding to the currently drawn screen. The client is configured to render a target screen based on the image data and display it to the user through the target application.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images