Data processing method, apparatus and electronic device

Abstract

This application discloses a data processing method, apparatus, and electronic device. The method is applied to a terminal running a target program, which includes a rendering thread and a computation thread. The rendering program loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The method includes: the rendering thread executing the second link library file to perform the following operations: if the rendering thread calls the first function within each frame, a hook operation is performed on the first function, and multiple frame buffers are obtained from multiple storage addresses; if the rendering thread calls the second function within each frame, a hook operation is performed on the second function, and the multiple frame buffers are sent to the computation thread; the computation thread determines the number of downsampling processes performed to obtain the floodlight image based on the obtained multiple frame buffers. This method is highly versatile, requires no manual intervention, and can determine the number of downsampling processes performed to obtain the floodlight image in real time.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to a data processing method, apparatus, and electronic device. Background Technology

[0002] Bloom is a common optical phenomenon, generally referring to the halo effect that occurs when a physical camera photographs a bright object. Applying bloom processing to an image can visually improve its contrast, enhance its expressiveness, and achieve better rendering results. With the development of image processing technology, image bloom processing has been widely used in fields such as 3D games and animation production. Currently, in terminal games, the terminal performs bloom processing on the image to be rendered to obtain the corresponding bloom image. Bloom processing can include the following steps: downsampling, blurring, and upsampling. The bloom effect presented by the bloom image is mainly determined by the number of downsampling processes; therefore, the number of downsampling processes performed to obtain the bloom image is also called the number of bloom layers in the bloom image.

[0003] Traditional techniques for determining the number of downsampling processes performed to obtain a floodlight image have the following problems: they require obtaining the application's source code and / or related configuration files, resulting in poor versatility; they require manual intervention for statistics, making it impossible to know in real time the number of downsampling processes performed to obtain the floodlight image.

[0004] Therefore, there is an urgent need for a data processing method that is versatile, requires no human intervention, and can determine in real time the number of downsampling processes performed to obtain the floodlight image. Summary of the Invention

[0005] This application provides a data processing method, apparatus, and electronic device. The method is highly versatile, requires no manual intervention, and can determine in real time the number of downsampling processes performed to obtain a floodlight image.

[0006] The first aspect of this application provides a data processing method applied to a terminal running a target program. The target program includes a rendering thread and a calculation thread. The rendering program loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of a plurality of frame buffers associated with each frame to obtain a floodlight image. The plurality of frame buffers correspond one-to-one with a plurality of storage addresses of the terminal, and any one frame buffer contains the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing on each frame. The method includes: the rendering thread executing the second link library file... The device performs the following operations: when the rendering thread calls the first function within each frame, it performs a hook operation on the first function and executes a data acquisition event to obtain the multiple frame buffers from the multiple storage addresses; and when the rendering thread calls the second function within each frame, it performs the hook operation on the second function and executes a data transmission event to send the acquired multiple frame buffers to the computing thread; in response to receiving the multiple frame buffers sent by the rendering thread, the computing thread determines the number of downsampling processes performed to obtain the floodlight image based on the multiple frame buffers.

[0007] A second aspect of this application provides a data processing apparatus applied to a terminal running a target program. The target program includes a rendering thread and a calculation thread. The rendering program loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of a plurality of frame buffers associated with each frame to obtain a floodlight image. The plurality of frame buffers correspond one-to-one with a plurality of storage addresses of the terminal, and any one frame buffer contains the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing on each frame. The apparatus includes: a first processing unit for executing the second link library file. The second processing unit is configured to: in each frame, if the rendering thread calls the first function, perform a hook operation on the first function and execute a data acquisition event to obtain the multiple frame buffers from the multiple storage addresses; and in each frame, if the rendering thread calls the second function, perform the hook operation on the second function and execute a data transmission event to send the acquired multiple frame buffers to the computing thread; the second processing unit is configured to: in response to receiving the multiple frame buffers sent by the rendering thread, determine the number of downsampling processes performed to obtain the floodlight image based on the multiple frame buffers.

[0008] A third aspect of this application also provides a computer-readable storage medium storing one or more computer instructions, characterized in that the instructions are executed by a processor to implement the data processing method described in any of the above technical solutions.

[0009] A fourth aspect of this application also provides an electronic device, including: a processor; and a memory for storing a data processing program, wherein after the server is powered on and runs the program through the processor, it executes the data processing method described above.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments disclosed in this application, nor is it intended to limit the scope of this application's disclosure. Other features disclosed in this application will become readily apparent from the following description.

[0011] In the technical solution of the data processing method provided in this application embodiment, the rendering thread in the target program running on the terminal loads a first dynamic link library file and a second dynamic link library file. When the rendering thread executes the second link library file, it performs the following operations: If the rendering thread calls a first function within each frame, a hook operation is performed on the first function, and a data acquisition event is executed to obtain multiple frame buffers from multiple storage addresses; and if the rendering thread calls a second function within each frame, a hook operation is performed on the second function, and a data transmission event is executed to send the acquired multiple frame buffers to the computing thread. The acquisition of multiple frame buffers associated with each frame is achieved by the rendering thread loading a custom second dynamic link library file. This process does not require modification of the target program's code, acquisition of relevant configuration files for the target program, or intrusive analysis of the target program, thus demonstrating the method's strong versatility. In the above implementation, execution is performed on a frame-by-frame basis, ensuring strong real-time performance. In the implementation where the computing thread processes the acquired multiple frame buffers to determine the number of downsampling operations performed to obtain the floodlight image, no manual intervention is required. In summary, the data processing method provided in this application is highly versatile, requires no manual intervention, and can determine in real time the number of downsampling processes performed to obtain the floodlight image. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1This is a schematic diagram illustrating an application scenario of the data processing method provided in the embodiments of this application.

[0014] Figure 2 This is a schematic diagram of a data processing method provided in an embodiment of this application.

[0015] Figure 2A Is to execute the above Figure 2 The schematic diagram of S230 is shown.

[0016] Figure 2B This is a schematic diagram of any edge included in any path of a directed acyclic graph provided in this application embodiment.

[0017] Figure 2C Is to execute the above Figure 2A The schematic diagram of S230-2 is shown.

[0018] Figure 3 This is a schematic diagram of another data processing method provided in the embodiments of this application.

[0019] Figure 4A This is one of the above-mentioned embodiments provided in this application. Figure 3 The described method involves a schematic diagram of a directed acyclic graph.

[0020] Figure 4B The above Figure 4A The diagram shown illustrates the type of each node in the directed acyclic graph.

[0021] Figure 5A This is one of the above-mentioned embodiments provided in this application. Figure 3 The described method involves a schematic diagram of a directed acyclic graph.

[0022] Figure 5B The above Figure 5A The diagram shown illustrates the type of each node in the directed acyclic graph.

[0023] Figure 6 This is a schematic diagram of a data processing device provided in an embodiment of this application.

[0024] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this application, the application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. However, this application can be implemented in many other ways different from those described above. Therefore, based on the embodiments provided in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0026] It should be noted that the terms "first," "second," "third," etc., in the claims, specification, and drawings of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. Such data are interchangeable where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown or described herein. Furthermore, the terms "comprising," "having," and their variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0027] For ease of understanding, the technical terms that may be involved in the embodiments of this application will be briefly introduced below.

[0028] 1. Image post-processing

[0029] Image post-processing refers to the process of optimizing images, primarily used to enhance characteristics such as anti-aliasing, high dynamic range (HDR) imaging, and flood illumination. Image post-processing techniques include flood illumination reduction, anti-aliasing, motion blur reduction, and depth-of-field processing. The objects of image post-processing can be images rendered from 3D scenes.

[0030] 2. Bloom

[0031] Flooding is a common optical phenomenon. Because physical cameras (i.e., actual video cameras or camcorders) typically cannot achieve perfect focus when capturing an image, light diffracts at the edges of objects as it passes through the camera lens, resulting in a halo effect. Flooding is not easily noticeable in low-light scenes, but it is more pronounced in high-light scenes. Therefore, flooding generally refers to the halo effect that occurs when a physical camera photographs a bright object.

[0032] 3. Flooding effect

[0033] In computer graphics, bloom effect, also known as specular highlight, is a computer graphics effect used in video games, demo animations, and High Dynamic Range Imaging (HDRI or HDR). Bloom effect produces stripes or feather-like rays of light around bright objects to blur image details, mimicking the bloom phenomenon in physical camera imaging, making images rendered by electronic devices appear more realistic.

[0034] 4. Open Graphics Library (OpenGL)

[0035] OpenGL refers to a professional graphics programming interface that defines a cross-programming language and cross-platform programming interface specification. It is used in industries including content creation, energy, entertainment, game development, manufacturing, pharmaceuticals, and virtual reality. OpenGL can help programmers develop high-performance, visually expressive graphics processing software on hardware devices such as personal computers (PCs), workstations, and supercomputers.

[0036] 5. OpenGL for Embedded Systems (OpenGL ES)

[0037] OpenGL ES is a subset of the OpenGL 3D graphics application programming interface (API), designed for embedded devices such as mobile phones and game consoles. OpenGL ES is based on OpenGL but removes many non-essential features, such as glBegin / glEnd, quadrilaterals (GL_QUADS), and polygons (GL_POLYGONS). After years of development, there are now three main versions: OpenGL ES 1.x for fixed-function pipeline hardware, OpenGL ES 2.x for programmable pipeline hardware, and OpenGL ES 3.x. OpenGL ES 1.0 is based on the OpenGL 1.3 specification, OpenGL ES 1.1 is based on the OpenGL 1.5 specification, and OpenGL ES 2.0 is defined according to the OpenGL 2.0 specification. OpenGL ES 3.x adds more features (such as compute shaders) to OpenGL ES 2.x.

[0038] OpenGL ES enables scenarios including but not limited to: image processing, such as image tone conversion and beautification; camera preview effect processing, such as beauty cameras; video processing; 3D games, etc.

[0039] 6. Directed Acyclic Graph (DAG)

[0040] In mathematics, particularly graph theory and computer science, a directed acyclic graph (DAG) is a directed graph without loops. If there is a non-directed acyclic graph where a path from point A to B via point C returns to A (forming a cycle), changing the direction of the edge from C to A to A to C transforms it into a DAG. The number of spanning trees in a DAG is equal to the product of the in-degrees of its nodes with non-zero in-degrees.

[0041] 7. Finite state machine (FSM)

[0042] Finite state automata are computational models abstracted for studying computational processes with limited memory and certain language classes. A finite state automaton has a finite number of states, each of which can transition to zero or more states. The input string determines which state transition is performed. A finite state automaton can be represented as a directed graph. Finite state automata are the subject of study in automata theory.

[0043] 8. Regular expressions

[0044] Regular expressions are patterns used to describe a set of string characteristics, used to match specific strings. They are tools that use a combination of special and ordinary characters to describe patterns, thereby achieving text matching. Regular expressions are currently integrated into various text editors and text processing tools.

[0045] 9. Dynamic Link Libraries

[0046] A dynamic link library is a library containing code and data that can be used by multiple programs simultaneously.

[0047] Dynamic link libraries (DLLs) have the .DLL extension on Windows platforms and the .SO extension on platforms such as Linux or Android. In this application, DLL refers to DLLs for multiple platforms, rather than specifically referring to DLLs for the Windows platform. That is, the DLL files described in the embodiments provided in this application can be either .DLL or .SO files.

[0048] Bloom is a common optical phenomenon, generally referring to the halo effect that occurs when a physical camera captures a bright object. Applying bloom processing to an image can visually improve its contrast, enhance its expressiveness, and achieve better rendering results. With the development of image processing technology, image bloom processing has been widely used in fields such as 3D games and animation production. Currently, the bloom processing performed by a terminal (e.g., a mobile device) on the bloom image associated with the application displayed on the terminal (e.g., a game bloom image) mainly includes the following operations: downsampling, Gaussian blurring, and downsampling image compositing. The number of downsampling operations performed on the bloom image is called the number of bloom layers, which is related to the bloom effect. More bloom layers generally result in a more delicate and realistic bloom effect. However, performing multiple downsampling operations on the terminal's graphics processing unit (GPU) involves computational complexity, high computational cost, and impacts on the terminal's GPU performance. Therefore, in testing game performance, if the game has a bottleneck on the GPU side, the number of bloom layers is a common consideration. If testers can know in real time the number of downsampling processes corresponding to the floodlight image, they can quickly locate the bottleneck problem of GPU performance degradation caused by excessive downsampling processes.

[0049] Traditional techniques involve determining the number of downsampling processes performed on the bloom image associated with the game scene based on the game source code and editor settings. However, due to issues of permissions, confidentiality, and security, obtaining the game source code and editor settings is often difficult. Furthermore, there may be issues such as overlapping settings priorities, invalid information, and useless information between settings in the code and configuration files, resulting in a significant workload for analysis. Another traditional technique involves intrusive analysis of the game scene to determine the number of downsampling processes performed on the bloom image associated with the game scene. This method includes the following steps: using a tool to capture frames of the game scene and saving the resulting frame information to a transferred file on a computer; using a tool to process the transferred file to obtain a list of rendering instructions; manually examining the rendering instruction list, guessing the function of each instruction, locating the instruction responsible for bloom, and obtaining the number of bloom layers (i.e., the number of downsampling processes performed on the bloom image). The above implementation requires the project team to provide a debugging package; otherwise, intrusive analysis is difficult. Furthermore, the transferred data is large in volume, with useful information distributed haphazardly and in small amounts, making automated processing difficult. In summary, the traditional methods for determining the number of downsampling processes performed to obtain a floodlight image have the following problems: (1) they require obtaining the application source code and / or related configuration files, resulting in poor versatility; (2) they require manual intervention for statistics, making it impossible to determine the number of downsampling processes performed to obtain a floodlight image in real time.

[0050] To address the aforementioned problems, this application provides a data processing method, apparatus, and electronic device.

[0051] The application scenarios and data processing methods applicable to the embodiments of this application will be described in detail below with reference to the accompanying drawings. It is understood that, where there is no conflict between the various embodiments provided in this application, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0052] First, the application scenarios of the data processing method applicable to the embodiments of this application will be described with reference to the accompanying drawings.

[0053] Figure 1 This is a schematic diagram illustrating an application scenario of the data processing method provided in the embodiments of this application. For example, Figure 1 The illustrated application scenario includes terminal 101 and server 102. Terminal 101 communicates and interacts with server 102 via network 103. Network 103 can be a wired network or a wireless network, and this application does not specifically limit it. The aforementioned wireless or wired network uses standard communication technologies and / or protocols.

[0054] Terminal 101 is a client application with an installed application. Terminal 101 can render the data to be rendered associated with the application and display the rendered result to the user through the user interface of Terminal 101. This application can be an application that requires downloading and installation, or an application that can be used immediately upon clicking; there is no limitation in this regard. The application can be any application that can provide a virtual environment for the user to immerse and manipulate virtual objects within that virtual environment. For example, the application can be a game application. Of course, besides game applications, other types of applications can also display virtual objects to the user and provide corresponding functions to these virtual objects. That is to say, the application installed on Terminal 101 can also be other types of applications besides game applications. For example, other types of applications can be, but are not limited to, any of the following: virtual reality (VR) applications, augmented reality (AR) applications, 3D mapping programs, military simulation programs, social applications, or interactive entertainment applications. The terminal 101 described above is equipped with at least the following processors: a central processing unit (CPU) and a graphics processing unit (GPU). The CPU performs computational analysis operations, such as performing computational analysis operations to obtain data to be rendered; the GPU performs rendering operations on the data to be rendered obtained by the CPU to obtain the corresponding rendered image. For example, the terminal 101 may be, but is not limited to, any of the following devices: smartphones, tablets, handheld computers, game consoles, e-book readers, multimedia playback devices, in-vehicle devices, wearable devices (e.g., smartwatches, smart bracelets, pedometers, etc.), smart home devices (e.g., refrigerators, televisions, air conditioners, electricity meters, etc.), smart robots, personal computers (PCs), and other electronic devices, as well as various forms of user equipment (UE), mobile stations (MS), and terminal equipment.

[0055] Server 102 is used to provide background services for applications in terminal 101. Server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Optionally, server 102 can provide background services for applications in multiple terminals 101 simultaneously.

[0056] It should be understood that the above Figure 1The application scenarios shown are for illustrative purposes only and do not constitute any limitation on the application scenarios applicable to the data processing methods provided in the embodiments of this application. Optionally, the above application scenarios may also include a greater number of terminals 101 or servers 102.

[0057] Next, the data processing method provided in the embodiments of this application will be described in conjunction with the accompanying drawings.

[0058] Figure 2 This is a schematic diagram of a data processing method provided in an embodiment of this application. For example, Figure 2 The entity executing the data processing method shown can be the one described above. Figure 1 Terminal 101 is shown. (As shown...) Figure 2 As shown, the data processing method includes S210 and S220. S210 and S220 will be described in detail below.

[0059] Before introducing the data processing method provided in the embodiments of this application, the execution subject and related concepts of the data processing method provided in the embodiments of this application will be introduced first. The data processing method provided in the embodiments of this application is applied to a terminal running a target program. The target program includes a rendering thread and a calculation thread. The rendering thread loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of the multiple frame buffers associated with each frame to obtain a floodlight image. The multiple frame buffers correspond one-to-one with multiple storage addresses of the terminal, and any one frame buffer contains the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing each frame. The rendering thread loads the first link library file and the second link library file. That is, in the above method, by setting the rendering thread to load the first link library file and the second link library file, the target program has the functions of the first link library file and the second link library file. In this way, it is not necessary to modify the code and configuration information of the target program.

[0060] The multiple frame buffers associated with each frame are drawn according to the drawing instructions corresponding to each frame buffer, allowing for real-time acquisition of the corresponding floodlight image. The number of frame buffers associated with each frame is not specifically limited and can be set according to actual needs. For example, each frame can be associated with 2, 5, or 7 frame buffers. Each frame buffer has a unique, non-repeatable, and unchangeable identity document (ID). That is, when the first function draws different frame buffers associated with each frame, the frame buffer IDs associated with the first function are different.

[0061] Multiple memory addresses correspond one-to-one with multiple framebuffers, with each framebuffer storing the data to be rendered at its corresponding memory address. Each framebuffer includes a framebuffer attachment, which is a portion of the memory address corresponding to that framebuffer that stores the data to be rendered. A framebuffer attachment can be a texture attachment or a renderbuffer object attachment. Specifically, when a framebuffer attachment is a texture attachment, if the framebuffer corresponding to that attachment is bound to other framebuffers, the rendering thread can draw the attachment onto at least one of the attachments in those other framebuffers using drawing commands. When a framebuffer attachment is a renderbuffer object attachment, if the framebuffer corresponding to that attachment is bound to other framebuffers, the rendering thread can use the attachment as the output of the current drawing command using drawing commands. Each framebuffer attachment also has a unique, non-repeatable, and unchangeable ID. The type of the aforementioned memory address is not specified; for example, the memory address could be a terminal memory address. Any framebuffer may include one or more framebuffer attachments, and any framebuffer attachment includes the following information: the ID of the framebuffer attachment and the attributes of the framebuffer attachment. In some implementations, the framebuffer attachment is a texture attachment, which includes the following information: the ID of the texture attachment and the attributes of the texture attachment (e.g., the texture attachment is one-dimensional, two-dimensional, or three-dimensional data). Optionally, in other implementations, the framebuffer attachment is a renderbuffer object attachment, which includes the following information: the ID of the renderbuffer object attachment and the attributes of the renderbuffer object attachment (e.g., the texture attachment is one-dimensional, two-dimensional, or three-dimensional data). Optionally, any of the above framebuffer attachments may also include other information, such as, but not limited to, the size of the framebuffer attachment.

[0062] Below, examples are given of the first and second functions included in the first link library file provided in the embodiments of this application.

[0063] In some implementations, when the first linker library file is OpenGL ES, the first function can be either the glBindFrameBuffer function or the glBindTexture function. Specifically, when the first function is glBindFrameBuffer, it is used to draw on any one of multiple framebuffers, and this framebuffer includes at least one framebuffer attachment that is a render buffer object attachment. When the first function is glBindTexture, it is used to draw on any one of multiple framebuffers, and this framebuffer includes at least one framebuffer attachment that is a texture attachment. For example, the following description uses OpenGL ES as the first linker library file and glBindFrameBuffer as the first function to illustrate the function's role. When the floodlight image is associated with two framebuffers, when the glBindFrameBuffer function is associated with framebuffer ID1, the rendering thread calls the glBindFrameBuffer function to draw the data to be rendered associated with framebuffer ID1; when the glBindFrameBuffer function is associated with framebuffer ID2, the rendering thread calls the glBindFrameBuffer function to draw the data to be rendered associated with framebuffer ID2. Framebuffer ID1 and framebuffer ID2 are different, and the framebuffer corresponding to framebuffer ID1 is different from the framebuffer corresponding to framebuffer ID2.

[0064] In some implementations, where the first linked library file is OpenGL ES, the second function can be the EGLSwapBuffer function defined in OpenGL ES.

[0065] In this embodiment, the number of threads included in the target program is not specifically limited. Optionally, in other implementations, the target program may also include a logic thread. In this implementation, the target program includes a rendering thread, the aforementioned calculation thread, and the logic thread. The logic thread is generally used to handle logic-related content, such as handling communication with the game server or calculating enemy AI in the game. Based on the aforementioned logic, the logic thread calculates and determines the object to be rendered (e.g., the model position and model structure of the object to be rendered) for use by the rendering thread. The rendering thread is composed of at least a CPU and a GPU; the calculation thread is composed of at least a CPU; and the logic thread is composed of at least a CPU. The relationship between the rendering thread and the calculation thread is not specifically limited. In some implementations, the rendering thread and the calculation thread are two different threads included in the target program. Optionally, in other implementations, the calculation thread is a sub-thread of the rendering thread. The target program can provide a graphical user interface to display a floodlight image; the type of the target program is not specifically limited. For example, the target program can be a game application, a virtual reality (VR) application, or an augmented reality (AR) application. There are no specific limitations on the form of the program on the aforementioned terminal. For example, the terminal can be a smartphone, a smart wearable device, an in-vehicle device, or a server.

[0066] The rendering engine used by the rendering thread running on the aforementioned terminal to perform rendering operations is not specifically limited; these rendering engines can utilize OpenGL or OpenES to draw images. For example, when the target program is a game application, the rendering engine can be the UE game engine or the Unity game engine. Optionally, in some implementations, the target program is a game application, and the first link library file is the OpenGL open graphics library file or the OpenGLES embedded system open graphics library. Optionally, in other implementations, the target program is a program other than a game application, and the first link library file is the OpenGL open graphics library file or the OpenGLES embedded system open graphics library. For example, other programs can be, but are not limited to, virtual reality programs.

[0067] In this embodiment, the types of the first and second library files are not specifically limited. In practical applications, the type of the library file depends on the platform on which it runs. For example, when the platform is Windows, the first and second library files can be .DLL type files. Similarly, when the platform is Linux or Android, the first and second library files can be .SO type files.

[0068] S210 and S220 will now be described in detail.

[0069] S210, the rendering thread executes the second linked library file to perform the following operations: if the rendering thread calls the first function within each frame, a hook operation is performed on the first function, and a data acquisition event is performed to obtain multiple frame buffers from multiple storage addresses; and if the rendering thread calls the second function within each frame, a hook operation is performed on the second function, and a data transfer event is performed to send the obtained multiple frame buffers to the computing thread.

[0070] Executing S210 above, that is, by using the second link library file loaded by the rendering thread, when the rendering thread calls the first function within each frame, it does not execute the first function immediately, but instead executes and retrieves data from multiple frame buffers from multiple storage addresses. The first function is used to draw any one of the multiple frame buffers associated with each frame to obtain a floodlight image. Furthermore, when the rendering thread calls the second function within each frame, it does not execute the second function immediately, but instead executes a data transfer event to send the retrieved multiple frame buffers to the computation thread. The second function indicates the end of drawing for each frame. In other words, executing S210 allows the rendering thread to send the multiple frame buffers corresponding to each frame, which are retrieved in real time, to the computation thread.

[0071] The second link library file includes: functions for listening for events that call the first and third functions within each frame, functions for executing data acquisition events, and functions for executing data transmission events. There is no specific limitation on the number of functions included in the second link library file. For example, the second link library file may include a function that has the following functions: listening for events that call the first and third functions within each frame, executing data acquisition events, and executing data transmission events. As another example, the second link library file may include function 1 and function 2, where function 1 is used to listen for events that call the first and third functions within each frame, and function 2 is used to execute data acquisition events and execute data transmission events. It is understood that the second link library file is a custom link library file provided in this application. Optionally, before executing S210 above, the following operation may also be performed: generating the second link library file.

[0072] The implementations of S210 described above, namely, "when the rendering thread calls the first function within each frame, perform a hook operation on the first function and execute a data acquisition event to retrieve multiple frame buffers from multiple storage addresses," and "when the rendering thread calls the second function within each frame, perform a hook operation on the second function and execute a data transfer event to send the retrieved multiple frame buffers to the computation thread," are not specifically limited. In some implementations, the method described in S210 can be achieved by setting the address of the function used to execute the data acquisition event to be the same as the address of the first function, and by setting the address of the function used to execute the data transfer event to be the same as the address of the second function.

[0073] Optionally, in some implementations, the computation thread is a sub-thread of the rendering thread. Before the rendering thread sends the acquired flood image associated with multiple frame buffers to the computation thread, the method further includes: in response to detecting a call to a third function within each frame, the rendering thread creates the computation thread. That is, in the implementation where the computation thread is a sub-thread of the rendering thread, the rendering thread performs the following operations: upon detecting a call to a third function of the first linked library file, it first creates the computation thread; then, it sends the acquired multiple frame buffers to the computation thread.

[0074] Executing S210 above, that is, when the rendering thread detects the end of the third function for drawing the flood image, the rendering thread sends the multiple frame buffers associated with the acquired flood image to the calculation thread, so that the calculation thread can analyze and process the multiple frame buffers associated with each frame. In this embodiment, when the rendering thread sends the acquired multiple frame buffers to the calculation thread, the rendering thread can also draw on the multiple frame buffers to obtain the flood image. That is, in some implementations, the method further includes: the rendering thread executing a second link library file to perform the following operations: when it detects that the rendering thread has sent the acquired multiple frame buffers to the calculation thread, it ends the Hook operation of the first and second functions, and calls the first and second functions, so that in each frame, any one of the multiple frame buffers is drawn to obtain the flood image.

[0075] In this embodiment, after executing S210, the rendering thread sends the multiple frame buffers associated with each frame to the calculation thread. Correspondingly, the calculation thread receives the multiple frame buffers associated with each frame sent by the rendering thread. In the process of the calculation thread obtaining the multiple frame buffers associated with each frame, the rendering thread loads a custom second dynamic link library file, which eliminates the need to modify the target program's code or obtain its configuration files. This method is highly versatile. Furthermore, the above implementation is executed on a frame-by-frame basis, ensuring the real-time nature of the obtained frame buffers; therefore, this method has strong real-time performance.

[0076] S220, in response to receiving multiple frame buffers sent by the rendering thread, the calculation thread determines the number of downsampling processes to be performed to obtain the floodlight image based on the multiple frame buffers.

[0077] Below, in conjunction with Figure 2A S220-1 and S220-2 shown illustrate the method described in S220 above. That is, performing S220 above includes at least S220-1 and S220-2.

[0078] S220-1, the computation thread generates a directed acyclic graph (DAG) containing multiple paths based on multiple framebuffers, wherein any path among the multiple paths includes at least one directed edge; the at least one directed edge is an edge pointing from a first node to a second node; at least one framebuffer attachment included in the first framebuffer corresponding to the first node is bound as a texture attachment to the second framebuffer corresponding to the second node, so that when the second framebuffer is drawn, at least one framebuffer attachment included in the first framebuffer is drawn to the second framebuffer; the second framebuffer is obtained by performing a preset process on the first framebuffer, and the type of the second node is associated with the preset process; the multiple framebuffers include the first framebuffer and the second framebuffer.

[0079] In S220-1 above, the directed acyclic graph includes multiple paths, where each path includes at least one directed edge; a directed edge consists of a first node and a second node, where the first node is also called the parent node of the second node. For example, Figure 2B This diagram illustrates any edge included in any path of a directed acyclic graph provided in this embodiment. When drawing the second frame buffer, at least one frame buffer attachment included in the first frame buffer is drawn to the second frame buffer; that is, at least one frame buffer attachment included in the first frame buffer is a texture attachment type.

[0080] In S220-1 above, the type of the second node is associated with the preset processing. The preset processing is associated with the rendering effect presented by the flood image. In some implementations, the flood image is a flood image. In practical applications, obtaining the flood image requires at least the following processing: initialization phase processing, downsampling processing, blurring processing (e.g., Gaussian blurring), upsampling processing, and post-processing. The number of downsampling processes performed to obtain the flood image is the same as the number of upsampling processes performed. Based on this, the preset processing can at least include: initialization processing, downsampling processing, blurring processing (e.g., Gaussian blurring), upsampling processing, and post-processing. The initialization processing can, but is not limited to, adjusting the framebuffer size; the post-processing is the process of combining other processes included in the preset processing with special effects.

[0081] Optionally, in some implementations, if obtaining the floodlight image allows for other processing besides the above-described processing, then the preset processing may also include such other processing. For example, other processing may include, but is not limited to, GPU particle computation processing or shadow computation processing. Below, examples are provided to describe the type of the second node and the preset processing. For instance, if the preset processing is upsampling, then the type of the second node is upsampling processing type; in this case, the second node is also called an upsampling processing node. Similarly, if the preset processing is GPU particle computation processing, then the type of the second node is GPU particle computation type; in this case, the second node is also called a GPU particle computation node.

[0082] Optionally, in some implementations, the target program further includes a logic thread. This logic thread determines the object to be drawn corresponding to the floodlight image. Multiple frame buffers indicate the drawing method for the object. The in-degree of the starting node of any path is zero, the out-degree of the ending node of any path is zero, and the frame buffer corresponding to the starting node of any path is determined based on the object to be drawn corresponding to the floodlight image. The type of the starting node of any path is associated with the method used to determine the frame buffer corresponding to the starting node based on the object to be drawn corresponding to the floodlight image. For example, if the frame buffer corresponding to the starting node is obtained by performing shadow mapping calculations on the object to be drawn, then the type of the starting node is the shadow mapping calculation type. Similarly, if the frame buffer corresponding to the starting node is obtained by performing GPU particle calculations on the object to be drawn, then the type of the starting node is the GPU particle calculation type. It should be noted that when any path includes only two nodes, the first node included in the path is the starting node of the path, and the second node included in the path is the ending node of the path.

[0083] The following describes, with reference to the accompanying drawings, an example of the directed acyclic graph (DAG) obtained after performing S220-1. For example, Figure 4A This illustration shows a schematic diagram of a directed acyclic graph (DAG) corresponding to multiple frame buffers associated with a floodlight image, as provided in an embodiment of this application. See also Figure 4A The DAG graph includes 12 nodes (i.e., nodes 401 to 412), and these 12 nodes correspond one-to-one with 12 different frame buffers. That is, the floodlight image is obtained by rendering these 12 frame buffers. Figure 4A The DAG graph shown includes multiple paths, for example Figure 4A The DAG shown includes a path: Node 401 -> Node 402 -> Node 404 -> Node 412, where node 401 has an in-degree of zero, meaning it is the starting node of this path; node 412 has an out-degree of zero, meaning it is the ending node of this path; and nodes 402 and 404 are intermediate nodes of this path. It should be understood that the above... Figure 4A The directed acyclic graph corresponding to the floodlight image shown is for illustrative purposes only and does not constitute any limitation. Below, we will discuss... Figure 4AThe types of directed edges and their associated nodes are described. For example, consider edge 401-402, which is associated with nodes 401 and 402. Node 401 has an in-degree of zero, meaning it is the starting node. Edge 401-402 is a directed edge pointing from node 401 to node 402. At least one framebuffer attachment corresponding to node 401 is bound as a texture attachment to the framebuffer corresponding to node 402. The framebuffer corresponding to node 402 is obtained by performing a preset process on the framebuffer corresponding to node 401. Node 402 is associated with this preset process, and node 401 is also referred to as the parent node of node 402. See also [example description]. Figure 4B When the frame buffer corresponding to node 402 is obtained by performing GPU particle computation processing on the frame buffer corresponding to node 401, the type of node 402 is GPU particle computation type. The frame buffer corresponding to node 401 is determined based on the object to be drawn, and the type of node 402 is associated with the method of determining the frame buffer corresponding to the starting node based on the object to be drawn corresponding to the floodlight image. For example, see... Figure 4B When the framebuffer corresponding to node 401 contains rendering data of the object to be drawn obtained through GPU particle computation, the type of node 401 is GPU particle computation type. For example, taking the edge 405-406 associated with nodes 405 and 406 as an example, edge 405-406 is a directed edge pointing from node 405 to node 406. At least one framebuffer attachment included in the framebuffer corresponding to node 405 is bound as a texture attachment to the framebuffer corresponding to node 406, and the framebuffer corresponding to node 406 is obtained by performing a preset process on the framebuffer corresponding to node 405. That is, when the framebuffer corresponding to node 406 is bound, at least one framebuffer attachment included in the framebuffer corresponding to node 405 will be drawn as a texture attachment onto the framebuffer corresponding to node 406. For example, see... Figure 4B When the frame buffer corresponding to node 406 is obtained by performing downsampling and blurring processing on the frame buffer corresponding to node 405, the type of node 406 is flood downsampled blur type. For example, as shown... Figure 5A As shown, the DAG graph includes 8 nodes (nodes 500 to 507), and these 8 nodes correspond one-to-one with 8 different framebuffers. That is, the floodlight image is obtained by rendering these 8 framebuffers. Figure 5A For an understanding of the directed acyclic graph shown, please refer to the explanation of the directed acyclic graph shown in 4A above, which will not be elaborated here.

[0084] S220-2, the calculation thread determines the number of downsampling processes to be performed to obtain the floodlight image based on the directed acyclic graph.

[0085] Below, in conjunction with Figure 2CThe steps S220-21 to S220-23 illustrate the method described in S220-2 above. That is, performing S220-2 above includes at least S220-21 to S220-23.

[0086] S220-21, the computation thread determines the target region of any path, wherein the type of any node in the target region of any path is associated with the first processing; the first processing is at least one of the following: downsampling processing, fuzzing processing, or upsampling processing, and the preset processing includes the first processing; the number of nodes of the first type in the target region of any path is equal to the number of nodes of the second type; the first type is associated with downsampling processing, and the second type is associated with upsampling processing.

[0087] The calculation thread determines the target region of any path, including: determining the type of any node based on a preset mapping relationship and the attribute information of any node included in the path; wherein the preset mapping relationship is a mapping relationship between the attribute information of a node and the type of a node; the attribute information of a node includes the out-degree of the node, the in-degree of the node, and the types of nodes other than the node in the directed edges associated with any node; the calculation thread determines the target region of any path based on the sequence information of any path and a preset expression; wherein the sequence information of any path includes multiple bits, and the i-th bit of the multiple bits is the type of the i-th node in the path from the starting node to the ending node, where i is a positive integer; the preset expression is used to identify the type of node associated with the first processing. Below is an example illustrating how the calculation thread determines the target region of any path. For example, Figure 4B The directed acyclic graph shown includes a path: node 403->node 404->node 405->node 406->node 407->node 408->node 409->node 410->node 412, and the target region of this path includes the following nodes: node 406, node 406, node 408, node 409, and node 410. For example, Figure 5B The directed acyclic graph shown includes a path: node 500 -> node 501 -> node 502 -> node 506 -> node 507, and the target region of this path includes the following nodes: node 502 and node 506. It should be noted that if a path in the above directed acyclic graph does not include nodes of the type associated with the first process, then that path does not include the target region. For example, see [link to example]. Figure 4B The directed acyclic graph shown includes a path: node 411 -> node 412. Neither node 411 nor node 412 is associated with the first process. Therefore, according to the above determination rules, this path does not include the target region.

[0088] Optionally, in some other implementations, the first process described above can also be an initialization process, wherein the initialization process can, but is not limited to, adjusting the size of the framebuffer.

[0089] In the above implementation, the preset mapping relationship is the mapping relationship between the attribute information of the node and the type of the node; the attribute information of the node includes the out-degree of the node, the in-degree of the node, and the type of the nodes in the directed edges associated with any node, excluding any node.

[0090] Table 1

[0091]

[0092]

[0093] For example, Table 1 above illustrates a preset mapping relationship. It should be understood that the preset mapping relationship shown in Table 1 is merely illustrative and does not constitute any limitation. It is understood that in the preset mapping relationship shown in Table 1, downsampling processing and blurring processing are performed by two different nodes.

[0094] Table 2

[0095]

[0096] For example, Table 2 above shows another preset mapping relationship. It should be understood that the preset mapping relationship shown in Table 2 is only illustrative and does not constitute any limitation. It is understood that in the preset mapping relationship shown in Table 2, downsampling processing and blurring processing are performed by the same node, and this same node is another node in the directed edge associated with the downsampling blurring processing node. This directed edge is an edge from the same node to the downsampling blurring processing node. In some implementations, the frame buffer corresponding to the intermediate node shown in Table 2 above is the same as the frame buffer corresponding to the downsampling blurring node. In other implementations, the frame buffer corresponding to the intermediate node shown in Table 2 above is obtained by performing downsampling blurring processing on the frame buffer corresponding to the downsampling blurring node. For example, based on the preset mapping relationship shown in Table 2 above, according to... Figure 4A The DAG shown can be obtained Figure 4A The DAG shown includes the type of each node. Figure 4A For details on the type of each node included in the DAG shown, please refer to [link / reference]. Figure 4B The DAG shown is... It is understandable that... Figure 4B In the DAG shown, nodes 401, 402, 403, 404, 411, and 412 are general nodes.

[0097] The presentation format of the aforementioned preset mapping method is not specifically limited. In some implementations, the preset mapping relationship can be implemented using a Finite State Automaton (FSM). In this approach, the state machine corresponding to the FSM is used to implement the preset mapping relationship, where the input of the FSM is the attribute information of a node, and the output of the FSM is the type of that node. In this implementation, the design of the finite state automaton enables the function of determining the target region of any path, avoiding manual review of the transferred data. For example, see S306-1 below, which describes a specific example of a computation thread using an FSM to determine the node type in a path included in a directed acyclic graph. Details not elaborated here can be found in the relevant description in S306-1 below. Optionally, in other implementations, the aforementioned preset mapping relationship can also be presented in the form of a data table stored in a database, i.e., the preset mapping relationship is stored in a data table (e.g., a two-dimensional data table). In this approach, the computation thread can determine the type of any node corresponding to the attribute information of any node included in any path by looking up the corresponding data table.

[0098] The aforementioned preset expression is used to identify nodes of the type associated with the first processing. In some implementations, the preset expression can be a regular expression. The sequence information of any path includes multiple bits, and the i-th bit of these multiple bits represents the type of the i-th node in any path from the starting node to the ending node, where i is a positive integer. For example, using... Figure 5B Taking the DAG diagram shown as an example, Figure 5B The path shown is: Node 500 -> Node 507. The sequence information of this path is: basic rendering node, post-processing node. The basic rendering node is responsible for drawing the 3D scene, such as drawing opaque objects, semi-transparent objects, etc., onto the framebuffer. The post-processing node is the node that mixes and synthesizes the results of multiple post-processing steps.

[0099] Optionally, in some implementations, before the computation thread determines the target region of any path, the following steps may be performed: the computation thread performs path search processing on the directed acyclic graph to obtain multiple paths. In this embodiment, the method by which the computation thread obtains the multiple paths included in the generated directed acyclic graph is not specifically limited. For example, the computation thread can traverse the directed acyclic graph to find all nodes with an in-degree of zero and all nodes with an out-degree of zero; based on the nodes with in-degree and out-degree of zero included in the directed acyclic graph, the computation thread finds all paths included in the directed acyclic graph through parallelized depth-first traversal.

[0100] S220-22, the calculation thread determines the number of downsampling processes to be performed to obtain the floodlight image based on the target area of ​​the first path among multiple paths, wherein the number of first-type nodes in the target area of ​​the first path is greater than or equal to the number of first-type nodes in the target area of ​​the second path; the second path is any path other than the first path among the multiple paths. S220-23, the calculation thread determines the number of first-type nodes in the target area of ​​the first path as the number of downsampling processes to be performed to obtain the floodlight image.

[0101] For example, with Figure 4B The directed acyclic graph shown is an example of the directed acyclic graph corresponding to multiple frame buffers associated with the floodlight image, and... Figure 4B The intermediate node shown is the downsampled blur node. Figure 4B The first path in a directed acyclic graph (DAG) can be: Node 403->Node 404->Node 405->Node 406->Node 407->Node 408->Node 409->Node 410->Node 412. The target region of this path includes the following nodes: Node 406, Node 407, Node 408, Node 409, and Node 410. Furthermore, the target region of this path includes three nodes associated with downsampling processing (i.e., nodes 406, 407, and 408). In other words, obtaining... Figure 4B The floodlight image corresponding to the finite acyclic graph shown is subjected to 3 downsampling operations.

[0102] In S220-21 to S220-23 above, which describes the method by which the computation thread determines the number of downsampling processes performed on the flood image based on the directed acyclic graph (DAG) corresponding to multiple frame buffers associated with the flood image: the computation thread first determines the target region of each path in the multiple paths included in the DAG, and this target region is the region associated with the first processing. Then, the computation thread determines the number of downsampling processes performed on the flood image corresponding to the DAG based on the number of nodes of the first type in the target region of each path in the multiple paths included in the DAG. This implementation process is achieved by the computation thread and requires no manual intervention. It should be understood that the above... Figure 3 This is for illustrative purposes only and does not constitute any limitation on the data processing methods provided in this application. For example, the above description uses the example of a rendering thread listening to and acquiring data, and a computing thread performing analysis on the acquired data. Optionally, in other implementations, the computing thread and the rendering thread can be the same thread, that is, the same thread can execute the steps performed by the rendering thread and the steps performed by the computing thread.

[0103] In this embodiment, the rendering thread of the target program running on the terminal loads a first dynamic link library file and a second dynamic link library file. When the rendering thread executes the second link library file, it performs the following operations: If the rendering thread calls a first function within each frame, a hook operation is performed on the first function, and a data acquisition event is executed to retrieve multiple frame buffers from multiple storage addresses; and if the rendering thread calls a second function within each frame, a hook operation is performed on the second function, and a data transfer event is executed to send the acquired multiple frame buffers to the computation thread. The acquisition of multiple frame buffers associated with each frame is achieved by the rendering thread loading a custom second dynamic link library file. This process does not require modification of the target program's code, acquisition of the target program's configuration files, or intrusive analysis of the target program, meaning the method is highly versatile. Furthermore, the above implementation is performed on a frame-by-frame basis, ensuring strong real-time performance. Furthermore, the computation thread processes the acquired multiple frame buffers to generate corresponding directed acyclic graphs (DAGs); and processes the DAGs to determine the number of downsampling operations performed when acquiring the floodlight image. In this process, the computation thread performs all operations without manual intervention, which helps reduce the workload of manual analysis. The rendering thread and the computation thread can be two different threads, effectively preventing subsequent steps such as DAG reconstruction from blocking the game's logic calculations and screen updates. In summary, the data processing method provided in this application is highly versatile, requires no manual intervention, and can determine the number of downsampling operations performed when acquiring the floodlight image in real time.

[0104] Below, in conjunction with Figure 3 This application introduces another data processing method provided by an embodiment. It is understood that... Figure 3 The data processing method described above is as follows. Figure 2 A specific example of the described data processing method. Specifically, see below. Figure 3 The described floodlight image is as described above. Figure 2 An example of a floodlight image is described below, and the floodlight image was obtained using the game engine UE. Figure 3 The described floodlight area is as described above. Figure 2 An example of the target region described, and Figure 3 The described floodlight region includes an initialization node, a downsampled blur node, an intermediate node, and an upsampled node.

[0105] Figure 3 This is a schematic diagram illustrating another data processing method provided in an embodiment of this application. It should be understood that... Figure 3The examples provided are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art will understand based on the following... Figure 3 The examples provided clearly demonstrate that various equivalent modifications or variations can be made, and such modifications and variations also fall within the scope of the embodiments of this application. It is understood that... Figure 3 The entity executing the data processing method shown can be the one described above. Figure 1 Terminal 101 is shown. (As shown...) Figure 3 As shown, the data processing method includes steps S301 to S306. Steps S301 to S306 will be described in detail below.

[0106] In the embodiments of this application, Figure 3 The described method applies to terminals running game applications. These game applications may include logic threads, rendering threads, and computation threads. The rendering thread in the game application loads OpenGL ES dynamic link library files and Hook dynamic link library files. Figure 3 The described method utilizes the game engine UE as the rendering engine. In UE, the data required for downsampling and blurring processes involved in bloom handling is stored in a single frame buffer. Optionally, when using Unity to implement bloom handling, the data required for downsampling and blurring processes are stored in two separate, consecutive frame buffers.

[0107] S301, the logic thread in the game application running in the terminal determines the objects that need to be rendered in the floodlight image and sends the determined objects that need to be rendered in the floodlight image to the rendering thread.

[0108] The logic thread and rendering thread in S301 above are two different threads in the game application. No specific restrictions are placed on how the logic thread determines the objects to be rendered in the floodlight image.

[0109] S302, the rendering thread in the game application running in the terminal determines the multiple frame buffer information to be rendered corresponding to the floodlight image, and stores the multiple frame buffer information to be rendered corresponding to the floodlight image into the terminal's memory.

[0110] The multiple frame buffer information described in S302 above is consistent with the above text. Figure 2The multiple frame buffers involved in the provided data processing method have the same definition; details not elaborated here can be found in the relevant descriptions above. In this application embodiment, the method principle for obtaining any one of the multiple frame buffer information is the same. Below, we will use S303 to obtain target frame buffer information as an example, where the target frame buffer information described below refers to any one of the aforementioned multiple frame buffer information.

[0111] S303, in response to the rendering thread of the game application running in the terminal calling OpenGL ES function #1 in the OpenGL ES dynamic link library file, execute the Hook function in the Hook dynamic link library file so that after obtaining the target frame buffer information associated with OpenGL ES function #1 from memory, execute OpenGL ES function #1 to draw the target frame buffer information.

[0112] Among them, OpenGL ES function #1 is mentioned above. Figure 2 An example of the first function in the described method; the Hook function is as described above. Figure 2 An example of the second function in the described method.

[0113] In this embodiment, during game application runtime, the rendering thread of the game application has already loaded the Hook dynamic link library file. That is, during game application runtime, the Hook function in the Hook dynamic link library file is constantly in a listening state. Executing S303, that is, when the Hook function detects that the rendering thread in the game application is calling OpenGL ES function #1, the rendering thread executes event 1 corresponding to the Hook function. Event 1 corresponding to the Hook function includes, in sequence: obtaining target framebuffer attachment information associated with OpenGL ES function #1 from memory; loading OpenGL ES function #1 from the OpenGL ES dynamic link library file, so that after obtaining the target framebuffer attachment information, the rendering thread executes OpenGL ES function #1 to draw the target framebuffer attachment information.

[0114] The rendering thread and the logic thread can be two separate threads included in a game application. Functions in OpenGL ES called by the rendering thread (e.g., OpenGL ES function #1) are not implemented in the game code associated with the game application, but rather in the terminal's system driver.

[0115] The OpenGL ES function #1 in the aforementioned OpenGL ES dynamic link library file is used to draw the target framebuffer information; that is, OpenGL ES function #1 is used to draw the data stored at the memory address corresponding to the target framebuffer. In some implementations, OpenGL ES function #1 is either the glBindFrameBuffer function or the glBindTexture function. When OpenGL ES function #1 is the glBindFrameBuffer function, when the target framebuffer associated with the glBindFrameBuffer function is bound, OpenGL ES function #1 is specifically used to output the framebuffer attachments included in the target framebuffer as the current drawing instruction. When OpenGL ES function #1 is the glBindTexture function, when the target framebuffer associated with the glBindFrameBuffer function is bound, OpenGL ES function #1 is specifically used to input the framebuffer attachments included in the target framebuffer as texture input. When other framebuffers besides the target framebuffer are bound, the drawing instructions are used to draw onto the attachments (or part of the attachments) of those other framebuffers.

[0116] In S303 above, the Hook function associated with the Hook dynamic link library is executed. That is, when the rendering thread calls OpenGL ES function #1, the Hook dynamic link library is mapped into the address space of the game application, and then the Hook function included in the Hook dynamic link library is accessed to execute the Hook event 1 corresponding to the Hook function.

[0117] Executing S303 above, that is, when the rendering thread calls OpenGL ES function #1, first use a custom Hook function to obtain the frame buffer attachment information associated with OpenGL ES function #1. In this way, the game code associated with the game application process is not modified. It only modifies the function address of the OpenGL ES function #1 associated with the OpenGL ES dynamic link library to the function address of the Hook function associated with the Hook dynamic link library when the OpenGL ES dynamic link library is loaded into the game application process at the system driver level.

[0118] After executing S303 above, target framebuffer information can be obtained. This target framebuffer information includes all framebuffer attachment information included in the target framebuffer. Any framebuffer attachment information includes at least one of the following: render buffer object attachment information, or texture attachment information. Render buffer object attachment information indicates that the arbitrary framebuffer attachment is used as a render buffer object, and texture attachment information indicates that the arbitrary framebuffer attachment is used as a texture attachment.

[0119] In this embodiment of the application, the floodlight image is an image obtained by drawing data stored in the memory addresses associated with multiple frame buffers corresponding to multiple frame buffer information. The frame buffer information corresponding to any one of the multiple frame buffers can be obtained by the method described in S303 above. The area is that the frame buffer IDs corresponding to different frame buffers are different, and the frame buffer attachment IDs corresponding to different frame buffer attachments are also different.

[0120] S304, in response to the rendering thread of the game application running in the terminal calling OpenGL ES function #2, execute the Hook function in the Hook dynamic link library file, so that the multiple frame buffer information corresponding to the acquired flood image is sent to the calculation thread. Here, OpenGL ES function #2 is used to indicate the end of the drawing of the flood image.

[0121] Among them, OpenGL ES function #2 is mentioned above. Figure 2 (An example of the third function in the described method). In this embodiment, the game application has already loaded the Hook dynamic link library file during runtime, meaning that the Hook function in the Hook dynamic link library file is always in a listening state during the game application's operation. Executing S304 above, that is, when the Hook function detects that the rendering thread in the game application calls OpenGL ES function #2, the rendering thread executes event 2 corresponding to the Hook function. Event 2 corresponding to the Hook function includes: sending multiple frame buffer information corresponding to the acquired flood image to the calculation thread.

[0122] The OpenGL ES function #2 is used to indicate the end of the rendering of the flood image. In other words, calling OpenGL ES function #2 signifies the end of the flood image rendering process, and no further rendering of the flood image will be performed. Specifically, OpenGL ES function #2 can be the eglSwapBuffer function in OpenGL ES. The eglSwapBuffer function is used to display the framebuffer UI rendered by S303 on the terminal screen to indicate the end of the flood image rendering.

[0123] S305, the computing thread in the game application running in the terminal generates a directed acyclic graph #1 according to the preset generation strategy and multiple frame buffer information corresponding to the floodlight image, and obtains all paths included in the directed acyclic graph #1.

[0124] The preset generation strategy includes: establishing multiple nodes corresponding to multiple frame buffers associated with multiple frame buffer information, wherein multiple frame buffers and multiple nodes correspond one-to-one, and any node represents the corresponding frame buffer; based on the multiple frame buffer information, determining the directed edge corresponding to any two nodes among the multiple nodes corresponding to the multiple frame buffer information, wherein the directed edge corresponding to any two nodes is an edge from node 1 to node 2, and any two nodes include node 1 and node 2, and at least one frame buffer attachment included in the frame buffer corresponding to node 1 is bound as a texture attachment to the frame buffer corresponding to node 2, wherein the frame buffer corresponding to node 1 is different from the frame buffer corresponding to node 2.

[0125] The directed acyclic graph #1 consists of multiple nodes and multiple edges. Each node corresponds one-to-one with multiple framebuffers associated with its corresponding framebuffer information. Any one of the edges represents a dependency between two framebuffers associated with that edge. This dependency can be either a framebuffer attachment (of type render buffer object) bound to another framebuffer, or a framebuffer attachment (of type texture attachment) bound to another framebuffer.

[0126] In this embodiment, the floodlight image is an image obtained through floodlight processing. The number of frame buffers associated with the multiple frame buffer information corresponding to the floodlight image can be 5, 7, or more, and is not specifically limited thereto. For example, when the floodlight image corresponds to 5 frame buffers, drawing the data stored in the memory addresses corresponding to these 5 frame buffers can obtain a floodlight image with a floodlight effect. Below, the directed acyclic graph obtained using the method described in S305 above is described using different data frame buffers corresponding to the floodlight image as an example. For example, Figure 4A It is shown that according to the above Figure 3 A schematic diagram of a directed acyclic graph generated by the method described in S305. See also Figure 4A The DAG diagram shown includes 12 nodes, which correspond to 12 frame buffers of the flood image. That is, drawing the data stored in the 12 frame buffers can obtain a flood image with flood effect. Figure 4AThe DAG graph shown also includes 14 edges, each representing the dependency between the framebuffers of the two nodes associated with that edge. The following explanation uses edge 405-406 as an example. Edge 405-406 associates nodes 405 and 406. Edge 405-406 is a directed edge from node 405 to node 406, meaning that at least one framebuffer attachment included in the framebuffer corresponding to node 405 is bound as a texture attachment to at least one framebuffer attachment included in the framebuffer corresponding to node 406. In other words, at least one framebuffer attachment included in the framebuffer corresponding to node 405 is drawn as a texture attachment onto the attachment of the framebuffer corresponding to node 406 using drawing instructions. For example... Figure 5A A schematic diagram of another directed acyclic graph generated according to the method described in S305 above is shown. See also Figure 5A The DAG diagram shown includes 8 nodes and 10 edges. In this method, the flood image corresponds to 8 frame buffers; that is, drawing the data stored in these 8 frame buffers can obtain a flood image with a flood effect. It is understandable that... Figure 5A The understanding of the nodes and edges shown is consistent with the above. Figure 4A The nodes and edges shown are understood in the same way; for details not elaborated here, please refer to the relevant descriptions above.

[0127] The directed acyclic graph #1 described in S305 above may include multiple paths, each path having an in-degree of 0 for its starting node and an out-degree of 0 for its ending node. The implementation method for obtaining all paths included in the directed acyclic graph #1 as described in S305 is not specifically limited. For example, a computation thread can traverse all nodes included in the directed acyclic graph #1 to obtain all nodes with an in-degree of 0 and all nodes with an out-degree of 0; for all nodes with an in-degree of 0 and all nodes with an out-degree of 0, all paths included in the directed acyclic graph #1 can be found through parallelized depth-first traversal.

[0128] After executing S305 above, a directed acyclic graph #1 and all paths included in the directed acyclic graph #1 can be obtained. The directed acyclic graph #1 describes multiple frame buffers corresponding to the floodlight image (represented by nodes in the directed acyclic graph #1) and the dependencies between these multiple frame buffers (represented by directed edges in the directed acyclic graph #1).

[0129] Next, in conjunction with S306, the number of downsampling operations performed to obtain the floodlight image is determined based on the finite state automaton #1 and the aforementioned finite acyclic graph #1.

[0130] S306, the computation thread in the game application running in the terminal determines the number of downsampling operations to be performed to obtain the floodlight image based on the directed acyclic graph #1 and the finite state automaton.

[0131] The computation thread in the game application running on the terminal determines the number of downsampling operations performed to obtain the floodlight image based on the directed acyclic graph #1 and the finite state automaton. This can include the following steps: The computation thread in the game application running on the terminal determines the floodlight interval corresponding to each path in the directed acyclic graph #1 based on the directed acyclic graph #1 and the finite state automaton. The floodlight interval includes nodes of any of the following types: initialization nodes, downsampling blur nodes, intermediate nodes, or upsampling nodes; and the downsampling blur nodes included in the floodlight interval... The number of downsampled blur nodes is the same as the number of upsampled nodes; the number of downsampled blur nodes included in the floodlight interval corresponding to each path in the directed acyclic graph #1 is determined as the number of downsampled blur nodes included in each path; the number of downsampled blur nodes included in the target path among the multiple paths in the directed acyclic graph #1 is determined as the number of downsampling operations performed to obtain the floodlight image, wherein the number of downsampled blur nodes included in the floodlight interval corresponding to the target path is greater than or equal to the number of downsampled blur nodes included in the floodlight interval corresponding to any path among the multiple paths other than the target path. Optionally, in some other implementations, the node type included in the above-mentioned floodlight interval can also be any of the following types: downsampled blur nodes, intermediate nodes, or upsampled nodes.

[0132] In this embodiment, the principle for determining the floodlight region corresponding to any path in the directed acyclic graph #1 is the same, as is the principle for determining the number of downsampled fuzzy nodes included in any path based on the floodlight region corresponding to that path. Below, taking the determination of path #1 in the directed acyclic graph #1 by a computational thread as an example, and combining S306-1, S306-2, and S306-3, we describe how to determine the number of downsampled fuzzy nodes included in path #1 using a finite state automaton.

[0133] S306-1, The computation thread in the game application running in the terminal determines the type of each node in path #1 in the directed acyclic graph #1 based on the directed acyclic graph #1 and the finite state automaton #1.

[0134] Executing S306-1 above, the computation thread determines the type of each node in the directed acyclic graph #1 based on the directed acyclic graph #1 and the finite state automaton #1. The type of each node is associated with a processing operation performed on the framebuffer corresponding to another node in the directed edge associated with that node. The data associated with the framebuffer corresponding to that node is the data obtained by performing the processing operation on the framebuffer corresponding to that other node, and the other node is the parent node of that node. The processing operation includes at least the following operations: initialization processing, downsampling processing, blurring processing, upsampling processing, and post-processing. For example, if a node in the directed acyclic graph #1 is a downsampling processing node, then the type of that node is determined by performing downsampling processing on the framebuffer corresponding to another node in the directed edge associated with that node, and the data associated with the framebuffer corresponding to that node is the data obtained by performing the downsampling processing on the framebuffer corresponding to that other node, and the other node is the parent node of that node.

[0135] In this process, the computation thread in the game application running on the terminal determines the node type of each node in the path #1 included in the directed acyclic graph #1 based on the input of the finite state automaton #1. The input of the finite state automaton #1 includes: the out-degree of each node in the path #1, the in-degree of each node, and the types of nodes that have dependencies on each node. The output of the finite state automaton #1 is the node type of each node in the path #1. Table 3 below shows the mapping relationship between the input and output of a finite state automaton #1 provided in an embodiment of this application.

[0136] Table 3

[0137]

[0138] Referring to Table 3 above, the current node can be uniquely determined as an upsampled node, a downsampled fuzzy node, or a general node based on its in-degree and out-degree. The current node can be uniquely determined as an initialization node or an intermediate node based on its in-degree, out-degree, and the type of the other node in the edge associated with it. It should be noted that if a node is of type initialization node, the associated processing operation is initialization processing; if a node is of type downsampled fuzzy node, the associated processing operations include downsampling and fuzzing; if a node is of type intermediate node, the associated processing operations include downsampling and fuzzing; and if a node is of type upsampled node, the associated processing operation is downsampling. When the associated processing operation of a node is any operation other than initialization, downsampling, fuzzing, or upsampling, the node is of type general node. For example, continuing with the above... Figure 4A Taking the DAG diagram shown above as an example, using the judgment criteria shown in Table 3 above, we can obtain... Figure 4A The DAG graph shown includes the type of each node. For example, Figure 4B The above is shown Figure 4A The DAG graph shown includes the type of each node. See also Figure 4B In this context, nodes 401, 402, 403, 404, 411, and 412 are all general nodes. For example, node 401 has an in-degree of 0 and an out-degree of 1, and its corresponding frame buffer is obtained by performing GPU particle calculations on the object to be rendered, as determined by the logical thread. Similarly, node 406 has an in-degree of 1 and an out-degree of 2, and its corresponding frame buffer is obtained by performing downsampling and blurring on the frame buffer corresponding to its parent node 405 in the directed edge associated with node 406. Continuing with the above example... Figure 5A Taking the DAG diagram shown above as an example, using the judgment criteria shown in Table 3 above, we can obtain... Figure 5A The DAG graph shown includes the type of each node. For example, Figure 5B The above is shown Figure 5A The DAG graph shown includes the type of each node.

[0139] After executing S306-1 above, the computation thread can obtain the types of the multiple nodes included in path #1. For example, using... Figure 5BFor example, path #1 can be: node 500 -> node 501 -> node 502 -> node 503 -> node 504 -> node 505 -> node 506 -> node 507, where node 501 is the initialization node, node 502 is the downsampling fuzzy node, node 503 is the downsampling fuzzy node, node 504 is the intermediate node, node 505 is the upsampling node, node 506 is the upsampling node, and nodes 500 and 507 are general nodes.

[0140] S306-2, The computing thread in the game application running in the terminal determines the floodlight interval corresponding to path #1 based on the node type of each node in path #1 and the finite state automaton #2.

[0141] The input to the finite state automaton #2 is: a path #1 including the type of each node. For example, with... Figure 5B For example, when path #1 is: node 500 -> node 501 -> node 502 -> node 503 -> node 504 -> node 505 -> node 506 -> node 507, the input to finite state automaton #2 is the following sequence: node 500 is a general node, node 501 is an initialization node, node 502 is a downsampled blur node, node 503 is a downsampled blur node, node 504 is an intermediate node, node 505 is an upsampled node, node 506 is an upsampled node, and node 507 is a general node. The output of finite state automaton #2 is the floodlight range corresponding to path #1.

[0142] The floodlight range corresponding to path #1 includes multiple nodes within path #1. In some implementations, these multiple nodes can be of any of the following types: initialization nodes, downsampled blur nodes, intermediate nodes, or upsampled nodes; the number of downsampled blur nodes included in the floodlight range corresponding to path #1 is equal to the number of upsampled nodes. The frame buffer corresponding to an intermediate node can be obtained by performing downsampling and blurring processing on the frame buffer corresponding to its parent node.

[0143] In this embodiment, the aforementioned finite state automaton #2 can be implemented using a regular expression. This regular expression is used to filter out nodes of the following types from the multiple nodes included in path #1: initialization nodes, downsampled fuzzy nodes, intermediate nodes, or upsampled nodes. It is understood that after filtering out the above types of nodes using the regular expression, the computation thread can further perform the following steps: determining that the number of downsampled fuzzy nodes filtered from the multiple nodes included in path #1 is equal to the number of upsampled nodes. For example, the regular expression can be: pd+mu+, where pd represents filtering out initialization nodes and downsampled fuzzy nodes, and mu represents filtering out intermediate nodes and upsampled nodes.

[0144] S306-3, The calculation thread in the game application running in the terminal determines the number of downsampled blurred nodes included in the floodlight interval corresponding to path #1 as the number of downsampled blurred nodes included in path #1.

[0145] For example, with Figure 5B For example, path #1 can be: node 500 -> node 501 -> node 502 -> node 503 -> node 504 -> node 505 -> node 506 -> node 507. The floodlight interval corresponding to path #1 includes the following nodes: node 502, node 503, node 504, node 505 and node 506, and the number of downsampled blurred nodes included in this floodlight interval is 3 (i.e., node 502, node 503 and node 504).

[0146] After executing S306-1 to S306-3 above, the computation thread can obtain the number of downsampled blurred nodes in each of the multiple paths included in the directed acyclic graph #1. Subsequently, the computation thread determines the number of downsampled blurred nodes included in the target path among the multiple paths in the directed acyclic graph #1 as the number of downsampling operations performed to obtain the floodlight image, wherein the number of downsampled blurred nodes included in the floodlight interval corresponding to the target path is greater than or equal to the number of downsampled blurred nodes included in the floodlight interval corresponding to any path other than the target path among the multiple paths. For example, continuing with... Figure 5B Taking the DAG diagram shown as an example, this DAG diagram includes 4 paths. Specifically, path 1: node 500 -> node 501 -> node 507. Path 2: node 500 -> node 501 -> node 502 -> node 506 -> node 507. Path 3: node 500 -> node 501 -> node 502 -> node 503 -> node 505 -> node 506 -> node 507. Path 4: node 500 -> node 501 -> node 502 -> node 503 -> node 504 -> node 505 -> node 506 -> node 507. Based on this, it can be determined that the number of downsampled blurred nodes included in the floodlight interval corresponding to path 4 is greater than the number of downsampled blurred nodes included in the floodlight interval corresponding to any one of paths 3, 2, or 1. That is to say, Figure 5B The number of layers of the floodlight corresponding to the floodlight image shown in the DAG diagram (i.e., the number of downsampling operations performed to obtain the floodlight image) is 3.

[0147] The above Figure 3Taking rendering using the UE engine as an example, the data required for downsampling and blurring processes involved in the bloom processing in the UE game engine is stored in a single frame buffer. That is, a node in the finite acyclic graph #1 is a downsampling / blurring node, and the associated processing operations include downsampling and blurring. Optionally, when using the Unity engine to implement bloom processing, the data required for the downsampling and blurring processes are stored in two separate frame buffers. That is, the first node in two consecutive nodes in the finite acyclic graph #1 is a downsampling node, and the associated processing operation is downsampling; the second node in the two consecutive nodes is a blurring node, and the associated processing operation is blurring (e.g., Gaussian blur).

[0148] It should be understood that the above Figure 3 The data processing methods shown are merely illustrative and do not constitute any limitation on the data processing methods provided in this application. For example, in some other implementations, the above... Figure 3 The rendering thread and the computation thread described above can also be the same thread.

[0149] In this embodiment, the rendering thread in the game application process loads the OpenGL ES dynamic link library file and the Hook dynamic link library file. During the game application's operation, when the rendering thread in the game application process is detected calling a drawing function in OpenGL ES to begin drawing the flood image within each frame, the rendering thread executes the Hook function in the Hook dynamic link library file to retrieve multiple frame buffer information corresponding to the flood image drawn in OpenGL ES from memory. This process continues until the rendering thread is detected calling the drawing function that finishes the first frame, at which point the process of retrieving frame buffer information from memory ends, and the obtained multiple frame buffer information corresponding to the flood image is sent to the calculation thread. After retrieving all frame buffer information corresponding to the flood image, the Hook function in the rendering thread continues to call the drawing function in OpenGL ES to complete the drawing operation of the multiple frame buffer information. This method of retrieving multiple frame buffer information does not require modification of the game code associated with the game application process, nor does it require intrusive analysis of the game code. Furthermore, the computation thread processes all framebuffer information corresponding to the acquired flood image to generate a directed acyclic graph #1 (DAG) for the flood image. DAG #1 describes all framebuffers corresponding to the flood image, as well as the generation and usage order information of framebuffer attachments included in all framebuffers. Next, the computation thread uses a finite state automaton to traverse all paths in the DAG, determining the type of framebuffer corresponding to each node in each path. Then, the computation thread uses the finite state automaton to analyze the order of multiple nodes in each path of DAG #1 and the types corresponding to these nodes, determining the number of downsampled blur nodes included in the flood interval corresponding to each path. Finally, based on the number of downsampled blur nodes included in the flood interval corresponding to each path in DAG #1, the computation thread determines the number of downsampling operations performed to obtain the flood image corresponding to DAG #1. In the above implementation, the Hook function, which hijacks the drawing function in OpenGL ES, is called every frame. Therefore, the directed acyclic graph generated based on the acquired information from multiple frame buffers, and the calculation of the finite automaton, are all performed frame by frame, ensuring real-time data processing. The rendering thread and the calculation thread can be two different threads to prevent subsequent steps such as DAG reconstruction from blocking the game's logic calculations and screen updates. The design using the finite automaton achieves automated floodlight range detection, avoiding manual review of the transferred data. In summary, the data processing method provided in this application does not require obtaining game code or related configuration files, thus exhibiting strong versatility; it requires no manual intervention, reducing the workload of manual analysis; and it can obtain the number of downsampling processes corresponding to the floodlight image in real time.

[0150] The above, combined with Figures 1 to 5BThis paper details the application scenarios and data processing methods applicable to the data processing methods provided in this application. Below, we will combine... Figure 6 and Figure 7 This application introduces the data processing apparatus and data processing device. It should be understood that the data processing method described above corresponds to the data processing apparatus and data processing device described below. Any content not described in detail below can be found in the relevant descriptions in the above method embodiments.

[0151] Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 6 As shown, the device is applied to a terminal running a target program. The target program includes a rendering thread and a calculation thread. The rendering program loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of the multiple frame buffers associated with each frame to obtain a floodlight image. The multiple frame buffers correspond one-to-one with multiple storage addresses of the terminal, and any one frame buffer contains the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing each frame. The device includes a first processing unit 601 and a second processing unit 602.

[0152] The first processing unit 601 is configured to execute the second link library file to perform the following operations: when the rendering thread calls the first function within each frame, perform a hook operation on the first function and execute a data acquisition event to obtain the multiple frame buffers from the multiple storage addresses; and when the rendering thread calls the second function within each frame, perform the hook operation on the second function and execute a data transmission event to send the acquired multiple frame buffers to the computing thread; the second processing unit 602 is configured to: in response to receiving the multiple frame buffers sent by the rendering thread, determine the number of downsampling processes performed to obtain the floodlight image based on the multiple frame buffers.

[0153] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 7 As shown, it includes a memory 701, a processor 702, a communication interface 703, and a communication bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the communication bus 704.

[0154] The memory 701 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 701 may store a program. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the data processing method of the embodiments of this application.

[0155] The processor 702 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the units in the data processing apparatus of this application embodiment, or to execute the various steps of the data processing method of this application embodiment.

[0156] The processor 702 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the data processing method provided in this application can be completed by the integrated logic circuits in the hardware of the processor 702 or by instructions in software form. The processor 702 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 701. The processor 702 reads the information in the memory 701 and, in conjunction with its hardware, performs the functions required by the units included in the data processing apparatus of this application embodiment, or executes the data processing method of the method embodiment of this application.

[0157] The communication interface 703 uses a transceiver device, such as, but not limited to, a transceiver, to implement... Figure 7The device shown communicates with other devices or communication networks.

[0158] Communication bus 704 may be included in Figure 7 The illustrated device shows a pathway for transmitting information between its various components (e.g., memory 701, processor 702, communication interface 703).

[0159] This application provides a computer-readable storage medium, which includes computer instructions. When executed by a processor, the computer instructions are used to implement any of the data processing techniques described in this application.

[0160] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored on a computer-readable medium and includes several instructions to cause a computing device (which may be a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments disclosed in this application.

[0161] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0162] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0163] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage media, or any other non-transferable media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0164] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A data processing method, characterized in that, The application is used on a terminal running a target program, the target program including a rendering thread and a calculation thread. The rendering thread loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of the multiple frame buffers associated with each frame to obtain a floodlight image. The multiple frame buffers correspond one-to-one with multiple storage addresses of the terminal, and any one frame buffer is the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing each frame; the method includes: The rendering thread executes the second linked library file to perform the following operations: If, within each frame, the rendering thread is detected calling the first function, a hook operation is performed on the first function, and a data retrieval event is executed to obtain the data from the multiple storage addresses of the multiple frame buffers; and, If the rendering thread calls the second function within each frame, the Hook operation is performed on the second function, and a data transmission event is executed to send the acquired multiple frame buffers to the computing thread. In response to receiving the plurality of frame buffers sent by the rendering thread, the calculation thread determines, based on the plurality of frame buffers, the number of downsampling processes to be performed to obtain the floodlight image.

2. The method according to claim 1, characterized in that, The computation thread determines the number of downsampling processes performed to obtain the floodlight image based on the multiple frame buffers, including: The computation thread generates a directed acyclic graph (DAG) comprising multiple paths based on the plurality of frame buffers. Each path includes at least one directed edge; the at least one directed edge is an edge pointing from a first node to a second node; at least one frame buffer attachment of the first frame buffer corresponding to the first node is bound as a texture attachment to the second frame buffer corresponding to the second node, so that when the second frame buffer is drawn, at least one frame buffer attachment of the first frame buffer is drawn onto the second frame buffer; the second frame buffer is obtained by performing a preset process on the first frame buffer, and the type of the second node is associated with the preset process; the plurality of frame buffers includes the first frame buffer and the second frame buffer. The computation thread determines the number of downsampling processes to be performed to obtain the floodlight image based on the directed acyclic graph.

3. The method according to claim 2, characterized in that, The target program further includes a logic thread, wherein the logic thread is used to determine the object to be drawn corresponding to the floodlight image, and the plurality of frame buffers are used to indicate the method of drawing the object to be drawn. The starting node of any path has an in-degree of zero, the ending node of any path has an out-degree of zero, and the frame buffer corresponding to the starting node of any path is determined based on the object to be drawn corresponding to the floodlight image.

4. The method according to claim 3, characterized in that, The computation thread determines the number of downsampling processes performed to obtain the floodlight image based on the directed acyclic graph, including: The computation thread determines the target region of any path, wherein the type of any node in the target region of any path is associated with a first process; the first process is at least one of the following: downsampling, fuzzing, or upsampling, and the preset process includes the first process; the number of nodes of the first type in the target region of any path is equal to the number of nodes of the second type; the first type is associated with the downsampling process, and the second type is associated with the upsampling process; The computing thread determines the number of downsampling processes to be performed to obtain the floodlight image based on the target region of the first path among the multiple paths, according to the multiple target regions of the multiple paths. The number of nodes of the first type in the target region of the first path is greater than or equal to the number of nodes of the first type in the target region of the second path. The second path is any path other than the first path among the multiple paths. The computing thread determines the number of nodes of the first type in the target area of ​​the first path as the number of downsampling processes performed to obtain the floodlight image.

5. The method according to claim 4, characterized in that, The computation thread determines the target region for any one of the paths, including: The computation thread determines the type of any node based on a preset mapping relationship, the attribute information of any node included in any path, and the types of nodes other than the node in the directed edges associated with the node. The preset mapping relationship is a mapping relationship between the attribute information of the node and the type of the node. The attribute information of the node includes the out-degree and in-degree of the node. The computing thread determines the target region of any path based on the sequence information of any path and a preset expression. The sequence information of any path includes multiple bits, and the i-th bit of the multiple bits is the type of the i-th node in any path from the starting node to the ending node, where i is a positive integer. The preset expression is used to identify nodes of the type associated with the first processing.

6. The method according to claim 4 or 5, characterized in that, Before the computation thread determines the target region for any of the paths, the method further includes: The computation thread performs path search processing on the directed acyclic graph to obtain the multiple paths.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The rendering thread executes the second linked library file to perform the following operations: If the rendering thread sends the acquired multiple frame buffers to the calculation thread, the Hook operation executed on the first function and the second function ends, and the first function and the second function are called, so that in each frame, any one of the multiple frame buffers is drawn to obtain the floodlight image.

8. The method according to any one of claims 1 to 5, characterized in that, The calculation thread is a sub-thread of the rendering thread. Before the rendering thread sends the multiple frame buffers associated with the acquired flood image to the calculation thread, the method further includes: In response to a call to the second function heard within each frame, the rendering thread creates the computation thread.

9. The method according to any one of claims 1 to 5, characterized in that, The rendering thread and the computation thread are two different threads included in the target program.

10. The method according to any one of claims 1 to 5, characterized in that, The target program is a game application, and the first link library file is an OpenGL or an embedded system OpenGL ES graphics library.

11. A data processing apparatus, characterized in that, The application is used on a terminal running a target program, the target program including a rendering thread and a calculation thread. The rendering thread loads a first link library file and a second link library file. The first link library file includes a first function and a second function. The first function is used to draw on any one of the multiple frame buffers associated with each frame to obtain a floodlight image. The multiple frame buffers correspond one-to-one with multiple storage addresses of the terminal, and any one frame buffer is the data to be rendered stored in the corresponding storage address. The second function indicates the end of drawing each frame; the apparatus includes: The first processing unit is used for: If, within each frame, the rendering thread is detected calling the first function, a hook operation is performed on the first function, and a data retrieval event is executed to obtain the data from the multiple storage addresses of the multiple frame buffers; and, If the rendering thread calls the second function within each frame, the Hook operation is performed on the second function, and a data transmission event is executed to send the acquired multiple frame buffers to the computing thread. The second processing unit is configured to: in response to receiving the plurality of frame buffers sent by the rendering thread, determine the number of downsampling processes performed to obtain the floodlight image based on the plurality of frame buffers.

12. A data processing device, characterized in that, include: The memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions to implement the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium storing one or more computer instructions thereon, characterized in that, The instruction is executed by the processor to implement the method as described in any one of claims 1 to 10.

Images