A window display method and device, electronic equipment and storage medium

Abstract

The application relates to the field of application display technology and provides a window display method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a target process corresponding to a full-screen application, and confirming a target function in the target process; setting a back function according to the target function, and adding the back function to the target process; if the back function is triggered, drawing an overlay display layer based on a backup buffer corresponding to the full-screen application, and displaying the overlay display layer in a display interface of the full-screen application. The application realizes controlled addition of a final display frame of the full-screen application without modifying the code of the full-screen application, realizes synthesis of an additional interface element at a pixel level and a main scene, guarantees visual consistency and a display sequence, and inserts processing in a back manner, so that compatibility and non-invasiveness are considered, and the application is convenient for rollback and life cycle management.

Description

Technical Field

[0001] This application relates to the field of application display technology, and in particular to a window display method, apparatus, electronic device and storage medium. Background Technology

[0002] With the continuous development of computer hardware and software technology, more and more companies and developers have launched various full-screen applications, such as large-scale full-screen games, immersive demonstration programs, and virtual reality environments. These full-screen applications typically have the characteristic of exclusive rendering, that is, the application directly occupies the graphics device interface and exclusively uses the display channel, thereby achieving higher rendering efficiency and display performance.

[0003] However, precisely because of this exclusive nature, it is difficult for the windows or interface content of other applications to be overlaid on top of the full-screen application. In other words, when a full-screen application is in exclusive mode, the regular desktop window system drawing path is blocked or bypassed, causing the user interface of external applications to be unable to be directly displayed on the full-screen screen.

[0004] In many scenarios, users need to view or interact with other interface information while the application is running in full-screen mode. For example, displaying instant messaging messages during gameplay, overlaying auxiliary prompts in immersive presentations, or presenting system notifications in virtual reality environments.

[0005] Therefore, how to enable other windows or interface elements to be drawn and displayed on the rendering result of the full-screen application without modifying the full-screen application itself has become an urgent technical problem to be solved. Summary of the Invention

[0006] In view of this, embodiments of this application provide a window display method, apparatus, electronic device, and storage medium to solve the problem in the prior art that other windows or interface elements cannot be drawn and displayed on the rendering result of a full-screen application.

[0007] A first aspect of this application provides a window display method, the method comprising: obtaining a target process corresponding to a full-screen application and identifying a target function in the target process; setting a wraparound function according to the target function and adding the wraparound function to the target process; if the wraparound function is triggered, drawing an overlay display layer based on a backup buffer corresponding to the full-screen application to display the overlay display layer in the display interface of the full-screen application.

[0008] A second aspect of this application provides a window display device, comprising: an acquisition module for acquiring a target process corresponding to a full-screen application and identifying a target function in the target process; an addition module for setting a wraparound function according to the target function and adding the wraparound function to the target process; and a drawing module for drawing an overlay display layer based on a backup buffer corresponding to the full-screen application when the wraparound function is triggered, so as to display the overlay display layer in the display interface of the full-screen application.

[0009] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0010] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0011] The beneficial effects of the embodiments of this application compared with the prior art are that the method in the embodiments of this application avoids the problems existing in the related art. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a window display method provided in the application embodiment;

[0014] Figure 2 This is a flowchart illustrating another window display method provided in the application embodiment;

[0015] Figure 3 This is a flowchart illustrating another window display method provided in the application embodiment;

[0016] Figure 4 This is a flowchart illustrating another window display method provided in the application embodiment;

[0017] Figure 5 This is a flowchart illustrating another window display method provided in the application embodiment;

[0018] Figure 6 This is a flowchart illustrating another optional window display method provided in the application embodiment;

[0019] Figure 7 This is a schematic diagram of the structure of a window display device provided in an embodiment of this application;

[0020] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] A window display method and apparatus according to embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] Figure 1 This is a flowchart illustrating a window display method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0024] S101. Obtain the target process corresponding to the full-screen application, and identify the target function in the target process;

[0025] S102. Set the wraparound function according to the objective function and add the wraparound function to the objective process;

[0026] S103. If the wraparound function is triggered, the overlay display layer is drawn based on the backup buffer corresponding to the full-screen application, so as to display the overlay display layer in the display interface of the full-screen application.

[0027] It is understood that the full-screen application mentioned in step S101 refers to an application running on the display system in an exclusive or near-exclusive rendering mode, such as a full-screen game or immersive presentation program used for high-performance rendering. Such applications typically directly call the graphics device interface and submit the rendering results to the display output. The target process of the full-screen application refers to the operating system process that hosts the full-screen application. The address space of this target process contains the executable code of the full-screen application, runtime data structures, and object handles related to rendering or input (such as device handles, swap chains, window handles, etc.). Since rendering-related handles and data structures reside in the address space of this process, and the address mappings of different processes may differ due to mechanisms such as Address Randomization (ASLR), the target process must be located before subsequent operations to obtain accessibility and accurate references to the aforementioned objects.

[0028] It is understandable that the target process contains multiple functions, and the target function refers to the key interface or method within the target process that is directly related to frame generation, frame submission, or input processing. This type of interface or method provides an effective entry point for implementing rendering observation or controlled intervention, such as a rendering interface for submitting frames to the display path or an entry point for window message processing. Although there are several potential entry points in the target process, to achieve maximum effect with minimal intrusion, the key interface directly coupled to the frame submission process or input path should be prioritized as the target function. Intervention in this target function can insert additional drawing at the end of the rendering submission or insert interception logic into the input path, thereby achieving controlled addition to the final displayed image or controlled processing of input behavior.

[0029] After confirming the target function, to insert controlled processing without disrupting the original rendering logic, this application sets up a wraparound function based on the target function. The wraparound function in this application refers to an alternative executable that can replace or wrap the entry point of the target function. It maintains compatibility with the target function in terms of calling convention, parameter layout, and return semantics. It can insert pre-processing or post-processing before and after the execution of the target function, and, when necessary, revert control flow back to the remaining execution path of the target function. Since CPU execution depends on the address space of the target process, the executable body of the wraparound function must be deployed in the executable memory area of ​​the target process to be stably invoked and to achieve control flow redirection. Therefore, deploying the wraparound function in the executable area of ​​the target process ensures the stability and controllability of the redirection and facilitates subsequent recovery and lifecycle management.

[0030] When the wraparound function is triggered, it indicates that the target application is about to or is performing operations related to frame submission. At this time, this application draws an overlay on the back buffer corresponding to the full-screen application to display the overlay in the full-screen application's display interface. The back buffer corresponding to the full-screen application refers to the frame buffer in the rendering pipeline used to prepare the frame to be submitted, and the final pixel data is stored in this buffer before the submit function is called; the overlay refers to the interface or primitives drawn on the back buffer, and the overlay content and the main rendering content (the pixel data that the full-screen application originally wanted to submit) are composited at the pixel level to form the final target frame to be submitted.

[0031] The backup buffer serves as the final rendering target for the final frame. Appending drawings to it avoids dependence on external window layers or desktop compositors, allowing the overlay layer to be directly composited with the main scene at the pixel level. This ensures display order (later drawn overwrites earlier drawn) and visual consistency. After the additional drawing is completed, the target frame containing the overlay layer is submitted to the display flow by calling the original execution path of the target function (i.e., the original call gateway) in the wraparound function. This achieves overlay display while minimizing intrusion into the original rendering flow, balancing compatibility and non-intrusiveness.

[0032] According to the solution provided in this application, the target process corresponding to the full-screen application is obtained, and the target function is identified in the target process; a wraparound function is set according to the target function, and the wraparound function is added to the target process; if the wraparound function is triggered, an overlay display layer is drawn based on the backup buffer corresponding to the full-screen application, so as to display the overlay display layer in the display interface of the full-screen application. This application achieves controlled appending of the final display frame of the full-screen application without modifying the full-screen application's own code, and realizes the synthesis of additional interface elements with the main scene at the pixel level, ensuring visual consistency and display order; and the wraparound insertion process takes into account compatibility and non-intrusiveness, and facilitates rollback and lifecycle management; and this application is applicable to multiple rendering modes (including full-screen, borderless windowing, etc.), avoiding the problem in related technologies that other windows or interface elements cannot be drawn and displayed on the rendering result of the full-screen application.

[0033] In some embodiments, such as Figure 2 As shown, the objective function is identified in the target process, including:

[0034] S201. Obtain the modules that have been loaded in the target process, determine the graphics rendering information based on the modules that have been loaded in the target process, and determine the type of graphics interface used by the full-screen application based on the graphics rendering information.

[0035] S202. Identify candidate functions related to the graphical interface type within the target process;

[0036] S203. Perform validity checks on each candidate function, and determine the candidate functions that pass the validity checks as the target function.

[0037] Step S201 first detects that a full-screen application has entered full-screen mode, then enumerates its loaded modules based on the target process handle, recording the module name, base address, size, and import / export table summary. Typical implementations include CreateToolhelp32Snapshot / Module32First / Module32Next, or EnumProcessModules / EnumProcessModulesEx. An example of the collected information is {Name, Base, Size, Imports, Exports, Timestamp}.

[0038] Subsequently, graphics rendering information is extracted from the module set to determine the interface type. For example, the presence of d3d11.dll and dxgi.dll indicates DirectX 11, the presence of d3d12.dll and dxgi.dll indicates DirectX 12, the presence of opengl32.dll indicates OpenGL (symbol set can be resolved by combining wglGetProcAddress if necessary), and the presence of vulkan-1.dll and the ability to resolve vkGetInstanceProcAddr / vkGetDeviceProcAddr indicates Vulkan. If multiple interface libraries are detected simultaneously (e.g., a game engine loading both D3D11 and OpenGL), the actual interface used can be selected based on its activity level: poll the thread call stack or the frequency of the most recently resolved rendering symbols (e.g., IDXGISwapChain::Present, wglSwapBuffers, vkQueuePresentKHR) and perform a lightweight probe count (counting only, without rewriting parameters) on key entry points for 1–2 seconds; the interface with the matching frame rate is considered the active interface. If multiple interfaces still exist, the interface context with a visible foreground window (e.g., SwapBuffers associated with the current foreground window DC) is prioritized. The final active interface ActiveAPI ∈ {DX11, DX12, OpenGL, Vulkan} is output.

[0039] Then, candidate functions related to the interface type are determined within the target process. This involves selecting a set of functions corresponding to the rendering path based on the ActiveAPI and resolving their addresses within the target process as candidate functions. For example: DX11 / DX12 (DXGI) candidate functions include IDXGISwapChain:Present, IDXGISwapChain1:Present1, and optional ResizeBuffers, whose addresses can be resolved through known virtual table slots, IAT / EAT, or exported symbol signature matching; OpenGL candidate functions include wglSwapBuffers (Win32) or glXSwapBuffers (Linux), which can be resolved through IAT lookup or exported tables; Vulkan candidate functions include vkQueuePresentKHR, whose function pointers are resolved at the device layer using vkGetDeviceProcAddr and filtered by active swap chain queue binding. The output is Candidates = { (addr, api, name, meta)}.

[0040] Each candidate function is then validated for validity, and the target function is ultimately confirmed. Validation may include at least two methods:

[0041] Frequency consistency verification: Within a limited time window (e.g., 1–2 seconds), only call counts and interval measurements are performed. The relative error between the candidate call frequency f_c and the observed frame rate f_v is calculated. If the error is lower than the threshold (e.g., 10%), the test is passed.

[0042] Call stack feature verification: Collect several shallow stacks (depth 5–15) and determine whether they contain the corresponding presentation path symbols. If the hit rate exceeds the threshold (e.g., 70%), it is considered passed.

[0043] Parameter signature / semantic matching verification: Checks function parameter size, handle validity, structural integrity, etc., and judges failure if there are anomalies or multiple dereference failures;

[0044] Optional Minimal Visibility Probe: In sandbox or diagnostic mode, flip corner pixels or write transparent markers to confirm the correlation between candidate calls and screen output, showing changes only when the candidate is triggered;

[0045] Resource reachability verification: Attempt to obtain the backup buffer (such as DX11 swapChain->GetBuffer, DX12RTV table, Vulkan imageIndex, etc.). Bonus points will be awarded for high success rate and low time consumption.

[0046] Then, based on the verification results, accumulate a score for each candidate function (e.g., frequency matching +40, stack feature +30, signature matching +20, resource availability +10), and select the candidate with the highest total score that exceeds the threshold (e.g., ≥70) to confirm it as the target function; if there is a tie, the candidate with more stable calls and located at the end of the presentation is selected first.

[0047] Through the above embodiments, the target function (such as IDXGISwapChain::Present, wglSwapBuffers, or vkQueuePresentKHR) can be reliably identified within the target process, providing a stable entry point for subsequent wraparound function settings and overlay display, thus achieving accuracy and reliability in overlay drawing.

[0048] According to the technical solution provided in this application, by acquiring the modules loaded in the target process, determining the graphics rendering information based on the loaded modules, and determining the graphics interface type used by the full-screen application based on the graphics rendering information; determining candidate functions related to the graphics interface type within the target process; validating each candidate function, and determining the candidate function that passes the validity check as the target function. Specifically, by acquiring the modules loaded in the target process and extracting the graphics rendering information, the graphics interface type used by the full-screen application can be accurately determined, avoiding functional failures or instability due to uncertain interface types. Within the target process, candidate functions are filtered based on interface types, and the target function is confirmed through multiple validity checks, ensuring that the selected function is closely related to frame generation, frame submission, or input processing, thereby providing a stable and controllable entry point for subsequent wraparound function settings and overlay display. This solution, through candidate function verification and precise filtering, operates only on necessary key interfaces, avoiding damage or interference to the original logic of the full-screen application, and improving system compatibility and security. This solution is applicable to various graphics interfaces (such as DirectX 11 / 12, OpenGL, Vulkan), and allows for flexible selection of target functions based on different full-screen application environments, achieving versatility and scalability. By reliably verifying the target function, wraparound functions can be inserted at key rendering points, thereby enabling monitoring of overlay display layer drawing or rendering, providing technical support for enhanced display, UI overlay, or debugging analysis of full-screen applications.

[0049] In some examples, such as Figure 3 As shown, the wraparound function is set according to the objective function, including:

[0050] S301. Determine the function prototype corresponding to the wraparound function based on the function prototype of the objective function;

[0051] S302. Generate a wraparound function based on the function prototype of the wraparound function, and establish a control flow redirection relationship between the call entry point corresponding to the target function and the wraparound function.

[0052] First, obtain the function prototype of the target function. The function prototype includes at least the calling convention (e.g., register or stack parameter passing rules), parameter types and order, return type, and necessary alignment constraints. Based on this function prototype information, determine the function prototype of the wraparound function compatible with the target function. This ensures that the target function and the wraparound function maintain consistency in calling convention, parameter arrangement, and return value semantics, thereby ensuring binary consistency during control flow fallback. A prototype description data structure is then formed to guide the subsequent generation of the wraparound function body and parameter encoding / decoding.

[0053] Subsequently, this application generates a wraparound function based on the function prototype of the wraparound function. Specifically, this application first allocates executable memory in the address space of the target process to hold the executable body of the wraparound function, and ensures that the memory area has executable permissions and alignment requirements. The executable body is generated based on the function prototype of the wraparound function. The generation stages of this executable body include: a pre-processing segment: performing predetermined processing before entering the original execution path, such as state acquisition, context checking, or overlay drawing preparation, while managing actual participants and return value placeholders according to prototype rules; an original path call segment: when functional equivalence needs to be maintained, falling back to the remaining execution path of the target function. Fallback is achieved through a preset "original call gateway," which guides the control flow to bypass the redirection entry and enter the subsequent execution of the target function; and a post-processing segment: performing finishing processing after returning from the original path and returning the result according to the prototype specification.

[0054] The above segments are combined into a complete instruction sequence and written into the executable area allocated by the target process, thus obtaining a wraparound function that can be directly called in the target process.

[0055] Then, a control flow redirection relationship is established between the call entry point corresponding to the target function and the wraparound function. Specifically, this application first selects a suitable redirection method according to the type and calling method of the target function, including but not limited to: entry point replacement: replacing the instruction or jump at the entry point of the target function with a jump pointing to the wraparound function; pointer replacement: replacing the reference pointer associated with the target function (such as import / export table entries, virtual function table entries or callback pointers) with the entry point of the wraparound function.

[0056] By using the above method, the call entry point of the target function is bound to the wraparound function, so that the call to the target function first enters the wraparound function, which performs pre-processing / post-processing according to a predetermined strategy, and falls back to the subsequent execution path of the target function through the original call gateway when necessary.

[0057] During the establishment of redirection relationships, consistency control is performed on the memory page attributes and multi-threaded access involved to ensure the atomicity and thread safety of binding operations, and the original state is recorded when needed to support the unbinding and restoration of wraparound functions.

[0058] According to the technical solution provided in this application, the function prototype corresponding to the wraparound function is determined based on the function prototype of the target function; the wraparound function is generated based on the function prototype of the wraparound function, and a control flow redirection relationship is established between the call entry point corresponding to the target function and the wraparound function. The wraparound function and the target function maintain consistency in calling convention, parameter type, and return value semantics, ensuring that the original execution path can still be correctly returned after inserting pre- / post-processing, thereby guaranteeing the integrity and stability of the original application logic. Control flow redirection is achieved at the entry point of the target function through the wraparound function, without modifying the original caller, avoiding damage to the existing program structure and reducing interference with the target process. Custom processing logic (such as state acquisition, resource preparation, or overlay rendering) is inserted before and after the execution of the target function, providing controllable expansion capabilities for the rendering or input process. The wraparound function is deployed to the executable area of ​​the target process, ensuring local execution in the target environment, making control flow redirection stable and controllable, and allowing for unbinding and restoration when necessary.

[0059] In some examples, such as Figure 4 As shown, adding the wraparound function to the target process includes:

[0060] S401. Determine the storage area within the address space of the target process to carry the executable body of the wraparound function;

[0061] S402. Deploy the executable of the wraparound function to the storage area, set the storage area to execute permissions, and establish the context information and resource management structure required for the wraparound function to run.

[0062] Specifically, a storage region is selected within the target process's address space to house the wraparound function's executable body. This storage region should meet page-level alignment and contiguity requirements, be within the target process's effective address range, and avoid conflicts with existing code or data segments. Simultaneously, the region must satisfy location reachability constraints to ensure that subsequent call points can correctly jump to the wraparound function. After region allocation, the region's basic attributes are recorded to form a region descriptor. The region descriptor includes at least: starting address, capacity, current permission status, occupancy flag, timestamp information, and the identifier of the module to which it belongs. This region descriptor provides queryable metadata for subsequent deployment and management.

[0063] The executable body of the wraparound function is deployed to the storage area and permissions are set. In this step, the executable body of the wraparound function is first written to the storage area determined by S401. To ensure consistency during the writing process, an atomic writing order is adopted, i.e., the function body content is written first, followed by the entry pointer, and data verification (e.g., length verification and checksum verification) is performed before and after writing to ensure the integrity and correctness of the deployment. Then, the memory permissions of the storage area are set to executable permissions, while retaining the permission information before the change for subsequent rollback or recovery. On some processor architectures, instruction cache coherency processing is also required to ensure that newly written instructions can be correctly fetched and executed. After deployment, a wraparound function entry descriptor is created. The descriptor includes at least: the wraparound function entry address, function prototype identifier (including calling convention, parameter arrangement, and return value type), version number, reference to the storage area, and an enable flag. This descriptor serves as an important basis for subsequent calls, debugging, and management.

[0064] In the steps of establishing the runtime context and resource management structure, the runtime context (Context) is first constructed to carry the state information required for the wraparound function to run. The context includes at least: a wraparound function entry descriptor, an executable region reference, a gateway / fallback information reference required for interaction with the original call path (if present, provided by the build phase), runtime flags (enabled / disabled, debug level), statistical counts, and timestamp information. Next, a resource management structure (Manager) is established to implement lifecycle management and concurrency control. The resource management structure includes at least: reference counting and start / stop interfaces; thread safety mechanisms (such as mutexes, spinlocks, or atomic flags) to ensure the serialization of context reading and entry point activation; rollback and recovery interfaces (supporting the recovery of permissions and entry point states); and monitoring and logging interfaces for persistent recording or circular buffering of deployment success / failure, permission changes, and start / stop events. Finally, the context and resource management structure are registered in the global or module-level registry through an initialization process, forming queryable and manageable instance entries, providing a unified calling interface for subsequent enable, disable, and uninstall operations.

[0065] According to the technical solution provided in this application, a storage area for carrying the executable body of the wraparound function is determined within the address space of the target process. The executable body of the wraparound function is deployed to the storage area, and execution permissions are set for the storage area. Context information and resource management structures required for the wraparound function to run are also established. Through this solution, the wraparound function can be deployed in a controllable manner within the target process, ensuring that the deployment location meets security and reachability requirements and avoiding disruption of the original process layout. Furthermore, the integrity of the executable body and its execution permissions are verified and guaranteed, improving deployment reliability. The context and resource management structures enable lifecycle management and thread safety control of the wraparound function, ensuring stable operation in multi-threaded scenarios. A global registration and logging mechanism ensures the controllability and traceability of subsequent operations and maintenance (enabling, disabling, debugging, and uninstallation).

[0066] In some examples, such as Figure 5 As shown, an overlay display layer is drawn based on the back buffer corresponding to the full-screen application to display the overlay display layer in the display interface of the full-screen application, including:

[0067] S501. Obtain frame buffer reference information associated with the full-screen application and determine the backup buffer to be used for display.

[0068] S502. Construct a drawing list for the overlay display layer, and draw in the backup buffer according to the drawing list to obtain the target frame containing the overlay display layer.

[0069] S503. The target function is called through the original call gateway of the target function, and the target frame is submitted to the display process so that the target frame can be displayed on the display interface.

[0070] In some examples, when the wraparound function is triggered, the frame buffer reference information used to identify the current frame is read from the frame commit context associated with the target function. Based on this frame buffer reference information, the back buffer used for this display is determined, and minimal drawing dependencies (including but not limited to the rendering target view, resource handles, or equivalent references) are established accordingly. When a change in display parameters is detected (such as a resolution change, buffer number adjustment, color space or multisampling configuration change), the resource references and drawing context associated with the back buffer are updated in a timely manner to ensure the continuity and consistency of subsequent drawing and commit.

[0071] In step S502, a drawing list for the overlay display layer is constructed. This drawing list includes at least: primitives to be drawn and their geometric / layout parameters, composition and blending attributes (e.g., transparency, blending rules), and binding information of required resources (e.g., text, textures, geometric data references, or equivalent descriptions). Before performing additional drawing, key information related to the current rendering state of the full-screen application (including but not limited to the current rendering target, graphics pipeline state, blending / sampling configuration, and bound resources) is saved to ensure non-intrusiveness to the original rendering process. Additional drawing is performed in the backup buffer according to the drawing list to generate a target frame containing the overlay display layer. After the overlay drawing is completed, the previously saved rendering state is restored to avoid continuous impact on subsequent frames of the full-screen application.

[0072] Then, step S503 is executed to call the target function through the original call gateway and submit it for display. Specifically, the wraparound function calls the target function through the original call gateway of the target function to submit the target frame obtained in S502 to the display process. If there are timing or synchronization constraints in the submission (such as needing to be aligned with vertical synchronization or waiting for the submission completion signal), the necessary synchronization operations are performed before and after the call according to the context strategy to ensure display stability. After the target frame is successfully submitted, S501-S503 are repeated when the wraparound is triggered in the next frame to achieve continuous presentation of the overlay display layer.

[0073] According to the technical solution provided in the embodiments of this application, through the above steps, without changing the image content already generated by the full-screen application, an overlay display layer is added to the same backup buffer, and the frame submission is completed using the objective function, thereby stably displaying the overlay display layer in the display interface of the full-screen application; at the same time, through state saving / restoration and parameter adaptation, the non-intrusiveness of the original rendering process and the continuity of display are maintained.

[0074] In some examples, such as Figure 6 As shown, the window display method provided in this application also includes:

[0075] S601. When the overlay display layer enters the input interception mode, the call status related to the input is recorded. The call status includes at least a snapshot of the keyboard key positions, mouse coordinates and key states, and window context. During the overlay display layer's input interception mode, the call status of the returned records is queried for inputs targeting a function.

[0076] S602. When the overlay display layer exits the intercept input mode, cancel the replacement logic for the input query and restore the call state of the previous record.

[0077] Specifically, when the overlay enters the input interception mode (e.g., when the overlay launches an interactive control or is activated by the user), the following operations are performed within a single critical section: a. Read and save the current input-related call state (InputSnapshot) atomically or mutually exclusively; b. Store the InputSnapshot in managed temporary storage (e.g., memory structure, registry key, or module metadata) and write it to the deployment log (recording time, triggering thread, snapshot summary).

[0078] After completing the above saving operations, enable the replacement logic for the input query of the target function, causing it to return the value saved in InputSnapshot. The replacement logic can be abstractly described as: short-circuiting the input query path at the wraparound layer or window procedure; or caching and returning the snapshot value at the call chain of the input query. Exemplary input APIs include, but are not limited to, GetKeyState, GetAsyncKeyState, GetCursorPos, or WndProc message queries.

[0079] Understandably, when the device enters the intercept input mode in the overlay display layer, all input queries (polling or event querying) targeting the target function return the corresponding field values ​​in the InputSnapshot. The Overlay itself can process user input normally, but by default, it does not forward real events to the target process (unless there is an explicit forwarding strategy). During the interception, key statistical information (such as interception duration, number of queries, abnormal events, etc.) is recorded for diagnosis and performance tuning.

[0080] When the overlay exits the input interception mode (e.g., the user closes the interaction or the overlay loses focus), the following steps are executed in the critical section: a. Disable the replacement / short-circuit logic to restore the input query to the original path of directly reading the real-time system or device status; b. If input-related caches in the target process were written during interception, restore them to the original values ​​saved in S601; c. Clean up and release the InputSnapshot data structure saved in S601, and write to the recovery log (recording the recovery time, result, and exception information).

[0081] During recovery, if necessary, short-term synchronization of concurrent inputs should be performed (e.g., discarding some events generated during the recovery moment) to avoid misprocessing of duplicate or lost inputs.

[0082] If an anomaly occurs in either S601 or S602 (such as acquisition failure, writing failure, recovery failure, or replacement logic anomaly), immediately stop all interception actions, cancel the replacement logic, release InputSnapshot, and restore the system to the "no interception" state.

[0083] Record diagnostic logs and error codes for analysis; prioritize the stability and responsiveness of the target process.

[0084] For example, taking the interception of keyboard and mouse input as an example, the input interception mode is entered, the keyboard KeyboardState

[256] is collected, and the replacement logic is disabled to restore normal query when the input interception mode is exited; the mouse (x,y) coordinates and buttonMask are recorded, the cached value returned by GetCursorPos or GetMouseState is intercepted, and the injection of Overlay click event into the target process is prohibited; when the input interception mode is exited, the replacement logic is disabled and the clip / capture state is released.

[0085] According to the technical solution provided in the embodiments of this application, the overlay display layer can safely intercept input when user interaction is required, and simultaneously return a snapshot of the input before interception to the target function, thereby ensuring that the target process is unaware of input changes during the interception period. After exiting the interception, the input state is fully restored, balancing the interactivity of the overlay with the non-intrusiveness of the target application, and ensuring system stability and controllability through concurrency control and abnormal rollback strategies.

[0086] In some examples, the window display method provided in this application also includes: using a non-queued message mechanism to pass input events; using dirty area update technology to refresh the modified texture area; and using shared texture technology to directly save the rendering output in video memory for use by the overlay display layer.

[0087] Specifically, when the overlay needs to pass input events (such as keyboard, mouse, or touch events) to the target process, a non-queued message mechanism such as SendMessage can be used instead of the standard message queue (PostMessage) mechanism. After the overlay captures the input event, it directly sends the event to the target window handle via SendMessage, allowing the target process to immediately receive and process the input. Since SendMessage is a synchronous call, it bypasses the queuing and polling delays of the message queue, thereby significantly reducing IPC latency and improving the overlay's response speed to user operations.

[0088] Specifically, when rendering the overlay content, only the texture areas that have actually changed are updated, rather than a full frame refresh. The overlay records the modified areas (DirtyRect) of each texture when building the draw manifest. When drawing to the back buffer or shared texture, only these dirty areas are updated, thereby reducing the overhead of copying video memory to system memory, reducing CPU load, and improving frame rate.

[0089] Specifically, for content output from web pages or external rendering sources, the rendering results can be directly stored in the GPU's video memory and provided to the Overlay via shared texture technology (D3D11 / DirectX shared resources or OpenGL shared context). The Overlay does not need to copy the rendering output to system memory and then upload it to video memory, avoiding wasted memory bandwidth. Direct access to shared textures through video memory enables efficient zero-copy rendering, improving rendering performance.

[0090] In some examples, this application can also implement multi-threaded rendering in a D3D11 rendering environment using DeferredContext and CommandList. This allows rendering commands to be constructed and submitted in parallel, improving CPU and GPU parallel utilization, thereby enhancing rendering performance and frame rate. Specifically, in a D3D11 rendering environment, drawing commands can be decomposed into multiple threads for execution: each thread uses DeferredContext to construct its own CommandList, and the drawing commands from each thread are merged and submitted to the main context (ImmediateContext). By constructing drawing commands in parallel using multiple threads, multi-core CPUs can be fully utilized, reducing the pressure on the main thread and improving overall rendering throughput and frame rate.

[0091] To better understand this application, this embodiment provides a more specific example for illustration. This example provides a window display method, as follows:

[0092] 1. DLL injection.

[0093] 1. The Win32 API call SetWindowsHookEx installs a Windows Hook for the full-screen application process. The system will inject a DLL of the same architecture (x86 / x64) into the process where the target Windows resides. Compared with other injection methods, such as CreateRemoteThread and QueueUserAPC, SetWindowsHookEx does not need to obtain read and write permissions for the target process. For many games with anti-cheat mechanisms, it is impossible to obtain their memory read and write permissions.

[0094] 2. API interception (Hook).

[0095] 1. The virtual memory address of the target function can be found by searching the function's instruction characteristics or indexing the virtual table.

[0096] 2. Write a detour function to replace the original target function. The calling convention and function signature of the detour function must be consistent with those of the original target function.

[0097] 3. Use the Detour library to replace the target function with a wraparound function. The principle is to rewrite the machine instruction at the beginning of the target function as the unconditional jump instruction jmp. As a result, all direct calls to the target function will jump to the prepared wraparound function.

[0098] 4. The wraparound function can choose to use the Detour library to call the actual original target function. The principle is to first execute the overwritten instructions at the beginning of the copied original function, and then unconditionally jump to the instruction after the overwritten address.

[0099] 3D rendering.

[0100] 1. For versions of D3D after D3D10, in addition to hooking the upper-level D3D API, it is also necessary to hook the lower-level DXGI API.

[0101] 2. For D3D12, the D3D12ON11 mechanism can be used to reuse a large amount of D3D11 code.

[0102] 3. Executing rendering instructions in the Present Hook ensures that the Overlay rendering instructions are submitted later than the full-screen application's rendering instructions, thus ensuring that the window is rendered on top of the full-screen application.

[0103] 4. Input Hook.

[0104] 1. Hook input-related APIs, such as GetKeyState, GetCursorPos, etc.

[0105] 2. SetWindowLongPtr can be used to replace the window procedure of a full-screen application to achieve hooking.

[0106] 3. When the Overlay needs to intercept input, record the call status and return the input status before interception.

[0107] 4. When the Overlay no longer needs to intercept input, the previously recorded call state is restored to ensure non-intrusiveness to the game logic.

[0108] 5. Performance optimization.

[0109] 1. Using non-queued message mechanisms such as SendMessage can effectively reduce IPC latency, thereby reducing input latency of the overlay.

[0110] 2. By utilizing the dirty area update technique, only modified textures are updated, which can greatly reduce memory copy overhead.

[0111] 3. Using Shared Texture can avoid copying the webpage rendering output to memory, and send the texture directly through video memory, thereby greatly improving performance.

[0112] 4. Use D3D11 DeferredContext and CommandList for multi-threaded rendering to improve performance.

[0113] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0114] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0115] This embodiment also provides a window display device, such as... Figure 7 As shown, the window display device includes:

[0116] The acquisition module 701 is used to acquire the target process corresponding to the full-screen application and identify the target function in the target process;

[0117] Add module 702 to set the wraparound function based on the target function and add the wraparound function to the target process;

[0118] The drawing module 703 is used to draw the overlay display layer based on the backup buffer corresponding to the full-screen application when the wraparound function is triggered, so as to display the overlay display layer in the display interface of the full-screen application.

[0119] In some examples, the target function is identified within the target process, including: obtaining the modules loaded in the target process, determining graphics rendering information based on the modules loaded in the target process, determining the graphics interface type used by the full-screen application based on the graphics rendering information; identifying candidate functions related to the graphics interface type within the target process; validating each candidate function, and determining the candidate functions that pass the validity check as the target function.

[0120] In some examples, the wraparound function is set based on the target function, including: determining the function prototype corresponding to the wraparound function based on the function prototype of the target function; generating the wraparound function based on the function prototype of the wraparound function; and establishing a control flow redirection relationship between the call entry point corresponding to the target function and the wraparound function.

[0121] In some examples, adding a wraparound function to the target process includes: determining a storage region within the target process's address space to host the wraparound function's executable body; deploying the wraparound function's executable body to the storage region, setting the storage region to execute permissions, and establishing the context information and resource management structure required for the wraparound function to run.

[0122] In some examples, an overlay display layer is drawn based on the back buffer corresponding to the full-screen application to display the overlay display layer in the display interface of the full-screen application. This includes: obtaining frame buffer reference information associated with the full-screen application and determining the back buffer used for display; constructing a drawing list for the overlay display layer and drawing in the back buffer according to the drawing list to obtain the target frame containing the overlay display layer; and calling the target function through the original call gateway of the target function to submit the target frame to the display process so as to display the target frame on the display interface.

[0123] In some examples, the window display device is also used to record input-related call states when the overlay layer enters an input interception mode. The call states include at least a snapshot of keyboard key positions, mouse coordinates and key states, and window context. While the overlay layer is in the input interception mode, it returns the recorded call states for input queries targeting a function. When the overlay layer exits the input interception mode, it cancels the replacement logic for the input query and restores the previously recorded call states.

[0124] In some examples, the window display device is also used to pass input events using a non-queued message mechanism; to refresh modified texture regions using dirty region update technology; and to save the rendered output directly in video memory for use by overlay display layers using shared texture technology.

[0125] According to the technical solution provided in this application, the device obtains the target process corresponding to the full-screen application and identifies the target function in the target process; sets a wraparound function according to the target function and adds the wraparound function to the target process; if the wraparound function is triggered, an overlay display layer is drawn based on the backup buffer corresponding to the full-screen application to display the overlay display layer in the display interface of the full-screen application. This application achieves controlled appending of the final display frame of the full-screen application without modifying the full-screen application's own code, and realizes the synthesis of additional interface elements with the main scene at the pixel level, ensuring visual consistency and display order; and the wraparound insertion process takes into account compatibility and non-intrusiveness, facilitating rollback and lifecycle management; and this application is applicable to multiple rendering modes (including full-screen, borderless windowing, etc.), avoiding the problem in related technologies that other windows or interface elements cannot be drawn and displayed on the rendering result of the full-screen application.

[0126] Figure 8 This is a schematic diagram of the electronic device 8 provided in an embodiment of this application. For example... Figure 8 As shown, the electronic device 8 of this embodiment includes a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, it implements the steps in the various method embodiments described above. Alternatively, when the processor 801 executes the computer program 803, it implements the functions of each module / unit in the various device embodiments described above.

[0127] Electronic device 8 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 8 may include, but is not limited to, processor 801 and memory 802. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or different components.

[0128] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0129] The memory 802 can be an internal storage unit of the electronic device 8, such as a hard disk or RAM of the electronic device 8. The memory 802 can also be an external storage device of the electronic device 8, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 8. The memory 802 can also include both internal and external storage units of the electronic device 8. The memory 802 is used to store computer programs and other programs and data required by the electronic device.

[0130] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0131] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0132] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A window display method, characterized in that, The method includes: Obtain the target process corresponding to the full-screen application, and identify the target function in the target process; A wraparound function is set according to the target function, and the wraparound function is added to the target process. The wraparound function refers to an alternative executable that can replace or wrap the entry point of the target function. If the wraparound function is triggered, an overlay display layer is drawn based on the backup buffer corresponding to the full-screen application, so as to display the overlay display layer in the display interface of the full-screen application. The objective function is identified in the target process, including: Obtain the modules that have been loaded in the target process, determine the graphics rendering information based on the modules that have been loaded in the target process, and determine the type of graphics interface used by the full-screen application based on the graphics rendering information. Within the target process, determine candidate functions related to the graphical interface type; Each candidate function is validated, and the candidate function that passes the validation is determined as the target function.

2. The method according to claim 1, characterized in that, The wraparound function is set according to the objective function, including: The function prototype corresponding to the wraparound function is determined based on the function prototype of the objective function; The wraparound function is generated based on its function prototype, and a control flow redirection relationship is established between the call entry point corresponding to the target function and the wraparound function.

3. The method according to claim 1, characterized in that, Adding the wraparound function to the target process includes: Within the address space of the target process, a storage region is determined to carry the wraparound function executable. The executable of the wraparound function is deployed to the storage area, the storage area is set to execute permissions, and the context information and resource management structure required for the wraparound function to run are established.

4. The method according to claim 1, characterized in that, An overlay display layer is drawn based on the back buffer corresponding to the full-screen application to display the overlay display layer in the full-screen application's display interface, including: Obtain frame buffer reference information associated with the full-screen application, and determine the backup buffer to be used for display; Construct a drawing list for the overlay display layer, and draw in the backup buffer according to the drawing list to obtain a target frame containing the overlay display layer; The target function is invoked through the original call gateway of the target function, and the target frame is submitted to the display process so that the target frame is displayed on the display interface.

5. The method according to claim 1, characterized in that, The method further includes: When the overlay display layer enters the input interception mode, it records the call status related to the input. The call status includes at least a keyboard key snapshot, mouse coordinates and key status, and window context. While the overlay display layer is in the input interception mode, it queries and returns the recorded call status for inputs directed to the target function. When the overlay display layer exits the intercepted input mode, it cancels the replacement logic for the input query and restores the previously recorded call state.

6. The method according to claim 1, characterized in that, The method further includes: Use a non-queued message mechanism to transmit input events; The dirty area update technique is used to refresh the modified texture area; By using shared texture technology, the rendering output is directly stored in video memory and used by overlay display layers.

7. A window display device, characterized in that, The device includes: An acquisition module is used to acquire the target process corresponding to the full-screen application and identify the target function in the target process. Identifying the target function in the target process includes: acquiring the modules loaded in the target process, determining graphics rendering information based on the modules loaded in the target process, determining the graphics interface type used by the full-screen application based on the graphics rendering information, identifying candidate functions related to the graphics interface type in the target process, validating the validity of each candidate function, and identifying the candidate functions that pass the validity validation as the target function. An add module is used to set a wraparound function according to the target function and add the wraparound function to the target process. The wraparound function refers to an alternative executable that can replace or wrap the entry point of the target function. The drawing module is used to draw an overlay display layer based on the backup buffer corresponding to the full-screen application when the wraparound function is triggered, so as to display the overlay display layer in the display interface of the full-screen application.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images