Method and system for implementing window rendering based on native windows on web operating system
Patent Information
- Application Number
- CN202310468087.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2043-04-26
AI Technical Summary
但Web应用无法通过Web引擎目前具有的接口直接访问到该原生窗口接口
[0012]本发明提供的基于Web操作系统上的原生窗口实现窗口绘制的方法及系统,可以在Web操作系统上,为Web应用提供访问原生窗口的能力,Web应用可以通过原生代码如C/C++访问原生窗口,使用硬件加速接口如OpenGLES、OpenMAX向原生窗口生产图形内容,扩展了Web应用的开发能力,提高了图形渲染场景和多媒体播放场景下的应用性能水平。本发明有效克服了web操作系统上现有绘制技术的缺点。
Smart Images

Figure CN116700860B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer information technology, and more specifically to a method and system for implementing window drawing based on native windows on a Web operating system. Background Technology
[0002] In web applications, web technologies such as HTML, JavaScript, and CSS are typically used to develop UI interaction logic, which can achieve excellent cross-platform performance to a certain extent. In specific scenarios, user interfaces need to be developed using direct-draw interfaces to meet highly customized Direct UI interaction requirements. Common scenarios include web games and maps. In web development, this requirement is generally met using Canvas2D and WebGL technologies. Canvas2D technology is based on the Skia drawing engine, translating Canvas2D drawing requests into Skia drawing instructions for execution, and then reading the Skia drawing results and sending them to the web engine for GPU compositing and display. WebGL technology is based on the OpenGL ES rendering interface, using JavaScript to directly call GPU instructions to complete drawing, compositing, and display.
[0003] In web-based operating systems, the Canvas2D technology described above often suffers from several drawbacks. Skia cannot fully utilize hardware acceleration, resulting in slow execution of drawing commands. This leads to UI drawing requests consuming a significant amount of JavaScript thread execution time, causing lag and impacting user experience. Furthermore, the highly encapsulated underlying data structures of Canvas2D and WebGL technologies prevent direct access to the native graphics hardware buffer. This prevents graphics data from being shared with other system modules, often requiring CPU-based reading and writing of the graphics hardware buffer for cross-module communication.
[0004] Furthermore, web-based terminal operating systems typically provide a native window interface based on a graphics buffer queue to uniformly manage the allocation, access, and release of graphics buffers. According to the producer-consumer model, the graphics buffer queue is divided into consumer ends and consumer ends. Through the inter-process communication capabilities provided by Binder, graphics buffers are sent and shared between different processes. In a multi-process model, this significantly reduces graphics data copying operations during IPC, improving graphics rendering performance. However, web applications cannot directly access this native window interface through the interfaces currently available in web engines.
[0005] The invention disclosed in CN115168756A is a local image rendering system based on a web mobile operating system. It includes an extended drawing window abstraction layer on top of the web mobile operating system's graphics rendering engine, comprising an image rendering module and a container management module. The image rendering module includes a drawing initialization module, a drawing window context module, and a redrawing module. The container management module includes a container registration module, a container deregistration module, and an event dispatch module. The container management module is responsible for managing the state of the drawing application's Canvas container window. The image rendering module is responsible for synchronously notifying the drawing application's Canvas container window to redraw after the third-party library completes the drawing, thus realizing the rendering of local image data. Summary of the Invention
[0006] To address the shortcomings of existing technologies, this invention provides a system for window rendering based on native windows on a Web operating system. The system includes a graphics data production module running on a Web application, and a native window compatibility module and a compositing module running on a Web engine. Both the graphics data production module and the compositing module are connected to the native window compatibility module. Under the management of the Web engine, the native window compatibility module creates a native window compatibility interface instance and returns it to the graphics data production module. This native window compatibility interface instance implements a graphics buffer queue module. The graphics data production module pushes graphics display content to the graphics buffer in the corresponding graphics buffer queue module for the compositing module to extract and complete the drawing of the window.
[0007] The native window compatibility module includes an interface compatibility module, a graphics buffer queue module, and a graphics buffer management module connected in sequence. The graphics buffer queue module is used to build a graphics buffer queue in the web engine based on the implementation of the native window of the web operating system. It implements the producer and consumer sides of the graphics buffer queue to implement the enqueue and dequeue operations of the graphics buffer respectively. The graphics buffer queue corresponds one-to-one with the native window compatible interface instance. The graphics buffer management module is used to maintain and manage the allocation and release operations of graphics buffers in all graphics buffer queue modules in the Web engine, and also implements the function of uploading graphics buffers as GPU textures; The interface compatibility module provides a producer interface for connecting to the producer end and a consumer interface for connecting to the consumer end. The producer interface is used by the web application to obtain new available graphics buffers from the graphics buffer queue module through the producer end and to enqueue graphics buffers that have been filled with graphics content into the graphics buffer queue module. The consumer interface is used by the consumer end to receive the event notification of the frame refresh and obtain the graphics buffer data corresponding to the current frame after enqueuing the graphics buffers that have been filled with graphics content into the graphics buffer queue module.
[0008] The graphics buffer management module allocates and releases graphics buffers based on the Allocate and Deallocate interfaces, respectively. The functions of the two interfaces are as follows: The Allocate interface receives requests from the graphics buffer queue module to create graphics buffers, calls the operating system's native graphics buffer management interface according to the request parameters, allocates and initializes the corresponding graphics buffers, creates browser texture objects and binds them to the graphics buffers, and returns the access handles of the browser texture objects and their owned graphics buffers to the web application through the producer end. The Deallocate interface is used to release the specified browser texture object and its corresponding graphics buffer. The browser texture object refers to a derived class based on the browser texture object interface definition, which holds and manages the graphics buffer and GPU texture.
[0009] The implementation method of uploading the graphics buffer as a GPU texture in the graphics buffer management module is as follows: Based on the browser texture object interface defined by the Web engine, the graphics buffer data access function is encapsulated. According to the GPU rendering interface of the operating system platform, the GPU texture upload function in the texture object interface type is implemented. The graphics buffer held by the browser texture object is uploaded as a GPU texture and the corresponding GPU texture access handle is obtained. The Web engine accesses and manages the graphics buffer and GPU texture access handle through the browser texture object.
[0010] The compositing module connects to the consumer end of the native window compatibility module via the consumer interface provided by the native window compatibility module. It listens for frame update notifications and, based on the frame update notifications, dequeues the current updated frame data of the native window compatibility interface instance from the graphics buffer queue. This data includes the graphics buffer and browser texture object corresponding to the frame. The GPU texture access handle held by the browser texture object is configured as the display content of the next compositing frame. When the next vertical sync signal is triggered, the GPU texture access handle held by the browser texture object is used to perform compositing with the GPU texture access handles of other Web graphics content, thereby achieving window rendering.
[0011] The present invention also provides a method for implementing window drawing based on the native window of the above system, comprising the following steps: Step S1: The native window compatibility module creates a native window compatibility interface instance and a graphics buffer queue corresponding to the native window compatibility interface instance, and returns the access handle of the graphics buffer, which is used by the graphics data generation module and the composition module to access the graphics buffer in the native window compatibility module through the access handle; the native window compatibility interface instance implements the interface compatibility module and the graphics buffer queue module. Step S2: The compositing module registers frame update notifications with the graphics buffer queue through the consumer interface in the native window compatibility module; Step S3: The web application accesses the producer interface in the native window compatibility module, dequeues an available graphics buffer from the producer end of the graphics buffer queue associated with the native window compatibility interface instance, fills the graphics buffer with the graphics content currently expected to be displayed, and then re-enqueues the graphics buffer filled with graphics content into the graphics buffer queue through the producer interface. Step S4: When the graphics buffer queue receives the graphics buffer enqueue event at the producer end, it sends a frame update notification to the composition module through the consumer end. Step S5: The compositing module responds to the frame update notification and calls the consumer interface to dequeue the graphics buffer at the head of the graphics buffer queue and its corresponding browser texture object. Through the GPU texture upload function of the browser texture object interface, the corresponding graphics buffer is uploaded to the GPU and the corresponding GPU texture access handle is obtained. Step S6: When the next vertical synchronization signal arrives, submit the GPU texture access handle obtained in step 5 and other Web graphics content as input parameters to the GPU instruction, and perform 2D compositing using the GPU rendering interface. Step S7: The compositing module marks the browser texture object obtained in step S5 as idle, releases its GPU texture access handle, and re-enqueues the graphics buffer held by the browser texture object to the graphics buffer queue through the consumer interface, waiting for the producer to re-enqueue the graphics buffer filled with valid graphics content.
[0012] The present invention provides a method and system for window rendering based on native windows on a Web operating system. This allows Web applications to access native windows on the Web operating system. Web applications can access native windows through native code such as C / C++ and use hardware acceleration interfaces such as OpenGLES and OpenMAX to produce graphical content for the native windows. This expands the development capabilities of Web applications and improves application performance in graphics rendering and multimedia playback scenarios. The present invention effectively overcomes the shortcomings of existing rendering technologies on Web operating systems. Attached Figure Description
[0013] Figure 1 The present invention provides a logical framework diagram of a system for implementing window drawing based on native windows on a Web operating system.
[0014] Figure 2 The present invention relates to a data interaction relationship diagram of a system for window drawing based on native windows on a Web operating system.
[0015] Figure 3 The present invention provides a logical framework diagram of the native window compatibility module of a system for implementing window drawing based on the native window on a Web operating system.
[0016] Figure 4 The following is a flowchart illustrating the implementation of the method for window drawing based on the native window of the Web operating system according to the present invention. Detailed Implementation
[0017] To gain a better understanding of the technical solution and beneficial effects of the present invention, the technical solution of the present invention and its beneficial effects are described in detail below with reference to the accompanying drawings.
[0018] To address the shortcomings of existing technologies, this invention proposes a system for window rendering based on native windows on the Web operating system. The core technical concept involves constructing a graphics buffer queue within the Web engine and implementing both a producer and a consumer side. Through the producer-side interface, data types compatible with the system's native windows are provided to the Web application; this is called a native window compatibility interface instance. Through this instance, the Web application can directly access the underlying hardware graphics buffer and share access across multiple system modules. Through the consumer-side interface, the graphics buffer content is submitted to the compositing module in the Web engine, and compositing operations are performed by the GPU.
[0019] Correspondingly, the core technical concept of the method for window rendering based on the native window of the Web operating system in this invention is that the third-party rendering code in the Web application fills the graphics buffer queue with graphics content and triggers the frame update notification of the graphics buffer queue based on the producer interface provided by the Web engine. The consumer end of the graphics buffer queue is managed by the Web engine. The frame update notification is sent to the compositing module for processing by the consumer end through the consumer interface. The compositing module extracts the graphics buffer and the corresponding browser texture object, calls the browser texture object interface to upload the graphics buffer to the GPU to obtain the GPU texture handle, and when the next vertical synchronization signal arrives, uses the GPU texture access handle held by the browser texture object to perform compositing operation with the GPU texture access handle of other Web graphics content and sends it for display, thus completing the display of the third-party rendered graphics content.
[0020] Figure 1 and Figure 2 The diagrams shown are the logical framework and data interaction diagram of the system for implementing window drawing based on the native window of the Web operating system according to the present invention. Figure 1 and Figure 2 As shown, the system for window rendering based on native windows on a Web operating system provided by this invention includes a graphics data production module running on a Web application, and a native window compatibility module and a compositing module running on a Web engine. Both the graphics data production module and the compositing module are connected to the native window compatibility module. Under the management of the Web engine, the native window compatibility module creates a native window compatibility interface instance and returns it to the graphics data production module. The native window compatibility interface instance implements a graphics buffer queue module. The graphics data production module pushes graphics display content to the graphics buffer in the corresponding graphics buffer queue module for the compositing module to extract and submit to the GPU to complete the window rendering.
[0021] Figure 3This is a logical framework diagram of the native window compatibility module of the system for implementing window drawing based on the native window on the Web operating system of the present invention, which also relates to the core inventive points of the present invention, and is described below in conjunction with... Figures 1-3 Detailed technical concept of the present invention: I. Graphic Data Production Module Integrated into web applications, it acts as a graphics producer, providing third-party drawing code capable of generating graphics data. It connects to the producer side of the native window compatibility module through the producer interface provided by the native window compatibility module. Specifically, since the web engine creates a native window compatibility interface instance through the native window compatibility module (which implements a graphics buffer queue one-to-one), the graphics data production module can directly call the native window interface to access the native window compatibility interface instance and push the graphics display content to the corresponding graphics buffer queue module. Alternatively, it can call a third-party graphics library that can indirectly access the native window interface, execute drawing instructions via CPU or GPU, and push the graphics display content to the graphics buffer queue module. Then, the graphics buffer queue notifies the consumer to extract and process the corresponding graphics buffer data.
[0022] II. Native Window Compatibility Module Based on the native window implementation of the Web operating system, a native window-compatible interface instance is created in the Web engine and returned to the graphics data production module. A graphics buffer queue is constructed and the allocation and release of graphics buffers are managed. The graphics buffer queue implements both the producer and consumer sides. Through the producer side, data types compatible with the native window interface of the Web operating system are provided to the Web application, i.e., native window-compatible interface instances. Through the consumer side, frame update notifications and frame data acquisition functions are provided to the composition module in the Web engine.
[0023] The native window compatibility module includes an interface compatibility module, a graphics buffer queue module, and a graphics buffer management module. The functions of each module are described below: Graphics Buffer Management Module: Provides graphics buffer and browser texture management functions. The graphics buffer management module maintains the allocation and release operations of all graphics buffer instances of the Web engine and provides two interfaces: Allocate and Deallocate.
[0024] The browser texture mentioned refers to a derived type based on the browser texture object interface of the Web engine, used to provide the Web engine with the function of accessing / managing graphics buffers. It holds an access handle to the corresponding graphics buffer and implements the GPU texture upload function according to the GPU rendering interface (such as OpenGL ES) of the Web operating system platform. This allows the graphics buffer to be uploaded as a GPU texture and obtain the corresponding GPU texture access handle, which is also stored in the browser texture object. The graphics buffer instance and GPU texture held by the browser texture object are released together with the destruction of the browser texture object.
[0025] In this invention, the browser texture object interface defines how the browser uses the graphics buffer, which includes a function interface for uploading as a GPU texture. This interface is implemented using the GPU rendering interface. In this invention, the GPU rendering interface mainly refers to OpenGL ES, which actually provides the function of uploading the graphics buffer to the GPU and obtaining the GPU texture access handle, that is, the step of obtaining the EGL Object (detailed below).
[0026] Allocate is used to receive requests from the browser application process to create a graphics buffer. Specifically, it requests a graphics buffer of a specified format and size. The graphics buffer management module calls the operating system's native graphics buffer management interface according to the request parameters, allocates and initializes the corresponding graphics buffer, then creates a browser texture object and binds it to the graphics buffer, and returns the access handle of the graphics buffer to the web application. Deallocate is used to destroy the specified browser texture object and release the graphics buffer and GPU texture it holds.
[0027] The graphics buffer queue module is based on the producer-consumer model defined by the native window interface of the Web operating system. Utilizing the IPC mechanism provided by Binder, it implements both the producer and consumer sides, which jointly manage a graphics buffer queue. The producer and consumer sides can respectively perform dequeue and enqueue operations on this queue. Figure 3Specifically, when the producer requests a dequeued graphics buffer, but the number of free graphics buffers in the queue is insufficient or the existing free graphics buffers in the queue do not match the request parameters, the graphics buffer queue module requests a new graphics buffer from the graphics buffer management module. Specifically, the graphics buffer queue module uses the Allocate interface provided by the graphics buffer management module to request a new graphics buffer object, obtain its access handle, register the handle to the graphics buffer queue, and return the access handle of the graphics buffer to the producer. When the producer enqueues a new valid graphics buffer, it enqueues the graphics buffer to the graphics buffer queue and sends a frame refresh notification to the consumer server.
[0028] Interface compatibility module: Provides consumer and producer interfaces.
[0029] The consumer interface is provided to the composition module. It is connected to the consumer end of the corresponding graphics buffer queue. It can obtain frame refresh event notifications from the consumer end and obtain the graphics buffer access handle corresponding to the current frame from the consumer end according to the current refresh status. The producer interface is provided for web applications to access. It connects to the corresponding producer end and can obtain new available buffers from the producer end and request queued graphics buffers filled with graphics content from the producer end. At the same time, the interface compatibility module encapsulates the producer interface with a set of interface families that are compatible with the native window interface of the web operating system. This interface family inherits from the native window type of the web operating system and provides the exact same interface and runtime behavior.
[0030] III. Synthesis Module Its data interaction with the graphics data production module is as follows: Figure 3 As shown, as a graphics consumer, it connects to the consumer end of the native window compatibility module based on the consumer interface provided by the native window compatibility module. It listens for frame update notifications and retrieves the graphics buffer access handle of the updated frame from the graphics buffer queue according to the frame update notification information. It then configures the GPU texture access handle held by the browser texture object to which the graphics buffer belongs as the display content of the next composite frame. When the next vertical sync signal is triggered, it uses this GPU texture access handle to complete the compositing with the GPU texture access handles of other web graphics content.
[0031] Figure 4 This is a flowchart illustrating the implementation of the method for window drawing based on the native window of the Web operating system according to the present invention. Figure 4 As shown, under a typical native window integration model of a web application, the execution steps of this invention are as follows: 1. The web application requests the web engine to create a native window compatibility interface instance by accessing the extended WebAPI. This native window compatibility interface instance is created and managed by the aforementioned native window compatibility module. Simultaneously with creating the native window compatibility interface instance, the web engine creates a corresponding graphics buffer queue instance and returns an access handle for that instance. There is a one-to-one correspondence between the native window compatibility interface instance and the graphics buffer queue instance. The web application obtains one access handle HP from this native window compatibility interface instance. This HP provides a producer interface for accessing the producer side of the graphics buffer queue, and it is compatible with the native window interface of the web operating system, allowing direct access to the producer side of the graphics buffer queue through the native window interface. The web engine retains another access handle HS from this native window compatibility interface instance. This HS provides a consumer interface for accessing the consumer side of the graphics buffer queue, allowing it to receive frame update notifications from the graphics buffer queue and dequeue the latest graphics buffer from the queue.
[0032] 2. The compositing module registers frame update notifications with the graphics buffer queue consumer through the consumer interface. Specifically, it calls the consumer interface provided by the access handle HS to register a callback function or callback function class with the corresponding graphics buffer queue consumer. This callback function or callback function class will be invoked when a new graphics buffer is enqueued in the graphics buffer queue, instructing the graphics buffer queue consumer to dequeue the new graphics buffer and the graphics content data in the graphics buffer as soon as possible.
[0033] 3. The web application dequeues an available graphics buffer from the corresponding graphics buffer queue producer end through the producer interface of the interface compatibility module, performs graphics data filling operation by accessing the virtual address of the graphics buffer, fills the graphics buffer with the graphics content that is currently expected to be displayed, and calls the producer interface to re-enqueue the graphics buffer with graphics content into the graphics buffer queue after the graphics content filling is completed.
[0034] Specifically, the native window compatibility interface instance implements the Web operating system's native window interface, and the graphics buffer content of the native window compatibility interface instance can be accessed using the Web operating system's native window interface.
[0035] 4. When the graphics buffer queue receives a graphics buffer enqueue event on the producer side, it sends a frame update notification to the consumer side and calls the callback function registered in the consumer interface. 5. The compositing module callback function is invoked in response to the frame update notification. Subsequently, the consumer interface is called to dequeue the graphics buffer at the head of the graphics buffer queue (i.e., the latest frame) and the corresponding browser texture object. The graphics buffer is uploaded as a GPU texture through the browser texture object, and the corresponding GPU texture access handle is obtained and stored in the browser texture object.
[0036] 6. When the next vertical synchronization signal arrives, call the OpenGL ES library interface, submit the GPU texture access handle obtained in step 5 and the GPU texture access handles of other Web graphics content as input parameters to the GPU instruction, and perform 2D compositing with other Web graphics content.
[0037] 7. After 2D compositing is completed, the compositing module marks the browser texture object dequeued in step 5 as idle, releases the GPU texture access handle held by the browser texture object, calls the consumer interface to re-enqueue the graphics buffer held by the browser texture object to the graphics buffer queue, and waits for the producer to fill in valid graphics content again.
[0038] Preferably, steps 3-7 can be completed using a GPU rendering interface that supports hardware acceleration.
[0039] Specifically, the EGL library interface and OpenGL ES library interface of the Web operating system can access the native window compatible interface instance described in step 3. The EGL library and OpenGL ES library implement the function of calling the dequeue graphics buffer in the native window interface and uploading it as a GPU texture. The GPU texture can receive image content from the GPU. After receiving valid graphics data, the EGL library and OpenGL ES library call the native window interface to enqueue the graphics buffer corresponding to the GPU texture to the producer end of the graphics buffer queue.
[0040] Specifically, when the browser texture object's function of uploading the graphics buffer as a GPU texture is implemented based on the OpenGL ES library interface, the GPU texture access handle mentioned in steps 6 and 7 refers to the identifier of the EGLImage returned by the OpenGL ES library interface after uploading the graphics buffer to the GPU module and creating the corresponding EGLImage instance through the OpenGL ES library interface.
[0041] In this invention, the technical meaning or difference between the access handle of the graphics buffer and the access handle of the native window compatibility interface is explained as follows: A graphics buffer is a space that holds graphics data, and also contains descriptive information about the graphics data, such as image data type, graphics data parsing format, graphics width, and graphics height. HP refers to the producer interface of the compatibility module, and HS refers to the consumer interface of the compatibility module; both actually point to the same instance of the native window compatibility interface. Both HP and HS provide the functions of enqueuing and dequeuing from the queue, but HP enqueues a buffer that holds new graphics data and dequeues an empty buffer without valid data; HS dequeues a buffer that holds new data and enqueues a used buffer without valid data. The difference between the access handle of the graphics buffer and the access handle of the native window compatibility interface is that the access handle of the native window compatibility interface instance, such as HP and HS, is used to pass (or exchange) the graphics buffer access handle with the queue; the graphics buffer access handle points to the actual graphics buffer that stores the graphics data.
[0042] Compared with the prior art, the present invention has the following beneficial effects: 1. It widely supports a large number of mobile terminal platforms based on OpenGL ES, and is not limited to a specific mobile terminal platform, demonstrating significant advantages in cross-hardware platform compatibility; 2. It fully leverages the scalability of the system's native window operation logic, expands the Web engine's ability to access native windows, and solves the problem of Web applications having difficulty sharing graphics buffers with other system modules. By utilizing the platform-independent characteristics of hardware graphics buffers and GPU rendering interfaces, Web applications can efficiently complete graphics-intensive tasks with the help of GPUs, thereby improving the graphics performance of the system and Web applications.
[0043] Although the present invention has been described using the above preferred embodiments, it is not intended to limit the scope of protection of the present invention. Any changes and modifications made by those skilled in the art to the above embodiments without departing from the spirit and scope of the present invention shall still fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.
Claims
1. A system for implementing window drawing based on native windows on a Web operating system, characterized in that, This includes a graphics data production module running on a web application, and a native window compatibility module and compositing module running on a web engine. Both the graphics data production module and the compositing module are connected to the native window compatibility module. Under the management of the Web engine, the native window compatibility module creates a native window compatibility interface instance and returns it to the graphics data production module. The native window compatibility interface instance implements a graphics buffer queue module. The graphics data production module pushes the graphics display content to the graphics buffer in the corresponding graphics buffer queue module for the compositing module to extract and complete the drawing of the window. The native window compatibility module includes an interface compatibility module, a graphics buffer queue module, and a graphics buffer management module connected in sequence. The graphics buffer queue module is used to build a graphics buffer queue in the web engine based on the implementation of the native window of the web operating system. It implements the producer and consumer sides of the graphics buffer queue to implement the enqueue and dequeue operations of the graphics buffer respectively. The graphics buffer queue corresponds one-to-one with the native window compatible interface instance. The graphics buffer management module is used to maintain and manage the allocation and release operations of graphics buffers in all graphics buffer queue modules in the Web engine, and also implements the function of uploading graphics buffers as GPU textures; The interface compatibility module provides a producer interface for connecting to the producer end and a consumer interface for connecting to the consumer end. The producer interface is used by the web application to obtain new available graphics buffers from the graphics buffer queue module through the producer end and to enqueue graphics buffers that have been filled with graphics content into the graphics buffer queue module. The consumer interface is used by the consumer end to receive event notifications of the current frame refresh and obtain the graphics buffer data corresponding to the current frame after enqueuing graphics buffers that have been filled with graphics content into the graphics buffer queue module.
2. The system for implementing window drawing based on native windows on a Web operating system as described in claim 1, characterized in that, In the graphics buffer management module, the allocation and release of graphics buffers are accomplished based on the Allocate and Deallocate interfaces, respectively. The functions of the two interfaces are as follows: The Allocate interface receives requests from the graphics buffer queue module to create graphics buffers, calls the operating system's native graphics buffer management interface according to the request parameters, allocates and initializes the corresponding graphics buffers, creates browser texture objects and binds them to the graphics buffers, and returns the access handles of the browser texture objects and their owned graphics buffers to the web application through the producer end. The Deallocate interface is used to release the specified browser texture object and its corresponding graphics buffer. The browser texture object refers to a derived class based on the browser texture object interface definition, which holds and manages the graphics buffer and GPU texture.
3. The system for implementing window drawing based on native windows on a Web operating system as described in claim 2, characterized in that, The implementation method of uploading the graphics buffer as a GPU texture in the graphics buffer management module is as follows: Based on the browser texture object interface defined by the Web engine, the graphics buffer data access function is encapsulated. According to the GPU rendering interface of the operating system platform, the GPU texture upload function in the texture object interface type is implemented. The graphics buffer held by the browser texture object is uploaded as a GPU texture and the corresponding GPU texture access handle is obtained. The Web engine accesses and manages the graphics buffer and GPU texture access handle through the browser texture object.
4. The system for implementing window drawing based on native windows on a Web operating system as described in claim 1, characterized in that, The compositing module connects to the consumer end of the native window compatibility module via the consumer interface provided by the native window compatibility module to listen for frame update notifications. Based on the frame update notifications, it dequeues the current updated frame data of the native window compatibility interface instance from the graphics buffer queue, including the graphics buffer and browser texture object corresponding to the frame. It then configures the GPU texture access handle held by the browser texture object as the display content of the next compositing frame. When the next vertical sync signal is triggered, the GPU texture access handle held by the browser texture object is used to perform compositing with the GPU texture access handles of other Web graphics content, thereby realizing window rendering.
5. A method for implementing window drawing based on the system according to any one of claims 1-4, characterized in that, Includes the following steps: Step S1: The native window compatibility module creates a native window compatibility interface instance and a graphics buffer queue corresponding to the native window compatibility interface instance, and returns the access handle of the graphics buffer, which is used by the graphics data generation module and the composition module to access the graphics buffer in the native window compatibility module through the access handle; the native window compatibility interface instance implements the interface compatibility module and the graphics buffer queue module. Step S2: The compositing module registers frame update notifications with the graphics buffer queue through the consumer interface in the native window compatibility module; Step S3: The web application accesses the producer interface in the native window compatibility module, dequeues an available graphics buffer from the producer end of the graphics buffer queue associated with the native window compatibility interface instance, fills the graphics buffer with the graphics content currently expected to be displayed, and then re-enqueues the graphics buffer filled with graphics content into the graphics buffer queue through the producer interface. Step S4: When the graphics buffer queue receives the graphics buffer enqueue event at the producer end, it sends a frame update notification to the compositing module through the consumer end; Step S5: The compositing module responds to the frame update notification and calls the consumer interface to dequeue the graphics buffer at the head of the graphics buffer queue and its corresponding browser texture object. Through the GPU texture upload function of the browser texture object interface, the corresponding graphics buffer is uploaded to the GPU and the corresponding GPU texture access handle is obtained. Step S6: When the next vertical synchronization signal arrives, submit the GPU texture access handle obtained in step 5 and other Web graphics content as input parameters to the GPU instruction, and perform 2D compositing using the GPU rendering interface. Step S7: The compositing module marks the browser texture object obtained in step S5 as idle, releases its GPU texture access handle, and re-enqueues the graphics buffer held by the browser texture object to the graphics buffer queue through the consumer interface, waiting for the producer to re-enqueue the graphics buffer filled with valid graphics content.
Citation Information
Patent Citations
Local image rendering system based on WEB mobile operating system
CN115168756A
Window information processing method and device, electronic equipment and storage medium
CN113822962A