Local image rendering system based on web mobile operating system

Abstract

The application provides a local image rendering system based on a WEB mobile operating system, relates to the technical fields of mobile terminal operating system and Web image processing, and comprises an extended drawing window abstract layer based on a WEB mobile operating system graphic rendering engine, wherein the drawing window abstract layer comprises an image rendering module and a container management module; the image rendering module comprises a drawing initialization module, a drawing window context module and a redrawing module; the container management module comprises a registration container module, a deregistration container module and an event distribution module; the container management module is responsible for managing the state of a drawing application Canvas container window; the image rendering module is responsible for synchronously notifying the drawing application Canvas container window to redraw after a third-party drawing library completes drawing, so as to realize the rendering of local image data. The application can solve the problem of Canvas2D standard interface drawing a large number of pixel data and can improve the local image rendering performance.

Description

Technical Field

[0001] This invention relates to the fields of mobile terminal operating system technology and Web image processing technology, specifically, to a local image rendering system based on a Web mobile operating system. Background Technology

[0002] Firefox OS is a web-based mobile operating system based on HTML5 technology. Its user interface and applications are entirely implemented using web technologies. The rendering engine used in this system provides interfaces for graphics drawing and image rendering to upper-layer applications, known as the Web Canvas standard interface. This primarily refers to WebGL and Canvas2D vector graphics rendering capabilities. Application developers use this standard interface to complete image drawing functions. However, because the Firefox OS open-source community has long ceased updates and maintenance, the web technologies it uses have also failed to be updated in a timely manner. The overall performance of its graphics drawing interface is relatively low, and it cannot meet the demanding real-time rendering and highly interactive requirements of most modern applications.

[0003] In HTML5 map applications, large amounts of map data need to be processed. A common technical solution is to first synthesize image data using a local binary library, and then use the existing HTML5 standard Canvas2D drawing interface to render and display the image. Because this requires allocating significant storage space to copy and upload image data to the Canvas container, map scrolling can experience stuttering, especially noticeable on high-resolution domestically produced devices, negatively impacting the user experience.

[0004] The invention disclosed in CN103778656B discloses an image rendering method, apparatus, and electronic device, comprising: detecting an image frame rate when rendering an image using a first image rendering effect; if the image frame rate is lower than a set image frame rate, adjusting the image rendering effect parameters to change the image rendering effect from the first image rendering effect to a second image rendering effect, wherein the second image rendering effect is lower than the first image rendering effect; and rendering an image using the second image rendering effect. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a local image rendering system based on a web mobile operating system.

[0006] According to the present invention, a local image rendering system based on a web mobile operating system is provided, the solution of which is as follows:

[0007] A local image rendering system based on a web mobile operating system extends the drawing window abstraction layer on the basis of the web mobile operating system's graphics rendering engine, including: an image rendering module and a container management module.

[0008] The image rendering module includes a drawing initialization module, a drawing window context module, and a redraw module; the container management module includes a container registration module, a container unregistration module, and an event dispatch module.

[0009] The container management module is responsible for managing the state of the drawing application's Canvas container window; the image rendering module is responsible for synchronously notifying the drawing application's Canvas container window to redraw after the third-party drawing library completes the drawing, thereby realizing the rendering of local image data.

[0010] Preferably, the drawing initialization module in the image rendering module is the starting entry point for local image rendering. It is triggered by the drawing application and completes the initialization preparation work of the drawing window abstraction layer. First, it initializes the drawing platform parameters, and then loads the third-party drawing library.

[0011] Preferably, if the loading of the third-party drawing library is successful, the native drawing interface native_itf, which is uniformly defined by the drawing window abstraction layer, is loaded. After the interface is successfully loaded, the local drawing initialization interface is called to complete the initialization of the third-party drawing library. At the same time, a notification event handle is injected for use in the event dispatch module in the container management module. If the loading process fails, the drawing platform is exited.

[0012] Preferably, the drawing window context module in the image rendering module provides rendering screen sharing;

[0013] After receiving the registration container event, the drawing window context module obtains a block of display memory from the cache queue managed by the Surface in the rendering engine as image data storage. It enables the local drawing window and the Canvas container in the HTML5 drawing application to share textures through the OpenGL shared context. The local library draws and renders to the OpenGL texture, and the Canvas container directly reuses this texture and uploads it to the rendering engine to wait for rendering.

[0014] Preferably, the redraw module in the image rendering module is based on the display memory management mechanism in the Surface system of the rendering engine. After listening to the frame refresh event, it sends a request to the rendering engine to redraw the content displayed in the Canvas container.

[0015] Preferably, after the local drawing library calls EGL SwapBuffer, the display cache is enqueued into the cache queue inside the Surface, and then the frame available event is triggered. After the redraw module receives the frame available event, it obtains the shared texture cache from the drawing window context, and all subsequent drawing content will be written to this cache.

[0016] The texture data is then converted into bitmap image data. The Canvas container uses this image data for drawing operations and notifies the rendering engine to redraw. The rendering engine will redraw when the next vertical synchronization VSYNC signal arrives.

[0017] Preferably, the container management module provides a Canvas container registration function, which associates and binds the Canvas container window with the local drawing and rendering window through the container index. After registration is completed, the drawing application Canvas container begins to receive local drawing library operation events.

[0018] The container registration module is triggered by the drawing application. First, the drawing application passes in the currently used Canvas container, the drawing platform caches the Canvas container context, then registers a buffer listener with the drawing window context module, and finally calls the local drawing registration interface to synchronize the container registration status.

[0019] Preferably, the container management module provides a Canvas container unregistration function in the container unregistration module. After unregistration, any operation of the drawing library will not be associated with this container.

[0020] The container unregistration module is also triggered by the drawing application. It removes the current Canvas container from the drawing platform and unbinds the Canvas container window from the local drawing rendering window. First, it removes the Canvas container context cache from the drawing platform through the Canvas container index, then unregisters the buffer listener to the drawing window context module, and finally calls the local drawing unregistration interface to synchronize the container unregistration status.

[0021] Preferably, the event dispatch module in the container management module handles event interactions between the upper-layer application and the drawing library;

[0022] In actual business scenarios, it is necessary to handle data interaction between the drawing application and the drawing library. The drawing window abstraction layer adopts the JSON data format commonly used in HTML5 applications, and the data communication interface between the two ends is agreed upon to realize data interaction between the two ends.

[0023] Preferably, the event dispatch module first starts a message receiving thread, registers a notification event callback in the HTML5 drawing application, registers an event listener with the event dispatch module, and the drawing application sends drawing event data to the third-party drawing library through the event dispatch module. The third-party drawing library requests the distribution of notification data from the event dispatch module by calling the notification event, and then the event dispatch module forwards the data to the drawing application.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The drawing window abstraction layer in this invention can be quickly extended and ported to other web mobile operating systems;

[0026] 2. This invention solves the problem of lag when drawing large amounts of data using the original HTML5 standard Canvas2D interface. At the same time, it optimizes and improves the implementation of the bitmap rendering interface in the Canvas2D standard. In practical use cases, it can replace the bitmap rendering interface in the Canvas2D standard, thereby improving local image rendering performance. Attached Figure Description

[0027] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0028] Figure 1 This is a block diagram of the overall technical solution;

[0029] Figure 2 To draw the flowchart for the initialization module;

[0030] Figure 3 To draw the functional flowchart of the window context module;

[0031] Figure 4 To redraw the module flowchart;

[0032] Figure 5 Flowchart for registering container modules;

[0033] Figure 6 Flowchart for unregistering the container module;

[0034] Figure 7 This is a flowchart of the event distribution module. Detailed Implementation

[0035] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0036] This invention provides a local image rendering system based on a web mobile operating system. It extends the drawing window abstraction layer on top of the web mobile operating system's graphics rendering engine. By binding the operating system application window and the local drawing rendering window, and using a frame refresh notification mechanism, it provides a unified local drawing interface operation based on the Canvas container, while shielding the implementation differences of third-party drawing libraries. (See also...) Figure 1As shown, the drawing window abstraction layer includes an image rendering module and a container management module. The container management module is responsible for managing the state of the drawing application's Canvas container window, while the image rendering module is responsible for synchronously notifying the drawing application's Canvas container window to redraw after the third-party drawing library has completed the drawing, thereby realizing the rendering of local image data.

[0037] (1) Image rendering module: mainly includes drawing initialization module, drawing window context module and redrawing module.

[0038] a) Draw the initialization module:

[0039] Reference Figure 2 As shown, the drawing initialization module provides the functionality to initialize the drawing abstraction layer, completing the loading of the drawing library and the injection of notification event handles. The drawing initialization module is the starting point for local image rendering, triggered by the drawing application. It completes the initialization preparation work for the drawing window abstraction layer. First, it initializes the drawing platform parameters, then loads the third-party drawing library. If loading is successful, it loads the local drawing interface `native_itf`, which is uniformly defined by the drawing window abstraction layer. After successful interface loading, it calls the local drawing initialization interface to complete the initialization of the third-party drawing library and injects notification event handles for use in the event dispatch module of the container management module. If the above loading process fails, the drawing platform exits.

[0040] b) Drawing Window Context Module

[0041] Reference Figure 3 As shown, the drawing window context module provides screen sharing for rendering. After receiving the registration container event, the drawing window context module obtains a block of display memory from the cache queue managed by the Surface in the rendering engine as image data storage. Through OpenGL shared context, the local drawing window and the Canvas container in the HTML5 drawing application share textures. The local library draws and renders to the OpenGL texture, and the Canvas container can directly reuse this texture and upload it to the rendering engine to wait for rendering.

[0042] c) Redraw module

[0043] Reference Figure 4As shown, the redraw module, based on the display memory management mechanism of the Surface rendering engine, sends a request to the rendering engine to redraw the content displayed in the Canvas container after listening for a frame refresh event. When the local drawing library calls EGLswapBuffer, the display cache is enqueued into the cache queue inside the Surface. Subsequently, a frame availability event is triggered. Upon receiving the frame availability event, the redraw module retrieves the shared texture cache from the drawing window context. All subsequent drawing content is written to this cache. Then, the texture data is converted into bitmap image data. The Canvas container uses this image data for drawing operations and notifies the rendering engine to redraw. The rendering engine will redraw when the next vertical synchronization (VSYNC) signal arrives.

[0044] (2) Container management module: mainly includes container registration module, container deregistration module and event dispatch module.

[0045] a) Register the container module:

[0046] Reference Figure 5 As shown, the container registration module provides Canvas container registration functionality. It associates and binds the Canvas container window with the local drawing rendering window via a container index. After registration, the drawing application's Canvas container begins receiving operation events from the local drawing library. The container registration module is triggered by a call from the drawing application. First, the drawing application passes in the currently used Canvas container, caching the Canvas container context in the drawing platform. Then, it registers a buffer listener with the drawing window context module. Finally, it calls the local drawing registration interface to synchronize the container registration status.

[0047] b) Unregister the container module

[0048] Reference Figure 6 As shown, the unregister container module provides the Canvas container unregistering function. After unregistration, any operations of the drawing library are no longer associated with this container. The unregister container module is also triggered by the drawing application, removing the current Canvas container from the drawing platform and unbinding this Canvas container window from the local drawing rendering window. First, the Canvas container context cache is removed from the drawing platform through the Canvas container index. Then, the buffer listener is unregistered with the drawing window context module. Finally, the local drawing unregister interface is called to synchronize the container unregistering status.

[0049] c) Event dispatch module

[0050] Reference Figure 7As shown, the event dispatch module is responsible for handling event interactions between the upper-layer application and the rendering library. In actual business scenarios, data interaction between the rendering application and the rendering library needs to be handled. The rendering window abstraction layer adopts the JSON data format commonly used in HTML5 applications, and data interaction between the two ends is achieved through agreed-upon data communication interfaces. The event dispatch module first starts a message receiving thread. After registering notification event callbacks in the HTML5 rendering application, it registers event listeners with the event dispatch module. The rendering application sends rendering event data, which is then passed through the event dispatch module to the third-party rendering library. The third-party rendering library, in turn, requests notification data from the event dispatch module by calling the notification event, and the event dispatch module then forwards the data to the rendering application.

[0051] The present invention will now be described in more detail.

[0052] Implementation Example 1:

[0053] I. Pseudocode for C++ Drawing Library Interface

[0054] (1) Notification Interface

[0055] Interface Description: This interface can be called when the local library needs to pass messages to the H5 application;

[0056] Parameter description: The parameter canvas is the canvas index passed when the registration interface is called, msg is a JSON format string, and length is the length of msg;

[0057] typedef void(*notify_func)(native_itf_t*,INDEX canvas,const char*msg,int32_t length);

[0058] (2) Initialize the interface

[0059] Interface description: This method will be called after the H5 application calls the initialization interface;

[0060] Parameter description: The notify parameter is a pointer to the notification interface function;

[0061] void(*init_)(native_itf_t*,notify_func notify);

[0062] (3) Registration Interface

[0063] Interface description: This method is called when an H5 application registers a canvas container;

[0064] Parameter description: The canvas parameter is the index of the canvas container specified by the application, and eglWindow is the EGL window provided by the drawing window abstraction layer that is bound to the canvas container;

[0065] void(*register_)(native_itf_t*,INDEX canvas,EGLNativeWindowTypeeglWindow);

[0066] (4) Cancel the interface

[0067] Interface description: This method is called when the canvas container is unregistered in the H5 application;

[0068] Parameter description: The canvas parameter is the index of the canvas container specified by the application.

[0069] void(*unregister_)(native_itf_t*,INDEX canvas);

[0070] (5) Event Interface

[0071] Interface Description: This interface is used to receive event messages from H5 applications;

[0072] Parameter description: The parameter canvas is the index of the canvas container that generates the event, msg is a JSON-formatted string, and length is the length of msg;

[0073] void(*event_)(native_itf_t*,INDEX canvas,const char*msg,int32_tlength);

[0074] II. HTML5 Application Interface Pseudocode

[0075] (1) Initialization interface pseudocode

[0076] Interface Description: Initializes the drawing window abstraction layer;

[0077] Parameter description: libname is the name of the drawing library;

[0078] void init(String libname);

[0079] (2) Register canvas

[0080] Interface Description: H5 registration canvas interface;

[0081] Parameter description: canvas is the H5 Canvas object, and Index is the Canvas index;

[0082] void registerCanvas(HTMLCanvas canvas,optional Number index=0);

[0083] (3) Cancel the canvas

[0084] Interface Description: H5 Cancellation Canvas Interface. This method is called if the canvas is no longer in use.

[0085] Parameter description: canvas is the H5 Canvas object, and Index is the Canvas index;

[0086] void unregisterCanvas(HTMLCanvas canvas,optional Number index=0);

[0087] (4) Send drawing event

[0088] Interface description: Distributes events to the local library, such as touch events;

[0089] Parameter description: msg H5 event message (JSON format recommended), Index is the Canvas index.

[0090] void event(String msg,optional Number index=0);

[0091] (5) Set up local library event callback methods

[0092] Interface Description: Sets the callback method for local library events. This interface is used to receive local library events.

[0093] Parameter description: cd is the callback method;

[0094] void setNativeDrawableEventHandler(NativeDrawableEventHandler cb);

[0095] The input parameter callback method is declared as follows:

[0096] / **

[0097] *@function Draw event callback method;

[0098] *@param msg Local library event message (JSON format recommended);

[0099] *@param args index The index of the Canvas. If none is specified, leave it blank; the default value is 0.

[0100] /

[0101] callback NativeDrawableEventHandler=void(Number index,String msg);

[0102] This invention provides a local image rendering system based on a web mobile operating system. Based on a drawing window abstraction layer, it binds the OpenGL context to the operating system application window through EGL, solving the problem of stuttering when drawing large amounts of pixel data using the Canvas2D standard interface and improving local image rendering performance.

[0103] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0104] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A local image rendering system based on a web mobile operating system, characterized in that, The drawing window abstraction layer is extended on the basis of the WEB mobile operating system graphics rendering engine. The drawing window abstraction layer includes: an image rendering module and a container management module. The image rendering module includes a drawing initialization module, a drawing window context module, and a redraw module; the container management module includes a container registration module, a container unregistration module, and an event dispatch module. The container management module is responsible for managing the state of the drawing application's Canvas container window; the image rendering module is responsible for synchronously notifying the drawing application's Canvas container window to redraw after the third-party drawing library completes the drawing, thereby realizing the rendering of local image data. The drawing initialization module in the image rendering module is the starting entry point for local image rendering. It is called and triggered by the drawing application to complete the initialization preparation work of the drawing window abstraction layer. First, it initializes the drawing platform parameters, and then loads the third-party drawing library. The image rendering module provides rendering screen sharing through the drawing window context module. After receiving the registration container event, the drawing window context module obtains a block of display memory from the cache queue managed by Surface in the rendering engine as image data storage. Through the OpenGL shared context, the local drawing rendering window and the Canvas container in the HTML5 drawing application share the texture. The local drawing library draws and renders to the OpenGL texture. The Canvas container directly reuses the OpenGL texture and uploads the texture to the rendering engine to wait for rendering. The redrawing module in the image rendering module is based on the display memory management mechanism of the rendering engine Surface system. After listening to the frame refresh event, it sends a redrawing request for the Canvas container to display the content of the rendering engine. The container management module provides a Canvas container registration function, which binds the Canvas container window and the local drawing rendering window through the container index. After registration is completed, the drawing application Canvas container begins to receive local drawing library operation events. The container registration module is triggered by the drawing application. First, the drawing application passes in the currently used Canvas container, the drawing platform caches the Canvas container context, then registers a buffer listener with the drawing window context module, and finally calls the local drawing registration interface to synchronize the container registration status. The event dispatch module in the container management module handles event interactions between the upper-layer application and the local drawing library. In actual business use cases, it is necessary to handle data interaction between the drawing application and the local drawing library. The drawing window abstraction layer adopts the JSON data format commonly used in HTML5 applications, and data interaction between the two ends is achieved by agreeing on the data communication interface between the two ends. When the local drawing library calls EGL SwapBuffer, the display cache is enqueued into the cache queue inside the Surface. Then, the frame available event is triggered. After the redraw module receives the frame available event, it obtains the shared texture cache from the drawing window context module. All subsequent drawing content will be written to the display cache. The texture data is then converted into bitmap image data. The Canvas container uses this image data for drawing operations and notifies the rendering engine to redraw. The rendering engine will redraw when the next vertical synchronization VSYNC signal arrives.

2. The local image rendering system based on a web mobile operating system according to claim 1, characterized in that, If the loading of the third-party drawing library is successful, the native drawing interface native_itf, which is uniformly defined by the drawing window abstraction layer, is loaded. After the interface is successfully loaded, the local drawing initialization interface is called to complete the initialization of the third-party drawing library. At the same time, a notification event handle is injected for use in the event dispatch module in the container management module. If the loading process fails, the drawing platform is exited.

3. The local image rendering system based on a web mobile operating system according to claim 1, characterized in that, The container management module provides a container unregistration module for the Canvas container. After unregistration, any operation of the local drawing library will not be associated with this container. The container unregistration module is also triggered by the drawing application. It removes the current Canvas container from the drawing platform and unbinds the Canvas container window from the local drawing rendering window. First, it removes the Canvas container context cache from the drawing platform through the Canvas container index, then unregisters the buffer listener to the drawing window context module, and finally calls the local drawing unregistration interface to synchronize the container unregistration status.

4. The local image rendering system based on a web mobile operating system according to claim 1, characterized in that, The event distribution module first starts a message receiving thread. After registering a notification event callback in the HTML5 drawing application, it registers an event listener to the event distribution module. The drawing application sends drawing event data, which is then passed to the third-party drawing library through the event distribution module. The third-party drawing library requests notification data from the event distribution module by calling the notification event, and then the event distribution module forwards the data to the drawing application.

Images