Page rendering method and device, electronic equipment and computer storage medium
By creating native rendering nodes and texture layer nodes and drawing native components using native systems, the problem that the rendering engine is difficult to render complex interactive scenes is solved, and the same-level display of native components and basic components is realized, improving rendering flexibility and performance.
Patent Information
- Application Number
- CN202410557622.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-07
- Publication Date
- 2024-12-03
AI Technical Summary
Existing rendering engines find it difficult to realize complex interactive scenes, such as video and map interactive scenes through self-rendering, and cannot effectively render native components, resulting in the problem that complex components block basic components.
By creating native rendering nodes and texture layer nodes, using the native system to draw native components on the virtual drawing surface and render them to the target page. Combining the design of the rendering tree and hierarchical tree, the same-level rendering of native components is achieved to avoid occlusion problems.
It realizes rendering of complex interactive scenes, ensures that the same level of native components and basic components are displayed, improves rendering flexibility and performance, and solves the problem of native component occlusion.
Smart Images

Figure CN119065758A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a page rendering method, device, electronic device, and computer storage medium. Background Art
[0002] With the continuous development of computer technology, users can realize various services through various interactive operations with client pages, such as video playback, picture and text reading, map search, map positioning, etc. In the process of developing the client, developers can design page display elements, display components and other information, and render them to the front-end page to realize various interactive operations.
[0003] In the related art, simple interactive scenarios can be implemented based on the basic components of the rendering engine itself. For example, for simple text and image rendering scenarios, the rendering of text and image information can be implemented using self-rendering based on the text components and image components of the rendering engine itself. For complex interactive scenarios, since the rendering engine itself does not have complex native components, it is difficult for the rendering engine to implement complex interactive scenarios such as video and map interactive scenarios using self-rendering. Summary of the invention
[0004] In view of the above problems, the present disclosure provides a page rendering method, device, electronic device and computer storage medium.
[0005] According to a first aspect of the present disclosure, a page rendering method is provided, comprising:
[0006] In response to receiving a request for rendering a target page, determining a node type of a current rendering node in a rendering tree, wherein the rendering tree includes at least one rendering node, each rendering node being used to render a component in the target page;
[0007] In the case where it is determined that the node type is a native rendering node, calling a native system to draw a native component to be rendered by the native rendering node on a virtual drawing surface, wherein the native system represents a terminal system for displaying a target page; and
[0008] A native component is rendered in a target page according to a texture layer node and a virtual drawing surface corresponding to a native rendering node, wherein the texture layer node is used to store texture information of the native rendering node.
[0009] According to an embodiment of the present disclosure, calling the native system to draw the native component to be rendered by the native rendering node on the virtual drawing surface includes:
[0010] The target drawing function in the native system is called by the first thread to draw the native component on the virtual drawing surface, wherein the virtual drawing surface is predefined in the target drawing function.
[0011] According to an embodiment of the present disclosure, after drawing the native component on the virtual drawing surface, the method further includes:
[0012] Store the virtual drawing surface into the message queue;
[0013] Sending a callback notification indicating that data preparation is complete to the texture layer node; and
[0014] Call the status update function to modify the frame data status of the texture layer node.
[0015] According to an embodiment of the present disclosure, rendering a native component in a target page according to a texture layer node and a virtual drawing surface corresponding to a native rendering node includes:
[0016] When it is determined that the frame data state is in the consumption state, using the second thread to obtain the virtual drawing surface from the message queue according to the texture layer node; and
[0017] The native components in the virtual drawing surface are drawn on the actual drawing surface, wherein the actual drawing surface is the drawing surface of the target page, and the actual drawing surface includes at least one component in the target page.
[0018] According to an embodiment of the present disclosure, obtaining a virtual drawing surface from a message queue according to a texture layer node includes:
[0019] Determine a binding state of the texture layer node, wherein the binding state indicates whether there is a binding relationship between the texture layer node and the virtual drawing surface;
[0020] When it is determined that there is no binding relationship between the texture layer node and the virtual drawing surface, a texture name generation function is called to generate a texture name of the texture layer node;
[0021] Calling a binding function to establish a binding relationship between the texture layer node and the virtual drawing surface; and
[0022] Get the virtual drawing surface from the message queue based on the binding relationship and the texture name of the texture layer node.
[0023] According to an embodiment of the present disclosure, drawing a native component in a virtual drawing surface on an actual drawing surface includes:
[0024] generating bitmap data corresponding to the virtual drawing surface; and
[0025] Call the bitmap drawing function to draw the native component at the preset position of the actual drawing surface, where the preset position is a parameter in the bitmap drawing function.
[0026] According to an embodiment of the present disclosure, before calling the native system to draw the native component corresponding to the current native rendering node on the virtual drawing surface, the method further includes:
[0027] Create a native render node;
[0028] Sending registration information of the native rendering node to the native system; and
[0029] According to the registration result returned by the native system, a texture layer node corresponding to the native rendering node is created, wherein the registration result is used to represent the result of the registration information.
[0030] According to an embodiment of the present disclosure, the native component is covered with an interactive component, and the interactive component is used to control the native component through interactive operations with the user; the method further includes:
[0031] In response to detecting that the user adjusts the display position of the target page on the terminal device through a preset interactive operation, re-determining the position information of the native component; and
[0032] Based on the position information, the position of the interactive component on the target page is adjusted so that the native component can be controlled through the interaction between the user and the interactive component when the display position of the target page is adjusted.
[0033] A second aspect of the present disclosure provides a page rendering device, comprising:
[0034] a determination module configured to determine, in response to receiving a request for rendering a target page, a node type of a current rendering node in a rendering tree, wherein the rendering tree includes at least one rendering node, each rendering node being used to render a component in the target page;
[0035] A calling module is configured to call a native system when determining that the node type is a native rendering node, and draw the native component to be rendered by the native rendering node on a virtual drawing surface, wherein the native system represents a terminal system for displaying a target page; and
[0036] The rendering module is configured to render a native component in a target page according to a texture layer node and a virtual drawing surface corresponding to the native rendering node, wherein the texture layer node is used to store texture information of the native rendering node.
[0037] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned page rendering method.
[0038] A fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned page rendering method.
[0039] A fifth aspect of the present disclosure further provides a computer program product, including a computer program, which implements the above-mentioned page rendering method when executed by a processor.
[0040] In an embodiment of the present disclosure, by creating a native rendering node for implementing a native component, the rendering tree may include the native rendering node, and the hierarchical tree corresponding to the rendering tree also includes the texture layer node corresponding to the native rendering node, thereby realizing the rendering of the native component. In addition, since the native rendering node and the basic node can be included during rendering, when rendering is performed using a rendering engine, the hierarchical relationship between the native component and the basic component can be flexibly controlled based on the rendering tree to avoid the native component blocking the basic component, thereby realizing the same-layer rendering of the native component. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0042] Figure 1 The application scenario of the page rendering method according to the embodiment of the present disclosure is schematically shown;
[0043] Figure 2 A flowchart of a page rendering method according to an embodiment of the present disclosure is schematically shown;
[0044] Figure 3 Schematically shows an application scenario diagram for implementing native component rendering based on a UI framework and a rendering engine according to an embodiment of the present disclosure;
[0045] Figure 4 A schematic diagram of a scene in which a native component to be rendered by a native rendering node is drawn on a virtual drawing surface according to an embodiment of the present disclosure is schematically shown;
[0046] Figure 5 A schematic diagram of a scene for rendering a native component according to an embodiment of the present disclosure is schematically shown;
[0047] Figure 6 A schematic diagram schematically shows the position adjustment of interactive components and native components in an interactive scenario according to an embodiment of the present disclosure;
[0048] Figure 7 A schematic diagram of a structure of a page rendering device according to an embodiment of the present disclosure is shown; and
[0049] Figure 8A block diagram of an electronic device suitable for a page rendering method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0050] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0051] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0052] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0053] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0054] In the field of computer technology, the design of front-end pages can be implemented through user interface (UI) frameworks such as ReactNative (RN). For example, RN can convert the front-end rendering tree (ShadowTree) into native components of Android and iOS for display. Alternatively, the rendering engine skia can be used to replace the native component display through skia self-rendering. However, some complex interactive scenes cannot be directly drawn through skia self-rendering, such as interactive scenes such as videos and maps.
[0055] In the related technology, a layer of CoverView can be overlaid on the SurfaceView or TextureView that carries the RN, and complex native components that cannot be directly drawn through the skia self-rendering method can be added to the CoverView to achieve the rendering and display of complex native components. However, in the above implementation, CoverView is always overlaid on the Surface of the Skia canvas, which makes it impossible for developers to control the hierarchy of these complex components through the front-end code. The effect is reflected as follows: video players, maps, etc. are always on the top layer, and the interactive information that should be displayed on them will be covered due to the hierarchy.
[0056] To this end, an embodiment of the present disclosure provides a page rendering method, including: in response to receiving a request for rendering a target page, determining a node type of a current rendering node in a rendering tree, wherein the rendering tree includes at least one rendering node, and each rendering node is used to render a component in the target page; when it is determined that the node type is a native rendering node, calling a native system to draw a native component to be rendered by the native rendering node on a virtual drawing surface, wherein the native system represents a terminal system for displaying the target page; and rendering a native component in the target page according to a texture layer node corresponding to the native rendering node and the virtual drawing surface, wherein the texture layer node is used to store texture information of the native rendering node.
[0057] Figure 1 The application scenario of the page rendering method according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown are merely examples of application scenarios to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0058] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0059] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).
[0060] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0061] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0062] It should be noted that the page rendering method provided in the embodiment of the present disclosure can generally be executed by a terminal device, such as one of the first terminal device 101, the second terminal device 102, and the third terminal device 103. Accordingly, the page rendering apparatus provided in the embodiment of the present disclosure can generally be set in a terminal device.
[0063] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0064] The following will be based on Figure 1 The scene described by Figure 2 to Figure 6 The page rendering method of the disclosed embodiment is described in detail.
[0065] Figure 2 The flowchart of the page rendering method according to the embodiment of the present disclosure is schematically shown.
[0066] like Figure 2 As shown, the method 200 includes operations S210 to S230.
[0067] In operation S210 , in response to receiving a request for rendering a target page, a node type of a current rendering node in a rendering tree is determined.
[0068] According to an embodiment of the present disclosure, the target page may include multiple components, such as a text component, an image component, a video player component, a map component, etc. There may be multiple upper and lower hierarchical display relationships between the multiple components, such as a video player component being located above a text component.
[0069] According to an embodiment of the present disclosure, in the process of designing a target page, a developer may form a rendering tree ShadowTree based on the upper and lower hierarchical display relationships between multiple components, so that a terminal device can implement rendering of multiple components based on ShadowTree.
[0070] According to an embodiment of the present disclosure, the rendering tree includes at least one rendering node, each rendering node is used to render a component in the target page. The node type of the rendering node is related to the component it renders. The node type of each rendering node can be a native rendering node or a basic node, the native rendering node is used to render the native component in the target page, and the basic node is used to render the basic component in the target page. The basic component can be understood as a component equipped by the rendering engine itself, that is, the rendering engine can directly draw the basic component. The native component can be understood as a component that the rendering engine itself is not equipped with and needs to be implemented with the help of the native system of the terminal device.
[0071] In operation S220 , when it is determined that the node type is a native rendering node, a native system is called to draw a native component to be rendered by the native rendering node on a virtual drawing surface.
[0072] According to an embodiment of the present disclosure, the native system represents a terminal system used to display a target page.
[0073] For example, the rendering engine skia cannot directly draw a video player component, but the terminal device's Android system is equipped with a video player component. Therefore, skia can use the video player component in the Android system to draw the video player component in the target page and implement the video playback function in the target page. Therefore, the video player component is a native component, and the Android system is the terminal device's terminal system, that is, the native system.
[0074] According to an embodiment of the present disclosure, when the node type is determined to be a base node, a base component is rendered in the target page based on the self-rendering method of the rendering engine. Since the rendering of the base component does not require the help of the native system, the calling of the native component in the native system is not involved when drawing the base component.
[0075] According to an embodiment of the present disclosure, the "virtual" in the virtual drawing surface refers to native components. The virtual drawing surface only includes native components to be rendered by the current native rendering node, and basic components do not need to be rendered on the virtual drawing surface.
[0076] In operation S230 , a native component is rendered in the target page according to the texture layer node and the virtual drawing surface corresponding to the native rendering node.
[0077] According to an embodiment of the present disclosure, the UI framework includes a rendering tree ShadowTree, and the rendering engine includes a hierarchical tree LayerTree corresponding to the rendering tree. ShadowTree facilitates the intuitive display of the front-end UI design, and LayerTree facilitates the hierarchical rendering of the back-end rendering.
[0078] According to an embodiment of the present disclosure, the texture layer node is used to store texture information of the native rendering node. LayerTree may include multiple texture layer nodes TextureLayer, each texture layer node corresponds to a rendering node at the same level in ShadowTree.
[0079] It should be noted that both the base node and the native rendering node correspond to a texture layer node to achieve component rendering. Developers can achieve the same-layer rendering of native components and base components by controlling the hierarchical position of TextureLayer in LayerTree.
[0080] According to an embodiment of the present disclosure, a native component is rendered in a target page based on a texture layer node and a virtual drawing surface corresponding to a native rendering node, including: based on the texture layer node, a virtual drawing surface including the native component is merged with an actual drawing surface for drawing a basic component, thereby enabling the native component and the basic component to be drawn on the actual drawing surface to be displayed with the help of the native system, thereby achieving rendering of the target page.
[0081] According to an embodiment of the present disclosure, merging the virtual drawing surface including the native component with the actual drawing surface for drawing the base component may include: redrawing the native component drawn in the virtual drawing surface on the actual drawing surface.
[0082] According to an embodiment of the present disclosure, while rendering the native component in the target page, the basic component can also be drawn.
[0083] In an embodiment of the present disclosure, by creating a native rendering node for implementing a native component, the rendering tree may include the native rendering node, and the hierarchical tree corresponding to the rendering tree also includes the texture layer node corresponding to the native rendering node, thereby realizing the rendering of the native component. In addition, since the native rendering node and the base node may be included during rendering, when rendering is performed using a rendering engine, the hierarchical relationship between the native component and the base component may be flexibly controlled based on the rendering tree to avoid the native component blocking the base component, or to cover other components on the upper layer of the native component / base component, thereby realizing the same-layer rendering of the native component.
[0084] Figure 3 An application scenario diagram for implementing native component rendering based on a UI framework and a rendering engine according to an embodiment of the present disclosure is schematically shown.
[0085] like Figure 3 As shown, in an application scenario 300 in which a rendering tree includes both basic nodes and native rendering nodes, the UI framework is RN and the rendering engine is skia.
[0086] For native rendering nodes, RN's ShadowTree sends commands to the native system when it encounters a native rendering node that needs to be implemented by a native component. After the native system creates a native component, it adds itself to a Surface, i.e. a virtual drawing surface; at the same time, it associates and creates a TextureLayer, i.e. a texture layer node, in Skia's RenderEngine, so that the native component can be refreshed and drawn later through the native system's onDraw driver, thereby achieving the effect of native component rendering on the same layer.
[0087] For basic nodes, when RN's ShadowTree encounters a basic node that requires basic components to implement, it does not need to communicate with the native system, and can directly use the rendering engine to implement component rendering.
[0088] According to an embodiment of the present disclosure, calling a native system to draw a native component to be rendered by a native rendering node on a virtual drawing surface includes: using a first thread to call a target drawing function in the native system to draw the native component on the virtual drawing surface, wherein the virtual drawing surface is predefined in the target drawing function.
[0089] According to an embodiment of the present disclosure, the first thread is a thread for the native system to perform operations such as data interaction and data control. Based on the system characteristics of the native system, such as the Android system, when rendering, the first thread includes the transfer of the target drawing function, which is the draw method. The target drawing function is used to define the shape, position and other information of the drawing component.
[0090] According to an embodiment of the present disclosure, the virtual drawing surface is predefined in the target drawing function. It can be understood that for multiple native rendering nodes, the native components to be rendered by the multiple native rendering nodes can be drawn on the corresponding virtual drawing surfaces. In an exemplary embodiment, the virtual drawing surface of each native rendering node can include the native components to be rendered by the native rendering node, and the basic components can be directly drawn on the actual drawing surface.
[0091] According to an embodiment of the present disclosure, the target drawing function is a rewritten draw method. The rewriting of the draw method includes: using the created virtual drawing surface and calling the Canvas obtained by lockHardwareCanvas to draw the native component.
[0092] For example, in the rewritten draw method, the Canvas surfaceCanvas = surface.lockHardwareCanvas() is declared in final. The contents of the try block executed by the draw method are super.draw(surfaceCanvas) and onFrameProduced(), which are used to draw native components on the virtual drawing surface. In addition, the rewritten draw method also includes a finally block, the content of which is surface.unlockCanvasAndPost(surfaceCanvas). The finally block is used to call unlockCanvasAndPost after the drawing is completed. At this time, the listener of the virtual drawing surface Surface will receive an onFrameAvailable callback notification. After the callback notification is generated, the callback notification is sent to the TextureLayer of the C++ framework layer, and the TextureLayer is marked as FrameReady, that is, the frame data is ready.
[0093] Figure 4 A schematic diagram of a scene in which a native component to be rendered by a native rendering node is drawn on a virtual drawing surface according to an embodiment of the present disclosure is schematically shown.
[0094] like Figure 4 As shown, when the RN layout receives the draw method, it will draw based on the draw method and pass the draw method to its child window, that is, the child View. The window WrapperView of the native component is also a child window of the RN layout. Therefore, after receiving the rendering request, the target drawing function of the native component can be passed to WrapperView based on the first thread, so that the native component is drawn on the virtual drawing surface.
[0095] After drawing is completed, unlockCanvasAndPost is called to generate an onFrameAvailable callback notification and send it to the texture layer node corresponding to the native component in the C++ framework layer, marking TextureLayer as FrameReady.
[0096] According to an embodiment of the present disclosure, after drawing the native component on the virtual drawing surface, it also includes: storing the virtual drawing surface in a message queue; sending a callback notification representing that data preparation is complete to the texture layer node; and calling a status update function to modify the frame data status of the texture layer node.
[0097] According to an embodiment of the present disclosure, the message queue may be a production and consumption queue of SurfaceTexture. After the native component is drawn on the virtual drawing surface, the virtual drawing surface may be stored in the message queue in the form of frame data.
[0098] In addition, after marking TextureLayer as FrameReady, call the status update function updateTexImage to modify the frame data status of TextureLayer, and at the same time modify the frame data stored in the SurfaceTexture production and consumption queue to the consumption status.
[0099] According to an embodiment of the present disclosure, a native component is rendered in a target page according to a texture layer node and a virtual drawing surface corresponding to a native rendering node, including: when it is determined that the frame data state is a consumption state, using a second thread to obtain a virtual drawing surface from a message queue according to the texture layer node; and drawing the native component in the virtual drawing surface on an actual drawing surface, wherein the actual drawing surface is a drawing surface of the target page, and the actual drawing surface includes at least one component in the target page.
[0100] According to an embodiment of the present disclosure, the thread in which the rendering engine renders the native component is different from the thread in which the native component is drawn to the virtual drawing surface. The rendering of the native component in the target page is implemented based on the second thread, namely the skia thread.
[0101] According to an embodiment of the present disclosure, the second thread calls the paint method for each TextureLayer during the rendering process. When the frame data state is the consumption state (that is, when TextureLayer is FrameReady), the second thread obtains the virtual drawing surface corresponding to the texture layer node from the message queue according to the identification information of the texture layer node, such as ID; then, the native components in the virtual drawing surface are drawn on the actual drawing surface.
[0102] In the embodiments of the present disclosure, since the LayerTree rendered by skia has levels, by controlling the hierarchical position of TextureLayer in the LayerTree, the same-layer rendering of native components and basic components can be achieved.
[0103] According to an embodiment of the present disclosure, obtaining a virtual drawing surface from a message queue according to a texture layer node includes: determining a binding state of the texture layer node, wherein the binding state represents whether there is a binding relationship between the texture layer node and the virtual drawing surface; in the case where it is determined that there is no binding relationship between the texture layer node and the virtual drawing surface, calling a texture name generation function to generate a texture name for the texture layer node; calling a binding function to establish a binding relationship between the texture layer node and the virtual drawing surface; and obtaining the virtual drawing surface from the message queue based on the binding relationship and the texture name of the texture layer node.
[0104] According to an embodiment of the present disclosure, during the rendering process, a binding relationship is established between the texture layer node and the virtual drawing surface obtained during the drawing process, so that the second thread can obtain the virtual drawing surface from the message queue based on the binding relationship according to the identification information of the texture layer node. When the binding relationship is established, the texture layer node TextureLayer is in the attach state; otherwise, it is not in the attach state.
[0105] When it is determined that there is no binding relationship between the texture layer node and the virtual drawing surface, the texture name generation function glGenTextures is called to generate the texture name texName of the texture layer node. The binding function attachToGLContext of the native layer SurfaceTexture is used to establish the binding relationship between Suface and the texture name texName. After that, the virtual drawing surface is obtained from the message queue based on the binding relationship and the texture name texName of the texture layer node.
[0106] When it is determined that there is a binding relationship between the texture layer node and the virtual drawing surface, the virtual drawing surface is directly obtained from the message queue based on the binding relationship and the texture name texName of the texture layer node.
[0107] According to an embodiment of the present disclosure, drawing a native component in a virtual drawing surface on an actual drawing surface includes: generating bitmap data corresponding to the virtual drawing surface; and calling a bitmap drawing function to draw the native component at a preset position on the actual drawing surface, wherein the preset position is a parameter in the bitmap drawing function.
[0108] According to an embodiment of the present disclosure, the bitmap creation function SkImage::MakeFromTexture can be called by the texture name texName, and the bitmap data can be created according to the prepared frame data obtained from the message queue. In an embodiment of the present disclosure, the created bitmap data is a graphics processing unit (GPU) texture type, so that the GPU can be used to render the SkImage later.
[0109] After the bitmap data is generated, the bitmap drawing function drawImage is called to draw the native component onto the actual drawing surface of the target page, thus realizing the rendering of the native component. The actual drawing surface is the Canvas included in the paint method when rendering based on the skia thread.
[0110] Figure 5 A schematic diagram of a scene for rendering a native component according to an embodiment of the present disclosure is schematically shown.
[0111] like Figure 5 As shown in the figure, for the texture layer node corresponding to the native rendering node, the Skia thread obtains the binding relationship between the virtual drawing surface surface and the texture texName, that is, determines whether the TextureLayer is in the attach state. If the frame data is ready, call updateTexImage to change the frame data in the SurfaceTexture production and consumption queue to the consumption state, and modify the frame data state of TextureLayer at the same time.
[0112] After that, SkImage::MakeFromTexture is called through texName to create a GPU texture type bitmap data SkImage based on the prepared frame data. Through the Canvas when skia is drawing, the drawImage method is called to draw the native component onto the Canvas to achieve the rendering of the native component.
[0113] In an embodiment of the present disclosure, the SurfaceTexture production and consumption queue is used to draw the native component on the actual drawing surface based on the second thread, which can not only ensure the same-layer rendering of the native component and the basic component, but also improve the rendering performance and rendering data consistency.
[0114] According to an embodiment of the present disclosure, before calling the native system and drawing the native component corresponding to the current native rendering node on the virtual drawing surface, it also includes: creating a native rendering node; sending registration information of the native rendering node to the native system; and creating a texture layer node corresponding to the native rendering node according to the registration result returned by the native system, wherein the registration result is used to characterize the result of the registration information.
[0115] According to an embodiment of the present disclosure, the native rendering nodes and the base nodes in the rendering tree are all inherited from PlatformConcreteViewShadowNode. The base components corresponding to the base nodes can be directly rendered by the rendering engine skia, while the native components corresponding to the native rendering nodes cannot be directly rendered by skia. It is necessary to create a relationship between the native rendering nodes and the native implementation, that is, to render the native components through the communication between the native rendering nodes and the native system.
[0116] According to an embodiment of the present disclosure, creating a native rendering node includes: creating a node of PlatformConcreteViewShadowNode type in the Framework of RN. The component name ComponentName of the node is RNTMyNativeView. It should be noted that the component name RNTMyNativeView is pre-declared in RN. Different types of native components have different component names. For example, for a video player component, the component name ComponentName can be video, and the component name ComponentName of a map component can be map.
[0117] Before referencing and using tags implemented by native components in RN, you need to declare the native component. For example, take the component name ComponentName as "RNTMyNativeView". <nativeprops>('RNTMyNativeView',):MyNativeViewType declares the component name, through<RNTMyNativeView ref={ref}style={{flex:1}}opacity={opacity} / > Reference and use tags implemented by native components.
[0118] Since the native rendering node needs to establish a relationship with the native system, RN sends the requestTextureLayer command to the native system and specifies the name of the native component as the name when it was created, that is, RNTMyNativeView; at the same time, the requestTextureLayer command also specifies the position and width and height of the native component in the window view. The requestTextureLayer command is the registration information.
[0119] After receiving the requestTextureLayer command, the native system creates the windows of the native component through two operations, namely the native View and a blank drawing surface Surface. The blank drawing surface Surface is the virtual drawing surface. Specifically, the ViewFactoryManager is used to find the registered ViewFactory of type RNTMyNativeView, and the native View is created through this Factory. On the other hand, a blank SurfacTexture is created through TextureRegistry, and Surface is used to encapsulate it as a blank drawing surface Surface.
[0120] After that, bind the native component and the native View to form a WrapperView, and add the formed WrapperView to the upper layer of RN's frame layout (FrameLayout) according to the width, height and position of the requestTextureLayer command.
[0121] After the creation is completed, the WrapperView and Texture ids are returned to the C++ framework layer, and the C++ framework layer creates a TextureLayer in the LayerTree for Skia drawing. As a result, the ShadowNode of the native rendering node has a corresponding TextureLayer in the LayerTree, which records the Texture id and the WrapperView containing the native component. The above creation process is also the process of registering the native component in the RN framework.
[0122] The embodiment of the present disclosure creates an independent native rendering node that is only used for native component rendering through the PlatformConcreteViewShadowNode inheritance method, communicates with the native layer through the native rendering node and generates a TextureLayer, thereby realizing the drawing of native components based on the rendering engine.
[0123] According to the embodiments of the present disclosure, when the content of a native component changes, the native system will be triggered to perform a partial refresh and re-call the draw method of WrapperView. The above refresh process is similar to the drawing process of the native component, except that when the callback notification of the frame data being ready is passed to the TextureLayer of the C++ framework layer, a partial drawing of the LayerTree is manually triggered to ensure that the updated content can be refreshed to the screen in time.
[0124] According to an embodiment of the present disclosure, an interactive component is covered on the upper layer of the native component, and the interactive component is used to control the native component through interactive operations with the user. The method also includes: in response to detecting that the user adjusts the display position of the target page on the terminal device through a preset interactive operation, re-determining the position information of the native component; and adjusting the position of the interactive component on the target page based on the position information, so that when the display position of the target page is adjusted, the native component is controlled through the interactive operation between the user and the interactive component.
[0125] There is a transparent WrapperView on the upper layer of RN's native layout. Users can operate terminal devices through gestures and implement interactive operations through interactive components. All gesture events will be passed to the above WrapperView, which will pass the gesture events to the corresponding native components to implement operations such as clicks and slides. For example, when reading the target page, the user can slide up to read the undisplayed part of the target page.
[0126] For complex interactive scenarios including video player components and map components, native components in related technologies can only be overlaid on the upper layer of basic components. In this complex scenario, it is impossible to control the video player component and the map component through interactive operations on the interactive components. However, in the embodiments of the present disclosure, the native components are not implemented by overlaying on the top layer of the basic components, and the upper and lower hierarchical relationship between the native components and the basic components can be flexibly adjusted. Therefore, the embodiments of the present disclosure can control the native components through the interactive components.
[0127] When the position of the native rendering node ShadowNode of the native component in the RN layout changes, it is necessary to notify the native layer to modify the position of WrapperView and adjust the position information of the interactive component to the display position of the native component to ensure that the gesture event can change with the position of the native component, thereby continuing to control the native component.
[0128] According to an embodiment of the present disclosure, adjustment parameters, such as the upward, downward, leftward, and rightward adjustment ranges, can be determined based on the position information of the native component before and after adjustment; based on the adjustment parameters, the position of the interactive component on the target page is adjusted.
[0129] Figure 6 The figure schematically shows the position adjustment of interactive components and native components in an interactive scenario according to an embodiment of the present disclosure.
[0130] like Figure 6 As shown, the target page 601 before the change includes a video player component 602 and an interactive component 603. When the target page 601 before the change is displayed through the terminal device, the interactive component 603 can be transparent, that is, the user cannot see the existence of the interactive component 603, but can interact with the interactive component through preset operations on the preset area. For example, the interactive component 603 is set at the center of the video player component 602 to control the play or stop of the video player component 602.
[0131] When the user slides the screen, the display position of the video player component 602 will change. In the target page 604 after the change, the video player component 602 slides to the upper side of the screen. When the position of the video player component 602 changes, if the position of the interactive component 603 is not adjusted, the user cannot control the play or stop of the video by clicking on the video player component 602. At this time, although the user cannot view the interactive component 603, in order to ensure the normal operation of the play or stop function in the target page 604 after the change, the position of the interactive component 603 needs to be readjusted based on the position change of the video player component 602.
[0132] When the embodiment of the present disclosure faces the scene of using the rendering engine skia to implement RN rendering, it avoids the problem of incomplete animation effect display caused by component level coverage, and can achieve the same-layer rendering of native components. In addition, the embodiment of the present disclosure can also control the native components in the target page through the interactive components, thereby improving the flexibility of the rendering process.
[0133] Figure 7 The structural block diagram of the page rendering device according to an embodiment of the present disclosure is schematically shown.
[0134] like Figure 7 As shown, the page rendering device 700 of this embodiment includes a determination module 710 , a calling module 720 and a rendering module 730 .
[0135] The determination module 710 is configured to determine the node type of the current rendering node in the rendering tree in response to receiving a request for rendering the target page, wherein the rendering tree includes at least one rendering node, and each rendering node is used to render a component in the target page.
[0136] The calling module 720 is configured to call the native system when determining that the node type is a native rendering node, and draw the native component to be rendered by the native rendering node on the virtual drawing surface, wherein the native system represents a terminal system for displaying the target page.
[0137] The rendering module 730 is configured to render a native component in a target page according to a texture layer node and a virtual drawing surface corresponding to the native rendering node, wherein the texture layer node is used to store texture information of the native rendering node.
[0138] According to an embodiment of the present disclosure, the calling module 720 includes a calling unit, which is used to use the first thread to call the target drawing function in the native system to draw the native component on the virtual drawing surface, wherein the virtual drawing surface is pre-defined in the target drawing function and the virtual drawing surface only includes the native component.
[0139] According to an embodiment of the present disclosure, the page rendering device 700 further includes a storage module, a callback module and a modification module. The storage module is used to store the virtual drawing surface in a message queue. The callback module is used to send a callback notification of the completion of the characterization data preparation to the texture layer node. The modification module is used to call a state update function to modify the frame data state of the texture layer node.
[0140] According to an embodiment of the present disclosure, the rendering module 730 includes an acquisition unit and a rendering unit.
[0141] The acquisition unit is used to acquire the virtual drawing surface from the message queue according to the texture layer node by using the second thread when the frame data state indicates that the virtual drawing surface is ready.
[0142] The rendering unit is used to draw the native component in the virtual drawing surface on the actual drawing surface, wherein the actual drawing surface is the drawing surface of the target page, and the actual drawing surface includes at least one component in the target page.
[0143] According to an embodiment of the present disclosure, the acquisition unit includes a state determination subunit, a generation subunit, a first calling subunit and an acquisition subunit.
[0144] The state determination subunit is used to determine the binding state of the texture layer node, wherein the binding state represents whether there is a binding relationship between the texture layer node and the virtual drawing surface.
[0145] The generating subunit is used to call the texture name generating function to generate the texture name of the texture layer node when it is determined that there is no binding relationship between the texture layer node and the virtual drawing surface.
[0146] The first calling subunit is used to call a binding function to establish a binding relationship between the texture layer node and the virtual drawing surface.
[0147] Gets a subunit for obtaining a virtual drawing surface from a message queue based on a binding relationship and a texture name of a texture layer node.
[0148] According to an embodiment of the present disclosure, the rendering unit includes a bitmap generation subunit and a second calling subunit. The bitmap generation subunit is used to generate bitmap data corresponding to the virtual drawing surface. The second calling subunit is used to call a bitmap drawing function to draw a native component at a preset position of the actual drawing surface, wherein the preset position is a parameter in the bitmap drawing function.
[0149] According to an embodiment of the present disclosure, the page rendering device 700 further includes a first creation module, a registration module and a second creation module.
[0150] The first creation module is used to create a native rendering node. The registration module is used to send registration information of the native rendering node to the native system. The second creation module is used to create a texture layer node corresponding to the native rendering node according to the registration result returned by the native system, wherein the registration result is used to represent the result of the registration information.
[0151] According to an embodiment of the present disclosure, the page rendering device 700 further includes a first adjustment module and a second adjustment module. According to an embodiment of the present disclosure, the native component is covered with an interactive component, and the interactive component is used to control the native component through interactive operations with the user.
[0152] The first adjustment module is used to re-determine the position information of the native component in response to detecting that the user adjusts the display position of the target page on the terminal device through a preset interactive operation.
[0153] The second adjustment module is used to adjust the position of the interactive component on the target page based on the position information, so that when the display position of the target page is adjusted, the native component can be controlled through the interaction between the user and the interactive component.
[0154] According to an embodiment of the present disclosure, any multiple modules among the determination module 710, the calling module 720 and the rendering module 730 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.
[0155] According to an embodiment of the present disclosure, at least one of the determination module 710, the calling module 720 and the rendering module 730 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the determination module 710, the calling module 720 and the rendering module 730 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.
[0156] It should be noted that the device part in the embodiments of the present disclosure corresponds to the method part in the embodiments of the present disclosure. The description of the device part specifically refers to the method part and will not be repeated here.
[0157] Figure 8 A block diagram of an electronic device suitable for a page rendering method according to an embodiment of the present disclosure is schematically shown.
[0158] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0159] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0160] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.
[0161] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0162] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.
[0163] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present disclosure.
[0164] The above functions defined in the system / device of the embodiment of the present disclosure are executed when the computer program is executed by the processor 801. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0165] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0166] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
[0167] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0168] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0169] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0170] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present disclosure. It should be understood that the above description is only a specific embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.< / nativeprops>
Claims
1. A page rendering method, comprising: In response to receiving a request for rendering a target page, determining a node type of a current rendering node in a rendering tree, wherein the rendering tree includes at least one rendering node, each of the rendering nodes being used to render a component in the target page; In the case where it is determined that the node type is a native rendering node, calling a native system to draw a native component to be rendered by the native rendering node on a virtual drawing surface, wherein the native system represents a terminal system for displaying the target page; and The native component is rendered in a target page according to a texture layer node corresponding to the native rendering node and the virtual drawing surface, wherein the texture layer node is used to store texture information of the native rendering node.
2. The method according to claim 1, wherein: The calling of the native system to draw the native component to be rendered by the native rendering node on the virtual drawing surface includes: The target drawing function in the native system is called by the first thread to draw the native component on the virtual drawing surface, wherein the virtual drawing surface is predefined in the target drawing function.
3. The method according to claim 1 or 2, wherein: After drawing the native component on the virtual drawing surface, the method further includes: Storing the virtual drawing surface in a message queue; Sending a callback notification indicating that data preparation is complete to the texture layer node; and Calling a state update function to modify the frame data state of the texture layer node.
4. The method according to claim 3, wherein: The rendering of the native component in the target page according to the texture layer node corresponding to the native rendering node and the virtual drawing surface includes: When it is determined that the frame data state is a consumption state, using a second thread to obtain the virtual drawing surface from the message queue according to the texture layer node; and The native component in the virtual drawing surface is drawn on an actual drawing surface, wherein the actual drawing surface is the drawing surface of the target page, and the actual drawing surface includes the at least one component in the target page.
5. The method according to claim 4, wherein: The acquiring the virtual drawing surface from the message queue according to the texture layer node comprises: Determining a binding state of the texture layer node, wherein the binding state indicates whether there is a binding relationship between the texture layer node and the virtual drawing surface; In the case where it is determined that there is no binding relationship between the texture layer node and the virtual drawing surface, calling a texture name generation function to generate a texture name for the texture layer node; Calling a binding function to establish a binding relationship between the texture layer node and the virtual drawing surface; and The virtual drawing surface is obtained from the message queue based on the binding relationship and the texture name of the texture layer node.
6. The method according to claim 4, wherein: Drawing the native components in the virtual drawing surface on the actual drawing surface includes: generating bitmap data corresponding to the virtual drawing surface; and A bitmap drawing function is called to draw the native component at a preset position on the actual drawing surface, wherein the preset position is a parameter in the bitmap drawing function.
7. The method according to claim 1, wherein: Before calling the native system to draw the native component corresponding to the current native rendering node on the virtual drawing surface, the method further includes: Creating the native rendering node; Sending registration information of the native rendering node to the native system; and A texture layer node corresponding to the native rendering node is created according to a registration result returned by the native system, wherein the registration result is used to represent a result of the registration information.
8. The method according to claim 1, wherein: The native component is covered with an interactive component, and the interactive component is used to control the native component through interactive operations with a user; the method further includes: In response to detecting that the user adjusts the display position of the target page on the terminal device through a preset interactive operation, re-determining the position information of the native component; and Based on the position information, the position of the interactive component on the target page is adjusted, so that when the display position of the target page is adjusted, the native component can be controlled through the interactive operation between the user and the interactive component.
9. A page rendering device, comprising: a determination module configured to, in response to receiving a request for rendering a target page, determine a node type of a current rendering node in a rendering tree, wherein the rendering tree includes at least one rendering node, each of the rendering nodes being used to render a component in the target page; a calling module configured to, when determining that the node type is a native rendering node, call a native system to draw a native component to be rendered by the native rendering node on a virtual drawing surface, wherein the native system represents a terminal system for displaying the target page; and The rendering module is configured to render the native component in a target page according to a texture layer node corresponding to the native rendering node and the virtual drawing surface, wherein the texture layer node is used to store texture information of the native rendering node.
10. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 8.
11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
