Method and apparatus for displaying view control

Abstract

The application provides a method and device for displaying a view control, and relates to the field of electronic equipment. The method can be applied to a terminal device, and the terminal device comprises a first engine, a first component, and a second engine. The method comprises the following steps: the first engine determines the plug-in attribute of the view control corresponding to the first application and the front-end DOM node corresponding to the view control. The first component determines the layout information corresponding to the view control according to the front-end DOM node, creates the third-party plug-in and the texture data corresponding to the view control according to the plug-in attribute, and registers the texture data in the GPU. The first component sends the layout information to the second engine. The first component displays the texture data according to the third-party plug-in. The second engine obtains the texture data from the GPU and renders the texture data according to the layout information. The terminal device can display various different view controls and has universality for different view controls.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of electronic devices, and in particular to a method and apparatus for displaying a view control. BACKGROUND

[0002] An operation system (OS) supports displaying various types of view controls, which is conducive to the construction of an OS ecosystem. For example, for an operation system A, when the operation system A supports displaying an Android view control (hereinafter referred to as an Android view control), a terminal device installed with the operation system A can support displaying an application page developed based on the Android view control, which is conducive to the construction of the OS ecosystem of the operation system A. The Android view control can be a web view control, a map view control, a camera view control, and the like.

[0003] Currently, a scheme for realizing that an operation system supports displaying view controls mainly includes: for each type of view control, an operation system development framework can provide a customized control corresponding to the view control, and the operation system can display the view control through the customized control corresponding to the view control. Different view controls correspond to different customized controls.

[0004] However, in the above scheme, the interfaces of the view controls are numerous, which leads to the fact that it is very complex to realize the customized controls corresponding to the view controls in the operation system development framework, and the development workload is extremely large. In addition, once the operation system has a demand for supporting a new view control, the operation system development framework needs to be newly customized and developed, which will seriously affect the development efficiency and is not conducive to the development of the OS ecosystem. SUMMARY

[0005] Embodiments of the present application provide a method and apparatus for displaying a view control, which can enable a terminal device to display various view controls and have universality for different view controls.

[0006] In a first aspect, embodiments of the present application provide a method for displaying a view control, which can be applied to a terminal device. The terminal device includes a first engine, a first component, and a second engine.

[0007] The method comprises: a first engine determining a plug-in attribute of a view control corresponding to a first application and a front-end document object model node corresponding to the view control. A first component determines layout information corresponding to the view control according to the front-end document object model node corresponding to the view control. The first component creates a third-party plug-in and texture data corresponding to the view control according to the plug-in attribute of the view control, and registers the texture data corresponding to the view control in a GPU. The first component sends the layout information corresponding to the view control to a second engine. The first component displays the texture data corresponding to the view control according to the third-party plug-in corresponding to the view control. The second engine obtains the texture data corresponding to the view control from the GPU, and renders the texture data corresponding to the view control according to the layout information corresponding to the view control.

[0008] The first application can be an application developed by a developer based on the view control.

[0009] Optionally, the view control can include any one of a webpage view control, a map view control, a camera view control, a video view control, a text view control, a live broadcast view control, an advertisement view control, and a screen projection view control.

[0010] In some embodiments, the view control can be an Android view control, or a Harmony view control, or a view control developed in other Java languages, which is not limited herein.

[0011] The first packaged file of the first application includes the plug-in attribute of the view control corresponding to the first application, and the plug-in attribute of the view control is preconfigured. For example, the developer of the first application can set the plug-in attribute of the view control to be a webpage view control or a map view control in the first packaged file of the first application, and preconfigure the plug-in attribute of the view control.

[0012] In the method, the developer of the first application only needs to set the plug-in attribute of the view control in the code of the first application, so that the terminal device can support displaying different view controls. When the terminal device displays different view controls according to the method, the terminal device does not need to consider the adaptation and connection of different view controls, and the entire implementation process can be automatically completed by the development framework of the operating system of the terminal device, the developer of the first application can not be aware, which greatly reduces the development difficulty of the first application adapted to the operating system of the terminal device, and is conducive to the construction of the operating system ecosystem of the terminal device.

[0013] In addition, the development framework of the operating system of the terminal device can realize one-time development, deploy various complex view controls, and greatly reduce the development cost. The developer of the first application sets the plugin attribute of the view control in the code of the first application, and the development framework of the operating system automatically matches the corresponding third-party plugin according to the plugin attribute of the view control set by the developer, which can also avoid the secondary development of the developer for different view controls, and effectively improve the development efficiency.

[0014] Further, the method can also avoid creating a texture cache in a central processing unit (CPU) by directly registering the texture data corresponding to the view control into the GPU. The second engine can directly obtain the texture data corresponding to the view control from the GPU for rendering, without the need for transmission of the texture data between the CPU and the GPU, which can save the copying time of the CPU between different texture caches and the time of uploading the texture to the GPU, greatly improving the rendering performance. At the same time, the memory and GPU memory occupied by the texture data are also saved.

[0015] Optionally, the step of registering the texture data corresponding to the view control into the GPU by the first component can include:

[0016] The first component generates a texture identifier of the texture data corresponding to the view control, and registers the texture data corresponding to the view control into the GPU according to the texture identifier of the texture data corresponding to the view control.

[0017] The method further includes: the first component sends the texture identifier of the texture data corresponding to the view control to the second engine.

[0018] The step of obtaining the texture data corresponding to the view control from the GPU by the second engine can include:

[0019] The second engine obtains the texture data corresponding to the view control from the GPU according to the texture identifier of the texture data corresponding to the view control.

[0020] The texture identifier of the texture data can be a texture ID, and the texture ID and the texture data have a one-to-one binding relationship. When the second engine obtains the texture data from the GPU, the texture data corresponding to the texture ID can be indexed from the GPU based on the texture ID.

[0021] Optionally, the step of displaying the texture data corresponding to the view control by the first component according to the third-party plugin corresponding to the view control can include:

[0022] The first component obtains a canvas texture object. The first component creates a virtual display controller in the third-party plug-in corresponding to the view control, and creates a corresponding virtual display control according to the canvas in the canvas texture object and layout information corresponding to the view control. The first component synthesizes the third-party plug-in corresponding to the view control and the virtual display control through the virtual display controller, and creates a corresponding display control. The first component adds the third-party plug-in corresponding to the view control to a container corresponding to the view in the display control through the display control, and sends texture data corresponding to the view control to the canvas in the canvas texture object for synthesis on the canvas.

[0023] Optionally, the second engine obtains the texture data corresponding to the view control from the GPU, and renders the texture data corresponding to the view control according to the layout information corresponding to the view control, including:

[0024] The second engine obtains the canvas from the canvas texture object and obtains the texture data corresponding to the view control from the GPU. The second engine renders the texture data corresponding to the view control according to the layout information corresponding to the view control.

[0025] Optionally, the first engine determines the plug-in attribute of the view control corresponding to the first application and a front-end document object model node corresponding to the view control, including:

[0026] The first engine parses a first packaged file in the first application to obtain the plug-in attribute of the view control corresponding to the first application and a first object corresponding to the view control and a style and attribute of the first object. The first engine generates a front-end document object model node corresponding to the view control according to the first object corresponding to the view control and the style and attribute of the first object.

[0027] Illustratively, the first application can be a JS application, and the first packaged file can be a JS package. The first object can be a JS object.

[0028] Optionally, the first component determines the layout information corresponding to the view control according to the front-end document object model node corresponding to the view control, including:

[0029] The first component interfaces the front-end document object model node corresponding to the view control to a back-end component node. The first component creates a corresponding element node according to the back-end component node, and creates a corresponding drawing node in the element node. The first component calculates the layout information corresponding to the view control in the drawing node.

[0030] Illustratively, the layout information corresponding to the view control can include a position and a size corresponding to the view control.

[0031] In some embodiments, the first component can include a component module, an element module, a drawing module, a resource registration module, a texture plug-in module, and a texture module.

[0032] The component module creates component nodes corresponding to view controls and sets the styles and properties of the first object parsed by the first engine to the corresponding component nodes. The element module creates element nodes corresponding to component nodes and mounts them to the element tree for layout calculation and style property updates of the entire tree. The drawing module creates drawing nodes corresponding to the view controls and calculates the layout information of the view controls at these drawing nodes. After calculating the layout information of the view controls, the drawing module can create a resource management module.

[0033] The resource registration module can create texture data corresponding to view controls, generate texture identifiers for the texture data corresponding to the view, and register the texture data corresponding to the view controls into the GPU based on the resource registration module and the texture plugin module. The texture module can create corresponding canvas texture (surfaceTexture) objects.

[0034] While creating the resource management module, the drawing module can match the view control plugin module and view control module corresponding to the plugin attribute of the view control from the first component based on the plugin attribute of the view control obtained by the first engine. Based on asynchronous callback technology, it can create the third-party plugin corresponding to the view control through the view control plugin module and view control module corresponding to the plugin attribute of the view control.

[0035] Among them, the view control plugin module and the view control module can be preset in the first component by the operating system developers.

[0036] For example, the first component can have a pre-defined webview control plugin module and webview control module corresponding to the webview control. When the plugin property of the view control obtained by the first engine is a webview control, the drawing module can match the webview control plugin module and webview control module from the first component.

[0037] For example, the first component can also have a pre-defined mapview control plugin module and mapview control module corresponding to the mapview control. When the plugin property of the view control obtained by the first engine is a mapview control, the drawing module can match the mapview control plugin module and mapview control module from the first component.

[0038] After creating the third-party plugin corresponding to the view control, the rendering module can create a virtual display controller within the third-party plugin, based on the `surfaceTexture` object and the view control plugin module corresponding to the view control's plugin property. It can also create the corresponding virtual display based on the `surface` in the `surfaceTexture` object and the layout information of the view control. The virtual display controller can composite the third-party plugin with the virtual display, creating the corresponding presentation. The presentation can add the third-party plugin to the container corresponding to the view within the presentation and send the texture data of the view control to the `surfaceTexture` object to composite onto the surface.

[0039] For example, the operating system of the terminal device may include system, system, system, system, system, Any one of the systems, etc.

[0040] For example, in one possible implementation scenario, the operating system could be HarmonyOS, the operating system's development framework could be a JavaScript development framework, the first engine could be a JavaScript engine, and the second engine could be a rendering engine.

[0041] Specifically, the rendering engine can be the Skia engine.

[0042] Secondly, embodiments of this application provide a terminal device that may include a first engine, a first component, and a second engine.

[0043] The system comprises the following components: a first engine determines the plugin attributes of the view control corresponding to the first application, and the front-end document object model node corresponding to the view control. A first component determines the layout information of the view control based on the front-end document object model node; creates the third-party plugin and texture data corresponding to the view control based on the plugin attributes, and registers the texture data of the view control to the GPU. The first component also sends the layout information of the view control to the second engine; and displays the texture data of the view control based on the third-party plugin. The second engine retrieves the texture data of the view control from the GPU and renders the texture data of the view control based on the layout information.

[0044] The first application can be an application developed by developers based on view controls.

[0045] Optionally, the view control may include any one of the following: webpage view control, map view control, camera view control, video view control, text view control, live streaming view control, advertising view control, screen mirroring view control, etc.

[0046] In some embodiments, the view control may be an Android view control, a HarmonyOS view control, or a view control developed in other Java languages, and there are no restrictions here.

[0047] The first package file of the first application includes the plugin properties of the view controls corresponding to the first application. The plugin properties of the view controls are pre-configured. For example, the developer of the first application can set the plugin properties of the view controls to web page view controls or map view controls in the first package file of the first application, thus pre-configuring the plugin properties of the view controls.

[0048] Optionally, the first component can be used to generate a texture identifier for the texture data corresponding to the view control, and register the texture data corresponding to the view control into the GPU based on the texture identifier. The first component is also used to send the texture identifier of the texture data corresponding to the view control to the second engine. The second engine can be used to obtain the texture data corresponding to the view control from the GPU based on the texture identifier.

[0049] Optionally, the first component can be used to obtain a canvas texture object; create a virtual controller in the third-party plugin corresponding to the view control, and create a corresponding virtual control based on the canvas in the canvas texture object and the layout information corresponding to the view control; composite the third-party plugin corresponding to the view control with the virtual control through the virtual controller, and create a corresponding display control; add the third-party plugin corresponding to the view control to the container corresponding to the view in the display control through the display control, and send the texture data corresponding to the view control to the canvas texture object to composite onto the canvas.

[0050] Optionally, the second engine can be used to obtain the canvas from the canvas texture object and obtain the texture data corresponding to the view control from the GPU; and render the texture data corresponding to the view control according to the layout information corresponding to the view control.

[0051] Optionally, the first engine can be used to parse the first packaged file in the first application to obtain the plugin properties of the view control corresponding to the first application, as well as the first object and its style and properties; and generate the front-end document object model node corresponding to the view control based on the first object and its style and properties.

[0052] For example, the first application can be a JS application, the first packaged file can be a JS package, and the first object can be a JS object.

[0053] Optionally, the first component can be used to connect the front-end document object model node corresponding to the view control to the back-end component node; create the corresponding element node according to the back-end component node, and create the corresponding drawing node in the element node; calculate the layout information corresponding to the view control in the drawing node.

[0054] For example, the layout information corresponding to a view control may include the position and size of the view control.

[0055] In some embodiments, the first component may include: a component module, an element module, a drawing module, a resource registration module, a texture plugin module, and a texture module.

[0056] The component module creates component nodes corresponding to view controls and sets the styles and properties of the first object parsed by the first engine to the corresponding component nodes. The element module creates element nodes corresponding to component nodes and mounts them to the element tree for layout calculation and style property updates of the entire tree. The drawing module creates drawing nodes corresponding to the view controls and calculates the layout information of the view controls at these drawing nodes. After calculating the layout information of the view controls, the drawing module can create a resource management module.

[0057] The resource registration module can create texture data corresponding to view controls, generate texture identifiers for the texture data corresponding to the view, and register the texture data corresponding to the view controls into the GPU based on the resource registration module and the texture plugin module. The texture module can create corresponding canvas texture (surfaceTexture) objects.

[0058] While creating the resource management module, the drawing module can match the view control plugin module and view control module corresponding to the plugin attribute of the view control from the first component based on the plugin attribute of the view control obtained by the first engine. Based on asynchronous callback technology, it can create the third-party plugin corresponding to the view control through the view control plugin module and view control module corresponding to the plugin attribute of the view control.

[0059] After creating the third-party plugin corresponding to the view control, the rendering module can create a virtual display controller within the third-party plugin, based on the `surfaceTexture` object and the view control plugin module corresponding to the view control's plugin property. It can also create the corresponding virtual display based on the `surface` in the `surfaceTexture` object and the layout information of the view control. The virtual display controller can composite the third-party plugin with the virtual display, creating the corresponding presentation. The presentation can add the third-party plugin to the container corresponding to the view within the presentation and send the texture data of the view control to the `surfaceTexture` object to composite onto the surface.

[0060] For example, the operating system of the terminal device may include system, system, system, system, system, Any one of the systems, etc.

[0061] For example, the first engine, the first component, and the second engine are all code within the development framework of the terminal device's operating system.

[0062] Thirdly, embodiments of this application provide an apparatus for displaying view controls, which can be used to implement the method for displaying view controls described in the first aspect. The functionality of this apparatus can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functions. For example, the apparatus may include modules such as a first engine, a first component, and a second engine. The apparatus can implement the methods described in the first aspect and any possible implementation thereof through the first engine, the first component, and the second engine. Further details will not be elaborated here.

[0063] Fourthly, embodiments of this application provide an electronic device, including: a processor; a memory; and a computer program; wherein the computer program is stored in the memory, and when the computer program is executed by the processor, it causes the electronic device to implement the method as described in the first aspect and any possible implementation thereof.

[0064] Fifthly, embodiments of this application provide a computer-readable storage medium comprising a computer program that, when executed on an electronic device, causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0065] In a sixth aspect, embodiments of this application also provide a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0066] The beneficial effects of the second to sixth aspects mentioned above can be referred to in the first aspect, and will not be repeated here.

[0067] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0068] Figure 1A This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0069] Figure 1A A schematic diagram illustrating the composition of the JS development framework provided in the embodiments of this application;

[0070] Figure 1A A schematic diagram of the operating system architecture of the terminal device provided in the embodiments of this application;

[0071] Figure 2 A flowchart illustrating the method for displaying a view control provided in an embodiment of this application;

[0072] Figure 3 This is a schematic diagram of the structure of the first component 200 provided in an embodiment of this application;

[0073] Figure 4 A schematic diagram illustrating the rendering effect of an Android view control provided in an embodiment of this application;

[0074] Figure 5 This is a schematic diagram illustrating another rendering effect of an Android view control provided in an embodiment of this application;

[0075] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0077] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0078] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0079] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0080] Operating systems (OS) support the display of various types of view controls, which is beneficial for building the OS ecosystem. A view control is a control that is responsible for drawing and handling events for a specific display area on an application page.

[0081] For example, for operating system A, when operating system A supports displaying Android view controls (hereinafter referred to as Android view controls), terminal devices with operating system A installed can support displaying application pages developed based on Android view controls, which is beneficial for operating system A to build its OS ecosystem.

[0082] For example, Android view controls may include webview controls, mapview controls, cameraview controls, textview controls, etc.

[0083] Currently, the main solutions for displaying Android view controls in operating systems are as follows: For each type of Android view control, the operating system's development framework can provide a customized control corresponding to that Android view control. The operating system can then display the Android view control through this customized control. Different Android view controls correspond to different customized controls. For example, for the WebView control, the operating system's development framework can create a customized control corresponding to the WebView control, and the operating system can then display the WebView control through this customized control.

[0084] However, the numerous interfaces of Android view controls in the above solutions make implementing customized controls corresponding to Android view controls within the operating system's development framework extremely complex and labor-intensive. Furthermore, if the operating system requires support for new Android view controls, the operating system's development framework needs to be completely redesigned, which will severely impact development efficiency and hinder the development of the OS ecosystem.

[0085] Therefore, this application provides a method for displaying view controls, which can be applied to terminal devices equipped with an operating system. When a first application developed by a developer based on view controls is installed on the terminal device, the terminal device can display the view controls corresponding to the first application based on the operating system's development framework, according to the method provided in this application. This method enables the terminal device's operating system to support the display of various view controls, providing versatility for different view controls. In this application, development efficiency is higher when developing within the operating system's development framework, which is beneficial for building the OS ecosystem.

[0086] Optionally, in this embodiment, the terminal device may include a mobile phone, a large screen (such as a smart screen), a tablet computer, a wearable device (such as a smartwatch, a smart bracelet, etc.), a television, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific form of the terminal device is not limited herein.

[0087] For example, taking a mobile phone as the terminal device, Figure 1A This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 1A As shown, a mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0088] The processor 110 may include multiple processing units, such as: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0089] The controller can serve as the nerve center and command center of the mobile phone. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0090] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0091] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0092] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the phone's storage capacity. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0093] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the mobile phone by running the instructions stored in internal memory 121. For example, by running the instructions stored in internal memory 121, processor 110 can enable the mobile phone to implement the method of displaying view controls described in the embodiments of this application.

[0094] The internal memory 121 may further include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as the first application described in this embodiment), etc. The data storage area may store data created during the use of the mobile phone (such as image data, phonebook data, etc.). Furthermore, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0095] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also power the mobile phone via the power management module 141. The power management module 141 connects to the battery 142, the charging management module 140, and the processor 110. The power management module 141 can also receive input from the battery 142 to power the mobile phone.

[0096] Mobile phones can perform audio functions, such as music playback and recording, through components like the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0097] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0098] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. For example, in this embodiment, display screen 194 can be used to display a JavaScript page of a JavaScript application.

[0099] The mobile phone implements its display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0100] Understandable Figure 1A The structure shown does not constitute a specific limitation on the mobile phone. In some embodiments, the mobile phone may also include... Figure 1A This could mean having more or fewer components, combining some components, separating some components, or having different component arrangements. Or, Figure 1A Some of the components shown can be implemented in hardware, software, or a combination of both.

[0101] Additionally, when the terminal device is another large-screen device (such as a smart screen), tablet computer, wearable device (such as a smartwatch, smart bracelet, etc.), television, in-vehicle equipment, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., the specific structure of these other terminal devices can also be referenced. Figure 1A As shown. For example, other terminal devices may be... Figure 1A The components were added or removed based on the given structure, which will not be elaborated here.

[0102] Optionally, in this embodiment, the view control can be an Android view control, a HarmonyOS view control, or other view controls (such as other view controls developed using Java). This embodiment does not limit the specific types of view controls that the operating system can support for display in this method. For example, the operating system can display different Android view controls using this method.

[0103] Optionally, in this embodiment of the application, the operating system may be system, system, or system, or system, or System, or Systems, etc., and there are no restrictions on the specific type of operating system.

[0104] The following uses HarmonyOS as an example to illustrate the method for displaying view controls provided in this application embodiment, combined with the scenario of displaying Android view controls in HarmonyOS.

[0105] HarmonyOS uses JavaScript as its development language, and its core development framework is the JS development framework. The JS development framework can include a JS engine and a rendering engine.

[0106] In this embodiment, a first component can be developed within a JavaScript development framework. This first component can work with a JavaScript engine and a rendering engine to implement the method for displaying the view control. The first component can be code capable of implementing the corresponding functions described in this embodiment. That is, in this embodiment, the terminal device can include a JavaScript engine, a first component, and a rendering engine. The terminal device can implement the method for implementing the display view control described in this embodiment using the JavaScript engine, the first component, and the rendering engine.

[0107] The following exemplary descriptions will involve the following concepts:

[0108] 1. JS Application

[0109] A JS application is an application developed using the JavaScript language. A JS application can provide JS pages, which are displayed by calling Android view controls; that is, the JS page is the display page for the Android view controls.

[0110] 2. JS bundle

[0111] A JS bundle, also known as a JS package, refers to a package containing Hypertext Markup Language (HTML) files, Cascading Style Sheets (CSS) files, and JS files within a JS application. A JS bundle can be created by bundling HTML, CSS, and JS files using a bundling tool.

[0112] For example, suppose the HTML file, CSS file, and JS file in a JS application are as follows:

[0113]

[0114] Therefore, the JS bundle formed by bundling the above HTML, CSS, and JS files using a bundling tool can be as follows:

[0115]

[0116]

[0117] It should be understood that the HTML, CSS, JS, and JS bundles mentioned above only provide partial code examples.

[0118] 3. JS objects

[0119] In JavaScript, apart from strings, numbers, booleans, null, and undefined values, all other data is a JS object, which can be an array, a date, or even a function.

[0120] 4. Virtual Document Object Model (VDOM) nodes and Document Object Model (DOM) nodes

[0121] VODM nodes are virtual / simulated DOM nodes.

[0122] DOM nodes can be used to connect to backend component nodes to enable the capabilities of frontend components.

[0123] 5. Component Node

[0124] The component node is a declarative description of the underlying UI component, describing its properties and styles. For example, the component node describes each front-end DOM node in each JS page, including the styles and properties of the corresponding JS object. The component node can be used to generate the component's entity elements.

[0125] 6. Element node

[0126] An element node is an instance of a component node, representing a specific component node.

[0127] 7. Render the node

[0128] The render node, also known as the rendering node, is used to calculate rendering data for each element node. This data can include the element node's position, size / dimension, and drawing commands within the JavaScript page.

[0129] 8. Virtual display: A control that captures content displayed on the screen.

[0130] 9. Presentation control: A special dialog box displayed on a specified display screen, used to display content on a secondary screen.

[0131] 10. SurfaceTexture: A type of canvas (surface) that can convert texture data into an external texture, which is not directly displayed.

[0132] 11. Surface: Similar to a canvas, all nodes on the page are drawn on this canvas.

[0133] 12. Context: refers to information about the environment in which an application is running.

[0134] For example, Figure 1B This is a schematic diagram illustrating the composition of the JS development framework provided in an embodiment of this application. For example... Figure 1B As shown in the embodiments of this application, the JS development framework may include: JS engine 100, first component 200, and rendering engine 300.

[0135] In this embodiment, when a terminal device installs a JS application developed by a developer based on an Android view control, the terminal device can load the JS application through a JS development framework. For example, the terminal device can provide an icon for the JS application, which the user can click. In response to the user clicking the icon, the terminal device can load the JS application through the JS development framework. After loading the JS application through the JS development framework, the JS engine 100 can parse the JS bundle in the JS application to determine the plugin attribute of the Android view control corresponding to the JS application, as well as the front-end DOM node corresponding to the Android view control. The first component 200 can determine the layout information corresponding to the Android view control based on the front-end DOM node, and create a third-party plugin and texture data corresponding to the Android view control based on the plugin attribute, and register the texture data corresponding to the Android view control in the GPU. The first component 200 can display the texture data corresponding to the Android view control based on the third-party plugin. The rendering engine 300 can obtain the texture data corresponding to the Android view control from the GPU and render the texture data corresponding to the Android view control based on the layout information determined by the first component 200.

[0136] For example, Figure 1CThis is a schematic diagram of the operating system architecture of the terminal device provided in an embodiment of this application. For example... Figure 1C As shown, the operating system of a terminal device may include an application layer, an application framework layer, system runtime libraries, and a kernel layer.

[0137] The application layer contains multiple applications, such as system applications like email, SMS, calendar, maps, browsers, and contact management, as well as third-party applications (like JS applications) designed and written by developers using Java. For each JS application within the application layer, developers can pre-configure the plugin properties of Android view controls in the JS bundle of that application during development.

[0138] The above Figure 1B The JS engine 100, first component 200, and rendering engine 300 shown can be deployed in Figure 1C The application framework layer or system runtime library layer shown here is not limited to any particular layer.

[0139] The following is about the above. Figure 1B The specific process of how the JS engine 100, the first component 200, and the rendering engine 300 cooperate to implement the method for displaying view controls provided in the embodiments of this application will be described.

[0140] Taking the Android view control as an example, Figure 2 This is a flowchart illustrating a method for displaying a view control provided in an embodiment of this application. Figure 2 As shown, the method for displaying the view control may include S201-S210.

[0141] S201 and JS Engine 100 parse the JS bundle in the JS application developed by the developer to obtain the plugin properties of the Android view control, as well as the JS object corresponding to the Android view control and the style and properties of the JS object.

[0142] In this context, the JS application developed by the developer is an application built using Android view controls. When developing a JS application, developers can pre-configure (pre-set) the plugin attributes of Android view controls in the HTML code of the JS application. For example, they can pre-configure the plugin attributes of Android view controls as WebView, MapView, CameraView, TextView, etc.

[0143] JS Engine 100 can load the JS bundle in the JS application developed by the developer into the JS development framework, parse the JS bundle, and obtain the JS object corresponding to the Android view control, as well as the style and properties of the JS object and the plugin properties of the Android view.

[0144] For example, a developer's JS application might be based on a WebView control. JS Engine 100 parses the JS bundle in the JS application, revealing that the Android view control's plugin property is a WebView control. Simultaneously, it obtains the corresponding JS object for the WebView control, along with its styles and properties.

[0145] For example, suppose a portion of the HTML code in a JS application designed (or developed) by a developer looks like this:

[0146]

[0147] Then, JS Engine 100 can parse the plugin property of the Android view control in the JS application to be: webview control.

[0148] Suppose that a developer has designed (or developed) a JavaScript application where part of the HTML file contains the following code:

[0149]

[0150] Then, JS Engine 100 can parse the plugin property of the Android view control in the JS application to be: mapview control.

[0151] S202 and JS Engine 100 generate the front-end document object model (DOM) node corresponding to the Android view control based on the JS object corresponding to the Android view control and the style and attributes of the JS object.

[0152] Optionally, the step of the JS engine 100 generating the front-end DOM node corresponding to the Android view control based on the JS object and its styles and attributes may include: the JS engine 100 generating a tree structure of VDOM nodes (referred to as the VDOM tree) based on the JS object corresponding to the Android view control, wherein the VDOM tree includes multiple VDOM nodes. The JS engine 100 sets the styles and attributes of the JS object to the corresponding VDOM nodes and performs data binding on each VDOM node in the VDOM tree. Simultaneously, the JS engine 100 creates the corresponding front-end DOM node while generating the VDOM node.

[0153] The VODM tree is a JavaScript object tree that simulates the DOM tree. Data binding refers to the process of establishing a connection between the application's UI and the data source, so that when the data of each VDOM node changes, the corresponding UI node will be automatically drawn and updated.

[0154] After the JS engine 100 generates the front-end DOM nodes corresponding to the Android view control based on the JS object and its styles and properties, the first component 200 can sequentially create corresponding back-end component nodes, element nodes, and render nodes based on the front-end DOM nodes to complete layout calculations, component style and property updates, etc. For example, the first component 200 can execute S203-S205.

[0155] S203, The first component 200 connects the front-end DOM node corresponding to the Android view control to the back-end component node.

[0156] S204. The first component 200 creates a corresponding element node based on the backend component node, and creates a corresponding render node in the element node.

[0157] S205, The first component 200 calculates the layout information corresponding to the Android view control at the drawing node.

[0158] The layout information can include position and size, with size referring to the layout dimensions. For example, the first component 200 can calculate the position and size of each DOM node corresponding to the Android view control at the drawing node.

[0159] S206. The first component 200 creates a third-party plugin corresponding to the Android view control based on the plugin attribute of the Android view control, and creates texture data corresponding to the Android view control.

[0160] For example, as described in the example in S201, suppose a portion of the HTML file in a JS application designed (or developed) by a developer is as follows:

[0161]

[0162] Therefore, the JS engine 100 can parse the plugin property of the Android view control in the JS application to be: webview control. Accordingly, in S206, the first component 200, based on the Android view control's plugin property being a webview control, creates an Android view control with the corresponding third-party plugin: a webview plugin.

[0163] For example, as illustrated in S201, suppose a portion of the HTML code in a JS application designed (or developed) by a developer is as follows:

[0164]

[0165] Therefore, the JS engine 100 can parse the plugin attribute of the Android view control in the JS application and find it to be a mapview control. Accordingly, the first component 200 in S206, based on the mapview control's plugin attribute, creates a third-party plugin for the corresponding Android view control: a mapview plugin.

[0166] S207. The first component 200 generates a texture identifier for the texture data corresponding to the Android view control, and registers the texture data corresponding to the Android view control to the graphics processing unit (GPU) based on the texture identifier.

[0167] For example, the texture identifier of the texture data corresponding to the Android view control can be the texture ID of the texture data, and the texture ID can represent the unique identity information of the texture data. After registering the texture data corresponding to the Android view control in the GPU according to the texture identifier of the texture data corresponding to the Android view control, the rendering engine 300 can subsequently index / retrieve the texture data corresponding to the Android view control from the GPU according to the texture identifier of the texture data corresponding to the Android view control.

[0168] It should be noted that this application does not restrict the execution order of S205 and S206-S207. For example, in some embodiments, S205 and S206-S207 can be executed simultaneously. In other embodiments, S205 can be executed before or after S206-S207.

[0169] S208, the first component 200 sends the layout information corresponding to the Android view control and the texture identifier of the texture data corresponding to the Android view control to the rendering engine 300.

[0170] S209, The first component 200 displays the texture data corresponding to the Android view control according to the third-party plugin corresponding to the Android view control.

[0171] Optionally, the step of the first component 200 displaying the texture data corresponding to the Android view control based on the third-party plugin may include: the first component 200 obtaining a surfaceTexture object; the first component 200 creating a virtual display controller in the third-party plugin, and creating a corresponding virtual display based on the surface in the surfaceTexture object and the layout information corresponding to the Android view control; the first component 200 compositing the third-party plugin and the virtual display control through the virtual display controller, and creating a corresponding presentation control; and the first component 200 adding the third-party plugin to the container corresponding to the view in the presentation control through the presentation control, and sending the texture data corresponding to the Android view control to the surfaceTexture object to be composited onto the surface.

[0172] For example, Figure 3 This is a schematic diagram of the structure of the first component 200 provided in an embodiment of this application. The following is in conjunction with... Figure 3 The structure shown provides a detailed explanation of the processes described in S203-S207 and S209 above. For example... Figure 3 As shown, the first component 200 may include:

[0173] Component module 210, also known as NativeTextureComponent module, can be used to create component nodes corresponding to Android view controls and set the styles and properties of JS objects parsed from the JS bundle by JS engine 100 to the corresponding component nodes.

[0174] Element module 220, also known as NativeTextureElement module, can be used to create element nodes corresponding to component nodes and mount element nodes on the element tree to facilitate layout calculations and updates of related style properties for the entire tree.

[0175] The rendering module 230, also known as the NativeTextureRender module, is used to create the render nodes corresponding to the Android view controls and calculate the layout information of the Android view controls at the render nodes. After calculating the layout information of the Android view controls, the rendering module 230 can create a resource management module 231. The resource management module 231 can also be called the NativeDelegate module or a resource manager.

[0176] The first component 200 also includes: a resource registration module 240, a texture plugin module 250, and a texture module 260. The resource registration module can also be called the ResourceRegister module, the texture plugin module can also be called the TexturePlugin module, and the texture module can also be called the Texture module.

[0177] The resource management module 231 can create texture data corresponding to Android view controls, generate texture identifiers for the texture data corresponding to Android view controls, and register the texture data corresponding to Android view controls in the GPU based on the resource registration module 240 and the texture plugin module 250. The texture module 260 can create corresponding canvas texture (surfaceTexture) objects.

[0178] While creating the resource management module 231, the drawing module 230 can match the Android view control plugin module and Android view control module corresponding to the plugin attribute of the Android view control from the first component 200 according to the plugin attribute of the Android view control parsed by the JS engine 100 in S201. Based on asynchronous callback technology, it can create the third-party plugin corresponding to the Android view control through the Android view control plugin module and Android view control module corresponding to the plugin attribute of the Android view control.

[0179] The Android view control plugin module and the Android view control module can be preset in the first component 200 by the operating system developers. For example, the first component 200 can preset a webview control plugin module and a webview control module corresponding to the webview control. When the plugin attribute of the Android view control parsed by the JS engine 100 in S201 is a webview control, the drawing module 230 can match the webview control plugin module (WebviewPlugin module) and the webview control module (Webview module) corresponding to the plugin attribute of the Android view control from the first component 200. Similarly, the first component 200 can also preset a mapview control plugin module (mapviewPlugin module) and a mapview control module (mapview module) corresponding to the mapview control. When the plugin attribute of the Android view control parsed by the JS engine 100 in S201 is a mapview control, the drawing module 230 can match the mapview control plugin module and the mapview control module corresponding to the plugin attribute of the Android view control from the first component 200, and so on.

[0180] After creating the third-party plugin corresponding to the Android view control, the drawing module 230 can create a virtual display controller within the third-party plugin based on the surfaceTexture object and the Android view control plugin module corresponding to the plugin property of the Android view control; and create a corresponding virtual display based on the surface in the surfaceTexture object and the layout information corresponding to the Android view control. The virtual display controller combines the third-party plugin and the virtual display control, and creates a corresponding presentation control. The presentation control can add the third-party plugin to the container corresponding to the view in the presentation control, and send the texture data corresponding to the Android view control to the surfaceTexture object to be composited onto the surface.

[0181] S210 and rendering engine 300 obtain the texture data corresponding to the Android view control from the GPU based on the texture identifier of the texture data corresponding to the Android view control, and render the texture data corresponding to the Android view control based on the layout information of the Android view control.

[0182] Optionally, the step of rendering engine 300 obtaining the texture data corresponding to the Android view control from the GPU based on the texture identifier of the texture data corresponding to the Android view control, and rendering the texture data corresponding to the Android view control based on the layout information corresponding to the Android view control, may include: rendering engine 300 obtaining the surface from the surfaceTexture object, and obtaining the texture data corresponding to the Android view control from the GPU based on the texture identifier of the texture data corresponding to the Android view control; rendering engine 300 rendering the texture data corresponding to the Android view control based on the layout information corresponding to the Android view control.

[0183] The following example illustrates how a developer sets the plugin property of a webview control in a JavaScript application. Figure 2 The specific application process of the method for displaying Android view controls provided in the embodiments of this application is illustrated by way of example.

[0184] In this embodiment, when a developer sets the plugin attribute to a webview control in a JS application, the JS engine 100 parses the JS bundle in the developer's JS application to obtain the Android view control's plugin attribute as a webview control, as well as the corresponding JS object and its style and attributes. Then, the JS engine 100 generates the front-end DOM node corresponding to the webview control based on the JS object and its style and attributes. The first component 200 connects the front-end DOM node corresponding to the webview control to the back-end component node, creates a corresponding element node based on the back-end component node, and creates a corresponding render node within the element node. Then, the first component 200 calculates the layout information corresponding to the webview control in the render node. Additionally, the first component 200 creates the webview plugin and texture data corresponding to the webview control. After creating the texture data corresponding to the webview control, the first component 200 generates a texture identifier for the texture data corresponding to the webview control and registers the texture data corresponding to the webview control in the GPU based on the texture identifier.

[0185] Subsequently, the first component 200 can send the layout information corresponding to the webview control and the texture identifier of the texture data corresponding to the webview control to the rendering engine 300. The first component 200 can also obtain a canvas texture (surfaceTexture) object, create a virtual display controller in the webview plugin, and create a corresponding virtual display control based on the canvas (surface) in the surfaceTexture object and the layout information corresponding to the webview control. After creating the virtual display control, the first component 200 can composite the webview plugin and the virtual display control through the virtual display controller and create a corresponding presentation control. After creating the corresponding presentation control, the first component 200 can add the webview plugin to the container corresponding to the view in the presentation control through the presentation control, and send the texture data corresponding to the webview control to the surfaceTexture object for compositing onto the surface. The rendering engine 300 can obtain the texture data corresponding to the webview control from the GPU based on the texture identifier of the texture data corresponding to the webview control, and render the texture data corresponding to the webview control based on the layout information corresponding to the webview control. Thus, the JS framework can support the display of webview controls.

[0186] For example, Figure 4 This is a schematic diagram illustrating the rendering effect of an Android view control provided in an embodiment of this application. For example... Figure 4 As shown, when the plugin attribute of the Android view control is preset to a webview control in the HTML file code of the JS application designed (or developed) by the developer (see the foregoing embodiments for the specific code), the method described in this application embodiment can render an image similar to... Figure 4 The webpage shown. It should be understood that different JavaScript applications, or JavaScript applications requesting different webpage addresses, Figure 4 The content of the pages shown is different from that of the web pages. Figure 4 This is just an example.

[0187] Similarly, when developers set the plugin property to other Android view controls (such as mapview controls) in a JS application, the operating system's development framework can support the display of other Android view controls in the manner described in the aforementioned embodiments.

[0188] For example, Figure 5This is a schematic diagram illustrating another rendering effect of an Android view control provided in an embodiment of this application. For example... Figure 5 As shown, when the pre-defined Android view control's plugin attribute in the HTML file of a JS application designed (or developed) by the developer is a mapview control (the specific code can be found in the aforementioned embodiments), the method described in this application embodiment can render an image similar to... Figure 5 The map page shown. Figure 5 This is also just an example.

[0189] In the method for displaying Android view controls provided in this application embodiment, JS application developers only need to set the plugin attribute of the Android view control in the code of the HTML file of the JS application to enable the operating system to support the display of different Android view controls. When the operating system displays different Android view controls according to the method provided in this application embodiment, it does not need to consider the adaptation and connection issues of different Android view controls. The entire implementation process can be automatically completed by the operating system's development framework, and JS application developers are unaware of it, which greatly reduces the development difficulty of JS applications adapted to the operating system and is conducive to attracting more developers to participate in the construction of the OS ecosystem.

[0190] Furthermore, the operating system-side development framework enables a single development effort to deploy various complex Android views, significantly reducing development costs. Developers of JS applications can set the plugin attributes of Android view controls within the HTML code of their JS applications. The operating system's development framework automatically matches the corresponding third-party plugins based on these plugin attributes, avoiding the need for developers to perform secondary development for different Android view controls and effectively improving development efficiency.

[0191] Furthermore, in the method for displaying an Android view control provided in this application embodiment, by directly registering the texture data corresponding to the Android view control (which can also be simply referred to as the Android-side texture data) to the GPU, the creation of a texture cache in the CPU can be avoided. The rendering engine can directly obtain the texture data corresponding to the Android view control from the GPU for rendering, without the need for texture data transfer between the CPU and GPU. This saves the copying time between different texture caches in the CPU and the time for uploading textures to the GPU, greatly improving rendering performance. At the same time, it also saves the memory occupied by texture data and GPU video memory.

[0192] It should be noted that although the foregoing embodiments exemplify Android view controls as WebView or MapView, it should be understood that in other scenarios, the view controls displayed by the method provided in this application embodiment can also be other Android view controls such as CameraView, TextView, live streaming view, advertising view, and screen casting view, or HarmonyOS view controls, other view controls developed in Java, etc. When the view control is one of the aforementioned other types, its implementation principle is the same as the principle of displaying WebView or MapView controls described in the foregoing embodiments, and will not be repeated here.

[0193] Additionally, it should be understood that the above explanation refers to operating systems other than HarmonyOS. TM When working with systems other than iOS, the JS development framework can be replaced with a framework for another language used by that system, and the JS engine can be an engine that parses code in that other language. For example, iOS... TM The system's corresponding development framework can be the Swift development framework, which can include the Swift engine, first components, rendering engine, etc.

[0194] In this embodiment, the application developed by the developer based on view controls can be referred to as the first application, and the corresponding packaged file of the first application (such as the JS bundle of a JS application) can be referred to as the first packaged file. The engine in the operating system's development framework that parses the first application (such as HarmonyOS) TM The JS engine in the system can be called the first engine, and the rendering engine can be called the second engine. The first engine parses the first packaged file in the first application, and the resulting object (such as a JS object) can be called the first object.

[0195] Corresponding to the method for displaying view controls described in the foregoing embodiments, this application also provides a terminal device, which may include a first engine, a first component, and a second engine.

[0196] The system comprises the following components: a first engine determines the plugin attributes of the view control corresponding to the first application, and the front-end document object model node corresponding to the view control. A first component determines the layout information of the view control based on the front-end document object model node; creates the third-party plugin and texture data corresponding to the view control based on the plugin attributes, and registers the texture data of the view control to the GPU. The first component also sends the layout information of the view control to the second engine; and displays the texture data of the view control based on the third-party plugin. The second engine retrieves the texture data of the view control from the GPU and renders the texture data of the view control based on the layout information.

[0197] The first application can be an application developed by developers based on view controls.

[0198] Optionally, the view control may include any one of the following: webpage view control, map view control, camera view control, video view control, text view control, live streaming view control, advertising view control, screen mirroring view control, etc.

[0199] In some embodiments, the view control may be an Android view control, a HarmonyOS view control, or a view control developed in other Java languages, and there are no restrictions here.

[0200] The first package file of the first application includes the plugin properties of the view controls corresponding to the first application. The plugin properties of the view controls are pre-configured. For example, the developer of the first application can set the plugin properties of the view controls to web page view controls or map view controls in the first package file of the first application, thus pre-configuring the plugin properties of the view controls.

[0201] Optionally, the first component can be used to generate a texture identifier for the texture data corresponding to the view control, and register the texture data corresponding to the view control into the GPU based on the texture identifier. The first component is also used to send the texture identifier of the texture data corresponding to the view control to the second engine. The second engine can be used to obtain the texture data corresponding to the view control from the GPU based on the texture identifier.

[0202] Optionally, the first component can be used to obtain a canvas texture object; create a virtual controller in the third-party plugin corresponding to the view control, and create a corresponding virtual control based on the canvas in the canvas texture object and the layout information corresponding to the view control; composite the third-party plugin corresponding to the view control with the virtual control through the virtual controller, and create a corresponding display control; add the third-party plugin corresponding to the view control to the container corresponding to the view in the display control through the display control, and send the texture data corresponding to the view control to the canvas texture object to composite onto the canvas.

[0203] Optionally, the second engine can be used to obtain the canvas from the canvas texture object and obtain the texture data corresponding to the view control from the GPU; and render the texture data corresponding to the view control according to the layout information corresponding to the view control.

[0204] Optionally, the first engine can be used to parse the first packaged file in the first application to obtain the plugin properties of the view control corresponding to the first application, as well as the first object and its style and properties; and generate the front-end document object model node corresponding to the view control based on the first object and its style and properties.

[0205] For example, the first application can be a JS application, the first packaged file can be a JS package, and the first object can be a JS object.

[0206] Optionally, the first component can be used to connect the front-end document object model node corresponding to the view control to the back-end component node; create the corresponding element node according to the back-end component node, and create the corresponding drawing node in the element node; calculate the layout information corresponding to the view control in the drawing node.

[0207] For example, the layout information corresponding to a view control may include the position and size of the view control.

[0208] In some embodiments, the first component may include: a component module, an element module, a drawing module, a resource registration module, a texture plugin module, and a texture module.

[0209] The component module creates component nodes corresponding to view controls and sets the styles and properties of the first object parsed by the first engine to the corresponding component nodes. The element module creates element nodes corresponding to component nodes and mounts them to the element tree for layout calculation and style property updates of the entire tree. The drawing module creates drawing nodes corresponding to the view controls and calculates the layout information of the view controls at these drawing nodes. After calculating the layout information of the view controls, the drawing module can create a resource management module.

[0210] The resource registration module can create texture data corresponding to view controls, generate texture identifiers for the texture data corresponding to the view, and register the texture data corresponding to the view controls into the GPU based on the resource registration module and the texture plugin module. The texture module can create corresponding canvas texture (surfaceTexture) objects.

[0211] While creating the resource management module, the drawing module can match the view control plugin module and view control module corresponding to the plugin attribute of the view control from the first component based on the plugin attribute of the view control obtained by the first engine. Based on asynchronous callback technology, it can create the third-party plugin corresponding to the view control through the view control plugin module and view control module corresponding to the plugin attribute of the view control.

[0212] After creating the third-party plugin corresponding to the view control, the rendering module can create a virtual display controller within the third-party plugin, based on the `surfaceTexture` object and the view control plugin module corresponding to the view control's plugin property. It can also create the corresponding virtual display based on the `surface` in the `surfaceTexture` object and the layout information of the view control. The virtual display controller can composite the third-party plugin with the virtual display, creating the corresponding presentation. The presentation can add the third-party plugin to the container corresponding to the view within the presentation and send the texture data of the view control to the `surfaceTexture` object to composite onto the surface.

[0213] For example, the operating system of the terminal device may include system, system, system, system, system, Any one of the systems, etc.

[0214] For example, the first engine, the first component, and the second engine are all code within the development framework of the terminal device's operating system.

[0215] For example, when the operating system's development framework is a JavaScript development framework, the first engine can be a JavaScript engine, and the second engine can be a rendering engine. See the foregoing embodiments for details. Figure 1B As shown in the image.

[0216] It should be understood that the terminal device can implement all the functions of the method for displaying view controls described in the foregoing method embodiments of this application through the first engine, the first component, and the second engine.

[0217] Optionally, embodiments of this application also provide an apparatus for displaying view controls, which can be used to implement the method for displaying view controls described in the foregoing method embodiments. The functionality of this apparatus can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functions.

[0218] For example, the device for displaying the view control may include modules such as a first engine, a first component, and a second engine. The device for displaying the view control can implement all the functions of the method for displaying the view control described in the foregoing method embodiments of this application through the first engine, the first component, and the second engine modules.

[0219] For example, a first engine can be used to determine the plugin properties of the view control corresponding to the first application, as well as the front-end document object model node corresponding to the view control. A first component can be used to determine the layout information corresponding to the view control based on the front-end document object model node corresponding to the view control; create the third-party plugin and texture data corresponding to the view control based on the plugin properties of the view control, and register the texture data corresponding to the view control to the GPU. The first component can also be used to send the layout information corresponding to the view control to the second engine; and display the texture data corresponding to the view control based on the third-party plugin corresponding to the view control. The second engine can be used to obtain the texture data corresponding to the view control from the GPU and render the texture data corresponding to the view control based on the layout information of the view control.

[0220] The first application can be an application developed by developers based on view controls.

[0221] Optionally, the view control may include any one of the following: webpage view control, map view control, camera view control, video view control, text view control, live streaming view control, advertising view control, screen mirroring view control, etc.

[0222] In some embodiments, the view control may be an Android view control, a HarmonyOS view control, or a view control developed in other Java languages, and there are no restrictions here.

[0223] The first package file of the first application includes the plugin properties of the view controls corresponding to the first application. The plugin properties of the view controls are pre-configured. For example, the developer of the first application can set the plugin properties of the view controls to web page view controls or map view controls in the first package file of the first application, thus pre-configuring the plugin properties of the view controls.

[0224] Optionally, the first component can be used to generate a texture identifier for the texture data corresponding to the view control, and register the texture data corresponding to the view control into the GPU based on the texture identifier. The first component is also used to send the texture identifier of the texture data corresponding to the view control to the second engine. The second engine can be used to obtain the texture data corresponding to the view control from the GPU based on the texture identifier.

[0225] Optionally, the first component can be used to obtain a canvas texture object; create a virtual controller in the third-party plugin corresponding to the view control, and create a corresponding virtual control based on the canvas in the canvas texture object and the layout information corresponding to the view control; composite the third-party plugin corresponding to the view control with the virtual control through the virtual controller, and create a corresponding display control; add the third-party plugin corresponding to the view control to the container corresponding to the view in the display control through the display control, and send the texture data corresponding to the view control to the canvas texture object to composite onto the canvas.

[0226] Optionally, the second engine can be used to obtain the canvas from the canvas texture object and obtain the texture data corresponding to the view control from the GPU; and render the texture data corresponding to the view control according to the layout information corresponding to the view control.

[0227] Optionally, the first engine can be used to parse the first packaged file in the first application to obtain the plugin properties of the view control corresponding to the first application, as well as the first object and its style and properties; and generate the front-end document object model node corresponding to the view control based on the first object and its style and properties.

[0228] Optionally, the first component can be used to connect the front-end document object model node corresponding to the view control to the back-end component node; create the corresponding element node according to the back-end component node, and create the corresponding drawing node in the element node; calculate the layout information corresponding to the view control in the drawing node.

[0229] For example, the layout information corresponding to a view control may include the position and size of the view control.

[0230] In some embodiments, the first component may include: a component module, an element module, a drawing module, a resource registration module, a texture plugin module, and a texture module. See the above for details. Figure 3 As shown in the image.

[0231] The component module creates component nodes corresponding to view controls and sets the styles and properties of the first object parsed by the first engine to the corresponding component nodes. The element module creates element nodes corresponding to component nodes and mounts them to the element tree for layout calculation and style property updates of the entire tree. The drawing module creates drawing nodes corresponding to the view controls and calculates the layout information of the view controls at these drawing nodes. After calculating the layout information of the view controls, the drawing module can create a resource management module.

[0232] The resource registration module can create texture data corresponding to view controls, generate texture identifiers for the texture data corresponding to the view, and register the texture data corresponding to the view controls into the GPU based on the resource registration module and the texture plugin module. The texture module can create corresponding canvas texture (surfaceTexture) objects.

[0233] While creating the resource management module, the drawing module can match the view control plugin module and view control module corresponding to the plugin attribute of the view control from the first component based on the plugin attribute of the view control obtained by the first engine. Based on asynchronous callback technology, it can create the third-party plugin corresponding to the view control through the view control plugin module and view control module corresponding to the plugin attribute of the view control.

[0234] After creating the third-party plugin corresponding to the view control, the rendering module can create a virtual display controller within the third-party plugin, based on the `surfaceTexture` object and the view control plugin module corresponding to the view control's plugin property. It can also create the corresponding virtual display based on the `surface` in the `surfaceTexture` object and the layout information of the view control. The virtual display controller can composite the third-party plugin with the virtual display, creating the corresponding presentation. The presentation can add the third-party plugin to the container corresponding to the view within the presentation and send the texture data of the view control to the `surfaceTexture` object to composite onto the surface.

[0235] For example, the operating system of the terminal device may include system, system, system, system, system, Any one of the systems, etc.

[0236] For example, the first engine, the first component, and the second engine are all code within the development framework of the terminal device's operating system.

[0237] It should be understood that the division of units (or modules) in the above device (e.g., dividing the device for displaying view controls into a first engine, a first component, and a second engine) is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, all units in the device can be implemented entirely in software through processing element calls; all units can be implemented entirely in hardware; or some units can be implemented in software through processing element calls, while others can be implemented in hardware.

[0238] For example, each unit can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, invoked and executed by a processing element within the device. Furthermore, these units can be integrated in whole or in part, or implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each of the above units can be implemented through integrated logic circuits in the processor element or through software invoked by the processing element.

[0239] In one example, the unit in the above device may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0240] For example, when the units in the device can be implemented through a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units can be integrated together to form a system-on-a-chip (SOC).

[0241] In one implementation, the units that implement the corresponding steps in the above methods can be implemented in the form of a processing element scheduler. For example, the device may include a processing element and a storage element, wherein the processing element calls a program stored in the storage element to execute the methods described in the above embodiments. The storage element may be a storage element located on the same chip as the processing element, i.e., an on-chip storage element.

[0242] In another implementation, the program used to perform the above methods can be located on a storage element on a different chip than the processing element, i.e., an off-chip storage element. In this case, the processing element calls or loads the program from the off-chip storage element onto the on-chip storage element to call and execute the methods described in the above method embodiments.

[0243] For example, embodiments of this application may also provide a device, such as an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device may include: a processor 601; a memory 602; and a computer program; wherein the computer program is stored in the memory 602, and when the computer program is executed by the processor 601, it causes the electronic device to implement the method of displaying view controls as described in the foregoing embodiments. The memory 602 may be located inside or outside the electronic device. The processor 601 may include one or more processors.

[0244] For example, the electronic device may be a mobile phone, tablet computer, wearable device (such as a smartwatch, smart bracelet, etc.), in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc.

[0245] In another implementation, the unit that implements the steps of the above method can be configured as one or more processing elements, which can be integrated circuits, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or combinations of these types of integrated circuits. These integrated circuits can be integrated together to form a chip.

[0246] For example, this application also provides a chip that can be applied to the aforementioned electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuits to implement the method for displaying view controls as described in the foregoing embodiments.

[0247] This application also provides a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to implement the method for displaying view controls as described in the foregoing embodiments.

[0248] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0249] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0250] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.

[0251] Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product, such as a program. This software product is stored in a program product, such as a computer-readable storage medium, and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0252] For example, embodiments of this application may also provide a computer-readable storage medium comprising a computer program that, when executed on an electronic device, causes the electronic device to implement the method for displaying view controls as described in the foregoing embodiments.

[0253] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of displaying a view control, the method comprising: The method is applied to a terminal device, and the terminal device comprises a first engine, a first component and a second engine. The method comprises: The first engine determines a plug-in attribute of a view control corresponding to a first application and a front-end document object model node corresponding to the view control; The first component determines layout information corresponding to the view control according to the front-end document object model node corresponding to the view control; The first component creates a third-party plug-in and texture data corresponding to the view control according to the plug-in attribute of the view control, and registers the texture data corresponding to the view control in a graphics processing unit (GPU); The first component sends the layout information corresponding to the view control to the second engine; The first component displays the texture data corresponding to the view control according to the third-party plug-in corresponding to the view control; The second engine acquires the texture data corresponding to the view control from the GPU, and renders the texture data corresponding to the view control according to the layout information corresponding to the view control.

2. The method of claim 1, wherein, The first component registers the texture data corresponding to the view control in the GPU, comprising: The first component generates a texture identifier of the texture data corresponding to the view control, and registers the texture data corresponding to the view control in the GPU according to the texture identifier of the texture data corresponding to the view control; The method further comprises: The first component sends the texture identifier of the texture data corresponding to the view control to the second engine; The second engine acquires the texture data corresponding to the view control from the GPU, comprising: The second engine acquires the texture data corresponding to the view control from the GPU according to the texture identifier of the texture data corresponding to the view control.

3. The method according to claim 1 or 2, characterized in that, The first component displays the texture data corresponding to the view control according to the third-party plug-in corresponding to the view control, comprising: The first component acquires a canvas texture object; The first component creates a virtual display controller in the third-party plug-in corresponding to the view control, and creates a corresponding virtual display control according to a canvas in the canvas texture object and the layout information corresponding to the view control; The first component synthesizes the third-party plug-in corresponding to the view control and the virtual display control through the virtual display controller, and creates a corresponding display control; The first component adds the third-party plug-in corresponding to the view control to a view corresponding container in the display control through the display control, and synthesizes the texture data corresponding to the view control to a canvas through the canvas texture object.

4. The method of claim 3, wherein, The second engine acquires the texture data corresponding to the view control from the GPU, and renders the texture data corresponding to the view control according to the layout information corresponding to the view control, comprising: The second engine acquires a canvas from the canvas texture object, and acquires the texture data corresponding to the view control from the GPU; The second engine renders the texture data corresponding to the view control according to the layout information corresponding to the view control.

5. The method according to claim 1 or 2, characterized in that, The first engine determines the plug-in attribute of the view control corresponding to the first application and a front-end document object model node corresponding to the view control, including: The first engine parses a first packaged file in the first application to obtain the plug-in attribute of the view control corresponding to the first application and a first object and a style and attribute of the first object corresponding to the view control; The first engine generates the front-end document object model node corresponding to the view control according to the first object and the style and attribute of the first object corresponding to the view control.

6. The method of claim 5, wherein, The first packaged file of the first application includes a preconfigured plug-in attribute of the view control corresponding to the first application.

7. The method according to claim 1 or 2, characterized in that, The first component determines the layout information corresponding to the view control according to the front-end document object model node corresponding to the view control, including: The first component interfaces the front-end document object model node corresponding to the view control to a back-end component node; The first component creates a corresponding element node according to the back-end component node and creates a corresponding drawing node in the element node; The first component calculates the layout information corresponding to the view control in the drawing node.

8. The method of claim 7, wherein, The layout information corresponding to the view control includes a position and a size corresponding to the view control.

9. The method of claim 1 or 2, wherein, The view control includes any one of the following: a webpage view control, a map view control, a camera view control, a video view control, a text view control, a live view control, an advertisement view control, and a screen projection view control.

10. The method of claim 1 or 2, wherein, The operating system of the terminal device includes any one of the following: an Android system, a Hongmeng system, an iOS system, a Windows system, a Mac system, and an EMUI system.

11. The method of claim 1 or 2, wherein, The first engine is a JS engine, and the second engine is a rendering engine.

12. A terminal device, comprising: including: a first engine, a first component, and a second engine; The first engine is configured to determine the plug-in attribute of the view control corresponding to the first application and a front-end document object model node corresponding to the view control; The first component is configured to determine the layout information corresponding to the view control according to the front-end document object model node corresponding to the view control; create a third-party plug-in and texture data corresponding to the view control according to the plug-in attribute of the view control, and register the texture data corresponding to the view control in a graphics processing unit (GPU); The first component is further configured to send the layout information corresponding to the view control to the second engine; and display the texture data corresponding to the view control according to the third-party plug-in corresponding to the view control; The second engine is configured to obtain the texture data corresponding to the view control from the GPU and render the texture data corresponding to the view control according to the layout information corresponding to the view control.

13. An electronic device, comprising: including: a processor; a memory; and a computer program; wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements the method according to any one of claims 1-11.

14. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program runs on the electronic device, the electronic device implements the method according to any one of claims 1-11.

Images