Interface drawing method, related device and communication system

Abstract

The embodiment of the present application provides an interface drawing method, related equipment and a communication system. Through the interface drawing method provided by the embodiment of the present application, an application process directly transmits lightweight drawing parameters of an interface element to a rendering service process, and then the rendering service process constructs a primitive object and a style object of a drawing engine based on the drawing parameters of the interface element. Compared with transmitting a large amount of drawing engine objects constructed based on the drawing parameters in the application process and the rendering service process, directly transmitting the lightweight data of the drawing parameters can significantly reduce the data transmission amount between the application process and the rendering service process, thereby reducing the load of the equipment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to interface drawing methods, related equipment and communication systems. Background Technology

[0002] As a core module of modern desktop operating systems, windows, with window management and display services at their core, provide the operating system with fundamental capabilities such as graphics rendering, interface interaction, and display management. A key function of display services is drawing the user interface and displaying the results on the screen. However, current interface drawing methods have significant limitations. Summary of the Invention

[0003] This application provides an interface drawing method, related devices, and a communication system. The application process directly passes lightweight drawing parameters of interface elements to the rendering service process, which then constructs primitive objects and style objects for the drawing engine based on these parameters. Compared to passing large amounts of drawing engine objects based on drawing parameters between the application process and the rendering service process, directly passing these lightweight drawing parameters significantly reduces the amount of data transmitted between them, thereby reducing the device load.

[0004] Firstly, embodiments of this application provide a method for drawing an interface. This method may include: parsing interface elements in a user interface through an application process to obtain drawing parameters for the interface elements to be drawn. The drawing parameters for the interface elements may include shape parameters and style parameters. After obtaining the drawing parameters, the drawing parameters can be passed from the application process to a rendering service process. Then, the rendering service process can construct a primitive object based on the shape parameters in the drawing parameters and a style object based on the style parameters in the shape parameters. The primitive object can be used to define the shape of the drawn interface element, and the style object can be used to define the display effect of the drawn interface element. Finally, the user interface corresponding to the aforementioned interface elements can be drawn using the constructed primitive object and style object.

[0005] By implementing the method provided in the first aspect, the application process directly passes lightweight drawing parameters of interface elements to the rendering service process, significantly reducing the amount of data transfer between the application process and the rendering service process, thereby reducing the load on the device.

[0006] In conjunction with the first aspect, in some embodiments, passing the drawing parameters of interface elements from the application process to the rendering service process may include: encoding the drawing parameters of the interface elements in a first data format, and then passing the encoded drawing parameters of the interface elements from the application process to the rendering service process. The first data format may include a shape parameter field and a style parameter field, wherein the shape parameter field can be used to fill the shape parameters in the drawing parameters, and the style parameter field can be used to fill the style parameters in the drawing parameters.

[0007] In conjunction with the first aspect, in some embodiments, before constructing a primitive object based on shape parameters and constructing style parameters based on style parameters by the rendering service process, the method further includes: parsing shape parameters and style parameters from drawing parameters of interface elements encoded in a first data format.

[0008] In the above embodiments, the rendering service process receives drawing parameters encoded according to the first data format. By using the shape parameter field and style parameter field to fill the shape parameter and style parameter respectively, the efficiency of the rendering service process in parsing the shape parameter and style parameter from the drawing parameters and constructing primitive objects and style objects can be improved, thereby improving the efficiency of interface drawing.

[0009] In conjunction with the first aspect, in some embodiments, the style parameter field in the first data format may further include a first mask. The first mask may include multiple bits. Each bit corresponds to a style parameter in the drawing parameters. Based on the setting of the first mask, the length of the style parameter field may be positively correlated with the value of the first mask, and each bit may be used to indicate whether the style parameter field includes a field of the style parameter corresponding to that bit.

[0010] In conjunction with the first aspect, in some embodiments, each bit can be used to indicate whether the style parameter field includes a field of the style parameter corresponding to that bit. This can include: when the bit corresponding to a style parameter is a first value, the style parameter field can include the field of that style parameter; when the bit corresponding to a style parameter is not a first value, the style parameter field may not include the field of that style parameter. The first value can be set to 1 for example. The so-called positive correlation between the length of the style parameter field and the value of the first mask can be interpreted as: the more bits in the first mask that are the first value, the more fields can be included in the style parameter field, and the longer the length of the style parameter field can be.

[0011] In the above embodiments, the style parameters included in the style parameter field can be effectively determined by the value of each bit in the first mask, reducing the difficulty of parameter parsing and thus improving the efficiency of interface rendering.

[0012] In conjunction with the first aspect, in some embodiments, the style parameters of an interface element may include a first style parameter, which may include color, anti-aliasing, blending mode, etc. When the value of the first style parameter in the drawing parameters of the interface element is not a default value, the bit corresponding to the first style parameter in the first mask can be a first value; correspondingly, when the value of the first style parameter in the drawing parameters of the interface element is a default value, the bit corresponding to the first style parameter in the first mask is not a first value.

[0013] In the above embodiments, for the first style parameter, the bit corresponding to the first style parameter in the first mask is set to the first value only when the value of the first style parameter in the drawing parameters is different from the default value. That is, the style parameter field only includes the first style parameter and is filled with it when the value of the first style parameter in the drawing parameters is different from the default value. When the value of the first style parameter in the drawing parameters is the same as the default value, the style parameter field may not be filled with the first style parameter, reducing unnecessary data transmission, thereby reducing the amount of data transmission and lowering the device load.

[0014] In conjunction with the first aspect, in some embodiments, the style parameters of the interface elements may further include second style parameters. The second style parameters may include one or more of the following: shader effects, color filtering, mask filtering, image filtering, color gamut, path effects, stroke, mixer, etc. The style parameter fields in the first data format include fields for both first and second style parameters. The fields for the first style parameters can be used to populate the data of the first style parameter. The fields for the second style parameters may further include an index field and a data field. The index field can be used to populate the address of the data field, and the data field can be used to populate the data of the second style parameter.

[0015] In the above embodiments, an index field for indicating the data filling position can be set in the field of the second style parameter to improve data readability, thereby improving the efficiency of interface rendering.

[0016] In conjunction with the first aspect, in some embodiments, the first data format may further include one or more of the following: a shape size field and a style size field. The shape size field can be used to fill the length of the shape parameter field, and the style size field can be used to fill the length of the style parameter field. The shape size field and the style size field can be placed at the beginning of the first data format, thus effectively determining the length and address of the shape parameter field and the style parameter field, thereby improving the parsing efficiency of the drawing parameters.

[0017] In conjunction with the first aspect, in some embodiments, the application process and the rendering service process can run on the same device. In this case, passing the drawing parameters of the interface elements from the application process to the rendering service process specifically includes: passing the drawing parameters of the interface elements from the application process to the rendering service process through inter-process communication. Inter-process communication may include: inter-process communication based on shared memory, etc.

[0018] In the above embodiments, the drawing parameters of interface elements can be directly passed from the application process to the rendering service process through inter-process communication. Passing lightweight drawing parameters can reduce the amount of data transmission, thereby reducing the device load.

[0019] In conjunction with the first aspect, in some embodiments, the application process runs on a first device, the rendering service process runs on a second device, and a first communication connection is established between the first device and the second device. Passing the drawing parameters of the interface elements from the application process to the rendering service process specifically includes: the first device passing the drawing parameters of the interface elements to the second device through the first communication connection. The first communication connection can be any of the following: Bluetooth (BT) communication connection, near field communication (NFC) connection, wireless local area network (WLAN) communication connection, cellular mobile communication connection, etc.

[0020] In the above embodiments, the first device transmits the drawing parameters of the interface elements to the second device through the first communication connection. The second device can directly construct the primitive object and style object of the drawing engine according to the drawing parameters of the interface elements, thereby avoiding the problem that the second device cannot parse the drawing engine object constructed by the first device due to the different drawing engines of the first device and the second device, which leads to the failure of interface drawing.

[0021] Secondly, embodiments of this application provide an electronic device, which may include a memory and a processor. The memory may store a computer program, and when the processor executes the computer program, it may perform the method described in the first aspect and any possible implementation thereof.

[0022] Thirdly, embodiments of this application provide a communication system, which may include a first device and a second device. An application process may run on the first device, and a rendering service process may run on the second device. The first and second devices can jointly implement interface rendering. The first device can parse interface elements in the user interface to obtain rendering parameters for the interface elements. These rendering parameters may include shape parameters and style parameters. After obtaining the rendering parameters, the first device can transmit them to the second device. Upon receiving the rendering parameters from the first device, the second device can construct a primitive object based on the shape parameters and a style object based on the style parameters. Then, the second device can use the constructed primitive object and style object to render the user interface of the corresponding interface element.

[0023] The communication system provided by the third party allows the interface elements parsed by the first device to be directly transmitted to the second device. The second device can then construct a rendering engine object, including primitive objects and style objects, based on the drawing parameters of the interface elements. The second device can then draw the user interface based on the constructed rendering engine object. During the interface drawing process, it does not depend on the drawing engine of the first device, thus eliminating the need for the first and second devices to have the same drawing engine type.

[0024] In conjunction with the third aspect, in some embodiments, the first device is used to transfer drawing parameters of interface elements to the second device, which may include: encoding the drawing parameters of the interface elements in a first data format, and then transferring the encoded drawing parameters from the first device to the second device. The first data format may include a shape parameter field and a style parameter field, wherein the shape parameter field can be used to fill in the shape parameters in the drawing parameters, and the style parameter field can be used to fill in the style parameters in the drawing parameters.

[0025] In conjunction with the third aspect, in some embodiments, before the second device is used to construct a primitive object based on shape parameters and a style parameter based on style parameters, the second device may also be used to parse shape parameters and style parameters from drawing parameters of interface elements encoded in a first data format.

[0026] In the above embodiments, the second device receives drawing parameters encoded according to the first data format. By using the shape parameter field and style parameter field to fill the shape parameter and style parameter respectively, the efficiency of the second device in parsing the shape parameter and style parameter from the drawing parameters and constructing primitive objects and style objects can be improved, thereby improving the efficiency of interface drawing.

[0027] In conjunction with the third aspect, in some embodiments, the style parameter field in the first data format may further include a first mask. The first mask may include multiple bits. Each bit corresponds to a style parameter in the drawing parameters. Based on the setting of the first mask, the length of the style parameter field may be positively correlated with the value of the first mask, and each bit may be used to indicate whether the style parameter field includes a field of the style parameter corresponding to that bit.

[0028] In conjunction with the third aspect, in some embodiments, each bit can be used to indicate whether the style parameter field includes a field of the style parameter corresponding to that bit. This can include: when the bit corresponding to a style parameter is a first value, the style parameter field can include the field of that style parameter; when the bit corresponding to a style parameter is not a first value, the style parameter field may not include the field of that style parameter. The first value can be set to 1 for example. The so-called positive correlation between the length of the style parameter field and the value of the first mask can be interpreted as: the more bits in the first mask that are the first value, the more fields can be included in the style parameter field, and the longer the length of the style parameter field can be.

[0029] In the above embodiments, the style parameters included in the style parameter field can be effectively determined by the value of each bit in the first mask, reducing the difficulty of parameter parsing and thus improving the efficiency of interface rendering.

[0030] In conjunction with the third aspect, in some embodiments, the style parameters of the interface element may include a first style parameter, which may include color, anti-aliasing, blending mode, etc. When the value of the first style parameter in the drawing parameters of the interface element is not the default value, the bit corresponding to the first style parameter in the first mask can be the first value; correspondingly, when the value of the first style parameter in the drawing parameters of the interface element is the default value, the bit corresponding to the first style parameter in the first mask is not the first value.

[0031] In the above embodiments, for the first style parameter, the bit corresponding to the first style parameter in the first mask is set to the first value only when the value of the first style parameter in the drawing parameters is different from the default value. That is, the style parameter field only includes the first style parameter and is filled with it when the value of the first style parameter in the drawing parameters is different from the default value. When the value of the first style parameter in the drawing parameters is the same as the default value, the style parameter field may not be filled with the first style parameter, reducing unnecessary data transmission, thereby reducing the amount of data transmission and lowering the device load.

[0032] In conjunction with the third aspect, in some embodiments, the style parameters of interface elements may further include second style parameters. The second style parameters may include one or more of the following: shader effects, color filtering, mask filtering, image filtering, color gamut, path effects, stroke, mixer, etc. The style parameter fields in the first data format include fields for both first and second style parameters. The fields for the first style parameters can be used to populate the data of the first style parameter. The fields for the second style parameters may further include an index field and a data field. The index field can be used to populate the address of the data field, and the data field can be used to populate the data of the second style parameter.

[0033] In the above embodiments, an index field for indicating the data filling position can be set in the field of the second style parameter to improve data readability, thereby improving the efficiency of interface rendering.

[0034] In conjunction with the third aspect, in some embodiments, the first data format may further include one or more of the following: a shape size field and a style size field. The shape size field can be used to fill the length of the shape parameter field, and the style size field can be used to fill the length of the style parameter field. The shape size field and the style size field can be placed at the beginning of the first data format, thus effectively determining the length and address of the shape parameter field and the style parameter field, thereby improving the parsing efficiency of the drawing parameters.

[0035] In conjunction with the third aspect, in some embodiments, a first communication connection can be established between the first device and the second device. The first device is used to transmit drawing parameters of interface elements to the second device. Specifically, the first device can be used to transmit drawing parameters of interface elements to the second device through the first communication connection. The first communication connection can be any of the following: Bluetooth (BT) communication connection, near field communication (NFC) connection, wireless local area network (WLAN) communication connection, cellular mobile communication connection, etc.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can implement the first aspect and any possible implementation thereof, or the third aspect and any possible implementation thereof.

[0037] Fifthly, embodiments of this application provide a computer program product that, when executed by a processor, can implement the first aspect and any possible implementation thereof, or the third aspect and any possible implementation thereof. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0039] Figure 1A A schematic diagram of a separate rendering architecture;

[0040] Figure 1B A schematic diagram for a unified rendering architecture;

[0041] Figure 2 The specific steps for implementing interface rendering based on a unified rendering architecture are shown.

[0042] Figure 3 The overall flow of the interface drawing method provided in the embodiments of this application is illustrated by way of example;

[0043] Figure 4 The interaction flow when the first and second devices implement interface drawing is shown;

[0044] Figure 5 The lightweight data format introduced in the embodiments of this application is illustrated by way of example;

[0045] Figure 6 This paper illustrates a specific implementation of the interface drawing method provided in an embodiment of this application;

[0046] Figure 7 A diagram illustrating how to pass drawing parameters of UI elements across processes;

[0047] Figure 8 A diagram illustrating how to pass drawing parameters of UI elements across devices;

[0048] Figure 9 An exemplary embodiment of the communication system 10 provided in this application is shown;

[0049] Figure 10 An exemplary schematic diagram of the hardware structure of electronic device 100;

[0050] Figure 11 This is an exemplary schematic diagram of the software architecture of an electronic device 100. Detailed Implementation

[0051] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0052] The user interface (UI) can include various visible interface elements displayed on the screen or virtual screen of an electronic device, such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, and navigation bars. The size, style, color, and other attributes designed by UI designers for these interface elements can be defined in the application's interface source code and resource files, with layout resources used to define the interface architecture.

[0053] Electronic devices can parse interface elements in the user interface to obtain the basic drawing parameters needed to draw these elements, such as shape, color, and position. Based on these parsed parameters, the electronic device can generate a series of drawing tasks and submit them to the graphics processing unit (GPU). The GPU can execute these tasks to complete the interface rendering. Ultimately, the application's user interface is presented to the user.

[0054] Currently, the rendering architectures used to draw user interfaces mainly include: separate rendering architecture and unified rendering architecture.

[0055] Figure 1A and Figure 1B The separate rendering architecture and the unified rendering architecture are shown respectively.

[0056] Separate rendering architecture:

[0057] In a separate rendering architecture, each application can generate its own drawing instructions, invoke the GPU to draw the user interface, and store the drawing results in its own buffer. These drawing results constitute the application's user interface. Then, the compositor can read the drawing results from the buffers of each application process, perform compositing, and finally display the composited result on the screen.

[0058] like Figure 1AAs shown, application A can parse the UI elements included in its user interface to generate a rendering node tree A. Rendering node tree A can be a data structure generated by the application's UI thread to generate the application's UI. Rendering node tree A records all the information needed to draw a frame of the user interface of application A. Rendering node tree A can include multiple rendering nodes, each of which can record the drawing operation of a UI element. Then, the UI thread of application A can pass rendering node tree A to the rendering thread of application A. The rendering thread of application A will call the GPU to execute the drawing task GPU-Task-A. Specifically, the rendering thread traverses the rendering nodes of rendering node tree A, converting the drawing operation of each rendering node into a corresponding drawing task. After traversal, the rendering thread submits the drawing tasks to the GPU either uniformly or in batches. The GPU executes the drawing task, draws the user interface of application A, and stores the drawing result in bufferA. Here, bufferA is the process buffer of application A, and the drawing result can be a bitmap or other data format that can store image information.

[0059] Similarly, application B can parse the UI elements included in its user interface to generate a render node tree B. The render node tree B records all the information needed to draw a frame of application B's user interface. Render node tree B can include multiple render nodes, each recording the drawing operation of a UI element. Then, application B's UI thread can pass the render node tree B to application B's rendering thread. Application B's rendering thread will then invoke the GPU to execute drawing tasks (GPU-Task-B). Specifically, the rendering thread traverses the render nodes of render node tree B, converting the drawing operation of each render node into a corresponding drawing task. After traversal, the rendering thread submits all drawing tasks to the GPU. The GPU executes the drawing tasks, draws application B's user interface, and stores the drawing results in bufferB. Here, bufferB is application B's process buffer.

[0060] Finally, for each frame of the interface, the compositor can obtain the drawing results of application A from bufferA and the drawing results of application B from bufferB, combine them, and then display the merged result on the screen.

[0061] Unified rendering architecture:

[0062] In a unified rendering architecture, an operating system-level rendering service manages the drawing operations of various applications on the electronic device. Applications are only responsible for sending the content to be drawn to the rendering service process via commands (cmds). Upon receiving the commands, the rendering service process can submit the drawing operations for each frame of the interface to the GPU for execution, rendering the interface that needs to be displayed. The rendering service process in the unified rendering architecture can draw the user interfaces of multiple applications simultaneously, and the interface drawn by the rendering service process can include the user interfaces of multiple applications.

[0063] like Figure 1B As shown, application A can parse the interface elements included in its user interface, generate command cmds-A, and pass cmds-A to the rendering service via inter-process communication (IPC). cmds-A contains all the drawing operations required to render the user interface of application A. Similarly, application B can also parse the interface elements included in its user interface, generate command cmds-B, and pass cmds-B to the rendering service via IPC. cmds-B contains all the drawing operations required to render the user interface of application B.

[0064] The rendering service constructs a unified rendering node tree based on the received cmds-A and cmds-B. This unified rendering node tree includes drawing operations from both application A and application B. It iterates through the rendering nodes of the unified rendering node tree, converting the drawing operations of each node into corresponding GPU-Tasks. The GPU executes the GPU-Tasks to render the user interfaces of application A and application B. After rendering is complete, the results can be temporarily stored in the rendering service process's buffer. For each frame of the interface, the compositor can read the rendering results from the buffer and display them on the screen.

[0065] In some examples, the compositor and the rendering service can reside in the same process; for instance, the compositor thread can be part of the rendering service process. It should be understood that the compositor's code can be loaded into the rendering service process at runtime; alternatively, the compositor can also be part of the rendering service.

[0066] In a separate rendering architecture, each application has its own rendering thread and buffer, and each application independently calls the GPU to perform drawing tasks, which increases the complexity of system management for electronic devices. In a unified rendering architecture, all application drawing tasks are handled by a unified rendering service process. This rendering service process can uniformly schedule and allocate GPU resources as needed, thereby managing and optimizing GPU resources more effectively.

[0067] Figure 2 This demonstrates the specific steps involved in implementing UI rendering based on a unified rendering architecture. The details are as follows:

[0068] S201. Parameter analysis for drawing.

[0069] First, electronic devices can read the attributes and layout information of interface elements from the application's interface source code or resource files. By parsing these attributes and layout information, the drawing parameters required to draw each interface element of the user interface can be obtained.

[0070] The drawing parameters of a UI element are a set of data describing the geometry and display effect of that element, and may include, but are not limited to, one or more of the following: shape, color, anti-aliasing, blending mode, shader effects, etc. For example, parsing a UI element can yield the following drawing parameters: a circle with center (a, b), radius c, border width d, border color black, fill color blue, etc.

[0071] S202. Construct the drawing engine object.

[0072] After parsing the drawing parameters of the interface elements, the electronic device can use the paint engine to construct a drawing engine object from these drawing parameters.

[0073] The rendering engine provides developers with a unified set of data structures and interfaces for implementing graphics rendering. It also offers multiple rendering methods, all of which require receiving a corresponding rendering engine object, including primitive objects and style objects. During the rendering service process, the rendering parameters of UI elements can be converted into the corresponding rendering engine object, and then the rendering interface can be called through the methods provided by the rendering engine to complete the UI rendering.

[0074] When constructing a drawing engine object, the drawing parameters of UI elements can be structurally encapsulated using the data structures provided by the drawing engine, thereby constructing one or more drawing engine objects. For example, for a rounded rectangle, the drawing parameters of the UI element can be structurally encapsulated to obtain a RoundRect object and a paint object. The RoundRect object contains geometric information such as the position, size, and corner radius of the rounded rectangle, while the paint object contains style information such as the color, blending mode, and anti-aliasing of the rounded rectangle.

[0075] S203. Pass the drawing engine object.

[0076] like Figure 2 As shown, S201 and S202 occur in the application process of the electronic device, while S204 occurs in the rendering service process of the electronic device. The electronic device needs to pass the drawing engine object from the application process to the rendering service process, and then the rendering service process calls the GPU to draw the interface elements of the user interface.

[0077] Specifically, within the application process, the rendering engine object can be converted into a byte stream through serialization, and then the byte stream can be transmitted to the rendering service process via IPC communication. Upon receiving the byte stream, the rendering service process can recover the original rendering engine object from the byte stream through deserialization. The application process can transmit the rendering engine object to the rendering service process using various IPC communication methods such as Binder, AIDL, shared memory, and Socket. For example, when transmitting the rendering engine object, the serialized rendering engine object can be written to a shared memory area, and then the rendering service process can be notified to read the data from that shared memory area.

[0078] If the application process and the rendering service process are on different devices, the rendering engine object needs to be sent from the device where the application process is located to the device where the rendering service process is located through an inter-device communication connection.

[0079] S204. Draw interface elements.

[0080] After the rendering service process recovers the drawing engine object from the byte stream, it can pass the drawing engine object to the drawing methods provided by the drawing engine. The drawing methods provided by the drawing engine generate drawing tasks for drawing UI elements. Then, the generated drawing tasks are converted into GPU-executable drawing commands, such as DrawCall, and sent to the GPU to drive it to draw the UI elements. Specifically, the rendering service process can add drawing commands to the GPU's command buffer, from which the GPU can read and execute the drawing tasks. The GPU can use one or more rendering pipelines to color each pixel on the screen, performing drawing tasks (including vertex processing, primitive assembly, rasterization, fragment shading, etc.), ultimately resulting in the drawn UI.

[0081] However, Figure 2 In the interface rendering methods shown, converting rendering parameters into rendering engine objects often increases the data volume, leading to larger data packets transmitted across processes and thus increasing system overhead. Furthermore, different rendering engines may be needed to achieve the best user experience on different types of devices. Different rendering engines construct their rendering engine objects differently; one rendering engine cannot parse the rendering engine object of another. Therefore, when utilizing… Figure 2 When the interface drawing method shown handles cross-device interface drawing tasks, it is necessary to ensure that the same drawing engine is used across devices. Otherwise, the original drawing engine object cannot be parsed from the byte stream through deserialization operations in the rendering service process.

[0082] This application provides a method for interface rendering in which the application process directly passes lightweight rendering parameters of interface elements to the rendering service process. The rendering service process then constructs primitive objects and style objects for the rendering engine based on these rendering parameters. Compared to passing large amounts of data-driven rendering engine objects based on rendering parameters between the application process and the rendering service process, directly passing these lightweight rendering parameters significantly reduces the amount of data transmitted between them, thereby reducing the device load.

[0083] Figure 3 The overall flow of the interface drawing method provided in the embodiments of this application is illustrated by way of example.

[0084] Figure 3 The interface rendering method shown can be implemented by a device, on which application processes and rendering service processes can be deployed; Figure 3 The interface drawing method shown can also be implemented by two devices based on cross-device communication. These two devices can be a first device and a second device. The first device can have an application process deployed, and the second device can have a rendering service process deployed.

[0085] S301. Drawing Parameter Analysis.

[0086] The application process parses the interface elements to obtain their drawing parameters. Here, the interface elements are the elements of the user interface to be drawn; the application process belongs to the first application, and the user interface belongs to the first application.

[0087] The first application can be a system application such as desktop, contacts, phone, camera, or text message, or it can be a third-party application.

[0088] The user interface of a first application can consist of one or more interface elements.

[0089] Drawing parameters for UI elements can include shape parameters and style parameters. Shape parameters describe the shape of the UI element. Shape parameters can include, but are not limited to: geometric type (e.g., rectangle, circle), coordinates, size, path description (e.g., Bézier curve control points), vertex set (e.g., vertex coordinates of a polygon), etc. Style parameters describe the display effect of the UI element. Style parameters can include color (e.g., stroke color, fill color), line thickness, shader effects, path effects, and other parameters related to display effects.

[0090] For example, parsing a rounded button yields the following drawing parameters: rounded rectangle, top-left corner coordinates (100, 200), width 200, height 50, corner radius 10, fill color red, stroke color blue, stroke width 2 pixels. The shape parameters can include: rounded rectangle, top-left corner coordinates (100, 200), width 200, height 50, corner radius 10, and the style parameters can include: fill color red, stroke color blue, stroke width 2 pixels.

[0091] S302. Pass drawing parameters.

[0092] After parsing the UI elements and obtaining their drawing parameters, these parameters can be directly passed from the application process to the rendering service process. For example, the drawing parameters can be packaged into a byte stream and then transmitted to the rendering service process. The rendering service process can be a system function on an electronic device that receives the drawing parameters, constructs a drawing engine object, and uses the drawing engine object to drive the GPU to draw the UI elements.

[0093] S303. Construct the drawing engine object.

[0094] After passing the drawing parameters from the application process to the rendering service process, the drawing parameters of the interface elements can be used to construct a drawing engine object.

[0095] The rendering engine includes two concepts: primitives and styles. Primitives describe the type, shape, and position of the drawn content, while styles describe display details such as color and whether anti-aliasing is used. Therefore, the constructed rendering engine object can include primitive objects (e.g., paths) and style objects (e.g., paint). Specifically, primitive objects can be constructed based on shape parameters through the rendering service process; style objects can be constructed based on style parameters through the rendering service process.

[0096] In some examples, primitive objects can be used to define the drawing path when drawing UI elements, such as lines, curves, rectangles, circles, custom paths, etc. Style objects can be used to define the display effects of drawing UI elements, which can include colors, anti-aliasing, color filters, shader effects, etc. In other words, primitive objects can be used to define "where to draw," and style objects can define "how to draw." For example, for a circular button, the shape parameters can be constructed into a primitive object to define the circular outline of the button, and the style parameters can be constructed into a style object to define that the fill color of this circle is red, the stroke color is blue, and the stroke width is 2 pixels.

[0097] A detailed explanation of how to construct primitive objects based on shape parameters and style objects based on style parameters will be provided in the specific embodiments later, and will not be repeated here.

[0098] S304. Draw interface elements.

[0099] After the primitive objects and style objects are constructed through the rendering service process, the GPU can be driven to draw the interface elements of the user interface using the primitive objects and style objects.

[0100] In some examples, the rendering service process can pass the constructed primitive objects and style objects as parameters to the corresponding drawing methods through the drawing engine to generate drawing tasks for drawing UI elements. For example, passing the primitive object `path1` and the style object `paint1` to the drawing method `drawRoundRect()` for drawing rounded rectangles generates the drawing task `drawRoundRect` for drawing rounded rectangles. Furthermore, the rendering service process can translate this drawing task into drawing commands that the GPU can understand, driving the GPU to draw UI elements according to the drawing paths and display effects defined by the primitive and style objects.

[0101] The above is an overview of the interface drawing method provided in the embodiments of this application.

[0102] When the interface drawing method is implemented by a single device, steps S301-S304 can all be executed by the same electronic device. During S302, the electronic device can use inter-process communication, such as shared memory-based inter-process communication, to transfer the drawing parameters of the interface elements from the application process of the electronic device to the rendering service process of the electronic device.

[0103] When the interface drawing method is implemented by two devices, such as a first device and a second device, as follows: Figure 4 As shown, S301 can be executed by the first device, while S303 and S304 can be executed by the second device. That is, the first device can parse the interface elements of the user interface and obtain the drawing parameters of the interface elements, and the second device can construct primitive objects based on the shape parameters in the drawing parameters, construct style objects based on the style parameters in the drawing parameters, and then use the constructed primitive objects and style objects to drive the GPU to draw the interface elements of the user interface.

[0104] like Figure 4 As shown, S302 can be implemented by the interaction of two devices. The first device and the second device can establish a first communication connection, which can be a Bluetooth (BT) communication connection, a near field communication (NFC) connection, a wireless local area network (WLAN) communication connection, a cellular mobile communication connection, etc. The first device can send drawing parameters of interface elements to the second device through the first communication connection. Correspondingly, the second device can receive drawing parameters of interface elements from the first device through the first communication connection.

[0105] This application also introduces a lightweight data format, which can be used to encode the drawing parameters of interface elements when transmitting them.

[0106] Figure 5 The lightweight data format introduced in the embodiments of this application is illustrated by way of example.

[0107] like Figure 5 As shown, the lightweight data format provided in this application embodiment may include: a data header field, a shape parameter field, and a style parameter field.

[0108] Header fields can be listed at the beginning and can include multiple subfields such as shape size, style size, and extended size. Specifically, the shape size field can be used to fill the length of the shape parameter field (shape size), the style size field can be used to fill the length of the style parameter field (paint size), and the extended size field can be used to fill the length of the extended field (extdata size).

[0109] The order of the shape parameter fields and style parameter fields is not restricted. Style parameter fields can be placed either before or after the shape parameter fields. For example... Figure 5 As shown, the shape parameter field is arranged after the data header field and before the style parameter field.

[0110] The shape parameter field can be used to populate the shape parameters of UI elements.

[0111] The shape parameters of a UI element can include shape type and shape data. Shape type describes the shape type of the UI element, such as rectangle, circle, triangle, line, curve, point, custom path, etc. Shape data describes the size and position of the UI element on the canvas. Different shape types can correspond to different shape data. For example, when the shape type is rectangle, the shape data can include the coordinates of the top-left vertex (left, top) and bottom-right vertex (right, bottom) of the rectangle on the canvas, or it can include the coordinates of the top-left vertex (left, top) and the length and width of the rectangle. When the shape type is circle, the shape data can include the coordinates of the center (x, y) and radius of the circle, or it can include the coordinates of three points on the circumference of the circle (x1, y1), (x2, y2), and (x3, y3).

[0112] In some embodiments, the shape parameter field may include two subfields: a shape type field and a shape data field. The shape type field can be used to fill in the shape type, and the shape data field can be used to fill in the shape data.

[0113] The above is merely an exemplary description of the shape section. In some embodiments, the shape section may also be filled with other shape parameters besides shape type and shape data.

[0114] The style parameter field can be used to populate style parameters for UI elements.

[0115] The style parameters of interface elements can include, but are not limited to, the following: antialiasing (AA), blend mode, color, as well as shader effects, color filters, mask filters, image filters, color space, path effects, strokes, blenders, etc.

[0116] A style parameter field can include multiple fields, each corresponding to a specific style parameter. Each style parameter's fields can be used to populate the data for that style parameter.

[0117] The style parameter field may also include a first mask. The first mask can consist of multiple bits, each corresponding to a style parameter. The style parameter field can be a variable-length field, dynamically including different fields based on the bit values ​​in the first mask. The bits corresponding to each style parameter in the first mask can be used to indicate whether the style parameter field includes that style parameter's field.

[0118] In some embodiments, when the bit corresponding to a style parameter in the first mask is a first value, the style parameter field includes the field of that style parameter. When the bit corresponding to a style parameter in the first mask is not a first value, the style parameter field does not include the field of that style parameter. The first value can be, for example, 1. The length of the style parameter field can be positively correlated with the value of the first mask. That is, the more bits in the first mask that are set to the first value, the more fields the style parameter field can contain, and the longer the style parameter field can be.

[0119] For example, Figure 5The mask in the style parameter field shown can consist of 11 bits, where bit 0 corresponds to color, bit 1 to anti-aliasing, bit 2 to blending mode, bit 3 to shader effect, bit 4 to color filter, bit 5 to mask filter, bit 6 to image filter, bit 7 to color gamut, bit 8 to path effect, bit 9 to stroke, and bit 10 to mixer. Each bit can be set to "1" or "0", with "1" being the first value. When a bit corresponding to a style parameter is set to "1", the style parameter field includes the field for that style parameter. When a bit corresponding to a style parameter is set to "0", the style parameter field does not include the field for that style parameter. For example, the value of mask can be "10001000100". In this case, bits 0, 4, and 8 are set to "1", and the remaining bits are set to "0". Therefore, the style parameter field only includes the color field, the color filter field, and the path effect field. In the style parameter field, the order of the fields can be the same as the order of the bits in the mask. That is, the color field, color filter field, and path effect field can be arranged in order from front to back.

[0120] In some embodiments, the above-mentioned multiple style parameters can be divided into two categories: first style parameters and second style parameters.

[0121] The first style parameter can include basic style parameters such as color, blending mode, and anti-aliasing. The data for the first style parameter is usually a single value. For example, color data can be represented by an RGB value (such as 0xFF0000 representing red) or a color name (such as "red"), blending mode data can be represented by an enumeration value (such as "clear mode") or a short string, and anti-aliasing data can be represented by a boolean value (such as "on / off").

[0122] In this embodiment, a default value can be set for each first style parameter. When the value of this first style parameter in the drawing parameters of the interface element is the default value, the corresponding bit in the first mask can be the first value, such as "1". When the value of this first style parameter in the drawing parameters of the interface element is not the default value, the corresponding bit in the first mask is not the first value. The default value can be set by the developer during development.

[0123] For example, blending modes can be used to define how source pixels (the content to be drawn) and target pixels (the content already drawn on the canvas) are combined, and can include clear mode, overlay mode, screen mode, darken / lighten mode, color dodge / burn mode, hard light / soft light mode, difference mode, etc. Developers can set the overlay mode as the default blending mode. When organizing the drawing parameters of interface elements in the lightweight data format provided in this application embodiment, if the blending mode in the drawing parameters is overlay mode, the bit (bit 2) corresponding to the blending mode in the first mask is set to "0". In this case, the style parameter field does not include the blending mode field. If the blending mode in the drawing parameters is a non-default mode such as clear mode, the bit corresponding to the blending mode in the first mask is set to "1". In this case, the style parameter field can include the blending mode field, filled with data indicating that the blending mode is clear mode. Correspondingly, when reading the blending mode data from the drawing parameters organized in the lightweight data format, the value of the first mask is first checked. If the value of bit 2 is "0", it can be determined that the blending mode is the default overlay mode and the style parameter field does not include the blending mode field. If the value of bit 2 is "1", it can be determined that the style parameter field contains a blending mode field. Then, the data in the blending mode field is queried and read to confirm that the blending mode is clear mode or other non-default mode.

[0124] The second style parameter can include shader effects, color filters, mask filters, image filters, color gamut, path effects, strokes, mixers, and more. The second style parameter can be used to describe complex drawing effects of interface elements, and it can contain more data items than the first style parameter.

[0125] For example, shader effects can achieve complex color effects, such as gradients. Shader effects can include linear gradient shader effects, radial gradient shader effects, and fan-shaped gradient shader effects. Different types of shader effects correspond to different paint data. For example, a linear gradient shader effect can generate a linear gradient color effect between two specified points, and its corresponding paint data can include the gradient start point (StartPoint), gradient end point (EndPoint), color array (colors), tiling mode (TileMode), relative position array (pos), and matrix object. A radial gradient shader effect can generate a circular gradient color effect outward from a point, and its corresponding paint data can include the center point (CenterPoint), radius (radius), color array, tiling mode, relative position array, and matrix object. The paint data for a fan-shaped gradient shader effect can include the center point coordinates, color array, tiling mode, fan gradient start angle (StartAngle), fan gradient end angle (EndAngle), relative position array, and matrix object.

[0126] Color filters can be used to process interface colors through a color matrix, such as adjusting brightness, contrast, saturation, hue, or achieving specific color effects. Mask filters can be used to achieve effects such as blurring and creating 3D effects. Image filters can be used to achieve blurring effects. Color gamut can provide richer color expression, such as sRGB and P3. Path effects can be used to set the effects of drawing paths, such as dashed lines, straight lines, and line offsets. Strokes can be used to set brush styles (such as round or square styles), brush thickness, and shadow effects. The rendering engine provides some default mixers through blending modes, and also allows for custom mixers.

[0127] For each second style parameter, when the drawing parameters of the interface element include this second style parameter, the corresponding bit of this second style parameter in the first mask can be set to the first value, and the style parameter field can include the field of this second style parameter. When the drawing parameters of the interface element do not include this second style parameter, the corresponding bit of this second style parameter in the first mask can be set to a non-first value, and the style parameter field does not include the field of this second style parameter.

[0128] For example, when drawing a UI element, if the element has a gradient effect, the drawing parameters obtained from parsing the element will include shader effect data. In this case, when organizing the UI element's drawing parameters in a lightweight data format, the bit corresponding to the shader effect in the first mask (bit 3) can be set to the first value ("1"), and the style parameter field includes the shader effect field. The shader effect field can be used to populate the shader effect data. If the UI element does not have a gradient effect, the drawing parameters obtained from parsing the element will not include shader effect data. In this case, when organizing the UI element's drawing parameters in a lightweight data format, the bit corresponding to the shader effect in the first mask (bit 3) can be set to a non-first value ("0"), and the style parameter field will not include the shader effect field.

[0129] In some embodiments, the fields of the second style parameter may include an index field and a data field.

[0130] An index field of a second style parameter can be used to populate the address of a data field for that second style parameter, and a data field of a second style parameter can be used to populate the data for that second style parameter. The address of the data field populated in the index field can be an offset relative to the start position of the entire style parameter field and its length, or an offset relative to the start position of the second style parameter field and its length, or a unique identifier (such as a hash value), etc. For example, Figure 5 The data format shown can use the (offset, size) index method to indicate the position of data fields.

[0131] When a style parameter field includes multiple second style parameter fields, the index fields of all second style parameters can be grouped into one area, and the data fields into another area. For example... Figure 5As shown, a region is set after the first style parameter field to fill the index field of the second style parameter, followed by the data field of the second style parameter. For example, (offset1, size1) in the index field can be used to indicate the address of the shader effect's data field data1. Here, offset1 is the offset of the shader effect's data field relative to the starting position of the entire style parameter field, and size1 is the length of the shader effect's data field. The position of data1 in the style parameter field can be determined by (offset1, data1) in the index field. The (offset2, size2) in the index field can be used to indicate the address of the path effect's data field data2, and the position of data2 in the style parameter field can be determined by (offset2, size2) in the index field.

[0132] like Figure 5 As shown in the embodiments of this application, the lightweight data format may also include extended fields (extdata). Extended fields can be used to fill in device information (such as device identifier, model, timestamp and other contextual information), checksums (which can be used to verify whether the data is complete), and other information.

[0133] When transmitting drawing parameters for interface elements, the lightweight data format introduced in this application embodiment is used to organize the drawing parameters. The presence of style parameters is indicated by bits in the first mask, and different field structures are used for different types of style parameters. The first type can directly contain data, while the second type can locate data through an index. By organizing and trimming the drawing parameters to be transmitted through data segmentation and masking, unnecessary data transmission is avoided, reducing the amount of data transmitted and effectively improving the transmission and parsing efficiency of drawing parameters. This, in turn, reduces system overhead and lowers the system load on the device.

[0134] Figure 6 This paper illustrates a specific implementation of the interface drawing method provided in an embodiment of this application. The details are as follows:

[0135] S601. The interface elements are parsed through the application process to obtain the drawing parameters of the interface elements, including shape parameters and style parameters. Here, the interface elements are the elements of the user interface to be drawn, the application process belongs to the first application program, and the user interface belongs to the first application program.

[0136] S602. Encode the drawing parameters of the interface elements in the first data format.

[0137] In this embodiment of the application, the first data format may be Figure 5 The lightweight data format shown.

[0138] For example, in S601, parsing a UI element yields the following drawing parameters: a rectangle with its top-left corner coordinates at (100, 100), a width of 100, a height of 50, a red stroke color, anti-aliasing enabled, a clear blending mode, and a linear gradient fill. Data describing the gradient effect includes: gradient start point x1, gradient end point x2, and a color array (colors). When organizing the drawing parameters in the first data format, the shape parameter field can be used to fill the shape parameters, including: shape type as rectangle, and shape data including top-left corner coordinates at (100, 100), a width of 100, and a height of 50. Style parameters include stroke color, anti-aliasing, blending mode, and shader effects. The developers set black as the default color, anti-aliasing as enabled by default, and clear blending mode as the default blending mode. Therefore, when organizing the style parameters according to the first data format, the mask value in the style parameter field is "00110000000". The style parameter field includes a color field and a shader effect field.

[0139] S603. The drawing parameters of the interface elements encoded in the first data format are passed from the application process to the rendering service process.

[0140] When transmitting drawing parameters for UI elements, one element's drawing parameters can be transmitted at a time, or multiple elements' drawing parameters can be packaged and transmitted. In some embodiments, after organizing the drawing parameters of the UI elements according to a first data format, the organized data can be serialized into a byte stream. The drawing parameters of the UI elements are then transmitted to the rendering service process via the byte stream.

[0141] When passing drawing parameters from the application process to the rendering service process, the following two scenarios can be included:

[0142] 1) Passing drawing parameters across processes

[0143] In some embodiments, such as Figure 7 As shown, the application process (process A) and the rendering service process (process B) can run on the same device. In this case, drawing parameters of interface elements can be passed from the application process to the rendering service process through inter-process communication.

[0144] On electronic devices, memory is not shared between different processes; data exchange between processes requires inter-process communication (IPC). IPC can be implemented using various methods such as Binder, AIDL, shared memory, and Socket. For example, drawing parameters for UI elements can be passed from the application process to the rendering service process using shared memory. After organizing the drawing parameters according to a first data format, a shared memory region can be allocated, and the drawing parameters can be written into this region. The rendering service process can then read the data from this shared memory region to obtain the drawing parameters for the UI elements.

[0145] 2) Transferring drawing parameters across devices

[0146] In some embodiments, an application process may run on the first device, and a rendering service process may run on the second device.

[0147] The first device can be a smart device with communication capabilities, such as a smartphone, tablet, or laptop. The second device can be a smart device with both communication and display capabilities, such as a smartphone, tablet, laptop, projector, or display screen. The first and second devices can establish a first communication connection. Through this first communication connection, drawing parameters for interface elements can be transferred from the first device to the second device.

[0148] For example, the first device is a tablet computer, and the second device is a separate display screen. The tablet computer and the separate display screen establish a wired communication connection via a data cable. The application process on the tablet computer (such as the process of a video application) can package the drawing parameters of the interface elements into a byte stream according to a first data format and send the byte stream to the separate display screen via the wired communication connection.

[0149] like Figure 8 As shown, in existing technical solutions, cross-device rendering requires device A (the first device) and device B (the second device) to use the same rendering engine; otherwise, after receiving the byte stream, device B cannot parse the rendering engine object generated by device A through deserialization. The rendering method provided in this application does not depend on the rendering engine of device A, therefore, there is no requirement that the rendering engines of device A and device B be identical. When implementing cross-device interface rendering using the interface rendering method provided in this application, device A only needs to package and transmit the rendering parameters to device B, and device B can directly create a rendering engine object based on the rendering parameters.

[0150] S604. In the rendering service process, the shape parameters and style parameters are parsed from the drawing parameters of the interface elements encoded in the first data format.

[0151] For example, shape and style parameters can be parsed out one by one by traversing the byte stream. First, based on the lengths of the shape and style parameter fields recorded in the header, their positions in the byte stream can be determined. Then, reading the data in the shape parameter field yields the shape type (rectangle) and shape data (top-left vertex coordinates (100, 100), width 100, height 50). For the style parameter field, the first mask is read as "00110000000". Based on the value of the first mask, it can be determined that anti-aliasing is enabled, the blending mode is cleared, and the style parameter field contains a color field and a shader effect field. Based on the data filled in the color field, the stroke color is determined to be red. Based on the index in the shader field, the position of the shader effect data field can be determined. Then, reading the data filled in the shader effect data field reveals that the filled area uses a linear gradient effect, with the gradient start point at x1, the gradient end point at x2, and the color array (colors).

[0152] S605. Construct primitive objects based on shape parameters, and construct style objects based on style parameters.

[0153] During the rendering service process, the rendering engine provides multiple methods for setting the drawing paths and display effects of UI elements, such as the `setShaderEffect()` method for setting the shader effects of UI elements. These methods correspond to the shape parameters and multiple style parameters (color, blending mode, etc.) in the drawing parameters. Each parameter is passed to the corresponding setting method, and the method for setting the drawing path is encapsulated as a primitive object, while the method for setting the display effect is encapsulated as a style object.

[0154] For example, data from the shape parameters can be passed to `Rect()`, which sets the drawing path for the rectangle, and then encapsulated into a primitive object `path`, which includes `path.Rect`. Data from the style parameters, including color, anti-aliasing, blending mode, and shader effects, can be passed to the following methods: `setColor()` for setting the drawing color, `setAntiAlias()` for setting anti-aliasing, `setBlendMode()` for setting the blending mode, and `setShaderEffect()` for setting the shader effects. These methods can then be uniformly encapsulated into a style object `paint`, which can include: `paint.setColor`, `paint.setAntiAlias`, `paint.setBlendMode`, and `paint.setShaderEffect`.

[0155] The above is merely an exemplary description of constructing primitive objects and style objects. In some embodiments, primitive objects and style objects can also be constructed using other methods. For example, drawing parameters can be used to customize drawing methods and encapsulate them into primitive objects and style objects.

[0156] For different rendering engines, different rendering engine objects can be constructed using rendering parameters during the rendering service process. For example... Figure 5 As shown, if the rendering engine is rendering engine A (such as Vulkan), the shape parameters can be constructed into a primitive object A (pathA), and the style parameters can be constructed into a style object (paintA). If the rendering engine is rendering engine B (such as OpenGL), the shape parameters can be constructed into a primitive object B (pathB), and the style parameters can be constructed into a style object (paintB). Both pathA and pathB can be used to define the drawing path when drawing UI elements, and both paintA and paintB can be used to define the display effect when drawing UI elements. The UI elements drawn by the GPU using pathA and paintA are the same as those drawn by the GPU using paintB and paintB. The difference lies in the methods used to define the drawing path and the methods used to define the display effect.

[0157] S606. Use primitive objects and style objects to drive the GPU to draw interface elements in the user interface.

[0158] The implementation method of S606 is the same as that of S304. The rendering service process can pass the constructed primitive objects and style objects as parameters to the corresponding drawing methods through the drawing engine to generate drawing tasks for drawing interface elements. The rendering service process can convert this drawing task into drawing commands that the GPU can understand, driving the GPU to draw interface elements according to the drawing paths and display effects defined by the primitive objects and style objects.

[0159] S607. Display user interface.

[0160] This application also provides a communication system 10.

[0161] Figure 9 An exemplary embodiment of the communication system 10 provided in this application is shown.

[0162] like Figure 9 As shown, the communication system 10 may include: a first device and a second device.

[0163] The first device can have application processes and communication services deployed on it.

[0164] The application process can be used to parse interface elements in the user interface and obtain the drawing parameters of the interface elements. In some embodiments, the application process can also be used to organize the drawing parameters of the interface elements in a first data format.

[0165] The communication service on the first device can be used to establish a communication connection with the second device and pass the drawing parameters of the interface elements to the second device.

[0166] The second device can be used to deploy rendering service processes and communication services.

[0167] The rendering service process can be used to construct primitive objects based on shape parameters in the drawing parameters, and to construct style objects based on style parameters in the drawing parameters. The rendering service process can also be used to drive the graphics processor to draw interface elements in the user interface using the primitive objects and style objects. In some embodiments, the rendering service process on the second device can also be used to parse the shape and style parameters of the interface elements from the drawing parameters encoded in a first data format.

[0168] The communication service on the second device can be used to establish a communication connection with the first device and receive drawing parameters of the interface elements from the first device.

[0169] This application also provides an electronic device 100.

[0170] Figure 10 This is an exemplary schematic diagram of the hardware structure of electronic device 100.

[0171] The device type of electronic device 100 can be a smartphone, tablet computer, personal computer (PC), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device (such as smartwatch, smart glasses), sports equipment, etc.

[0172] The electronic device 100 may be a device that implements the interface drawing method provided in the embodiments of this application, or it may be a first device or a second device that implements the interface drawing method provided in the embodiments of this application.

[0173] like Figure 10As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone sensor 180M, etc.

[0174] The processor 110 can be one or more, and they can be integrated into an integrated circuit of a system-on-a-chip (SOC). An SOC is a system-on-a-chip. The processor 110 may include a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processing unit (NPU). The CPU may include an application processor (AP) and a baseband processor (BP). The AP is responsible for running the operating system, user interface, and applications on the terminal device; the BP is responsible for transmitting and receiving wireless signals and managing radio frequency services. The GPU is responsible for graphics rendering, performing tasks such as shading, material filling, rendering, and output based on rendering instructions and data from the CPU. The NPU, by referencing biological neural network structures, such as the transmission patterns between neurons in the human brain, can quickly process input information and continuously learn. The NPU can be used to run artificial intelligence algorithms, such as instruction recommendation algorithms, image processing algorithms, and image understanding algorithms. The CPU and GPU can be used to render and synthesize the image to be displayed on the screen 194.

[0175] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, photos, videos, and other data can be stored on the external memory card.

[0176] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the instructions stored in the internal memory 121, thereby causing the electronic device 100 to perform the photographic preview method, various functional applications, and data processing provided in some embodiments of this application. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system; it may also store one or more application programs (such as a gallery, contacts, etc.). The data storage area may store data (such as photos, contacts, etc.) created during the use of the electronic device 100.

[0177] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0178] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for applications on electronic device 100. The modem processor can include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The low-frequency baseband signal, after processing by the baseband processor, is transmitted to the application processor. Wireless communication module 160 can provide wireless communication solutions, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies, for applications on electronic device 100.

[0179] Electronic device 100 can realize display functions through GPU, display screen 194, and application processor. GPU is a microprocessor for image processing, connected to display screen 194 and application processor. GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute instructions to generate or modify display information.

[0180] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0181] In some embodiments, after generating a drawing command for creating interface elements, the CPU can pass the drawing command to the GPU and instruct the GPU to render according to the drawing command. Here, the drawing command can be used to instruct how to use drawing elements, such as graphics, text, and images, to draw specific interface elements in the user interface.

[0182] The components in the electronic device 100 can be implemented as hardware modules, software modules, or a combination of software and hardware modules. Figure 10 The illustrated device structure does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements.

[0183] The above is merely an exemplary description of the hardware structure of the electronic device 100. In some embodiments, the electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0184] Figure 11 This is an exemplary schematic diagram of the software architecture of an electronic device 100.

[0185] The software architecture of electronic device 100 can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem level depending on the deployment scenario of different device forms. Each subsystem can also be tailored, added, or combined at the functional level.

[0186] The kernel layer can include the Kernel Abstraction Layer (KAL), kernel subsystems, and driver subsystems. The KAL provides basic kernel capabilities to upper layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management. The kernel subsystems can support the selection of suitable OS kernels for different resource-constrained devices, and are not limited to Linux, HarmonyOS, or LiteOS (LiteOperating System).

[0187] The driver subsystem may include a driver framework. The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: a display driver, and a camera system service layer comprising the system's core capabilities, providing services to applications through the framework layer. In some embodiments, the driver subsystem may utilize a rendering engine object to drive the GPU to render user interface elements.

[0188] The system service layer may include a set of subsystems such as the system basic capability subsystem, the basic software service subsystem, the enhanced software service subsystem, and the hardware service subsystem.

[0189] The system's basic capabilities subsystem set provides fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include a distributed soft bus, distributed data management, distributed task scheduling, and the Ark multi-language runtime, as well as a multi-modal input subsystem, a graphics subsystem, a security subsystem, and an AI subsystem.

[0190] The graphics subsystem provides graphics rendering and display output functions. Internally, it provides a smooth and efficient display experience by making reasonable use of system hardware resources. The graphics subsystem may include modules such as UI components, layout, animation, input events, window management, rendering service, and drawing. In some embodiments, the rendering service module can provide the drawing capabilities of the UI framework, converting the attribute descriptions of interface elements into drawing parameters. In some embodiments, the drawing module can provide standardized interfaces within the graphics subsystem to complete basic functions such as 2D graphics drawing, 3D graphics drawing, and drawing engine management.

[0191] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices. Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices. The distributed soft bus provides communication capabilities for seamless interconnection between multiple devices, including WLAN service capabilities, Bluetooth service capabilities, soft bus, inter-process communication, and StarFlash communication capabilities. The Ark Multi-Language Runtime is a unified compilation and runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0192] The basic software service subsystem set can provide public and general software services. This set may include an event notification subsystem, a telephone service subsystem, a multimedia subsystem, etc.

[0193] The enhanced software service subsystem suite can provide differentiated capability-enhancing software services for different devices, including proprietary business subsystems for smart screens, wearables, and IoT.

[0194] The hardware service subsystem set can provide hardware services, including location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, and so on.

[0195] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes the ArkUI framework, the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of an Ability). Different devices may have different operating systems, and therefore support different APIs.

[0196] The ArkUI framework provides a complete infrastructure for UI development of system applications, including UI functions (such as components, layouts, animations, and interactive events) and real-time interface preview tools. The ArkUI framework provides at least two development paradigms, including a declarative development paradigm based on ArkTS (referred to as "declarative development paradigm") and a JS-compatible web-like development paradigm (referred to as "web-like development paradigm").

[0197] The application layer can include system applications and extended / third-party applications. System applications can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party applications can include social applications, travel applications, etc. In some embodiments, the application layer can pass the attribute descriptions of interface elements to the system service layer, and the system service layer can parse the drawing parameters of the interface elements based on the attribute descriptions.

[0198] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the interface drawing method provided in this application.

[0199] This application also provides a computer program product that, when run, can implement the interface drawing method provided in this application.

[0200] This application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the interface drawing method provided in this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0201] The steps of the method described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, read-only optical discs (CD-ROMs), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a transceiver or relay device. Alternatively, the processor and storage medium can exist as discrete components in a wireless access network device or user equipment.

[0202] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0203] In this application's embodiments, the term "user interface (UI)" refers to the medium through which an application or operating system interacts and exchanges information with the user. It converts the internal form of information into a form acceptable to the user. The user interface is source code written in specific computer languages ​​such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on an electronic device, ultimately presenting content that the user can recognize. Examples include images, text, buttons, and other controls. Controls, also called widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, images, and text. The attributes and content of controls in the interface are defined using tags or nodes, such as XML tags. <textview> 、 <imgview> 、 <videoview>Nodes define the controls contained in the interface. A node corresponds to a control or property in the interface, and after parsing and rendering, the node is presented as the content visible to the user. In addition, many applications, such as hybrid applications, often contain web pages within their interfaces. A web page, also known as a webpage, can be understood as a special control embedded in the application interface. Web pages are source code written in a specific computer language, such as Hypertext Markup Language (HTML) or Cascading Style Sheets (CSS). The source code of a webpage can be loaded and displayed as user-readable content by a browser or a webpage display component with browser-like functionality. The specific content contained in a webpage is also defined through tags or nodes in the webpage's source code; for example, HTML uses tags or nodes to define the content. 、 、 <video> 、 <canvas>To define the elements and attributes of a webpage.

[0204] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A method for drawing an interface, characterized in that, include: The application process parses the interface elements to obtain the drawing parameters of the interface elements. The interface elements are the elements of the user interface to be drawn. The drawing parameters of the interface elements include shape parameters and style parameters. The drawing parameters of the interface elements are passed from the application process to the rendering service process; Based on the shape parameters, a primitive object is constructed through the rendering service process; Based on the style parameters, a style object is constructed through the rendering service process; Using the primitive object and the style object, draw the user interface corresponding to the interface element.

2. The method as described in claim 1, characterized in that, The step of passing the drawing parameters of the interface elements from the application process to the rendering service process includes: The drawing parameters of the interface elements are encoded in a first data format; the first data format includes a shape parameter field and a style parameter field, wherein the shape parameter field is used to fill the shape parameters and the style parameter field is used to fill the style parameters. The drawing parameters of the interface elements, encoded in the first data format, are passed from the application process to the rendering service process.

3. The method as described in claim 2, characterized in that, Before constructing the primitive object through the rendering service process based on the shape parameters, the process further includes: The shape parameters and style parameters are parsed from the drawing parameters of the interface elements encoded in the first data format.

4. The method as described in claim 2 or 3, characterized in that, The style parameter field includes a first mask, which includes multiple bits, each bit corresponding to a style parameter. The length of the style parameter field is positively correlated with the value of the first mask. The bit corresponding to each style parameter is used to indicate whether the style parameter field includes the field of the corresponding style parameter.

5. The method as described in claim 4, characterized in that, The bits corresponding to each style parameter are used to indicate whether the style parameter field includes the field of the corresponding style parameter, specifically including: When the bit corresponding to a style parameter is a first value, the style parameter field includes the field of the style parameter; when the bit corresponding to a style parameter is not a first value, the style parameter field does not include the field of the style parameter.

6. The method as described in claim 5, characterized in that, The style parameters include a first style parameter, which includes: color, anti-aliasing, and blending mode; When the value of the first style parameter in the drawing parameters of the interface element is not the default value, the bit corresponding to the first style parameter in the first mask is the first value. When the first style parameter takes the default value in the drawing parameters of the interface element, the bit corresponding to the first style parameter in the first mask is not the first value.

7. The method as described in claim 6, characterized in that, The style parameters also include a second style parameter, which includes one or more of the following: shader effects, color filtering, mask filtering, image filtering, color gamut, path effects, stroke, and mixer. The fields of the first style parameter are used to fill the data of the first style parameter. The fields of the second style parameter include an index field and a data field. The index field is used to fill the address of the data field, and the data field is used to fill the data of the second style parameter.

8. The method according to any one of claims 4-7, characterized in that, The first data format further includes one or more of the following: a shape size field and a style size field; wherein the shape size field is used to fill the length of the shape parameter field, and the style size field is used to fill the length of the style parameter field.

9. The method according to any one of claims 1-8, characterized in that, The application process and the rendering service process run on the same device. The step of passing the drawing parameters of the interface elements from the application process to the rendering service process specifically includes: The drawing parameters of the interface elements are passed from the application process to the rendering service process through inter-process communication; the inter-process communication includes: inter-process communication based on shared memory.

10. The method according to any one of claims 1-8, characterized in that, The application process runs on the first device, the rendering service process runs on the second device, and a first communication connection is established between the first device and the second device; The step of passing the drawing parameters of the interface elements from the application process to the rendering service process specifically includes: Through the first communication connection, the first device transmits the drawing parameters of the interface elements to the second device.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-10.

12. A communication system, comprising a first device and a second device, characterized in that, The first device is used to parse the interface elements in the user interface to obtain the drawing parameters of the interface elements, which include shape parameters and style parameters. The first device is also used to pass the drawing parameters of the interface elements to the second device; The second device is used to construct a primitive object based on the shape parameters; The second device is also used to construct a style object based on the style parameters; The second device is also used to draw a user interface corresponding to the interface element using the primitive object and the style object.

13. The system as described in claim 12, characterized in that, The first device is further configured to transmit the drawing parameters of the interface elements to the second device, specifically including: The first device is specifically used to encode the drawing parameters of the interface element in a first data format, and to transmit the drawing parameters of the interface element encoded in the first data format to the second device. The first data format includes a shape parameter field and a style parameter field. The shape parameter field is used to fill the shape parameter, and the style parameter field is used to fill the style parameter.

14. The system as described in claim 13, characterized in that, Before the second device is used to construct a primitive object based on the shape parameters, the second device is also used to parse the shape parameters and the style parameters from drawing parameters encoded in the first data format.

15. The system as described in claim 14, characterized in that, The style parameter field includes a first mask, which includes multiple bits, each bit corresponding to a style parameter. The length of the style parameter field is positively correlated with the value of the first mask. The bit corresponding to each style parameter is used to indicate whether the style parameter field includes the field of the corresponding style parameter.

16. The system as described in claim 15, characterized in that, The bits corresponding to each style parameter are used to indicate whether the style parameter field includes the field of the corresponding style parameter, specifically including: When the bit corresponding to a style parameter is a first value, the style parameter field includes the field of the style parameter; when the bit corresponding to a style parameter is not a first value, the style parameter field does not include the field of the style parameter.

17. The system as claimed in claim 16, characterized in that, The style parameters include a first style parameter, which includes: color, anti-aliasing, and blending mode; When the value of the first style parameter in the drawing parameters of the interface element is not the default value, the bit corresponding to the first style parameter in the first mask is the first value. When the first style parameter takes the default value in the drawing parameters of the interface element, the bit corresponding to the first style parameter in the first mask is not the first value.

18. The system as claimed in claim 17, characterized in that, The style parameters also include a second style parameter, which includes one or more of the following: shader effects, color filtering, mask filtering, image filtering, color gamut, path effects, stroke, and mixer. The fields of the first style parameter are used to fill the data of the first style parameter. The fields of the second style parameter include an index field and a data field. The index field is used to fill the address of the data field, and the data field is used to fill the data of the second style parameter.

19. The system according to any one of claims 13-18, characterized in that, The first data format further includes one or more of the following: a shape size field and a style size field; wherein the shape size field is used to fill the length of the shape parameter field, and the style size field is used to fill the length of the style parameter field.

20. The system according to any one of claims 12-19, characterized in that, A first communication connection is established between the first device and the second device; the first device is further configured to transmit the drawing parameters of the interface elements to the second device, specifically including: The first device is used to transmit the drawing parameters of the interface elements to the second device through the first communication connection.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

Images