User interface implementation method and device

Abstract

The embodiments of the present application provide a method and device for implementing a user interface, which relate to the field of terminal technology. The application installation package of the first application includes a first description file and a second description file for performing interface description and interface behavior definition on the UI of the first application; the first description file and the second description file use different interface description languages; wherein the first description file uses a first interface description language, and the second description file uses a second interface description language. When the electronic device runs the first application; the first UI engine reads, parses and executes the first description file to generate the first part of the first UI; the second UI engine reads, parses and executes the second description file to generate the second part of the first UI. The first UI engine can be a general operating system engine, and the second UI engine is not related to the operating system platform, can adapt to a variety of operating system platforms, and has low technical implementation difficulty.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on August 25, 2020, with application number 202010862489.9 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference in this application.

[0002] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on September 30, 2020, with application number 202011064544.6 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference in this application.

[0003] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office on October 22, 2020, with application number 202011141010.9 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference in this application.

[0004] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 22, 2020, with application number 202011142718.6 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference in this application.

[0005] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on November 30, 2020, with application number 202011381146.7 and application name “A User Interface Implementation Method and Device”, all contents of which are incorporated by reference in this application.

[0006] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on November 30, 2020, with application number 202011384490.1 and application name “A User Interface Implementation Method and Device”, all contents of which are incorporated by reference in this application. Technical Field

[0007] The present application relates to the field of terminal technology, and in particular to a method and device for implementing a user interface. Background Art

[0008] Developers usually develop applications (applications, Apps) based on a certain operating system (operator system, OS) platform. In the process of App development, a very important task is to develop the user interface (user interface, UI) of the App. Usually, developers use the software development kit (software development kit, SDK) provided by the OS platform to develop the UI of the App. UI development mainly includes interface description and interface behavior definition. Interface description refers to the use of interface description language to describe the layout of the UI, the controls used, and the visual style of the layout and controls. Interface behavior definition refers to the use of interface description language to define interface behavior; interface behavior includes dynamic changes of UI and the response of electronic devices to dynamic changes of UI (such as response to user operations on the UI). Each OS platform has its corresponding interface description language; for example Use XML (extensible markup language) format, Use the embedded domain specific language (EDSL) built with Swift to describe the interface and define the interface behavior. The UI engine provided by the OS platform can interpret and execute the UI interface description language, render the UI and present it to the user. In addition, each OS platform also has a corresponding programming language to implement interface behavior, realize dynamic changes of the UI and respond to user operations on the UI; for example Using JAVA, Use Swift programming language to implement interface behaviors.

[0009] How to provide convenience for developers so that they can easily develop a UI that is compatible with the operating system and rich in functions is a problem that needs to be solved. Summary of the invention

[0010] The embodiment of the present application provides a method and device for implementing a user interface, which can provide rich UI programming capabilities and facilitate developers to develop a UI that is compatible with the operating system and has rich functions. To achieve the above purpose, the present application adopts the following technical solutions:

[0011] In the first aspect, the present application provides a user interface implementation method, including: an application installation package of a first application of an electronic device, the application installation package including a first description file and a second description file; wherein the first description file and the second description file are used to describe the interface and define the interface behavior of a first user interface UI of the first application; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language. The electronic device runs the first application; wherein the first UI engine of the electronic device reads, parses and executes the first description file to generate the first part of the first UI; the second UI engine of the electronic device reads, parses and executes the second description file to generate the second part of the first UI; the electronic device displays the first UI.

[0012] In this method, two different interface description languages ​​are supported for co-developing UI. The operating system of the electronic device includes two UI engines, which respectively parse and execute the two different interface description languages. One of the UI engines can be a general-purpose OS (e.g. ) can parse a general interface description language; another UI engine is an extended UI engine that is not related to the OS platform and can parse DSL. In this way, developers can use the basic interface description language to describe the UI layout, the included controls, etc.; and selectively use DSL to apply custom UI programming capabilities to some controls, add some dynamic effects to the UI, etc. The extended UI engine provided in the embodiment of the present application is not related to the OS platform, so it can adapt to multiple OS platforms, has low technical implementation difficulty, and is convenient for developers to use.

[0013] In a possible implementation, the first UI engine of the electronic device generates the first part of the first UI, including: the first UI engine of the electronic device generates one or more first controls in the first UI according to the first description file; wherein the one or more first controls have first UI programming capabilities.

[0014] The first control is a general-purpose OS (such as ) generated controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of controls; selecting controls, and entering text on controls.

[0015] In a possible implementation, the second UI engine of the electronic device generates the second part of the first UI, including: the second UI engine of the electronic device applies the second UI programming capability to one or more first controls according to the second description file. That is, the developer can use the custom interface description language to apply the custom UI programming capability to the controls generated by the general OS in the second description file, expand the capabilities of the general OS controls, and enrich the use effects of the general OS controls.

[0016] In a possible implementation, the second UI engine of the electronic device generates the second part of the first UI, including: the second UI engine of the electronic device generates one or more second controls in the first UI according to the second description file; wherein the one or more second controls have second UI programming capabilities.

[0017] The second control is a custom control provided by the OEM OS in the embodiment of the present application, supports custom second UI programming capabilities, and supports rich control effects.

[0018] In a possible implementation, the second UI programming capability includes at least one of: visual attribute capability, layout capability, unified interaction capability and motion effect capability.

[0019] Among them, layout capability is used to describe the layout of controls in the UI, such as the shape, position, and size of controls. Visual attribute capability is used to describe the visual attributes of controls, such as the color, grayscale, and other visual effects of controls. Unified interaction capability is used to provide control responses based on user behavior, such as performing searches based on the user's "confirmation" behavior. Animation effect capability is used to display animation effects on controls, such as displaying click rebound animation effects on controls.

[0020] In a possible implementation, the layout capability includes: at least one of stretching, hiding, wrapping, equal distribution, proportion and extension. Among them, stretching refers to the display capability of the width and height of the control to be enlarged or reduced according to different proportions; hiding refers to the display capability of the control to be visible or invisible in the display interface; wrapping refers to the display capability of the content in the control to be displayed in one or more lines in the display interface; equal distribution refers to the display capability of the control to be evenly distributed in the display interface; proportion refers to the ability of the control to occupy the total layout according to a specified percentage in a specified direction; extension refers to the ability of the control to be arranged and displayed in one direction on the UI.

[0021] In a possible implementation, if the second UI engine determines that the second description file exists, the first UI engine is triggered to read the first description file, and the second UI engine is triggered to read the second description file. In other words, the second UI engine controls the distribution process, triggering the first UI engine and the second UI engine to parse and execute the description file.

[0022] In a possible implementation, the first description file and the second description file are located in different paths in the application installation package, and the first UI engine and the second UI engine read the description files in different paths respectively according to a preset rule.

[0023] In a possible implementation, different tags are preset in the first description file and the second description file. The first UI engine and the second UI engine read the corresponding description files respectively according to the preset tags.

[0024] In a possible implementation, the second UI engine further performs a syntax check on the second interface description language; if the syntax check passes, the second UI engine parses and executes the second description file.

[0025] In a possible implementation, the second UI engine of the electronic device implements the mapping between the device event and the user behavior in the second description file; in response to the device event, the control action corresponding to the user behavior in the second description file is executed. In other words, the OEM OS can map the events triggered by electronic devices of different forms to the same user behavior (for example, mapping the mouse double-click event on the PC to the "confirmation" behavior, and mapping the finger single-click event on the mobile phone to the "confirmation" behavior), avoiding the developer from defining the correspondence between device events and user behaviors for electronic devices of different forms, which brings duplication of work; so that the same description file can be applied to electronic devices of various forms, reducing the difficulty of development and bringing convenience to developers.

[0026] In a possible implementation, the second UI engine includes a set of syntax and semantic specifications of the fields in the second description file, so that the developer can develop the UI on the OEM OS platform according to the syntax and semantic specifications of the OEM OS.

[0027] In a possible implementation, the first interface description language is an extensible markup language (xml), and the second interface description language is a domain-specific language (DSL).

[0028] In a second aspect, the present application provides a user interface implementation method, including: displaying a development interface of a first application; the development interface of the first application includes a first description file and a second description file; the first description file and the second description file are used to describe the interface and define the interface behavior of a first user interface UI of the first application; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; in response to a first operation input by a user, adding a description of a first part of the first UI to the first description file; in response to a second operation input by the user, adding a description of the second part of the first UI to the second description file; and generating an application installation package for the first application according to the first description file and the second description file.

[0029] In this method, developers can use two different interface description languages ​​to jointly develop UI. One of the languages ​​is a common OS (such as ) and the other language is the custom interface description language. Developers can use the basic interface description language to describe the UI layout, the included controls, etc., and selectively use DSL to apply custom UI programming capabilities to some controls, add some dynamic effects to the UI, etc. Since the custom interface description language is not related to the OS platform, it can adapt to multiple OS platforms, with low technical implementation difficulty, and is convenient for developers to use.

[0030] In a possible implementation, adding a description of the first part of the first UI in the first description file includes: adding a description of one or more first controls in the first UI in the first description file; and applying the first UI programming capability to the one or more first controls.

[0031] The first control is a general OS (such as ) supports controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of controls; selecting controls, and entering text on controls.

[0032] In a possible implementation, adding a description of the second part of the first UI in the second description file includes: adding a description of one or more first controls in the second description file; and applying the second UI programming capability to the one or more first controls.

[0033] That is, developers can use custom interface description languages ​​to apply custom UI programming capabilities to controls generated by the universal OS in the second description file, thereby expanding the capabilities of universal OS controls and enriching the usage effects of universal OS controls.

[0034] In a possible implementation, adding a description of the second part of the first UI in the second description file includes: adding a description of one or more second controls in the second description file; and applying the second UI programming capability to the one or more second controls. The second control is a custom control provided by the OEM OS in the embodiment of the present application, supports custom second UI programming capabilities, and supports rich control effects.

[0035] In a possible implementation, the second UI programming capability includes at least one of: visual attribute capability, layout capability, unified interaction capability and motion effect capability.

[0036] In a possible implementation, the layout capability includes at least one of stretching, hiding, folding, dividing, proportioning, and extending.

[0037] In a possible implementation, the first description file and the second description file are in different paths in the application installation package. In this way, the first UI engine and the second UI engine of the OEM OS can read files in different paths according to preset rules to obtain corresponding description files.

[0038] In a possible implementation, different tags are preset in the first description file and the second description file, so that the first UI engine and the second UI engine of the OEM OS can read the corresponding description files respectively according to the preset tags.

[0039] In a third aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which are used to perform an interface description and interface behavior definition for a first user interface UI of a first application, wherein the computer instructions include a first instruction stored in a first description file, and a second instruction stored in a second description file; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; the first instruction is used to describe a first part of the first UI, and the second instruction is used to describe the second part of the first UI.

[0040] In this method, developers use two different interface description languages ​​to jointly develop the UI. One of the interface description languages ​​is a common OS (such as ) and the other interface description language is the custom interface description language. Developers use the basic interface description language to describe the UI layout, the included controls, etc., and selectively use DSL to apply custom UI programming capabilities to some controls, add some dynamic effects to the UI, etc. Since the custom interface description language is not related to the OS platform, it can adapt to multiple OS platforms, with low technical implementation difficulty and convenient for developers to use.

[0041] In a possible implementation, the first instruction is specifically used to: describe one or more first controls in the first UI, and apply the first UI programming capability to the one or more first controls.

[0042] The first control is a general OS (such as ) supports controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of controls; selecting controls, and entering text on controls.

[0043] In one possible implementation, the second instruction is specifically used to: apply the second UI programming capability to one or more first controls. In another possible implementation, the second instruction is specifically used to: describe one or more second controls in the first UI, and apply the second UI programming capability to the one or more second controls.

[0044] That is, developers can use custom interface description languages ​​to apply custom UI programming capabilities to controls generated by the general OS in the second description file, thereby expanding the capabilities of general OS controls; they can also add custom controls with rich control effects.

[0045] In a possible implementation, the second UI programming capability includes: at least one of visual attribute capability, layout capability, unified interaction capability and motion effect capability. The layout capability includes: at least one of stretching, hiding, line breaking, equal division, proportion and extension.

[0046] In a possible implementation, the first description file and the second description file are in different paths in the computer-readable storage medium. In this way, the first UI engine and the second UI engine of the OEM OS can read files in different paths according to preset rules to obtain corresponding description files.

[0047] In a possible implementation, different tags are preset in the first description file and the second description file, so that the first UI engine and the second UI engine of the OEM OS can read the corresponding description files respectively according to the preset tags.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned first aspect.

[0049] In a fifth aspect, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the first aspect above.

[0050] In a sixth aspect, the present application provides an electronic device comprising: a display screen, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the display screen and the memory, and the one or more computer programs are stored in the memory, and when the electronic device runs the first application, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the second aspect.

[0051] The embodiment of the present application provides a method and device for implementing a user interface, which can realize one-time development and multi-device deployment; that is, develop a set of interface description files suitable for various types of electronic devices; and reduce the development difficulty of developers. To achieve the above purpose, the present application adopts the following technical solutions:

[0052] In the seventh aspect, the present application provides a method for implementing a user interface, including: a first electronic device and a second electronic device respectively download an application installation package of a first application from a server; and respectively install the application installation package. The application installation package includes a description file and a resource file; wherein the description file is used to perform an interface description and an interface behavior definition for a first UI of the first application; and the resource file includes resources used to generate the UI of the first application. The first electronic device reads a first code corresponding to a device type of the first electronic device in the description file, and uses the resources of the resource file according to the definition of the first code to generate the first UI of the first electronic device; the second electronic device reads a second code corresponding to a device type of the second electronic device in the description file, and uses the resources of the resource file according to the definition of the second code to generate the first UI of the second electronic device; the device type of the first electronic device is different from the device type of the second electronic device. Among them, the device types of electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screens, car computers, etc.

[0053] In this method, different types of electronic devices read the same description file of the same UI and present different UI layouts. It is possible to develop a set of description files suitable for various types of electronic devices, reducing the development difficulty for developers.

[0054] In combination with the seventh aspect, in a possible implementation, the method also includes: the first electronic device generates a first control in a first UI of the first electronic device according to a definition of a third code in the description file, and the first control in the first UI of the first electronic device has control properties customized by the operating system of the first electronic device; the third electronic device generates a first control in a first UI of the third electronic device according to a definition of the third code in the description file, and the first control in the first UI of the third electronic device has control properties of a general operating system; wherein the third code is part or all of the first code.

[0055] In this method, the description file defines that the first control supports the control attributes customized by the operating system. The operating system of the first electronic device provides customized control attributes, and the first control in the first UI of the first electronic device has the control attributes customized by the operating system of the first electronic device; the third electronic device supports a general operating system (such as ), the first control in the first UI of the third electronic device has the control properties of the general operating system. In this way, the same description file can be successfully run in different operating systems, achieving cross-operating system platform operation and reducing the difficulty of development for developers.

[0056] In an eighth aspect, the present application provides a user interface implementation method, including: a first electronic device downloads an application installation package of a first application; and installs the application installation package; wherein the application installation package includes a description file and a resource file; the description file is used to describe the interface and define the interface behavior of a first user interface UI of the first application; the resource file includes resources used to generate the UI of the first application. The first electronic device reads a first code corresponding to the device type of the first electronic device in the description file, and uses the resources of the resource file to generate the first UI of the first electronic device according to the definition of the first code.

[0057] In this method, the electronic device reads the code corresponding to the device type of the electronic device in the description file. In this way, different electronic devices can present different UI layouts when reading the same description file. It is possible to develop a set of description files that are applicable to various types of electronic devices, reducing the development difficulty for developers.

[0058] In combination with the eighth aspect, in a possible implementation method, the first electronic device generates a first control in the first UI of the first electronic device according to the definition of the third code in the description file, and the first control in the first UI of the first electronic device has control properties customized by the operating system of the first electronic device; wherein the third code is part or all of the first code.

[0059] In the method, the description file defines that the first control supports control properties customized by the operating system. The operating system of the first electronic device provides customized control properties, and the first control in the first UI of the first electronic device has the control properties customized by the operating system of the first electronic device.

[0060] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the operating system of the first electronic device includes a custom UI programming capability, and the custom UI programming capability is used to provide custom control properties of the operating system of the first electronic device.

[0061] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the customized control attributes include: at least one of: visual attributes, layout attributes, interaction attributes, motion effect attributes, and software and hardware dependency attributes.

[0062] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the layout attributes include: at least one of stretching, hiding, wrapping, dividing, proportioning, and extending.

[0063] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the first UI of the first electronic device includes a second control, and the second control has control attributes of a general operating system. That is, the first UI generated by the first electronic device according to the description file may include controls with control attributes customized by the operating system of the first electronic device, and controls with control attributes of a general operating system. A richer variety of controls are provided.

[0064] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the first UI of the first electronic device includes a third control, and the third control has a control attribute customized in the first application. In this method, the developer can customize the control attribute belonging to the first application in the file of the installation package to make the UI richer.

[0065] In combination with the seventh aspect or the eighth aspect, in a possible implementation manner, the description file includes a fourth code for defining a correspondence between a control attribute of a fourth control in a first UI of a first electronic device and first data in an operating system of the first electronic device. The method further includes: the first electronic device receives a first input from a user on the fourth control; and modifies a value of the first data according to the first input.

[0066] In this method, the developer defines the corresponding relationship between the control properties of the control and the background data in the operating system in the description file; the UI engine of the electronic device implements the function of modifying the background data according to the user input. This avoids the developer from describing the implementation of modifying the background data according to the user input in the description file, and reduces the development difficulty of the developer.

[0067] In combination with the seventh aspect or the eighth aspect, in a possible implementation manner, the method further includes: a control property of a fourth control in a first UI of the first electronic device changes as first data in an operating system of the first electronic device changes.

[0068] In this method, the developer defines the corresponding relationship between the control properties of the control and the background data in the operating system in the description file; the UI engine of the electronic device implements the control properties of the control to change with the change of the background data in the operating system of the electronic device. The control in the UI can change with the change of the electronic device parameters, and avoids the developer's description in the description file that the control properties of the control change with the change of the electronic device parameters, reducing the development difficulty of the developer.

[0069] In a ninth aspect, the present application provides a method for implementing a user interface, including: displaying a development interface of a first application; the development interface of the first application includes a description file for performing an interface description and interface behavior definition on a first user interface UI of the first application; in response to a first operation input by a user, adding a first code corresponding to a device type of a first electronic device to the description file; in response to a second operation input by a user, adding a second code corresponding to a device type of a second electronic device to the description file; generating an application installation package for the first application according to the description file. The device type of the first electronic device is different from the device type of the second electronic device. The device types of electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screens, car computers, etc.

[0070] In this method, a description file includes codes corresponding to different types of electronic devices. Different types of electronic devices can present different UI layouts by reading the same description file of the same UI. It is possible to develop a set of description files suitable for various types of electronic devices, reducing the development difficulty for developers.

[0071] In conjunction with the ninth aspect, in a possible implementation, the application installation package of the first application also includes a resource file, and the resource file includes resources used to generate a UI of the first application.

[0072] In conjunction with the ninth aspect, in a possible implementation, the description file includes a third code defining that the first control has control properties customized by the operating system of the first electronic device, and a fourth code defining that the second control has control properties of a general operating system. In other words, the first UI generated by the electronic device according to the description file may include controls with control properties customized by the operating system of the first electronic device, and controls with control properties of a general operating system. A richer variety of controls are provided.

[0073] In conjunction with the ninth aspect, in a possible implementation, the control attributes customized by the operating system of the first electronic device include: at least one of: visual attributes, layout attributes, interaction attributes, motion effect attributes, and software and hardware dependency attributes.

[0074] In conjunction with the ninth aspect, in a possible implementation, the layout attributes include: at least one of stretching, hiding, wrapping, dividing, proportioning, and extending.

[0075] In a tenth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned ninth aspect.

[0076] In an eleventh aspect, the present application provides a computer-readable storage medium, including computer instructions, the computer instructions are used to describe the interface and define the interface behavior of a first user interface UI of a first application; wherein the computer instructions include a first code corresponding to the device type of a first electronic device and a second code corresponding to the device type of a second electronic device; wherein the device type of the first electronic device is different from the device type of the second electronic device. The device types of the electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screens, car computers, etc.

[0077] In conjunction with the eleventh aspect, in a possible implementation, the computer instructions also include generating resources used by the UI of the first application.

[0078] In combination with the eleventh aspect, in a possible implementation, the computer instructions also include a third code defining that the first control has control properties customized by the operating system of the first electronic device, and a fourth code defining that the second control has control properties of a general operating system.

[0079] In the twelfth aspect, the present application provides an electronic device, comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the eighth aspect.

[0080] In the thirteenth aspect, the present application provides an electronic device comprising: a display screen, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the display screen and the memory, and the one or more computer programs are stored in the memory, and when the electronic device runs the first application, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the ninth aspect.

[0081] The embodiment of the present application provides a method and device for implementing a user interface, which supports displaying various layout modes and control types on the UI of an application widget, facilitates users to use the application widget, and improves user experience. To achieve the above purpose, the present application adopts the following technical solutions:

[0082] In a fourteenth aspect, the present application provides a user interface implementation method, including: a first application process of an electronic device reads a component interface description file, generates first widget UI data based on the component interface description file, and binds controls in the first widget UI data to background data in an operating system of the electronic device; wherein the component interface description file is used to perform interface description and interface behavior definition on a first UI of an application widget of a first application; then, the first application process sends first data to an application widget process; the application widget process receives the first data, obtains first widget UI data based on the first data, and displays the first UI of the application widget according to the first widget UI data.

[0083] In this method, both the application process and the application widget process generate widget UI data according to the component interface description file. The application process binds the controls in the widget UI data to the background data, and the application widget process displays the widget UI data as the application widget UI. In this way, the developer can define various types of controls in the component interface description file so that the UI of the application widget supports various types of controls. When the user operates on the UI of the application widget, the application process can execute the corresponding business logic according to the correspondence between the controls in the widget UI data and the background data.

[0084] In one possible implementation, the first application process sends a component interface description file to the application widget process; the application widget process receives the component interface description file, generates first widget UI data based on the component interface description file, and displays the first UI of the application widget according to the first widget UI data.

[0085] In a possible implementation, the first application process sends first widget UI data to the application widget process; the application widget process receives the first widget UI data, and displays the first UI of the application widget according to the first widget UI data.

[0086] In conjunction with the fourteenth aspect, in a possible implementation, the method further includes: the first application process generates a first control in the first widget UI data according to the definition of the first code in the component interface description file, and the first control has the native control attributes of the operating system of the electronic device. Among them, the native controls of the operating system include: input box, check box, slide selector, scroll view, radio button, rating bar, search box, drag bar, or switch, etc.

[0087] In other words, developers can define various native operating system control properties in the component interface description file, so that the UI of the application widget supports various native operating system controls.

[0088] In conjunction with the fourteenth aspect, in a possible implementation, the method further includes: the first application process generates a second control in the first widget UI data according to the definition of the second code in the component interface description file, and the second control has custom control properties in the operating system of the electronic device. The custom control properties include: at least one of visual properties, layout properties, interaction properties, motion properties, and software and hardware dependency properties. The layout properties include: at least one of stretching, hiding, folding, equal distribution, proportion, and extension.

[0089] That is to say, the developer can define various custom control properties in the operating system in the component interface description file, so that the UI of the application widget supports various custom controls in the operating system.

[0090] In combination with the fourteenth aspect, in a possible implementation, the component interface description file includes a third code for defining the correspondence between the control properties of the third control in the first UI of the application widget and the first data in the operating system of the electronic device, and the method also includes: the electronic device receives a first input from the user on the third control; and modifies the value of the first data based on the first input.

[0091] In combination with the fourteenth aspect, in a possible implementation manner, the control property of the third control in the first UI of the application widget changes as the first data in the operating system of the electronic device changes.

[0092] In combination with the fourteenth aspect, in a possible implementation, the method also includes: the electronic device downloads an application installation package of the first application from the server; the application installation package includes a component interface description file; and the electronic device uses the application installation package to install the first application.

[0093] In this method, the component interface description file is obtained from the application installation package.

[0094] In a fifteenth aspect, the present application provides a method for implementing a user interface, including: displaying a development interface of a first application; the development interface of the first application includes a component interface description file; the component interface description file is used to perform an interface description and interface behavior definition for a first UI of an application widget of the first application. In response to a first operation input by a user, a first code defining a first control in the first widget UI is added to the component interface description file; wherein the first control has control properties native to the operating system; controls native to the operating system include: an input box, a check box, a sliding selector, a scroll view, a radio button, a rating bar, a search box, a drag bar, or a switch, etc. An application installation package for the first application is generated according to the component interface description file.

[0095] In this method, the developer can define in the component interface description file that the control has the control properties native to the operating system, so that the UI of the application widget running on the electronic device includes various controls with the control properties native to the operating system.

[0096] In conjunction with the fifteenth aspect, in a possible implementation, the method further includes: in response to a second operation input by the user, adding a second code defining a second control in the first widget UI in the component interface description file, wherein the second control has custom control properties in the operating system; the custom control properties include: at least one of visual properties, layout properties, interaction properties, dynamic properties, and software and hardware dependency properties. Among them, the layout properties include: at least one of stretching, hiding, line breaking, equal division, proportion, and extension.

[0097] In this method, the developer can define in the component interface description file that the control has control properties customized by the operating system, so that the UI of the application widget running on the electronic device includes various controls with control properties customized by the operating system.

[0098] In the sixteenth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned fifteenth aspect.

[0099] In the seventeenth aspect, the present application provides a computer-readable storage medium, including computer instructions, which are used to describe the interface and define the interface behavior of a first user interface UI of an application widget of a first application; wherein the computer instructions include a first code for generating a first control in the first widget UI, and the first control has control properties native to the operating system; controls native to the operating system include: input boxes, check boxes, sliding selectors, scroll views, radio buttons, rating bars, search boxes, drag bars, or switches, etc.

[0100] In conjunction with the seventeenth aspect, in a possible implementation, the computer instructions further include a second code for generating a second control in the first widget UI, wherein the second control has control properties customized in the operating system; the customized control properties include: at least one of visual properties, layout properties, interaction properties, dynamic properties, and software and hardware dependency properties. Among them, the layout properties include: at least one of stretching, hiding, folding, equal division, proportion, and extension.

[0101] In an eighteenth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes any one of the methods described in the fourteenth aspect.

[0102] In the nineteenth aspect, the present application provides an electronic device, comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes the method described in any one of the above fourteenth or fifteenth aspects.

[0103] The embodiment of the present application provides a method and device for implementing a user interface, which supports projecting various UIs on a control device to an IoT device for playback, thereby improving the user experience. To achieve the above purpose, the present application adopts the following technical solutions:

[0104] In the twentieth aspect, the present application provides a user interface implementation method, including: a first electronic device reads a first playback end interface description file of a first application, and generates first playback end UI data according to the first playback end interface description file, and binds the controls in the first playback end UI data to the background data in the operating system of the first electronic device; wherein the first playback end interface description file is used to perform interface description and interface behavior definition for the first playback end UI of the first application played on the second electronic device; the first electronic device sends the first data to the second electronic device; the second electronic device receives the first data, obtains the first playback end UI data according to the first data, and displays the first playback end UI according to the first playback end UI data.

[0105] In this method, both the control device and the player generate player UI data according to the player interface description file. The control device binds the controls in the player UI data to the background data, and the player displays the player UI data as the player UI. In this way, developers can define various UIs in the player interface description file to make the player UI richer. Different UI layouts can also be defined for players of different device types so that the player UI matches the size and shape of the player screen. When the user operates on the player UI, the control device can execute the corresponding business logic according to the correspondence between the controls in the player UI data and the background data.

[0106] In a possible implementation, the first electronic device sends a first playback end interface description file to the second electronic device, and the second electronic device generates first playback end UI data according to the first playback end interface description file and displays the first playback end UI according to the first playback end UI data.

[0107] In a possible implementation, the first electronic device sends first playback end UI data to the second electronic device, and the second electronic device receives the first playback end UI data and displays the first playback end UI according to the first playback end UI data.

[0108] In combination with the twentieth aspect, in a possible implementation, the method also includes: the second electronic device receives a first operation of a user on a first playback end UI; in response to the first operation of the user on the first playback end UI, the second electronic device sends a first instruction to the first electronic device; the first electronic device receives the first instruction, reads a second playback end interface description file, generates second playback end UI data according to the second playback end interface description file, and binds controls in the second playback end UI data to background data in the operating system of the first electronic device; wherein the second playback end interface description file is used to perform interface description and interface behavior definition for the second playback end UI that plays the first application on the second electronic device; the first electronic device sends the second playback end interface description file to the second electronic device; the second electronic device receives the second playback end interface description file, generates second playback end UI data according to the second playback end interface description file, and displays the second playback end UI according to the second playback end UI data.

[0109] In this method, the user can directly operate the player UI on the playback device, control the device to execute the business logic corresponding to the operation, and send the playback device the playback interface description file corresponding to the updated playback UI. The playback device generates an updated playback UI according to the updated playback interface description file. This allows the playback UI to be directly operated on the playback device and successfully switched.

[0110] In combination with the twentieth aspect, in a possible implementation, the method also includes: the second electronic device receives a first operation of a user on a first playback end UI; in response to the first operation of the user on the first playback end UI, the second electronic device sends a first instruction to the first electronic device; the first electronic device receives the first instruction and reads a second playback end interface description file; the second playback end interface description file is used to perform interface description and interface behavior definition for the second playback end UI that plays the first application on the second electronic device; the first electronic device generates second playback end UI data based on the second playback end interface description file, and binds the controls in the second playback end UI data to the background data in the operating system of the first electronic device; the first electronic device sends the second playback end UI data to the second electronic device; the second electronic device receives the second playback end UI data, and displays the second playback end UI according to the second playback end UI data.

[0111] In this method, the user can directly operate the playback UI on the playback device, control the device to execute the business logic corresponding to the operation, and send updated playback UI data to the playback device, and the playback device updates the playback UI according to the updated playback UI data. This allows the user to directly operate the playback UI on the playback device and successfully switch the playback UI.

[0112] In combination with the twentieth aspect, in a possible implementation, the method also includes: the first electronic device downloads an application installation package of the first application from the server; the application installation package includes a first playback interface description file and a resource file; the resource file includes resources used to generate the playback UI of the first application; the first electronic device uses the application installation package to install the first application.

[0113] In combination with the twentieth aspect, in a possible implementation, the method further includes: the first electronic device reads a first code corresponding to the device type of the third electronic device in the first playback end interface description file, and generates third playback end UI data using resources of the resource file according to the definition of the first code; the first electronic device reads a second code corresponding to the device type of the fourth electronic device in the first playback end interface description file, and generates fourth playback end UI data using resources of the resource file according to the definition of the second code; wherein the device type of the fourth electronic device is different from the device type of the third electronic device. The first electronic device sends the first playback end interface description file and the resource file to the third electronic device and the fourth electronic device respectively; the third electronic device generates third playback end UI data using resources of the resource file according to the definition of the first code corresponding to the device type of the third electronic device in the first playback end interface description file; the first playback end UI is displayed according to the third playback end UI data; the fourth electronic device generates fourth playback end UI data using resources of the resource file according to the definition of the second code corresponding to the device type of the fourth electronic device in the first playback end interface description file; the first playback end UI is displayed according to the fourth playback end UI data.

[0114] In this method, different types of playback devices read the same playback end interface description file of the same UI and present different playback end UI layouts. It is possible to develop a set of playback end interface description files that are applicable to various types of playback devices, reducing the development difficulty for developers.

[0115] In conjunction with the twentieth aspect, in a possible implementation, the method further includes: the first electronic device reads a first code corresponding to the device type of the third electronic device in the first playback end interface description file, and generates third playback end UI data using resources of the resource file according to the definition of the first code; the first electronic device reads a second code corresponding to the device type of the fourth electronic device in the first playback end interface description file, and generates fourth playback end UI data using resources of the resource file according to the definition of the second code; wherein the device type of the fourth electronic device is different from the device type of the third electronic device. The first electronic device sends the third playback end UI data to the third electronic device; the third electronic device displays the first playback end UI according to the third playback end UI data; the first electronic device sends the fourth playback end UI data to the fourth electronic device; the fourth electronic device displays the first playback end UI according to the fourth playback end UI data.

[0116] In this method, different types of playback devices present different playback UI layouts according to the same playback interface description file of the same UI. It is possible to develop a set of playback interface description files suitable for various types of playback devices, reducing the development difficulty for developers.

[0117] In conjunction with the twentieth aspect, in a possible implementation, the method further includes: the first electronic device generates a first control in the first playback end UI according to the definition of the third code in the first playback end interface description file, and the first control has control properties customized by the operating system of the first electronic device; wherein the control properties customized by the operating system of the first electronic device include: at least one of visual properties, layout properties, interaction properties, dynamic properties, and software and hardware dependency properties. The layout properties include: at least one of stretching, hiding, line breaking, equal division, proportion, and extension.

[0118] In conjunction with the twentieth aspect, in a possible implementation, the method further includes: the first electronic device displays the first playback end UI according to the first playback end UI data. In this method, the control device and the playback device synchronously play the playback end UI, which can realize mirroring screen projection, and the control device and the playback device work together.

[0119] In the twenty-first aspect, the present application provides a user interface implementation method, including: displaying the development interface of a first application; the development interface of the first application includes a playback end interface description file; the playback end interface description file is used to perform interface description and interface behavior definition for the playback end UI of the first application played on the playback end; in response to a first input from a user, adding a first code corresponding to a device type of a first electronic device to the playback end interface description file; in response to a second input from a user, adding a second code corresponding to a device type of a second electronic device to the playback end interface description file; the device type of the first electronic device is different from the device type of the second electronic device; and generating an application installation package for the first application according to the playback end interface description file.

[0120] In this method, a player interface description file includes codes corresponding to different types of player devices. Different types of player devices can present different player UI layouts by reading the same player interface description file of the same UI. It is possible to develop a set of player interface description files suitable for various types of player devices, reducing the development difficulty for developers.

[0121] In combination with the twenty-first aspect, in a possible implementation, the application installation package of the first application also includes a resource file, and the resource file includes resources used to generate a playback-end UI of the first application.

[0122] In conjunction with the twenty-first aspect, in a possible implementation, the playback end interface description file includes a third code defining that a first control in the first playback end UI has control attributes customized by an operating system of a first electronic device; the control attributes customized by the operating system of the first electronic device include: at least one of visual attributes, layout attributes, interaction attributes, dynamic effect attributes, and software and hardware dependency attributes. The layout attributes include: at least one of stretching, hiding, line folding, equal division, proportion, and extension.

[0123] In aspect 22, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device may execute any one of the methods described in aspect 21 above.

[0124] In a twenty-third aspect, the present application provides a computer-readable storage medium, including computer instructions, the computer instructions are used to perform interface description and interface behavior definition for a first playback end UI of a first application; wherein the computer instructions include a first code corresponding to a device type of a first electronic device and a second code corresponding to a device type of a second electronic device; wherein the device type of the first electronic device is different from the device type of the second electronic device. The device types of the electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptop computers, netbooks, large screens, car computers, etc.

[0125] In combination with the twenty-third aspect, in one possible implementation, the computer instructions also include generating resources used by the playback-end UI of the first application.

[0126] In conjunction with the twenty-third aspect, in a possible implementation, the computer instructions further include a third code defining that the first control in the first playback end UI has control properties customized by the operating system of the first electronic device; the control properties customized by the operating system of the first electronic device include: at least one of visual properties, layout properties, interaction properties, dynamic properties, and software and hardware dependency properties. The layout properties include: at least one of stretching, hiding, folding, equal distribution, proportion, and extension.

[0127] In a twenty-fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes any one of the methods described in the twentieth aspect.

[0128] In aspect 25, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor can execute one or more computer programs stored in the memory so that the electronic device executes the method described in any one of aspect 20 or aspect 21.

[0129] It can be understood that the electronic devices and computer-readable storage media provided in the above aspects are all applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0130] Figure 1 A schematic diagram of a scenario for implementing a method for a user interface provided in an embodiment of the present application;

[0131] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0132] Figure 3 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0133] Figure 4 A schematic diagram of a method for implementing a user interface;

[0134] Figure 5 A schematic diagram of a method for implementing a user interface;

[0135] Figure 6 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0136] Figure 7 A schematic diagram of the architecture of a method for implementing a user interface provided in an embodiment of the present application;

[0137] Figure 8 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0138] Fig. 9 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0139] Fig.10 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0140] Fig.11A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0141] Fig.12 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0142] Fig.13 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0143] Fig.14 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0144] Fig.15 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0145] Fig.16 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0146] Fig.17 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0147] Fig.18 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0148] Fig.19 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0149] Fig. 20A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0150] Fig. 20B A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0151] Fig. 20C A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0152] Fig.21 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0153] Fig. 22 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0154] Fig.23 A schematic diagram of a scenario for implementing a method for a user interface provided in an embodiment of the present application;

[0155] Fig.24AA schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0156] Fig. 24B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0157] Fig.24C A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0158] Fig.24D A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0159] Fig.25 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0160] Fig.26 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0161] Fig. 27 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0162] Fig.28 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0163] Fig.29A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0164] Fig.29B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0165] Fig.29C A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0166] Fig.29D A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0167] Fig.30 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0168] Fig.31 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0169] Fig.32 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0170] Fig.33A schematic diagram of a scenario for implementing a method for a user interface provided in an embodiment of the present application;

[0171] Fig.34 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0172] Fig.35A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0173] Fig.35B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0174] Fig.36 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0175] Fig.37A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0176] Fig.37B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0177] Fig.38A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0178] Fig.38B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0179] Fig.39A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0180] Fig.39B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0181] Fig.40A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0182] Fig.40B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0183] Fig.40C A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0184] Fig.40D A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0185] Fig.41AA flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0186] Fig.41B A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0187] Fig.42A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0188] Fig.42B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0189] Fig.43 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0190] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0191] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connected" includes direct and indirect connections, unless otherwise stated.

[0192] Please refer to Figure 1, an application development tool (such as Android Studio, DevEcoStudio, etc.) is installed on the electronic device 200. Usually, the developer uses an interface description language to develop the UI in the application development tool to form an interface description file. In this application, the electronic device 200 may also be referred to as a developer device. The interface description file may also be referred to as a description file.

[0193] UI development mainly includes interface description and interface behavior definition. Interface description refers to the use of interface description language to describe the layout of the UI, the controls used, and the visual style of the layout and controls. Interface behavior definition refers to the use of interface description language to define interface behavior; interface behavior includes dynamic changes in the UI and the response of electronic devices to dynamic changes in the UI (such as the response to user operations on the UI). Each OS platform has its corresponding interface description language; for example Using XML (extensible markup language) format, Use the embedded domain specific language (EDSL) built with Swift to describe the interface and define the interface behavior.

[0194] The developer packages the interface description file into the installation package of the App and publishes the App in the application market provided by the server 300. The application market can provide installation packages of various Apps for users to download. For example, the installation package can be Application package (Android application package, APK) file.

[0195] Taking a mobile phone as an example of electronic device 100, a user can use a mobile phone to download an installation package of an App in the application market. Taking a video App as an example, after the mobile phone downloads the installation package of the video App, the video App can be installed in the mobile phone by running the installation package. In this way, the mobile phone also obtains the interface description file in the installation package. The mobile phone can build a UI according to the interface description file. The UI engine provided by the OS platform of the mobile phone interprets and executes the interface description language, and renders the UI to present it to the user. The constructed UI is presented on the display device (such as a display screen) of the mobile phone. The OS platform of the mobile phone also executes the programming language that implements the interface behavior, realizes the dynamic change of the UI and responds to the user's operation on the UI.

[0196] Exemplarily, a developer develops the UI of a video app using an interface description language supported by an OS platform on an electronic device 200 and publishes the video app. A user uses the installation package of the "video" app to install the video app on a mobile phone, and a "video" icon 101 is generated on the mobile phone desktop. The user can click on the "video" icon 101 to open the video app. In response to the user's click on the "video" icon 101, the mobile phone runs the video app. The mobile phone is installed with an OS platform, which reads an interface description file, parses and executes an interface description language, renders the UI of the video app according to the interface description in the interface description file, and presents the UI 102 of the video app on the display screen. Furthermore, the interface description file may also include a definition of interface behavior. In response to the user's operation on UI 102, the mobile phone can execute corresponding interface actions and implement interface behavior according to the interface behavior defined in the interface description file. Usually, the OS platform also has a corresponding programming language for implementing interface behavior, realizing dynamic changes of UI 102, and responding to user operations on UI 102; for example Using JAVA, Use Swift programming language to implement interface behaviors.

[0197] It is understandable that in some embodiments, the developer can directly develop the UI of the App on the electronic device 100 and run the App on the electronic device 100; that is, the electronic device 200 and the electronic device 100 can be the same electronic device. This embodiment of the application is not limited to this.

[0198] The electronic device 100 may include a portable computer (such as a mobile phone, etc.), a handheld computer, a tablet computer, a laptop computer, a netbook, a personal computer (PC), a smart home device (such as a smart TV, a smart screen, a large screen, a smart speaker, etc.), a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), an augmented reality (AR)\virtual reality (VR) device, a car computer, etc., and the present application embodiment does not impose any restrictions on this. Exemplary embodiments of the electronic device 100 include but are not limited to a computer equipped with Or a portable electronic device with other operating systems. It is understandable that in some other embodiments, the electronic device 100 may not be a portable electronic device, but a desktop computer.

[0199] For example, please refer to Figure 2, which shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, an audio module 130, a speaker 130A, a microphone 130B, a display screen 140, a wireless communication module 150, a power module 160, and the like.

[0200] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0201] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110.

[0202] The controller is the nerve center and command center of the electronic device 100. It can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching and executing instructions.

[0203] The application processor can run the operating system of the electronic device 100, which is used to manage the hardware and software resources of the electronic device 100. For example, it manages and configures memory, determines the priority of system resource supply and demand, controls input and output devices, operates the network, manages the file system, manages drivers, etc. The operating system can also be used to provide an operating interface for users to interact with the system. Among them, various types of software can be installed in the operating system, such as drivers, applications (applications, Apps), etc.

[0204] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0205] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc.

[0206] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0207] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0208] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can enable the electronic device 100 to execute the user interface implementation method provided in some embodiments of the present application, as well as various applications and data processing, etc. by running the above instructions stored in the internal memory 121. The internal memory 121 may include a code storage area and a data storage area. Among them, the data storage area can store data created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, a flash memory component, a universal flash storage (UFS), etc. In some embodiments, the processor 110 can enable the electronic device 100 to execute the user interface implementation method provided in the embodiment of the present application, as well as other applications and data processing by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor 110.

[0209] The electronic device 100 can implement audio functions through the audio module 130, the speaker 130A, the microphone 130B, and the application processor, etc. For example, music playing, recording, etc. The audio module 130 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 130 can also be used to encode and decode audio signals. In some embodiments, the audio module 130 can be arranged in the processor 110, or some functional modules of the audio module 130 can be arranged in the processor 110.

[0210] The speaker 130A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0211] The microphone 130B, also called a "microphone" or "microphone", is used to convert a sound signal into an electrical signal. A user can make a sound by approaching the microphone 130B with his or her mouth, and input the sound signal into the microphone 130B.

[0212] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2 and the wireless communication module 150.

[0213] The wireless communication module 150 can provide wireless communication solutions including Wi-Fi, Bluetooth (BT), wireless data transmission modules (e.g., 433MHz, 868MHz, 915MHz) applied to the electronic device 100. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via antenna 1 or antenna 2, filters and frequency modulates the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through antenna 1 or antenna 2.

[0214] The electronic device 100 implements the display function through a GPU, a display screen 140, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 140 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0215] The display screen 140 is used to display images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 140, where N is a positive integer greater than 1. In the embodiment of the present application, the display screen 140 can be used to display a UI and receive user operations on the UI.

[0216] In some embodiments, a pressure sensor 170A, a touch sensor 170B, etc. are provided on the display screen 140. The pressure sensor 170A is used to sense pressure signals and can convert pressure signals into electrical signals. When a touch operation is applied to the display screen 140, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 170A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 170A. The touch sensor 170B, also known as a "touch panel", can form a touch screen, also known as a "touch screen", with the display screen 140. The touch sensor 170B is used to detect touch operations applied thereto or thereabout. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can also be provided through the display screen 140.

[0217] The power module 160 can be used to supply power to various components included in the electronic device 100. In some embodiments, the power module 160 can be a battery, such as a rechargeable battery.

[0218] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0219] Figure 3 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0220] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system library, and kernel layer.

[0221] The application layer can include a series of application packages.

[0222] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, negative one screen, WLAN, desktop, music, video, short message, and other applications.

[0223] The application framework layer includes the OS, which provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions to implement predefined functions. For example, obtaining the size of the display screen, determining whether there is a status bar, locking the screen, capturing the screen, etc.; providing data for application access; providing various resources for applications, such as localized strings, icons, pictures, interface description files, video files, etc. The view system of the OS includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures. The OS can also enable applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction; notifications can also appear in the system top status bar in the form of charts or scroll bar text, such as notifications of applications running in the background; notifications can also appear on the screen in the form of dialog windows. For example, prompting text information in the status bar, issuing a prompt sound, vibrating electronic devices, flashing indicator lights, etc.

[0224] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0225] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0226] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0227] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0228] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0229] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0230] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0231] A 2D graphics engine is a drawing engine for 2D drawings.

[0232] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0233] As an open source OS, it is widely used in portable electronic devices. At the same time, many manufacturers have also launched their own enhanced systems (OEM OS); for example, Huawei's EMUI is based on OEM OS can provide more than the base OS (such as ) A more optimized and enhanced SDK, providing manufacturers with customized UI programming capabilities.

[0234] In one implementation, the OEM OS released by the manufacturer supports An interface description language (such as XML) can provide Basic UI programming capabilities and manufacturer-defined UI programming capabilities. Please refer to Figure 4 , the UI development language of the App is Applicable interface description language (such as XML) for declaration Provides basic UI programming capabilities. Add a manufacturer-defined field to the interface description language (such as XML) to declare the manufacturer's customized UI programming capabilities. The basic UI engine provided interprets and executes the interface description language; the basic UI engine adds interpretation and execution of manufacturer-defined fields. OEM OS support Provides basic UI programming capabilities and provides manufacturers with customized UI programming capabilities. In this method, use Native development interface, interface description language and UI engine, using The native interface description language and the method of adding custom fields to the UI engine provide manufacturers with the ability to customize UI programming.

[0235] In another implementation, the OEM OS released by the manufacturer is independent of the general OS platform (such as ), providing manufacturer-defined UI programming capabilities. Please refer to Figure 5 , the UI development language of the App is the manufacturer's custom interface description language. The OEMOS platform provides a custom UI engine to parse and execute the custom interface description language; it also provides manufacturer-customized UI programming capabilities. In this method, the manufacturer customizes a complete UI programming framework that is independent of the general OS platform, which can meet the developer's demands for cross-platform development and operation of Apps.

[0236] The embodiments of the present application provide a method and device for implementing a user interface, which can provide rich UI programming capabilities and can adapt to a variety of OS platforms. The technical implementation difficulty is low and it is convenient for developers to use.

[0237] Please refer to Figure 6 The OEM OS platform provided in the embodiment of the present application supports a basic interface description language and a custom interface description language. The basic interface description language is an interface description language supported by a general OS platform; for example, xml, Swift, etc. In the embodiment of the present application, the custom interface description language is a domain specific language (DSL), and the custom interface description language is not related to the general OS platform. In the following embodiments of the present application, the custom interface description language is called DSL. Developers can use the basic interface description language and DSL to jointly develop the UI of the App. For example, developers use the basic interface description language to describe the UI layout, the included controls, etc.; and selectively use DSL to apply custom UI programming capabilities to some controls, add some dynamic effects to the UI, etc. For example, custom UI programming capabilities may include layout capabilities, visual attribute capabilities, unified interaction capabilities, dynamic effect capabilities, etc. Layout capabilities are used to describe the layout of controls in the UI; such as the shape, position, size, etc. of controls. Visual attribute capabilities are used to describe the visual attributes of controls; for example, visual effects such as color and grayscale of controls. Unified interaction capabilities are used to provide control responses based on user behavior; for example, perform searches based on the user's "confirmation" behavior. Dynamic effect capabilities are used to display animation effects on controls; for example, display click rebound dynamic effects on controls, etc.

[0238] The OEM OS provided in the embodiment of the present application can realize both the basic UI programming capabilities provided by the general OS platform and the customized UI programming capabilities extended relative to the OS platform. The OEM OS platform includes a basic UI engine and an extended UI engine. When an electronic device builds a UI, the basic UI engine is used to interpret and execute the basic interface description language to generate a basic UI (with basic UI programming capabilities); the extended UI engine is used to interpret and execute DSL, and superimpose customized UI programming capabilities on the basic UI.

[0239] The user interface implementation method provided in the embodiment of the present application, the custom interface description language and the extended UI engine only need to cover the custom UI programming capabilities, so the manufacturer's release difficulty is low and easy to expand; and the developer access threshold is low. The custom interface description language and the extended UI engine are not related to the general OS platform, which can be It can also be other general-purpose OS platforms. Customized interface description language and extended UI engine can be easily applied to a variety of general-purpose OS platforms.

[0240] Developers use basic interface description language and custom interface description language to develop apps. When the app runs on an electronic device, the UI engine of the OEM OS on the electronic device parses and executes the interface description language (basic interface description language and custom interface description language) to generate a UI. The basic UI engine is used to interpret and execute the basic interface description language, and the extended UI engine provided by the OEM OS in the embodiment of the present application is used to parse and execute the custom interface description language. Please refer to Figure 7In one example, the extended UI engine 310 includes modules such as a process control 311, a DSL file loading 312, a parsing engine 313, and an execution engine 314. Among them, the process control 311 is used to control the execution process of each module in the extended UI engine 310, as well as the interaction process between the extended UI engine 310 and other modules in the OEM OS. The DSL file loading 312 is used to read the DSL file. The parsing engine 313 includes sub-modules such as DSL syntax checking and DSL parsing. Among them, the DSL syntax checking sub-module is used to perform syntax checking on the content in the DSL file. The DSL parsing sub-module is used to parse the DSL file and convert the content in the DSL file into a data format that matches the execution engine. In one implementation, if the DSL syntax checking sub-module successfully checks the syntax of the DSL file, the DSL parsing sub-module parses the DSL file; if the DSL syntax checking sub-module fails to check the syntax of the DSL file, the DSL parsing sub-module does not perform parsing of the DSL file. The parsing engine 313 can also include sub-modules such as DSL preprocessing. For example, the DSL preprocessing sub-module is used to precompile the DSL file. The execution engine 314 includes sub-modules such as version management, control construction, event proxy, interpretation execution engine, and semantic support library. Among them, the version management sub-module is used to match the version of the extended UI engine 310 with the DSL file in the App. The version of the extended UI engine 310 needs to be consistent with the version of the DSL file or newer than the version of the DSL file in order to run normally. The control construction sub-module is used to build the UI control according to the content of the DSL file. The event proxy sub-module is used to realize the mapping between device events and user behaviors. For example, a mouse double-click event and a finger click event on the display screen can be mapped to the user's "confirmation" behavior on the electronic device. The interpretation execution engine is used to interpret and execute the code in the DSL file, and in response to the user behavior, execute the action corresponding to the user behavior defined in the DSL file. The semantic support library includes a set of grammatical semantic specifications for all fields in the DSL file, such as field definitions and syntax such as environment variable interfaces, public fields, layout template attributes, visual attributes, layout attributes, interaction attributes, and animation attributes.

[0241] Please continue to refer to Figure 7In the embodiment of the present application, the OEM OS also includes a custom UI programming capability 320. The custom UI programming capability 320 includes a DSL adaptation layer, which is used to provide an adaptation interface for the extended UI engine 310 with the custom UI programming capability. The custom UI programming capability 320 also provides for the implementation of custom UI programming capabilities such as visual attribute capabilities, layout capabilities, unified interaction capabilities, and dynamic effect capabilities. For example, a control is declared in the DSL file to enable vertical stretching capabilities, and the implementation of the custom UI programming capability (vertical stretching capability) is completed by the custom UI programming capability 320; that is, when the display window of the control changes, the custom UI programming capability 320 implements the vertical stretching of the control, and the developer does not need to implement the vertical stretching of the control in the DSL.

[0242] Developers can use the basic interface description language and DSL to jointly develop apps. The grammatical rules and development tools of the basic interface description language can use conventional technologies. The embodiments of the present application also provide grammatical rules and development tools for DSL. In one example, the embodiments of the present application provide a development tool that supports the grammatical rules of the basic interface description language and DSL, and provides an editing and compilation environment for the basic interface description language and DSL.

[0243] In some embodiments, embodiments of the present application provide a development tool, in which the development interface of the development tool includes a basic interface description language file and a DSL file. Exemplarily, the developer opens the development interface of the development tool, and the development interface includes an initial version of the basic interface description language file and an initial version of the DSL file. Further, the developer can use the basic interface description language in the initial version of the basic interface description language file to add control descriptions, and can also use DSL in the initial version of the DSL file to add control descriptions. It can be understood that the initial version of the DSL file can be preset in the development tool, or it can be added by the developer in the development tool. In some instances, the development tool can also include DSL templates, DSL syntax rule description files, interface description examples, etc.

[0244] In some examples, the basic interface description language file is used to describe native controls and apply basic UI programming capabilities to native controls. The DSL file is used to declare custom UI programming capabilities of controls. For example, you can apply custom UI programming capabilities to native controls in the DSL file; for another example, you can also declare custom controls in the DSL file and apply custom UI programming capabilities to custom controls.

[0245] In one implementation, the basic interface description language file and the DSL file are respectively set in different paths of the development tool folder. Exemplarily, the basic interface description language is carried by one or more xml format files, and the DSL is carried by one or more json format files. Figure 8 As shown, Taking the platform as a general OS platform, the developer creates an App folder app in the development tool. An AndroidManifest.xml file is integrated in the res directory of the folder app. The developer can declare the basic UI programming capabilities used in this XML file. A huawei_dsl.json file is integrated in the assets directory of the folder app. The developer can declare the custom UI programming capabilities used in this json DSL file.

[0246] It is understandable that the above-mentioned setting of the basic interface description language file and the DSL file in different paths of the development tool folder is only for the UI engine of the OEM OS to distinguish between the basic interface description language file and the DSL file. In practical applications, other methods can also be used to distinguish between the basic interface description language file and the DSL file. For example, different labels are preset for the basic interface description language file and the DSL file, and the UI engine of the OEM OS can obtain the basic interface description language file and the DSL file respectively according to the preset labels. The embodiments of the present application are not limited to this.

[0247] The developer completes the App development in the development tool, compiles it and generates the App installation package. The basic interface description language file and the DSL file are integrated into the App installation package so that the UI engine of the OEM OS can read the basic interface description language file and the DSL file. In one implementation, the storage location of the basic interface description language file and the DSL file in the App installation package is consistent with its location in the folder of the App in the development tool.

[0248] In one example, the DSL file uses a standard JSON format; illustratively, the DSL file includes content such as version, app, and layout.

[0249] Wherein, version indicates the version number of the DSL file; illustratively, the version format is xyz, where x indicates the product, y indicates the subsystem of the product, and z indicates the number of developments; for example, it may be 101.1.003.

[0250] The app content block is used to declare the custom UI programming capabilities that act on the App global controls in the App installation package where the DSL file is located. For example, the format of the app content block is:

[0251]

[0252] Among them, feature_name is the attribute of the custom UI programming capability. value is the attribute value of the custom UI programming capability.

[0253] The layout content block is used to declare custom UI programming capabilities that act on controls in a layout. For example, the layout content block format is:

[0254]

[0255] Among them, layoutId is used to indicate a layout; for example, layoutId is the identifier of a layout. widgetId is used to indicate a control in the layout; for example, widgetId is the identifier of the control. prop_name is the property of the custom UI programming capability, which indicates a feature of the custom UI programming capability; for example, custom UI programming capability enable, custom UI programming capability priority, custom UI programming capability parameters, etc. value is the attribute value of the custom UI programming capability, and the attribute value is used to specify the value of the attribute; for example, the attribute is custom UI programming capability enable, and accordingly, the attribute value of true indicates that the custom UI programming capability is enabled, and the attribute value of false indicates that the custom UI programming capability is not enabled.

[0256] For example, the DSL file includes the following code segment:

[0257]

[0258]

[0259] The version number is 101.1.003. The attribute value of the custom UI programming capability zoom is enabled, that is, the global control of the App enables the zoom capability. The control named R.id.edit_text in the layout named R.layout.mainpage in the App enables the onSearch capability, and the attribute value of onSearch is com.app.Search$onSearchPrice (that is, the specific execution action of the search function is defined in com.app.Search$onSearchPrice).

[0260] It is understandable that a DSL file may include fewer or more fields. For example, a DSL file may include version and layout content blocks, but not app content blocks. For another example, a layout content block may include description fields for multiple controls. For another example, multiple custom UI programming capabilities may be enabled for a control. The embodiments of the present application do not limit this.

[0261] The custom UI programming capabilities of the embodiment of the present application may include visual attribute capabilities, layout capabilities, unified interaction capabilities, dynamic effect capabilities, etc. Developers can declare custom UI programming capabilities in a DSL file to use the custom UI programming capabilities provided by the OEM OS.

[0262] Exemplarily, the visual attributes of the UI are reflected in the visual attributes of the control. The OEM OS defines a set of visual parameter variables for the control to describe the visual attributes of the control; this set of visual parameter variables can be used to switch the visual attributes of multiple brands or multiple devices. When developers describe the visual attributes of the UI, they can use visual parameter variables (dynamically obtain attribute values ​​that match the brand or the electronic device itself when the electronic device is running), without the need for developers to specify specific variable values. Declaring visual parameter variables in the DSL file enables the control to have the corresponding visual attribute capabilities.

[0263] An example of using the visual attribute capability in a DSL file is as follows:

[0264]

[0265] In this example, the attribute value of the visual attribute textColor of the control R.id.textview is emuiColor1; the attribute value of the visual attribute foreground of the control R.id.image is emui_color_bg. Among them, emuiColor1 and emui_color_bg are visual parameter variables, which are mapped to different color values ​​on different brands or devices. The mapping relationship between visual parameter variables and color values ​​on different brands or devices is preset in the OEM OS, which avoids the repeated work of developers specifying textColor and foreground attribute values ​​on different brands or devices.

[0266] OEM OS provides adaptive layout capabilities to build responsive UI, so that the layout of UI can adapt to display screens of different sizes and shapes, avoiding developers from having to perform different layout work for different devices. Exemplarily, adaptive layout capabilities include automatic stretching, hiding, line folding, equal division, proportion, extension and other layout capabilities. In one example, the adaptive layout capabilities provided by OEM OS are applicable to LinearLayout layout and controls in the layout.

[0267] Several layout capabilities are exemplified below.

[0268] 1. The adaptive layout capability master switch is used to turn on or off the adaptive layout capability of the control. In one example, the adaptive layout capability master switch is turned on to enable any of the layout capabilities.

[0269] 1. Field definition

[0270] ability property Field Definition Attribute belongs to Main switch Enable adaptive layout capability use_HwConstraintLayout layout

[0271] Among them, "capability" is used to indicate the custom UI programming capability. "Attribute" indicates the characteristic parameter of the custom UI programming capability. "Attribute belongs to" indicates the classification of the attribute function; for example, if the attribute belongs to layout, it means that the attribute is used for control layout; for example, if the attribute belongs to sub-element, it means that the attribute is used to describe the control.

[0272] 2. Automatic stretching capability: If the control enables automatic stretching, the control can be automatically stretched in the UI according to the window size to adapt to the window size.

[0273] 1. Field definition

[0274]

[0275] 2. Example of using automatic stretching capability in DSL files

[0276]

[0277] In this example, the vertical stretching capability of the control is enabled in the R.layout.linearlayout_vertical layout. When the display window changes, the control in this layout can automatically stretch vertically to adapt to the display window size.

[0278] 3. Hiding capability: If the control has the hiding capability enabled, it can be hidden in the UI.

[0279] 1. Field definition

[0280]

[0281] 2. Example of using hidden capabilities in DSL files

[0282]

[0283]

[0284] In this example, the control R.id.container in the R.layout.mainpage layout enables vertical hiding. The vertical hiding priority of R.id.image1 in R.id.container is 2, and the vertical hiding priority of R.id.image2 is 1.

[0285] 4. Line folding capability. If the control enables line folding capability, the control can be folded into multiple lines in the UI. In one example, the line folding width limit value can be used to specify the maximum width of the control displayed in each line.

[0286] 1. Field definition

[0287]

[0288] 2. Example of using line break capability in DSL files

[0289]

[0290] In this example, the controls in the R.layout.mainpage layout enable line wrapping. The line wrapping width limit of R.id.image1 is 160dp, and the line wrapping width limit of R.id.image2 is 160dp, which means that the maximum width of R.id.image1 displayed in each line is 160dp, and the maximum width of R.id.image2 displayed in each line is 160dp.

[0291] 5. Equalization capability: If the control is enabled with the equalization capability, the control can be evenly displayed in the UI.

[0292] 1. Field definition

[0293]

[0294] 2. Example of using the averaging capability in a DSL file

[0295]

[0296] In this example, the control in the R.layout.mainpage layout enables the balancing capability. Among them, the balancing type of R.id.image1 is spread.

[0297] 6. Proportion capability: The control enables the proportion capability, which means that the control can occupy the total size of the layout according to the specified percentage in the specified direction.

[0298] 1. Field definition

[0299]

[0300] 2. Example of using the proportion capability in a DSL file

[0301]

[0302] In this example, the vertical proportion capability is enabled in the R.layout.mainpage layout, where the vertical proportion of R.id.image1 is 33.33%.

[0303] 7. Extension capability. If the control enables extension capability, the control can be extended and displayed on the UI according to the display screen size. In one example, the exposure value is used to specify the exposure characteristics of the last control that can be displayed on the UI.

[0304] 1. Field definition

[0305]

[0306] 2. Example of using extension capabilities in DSL files

[0307]

[0308] In this example, the controls in the R.layout.mainpage layout enable the extension capability. Among them, R.id.image1 enables the exposure feature capability, and the exposure value is 40dp.

[0309] OEM OS also provides unified interaction capabilities, which support developers to define the response of controls based on behaviors. In one example, the unified interaction capabilities include search, zoom, etc. Developers can declare unified interaction capabilities in DSL files, so that controls have the capabilities of search, zoom, etc.

[0310] When developers develop UI based on a general OS platform, developers define the behavior corresponding to the event. For example, define a "confirm" behavior corresponding to a mouse double-click event, define a "confirm" behavior corresponding to a finger single-click event on the display screen, and define the correspondence between other events and the "confirm" behavior. The workload of developers is large. The OEM OS provided in the embodiment of the present application supports developers to directly define the response to the "confirm" behavior (i.e., define the unified interactive capability corresponding to the behavior) without defining the event corresponding to the "confirm" behavior; the mapping relationship between events and behaviors is completed by the OEM OS. The OEM OS maps events triggered by electronic devices of different forms to the same behavior (for example, mapping a mouse double-click event on a PC to a "confirm" behavior, and mapping a finger single-click event on a mobile phone to a "confirm" behavior), avoiding developers from defining the correspondence between events and behaviors for electronic devices of different forms, which brings duplication of work.

[0311] An example of using the search capability in a DSL file is as follows:

[0312]

[0313] In this example, the control R.id.sample_text has onSearch capability. When the electronic device receives the user's "confirmation" behavior on the control R.id.sample_text (for example, receiving a mouse double-click event on R.id.sample_text, receiving a finger single-click event on R.id.sample_text, etc.), it executes the search function defined in com.sample.SearchImplSample$onSearchSample.

[0314] An example of using the zoom capability in a DSL file is as follows:

[0315]

[0316]

[0317] In this example, the control R.id.sample_text has the onZoom capability. When the electronic device receives the user's "confirmation" behavior on the control R.id.sample_text (for example, receiving a mouse double-click event on R.id.sample_text, receiving a finger single-click event on R.id.sample_text, etc.), it executes the zoom function defined in com.sample.ZoomImplSample$onZoomSample.

[0318] OEM OS also provides enhanced motion effects, making the motion effects of controls more expressive. The motion effects provided by OEM OS are applicable to Button and its subclasses; they can be enabled globally in the App or for controls.

[0319] In one example, the animation capability includes a click rebound micro animation of a Button control (field definition: reboundAnimation).

[0320] An example of using the animation capability in a DSL file is as follows:

[0321]

[0322] Declare reboundAnimation in the layout content block, and enable the click rebound micro-animation effect for the target control.

[0323] For example, Fig. 9 A schematic diagram showing a process of generating a UI when an App is running.

[0324] S401. The process control module of the extended UI engine reads the basic interface description language file, and calls the basic UI engine to parse and execute the basic interface description language to construct a basic UI.

[0325] Among them, the basic UI controls use basic UI programming capabilities.

[0326] S402 . The process control module of the extended UI engine calls the DSL file loading module to read and load the DSL file.

[0327] S403: The parsing engine performs syntax verification, parsing and preprocessing on the content in the DSL file to obtain a data format that matches the execution engine.

[0328] In one example, the DSL syntax check submodule performs syntax check on the content in the DSL file, and if the check passes, the DSL parsing submodule parses the fields in the DSL file. Furthermore, the DSL preprocessing submodule preprocesses the DSL file to obtain a data format that matches the execution engine.

[0329] S404: The execution engine builds an enhanced UI based on the basic UI built in S401 and in units of controls according to the content of the DSL file.

[0330] In one implementation, the control construction submodule sequentially obtains the semantic processing components corresponding to the fields in the DSL file from the semantic support library. For example, the semantic processing component SearchHandler of the "onSearch" field is obtained from the semantic support library. Furthermore, the control construction submodule applies the custom UI programming capability to the control through the DSL adaptation layer to build an enhanced UI.

[0331] For example, Fig.10 A schematic diagram showing a flow chart of an electronic device responding to a user's operation on a UI.

[0332] S501. The execution engine creates an event proxy and registers the event proxy to the UI through the DSL adaptation layer.

[0333] S502: The OEM OS monitors user operation events of the UI, and reports the user operation events to the event proxy.

[0334] S503: The event proxy realizes the mapping between events and behaviors.

[0335] S504: The interpretation and execution engine interprets and executes the code in the DSL file, implements the response specified in the DSL file according to the behavior, and completes the response to the user's operation in the UI.

[0336] The user interface implementation method provided in the embodiment of the present application supports native controls and custom controls in the App, and also supports the application of custom UI programming capabilities in native controls. ) supports the general OS, and the general OS provides basic UI programming capabilities for native controls. Custom controls are controls that are not supported by the general OS but supported by the OEM OS, and the OEM OS provides custom UI programming capabilities for custom controls.

[0337] Please refer to Fig.11 , which shows a flow diagram of OEM OS building native controls. The App development engineering package of OEM OS 1101 includes a basic interface description language file, and the process control 1111 of the basic UI engine 1110 instructs the parsing engine 1112 to process the basic interface description language file. The parsing engine 1112 reads and loads the basic interface description language file, and converts the basic interface description language file into a data format that matches the execution engine 1113. The execution engine 1113 builds a basic UI according to the content of the basic interface description language file and generates a native control 1130.

[0338] Please refer to Fig.12 , which shows a flow diagram of the OEM OS applying custom UI programming capabilities in native controls. The App development engineering package of the OEM OS 1101 includes a basic interface description language file and a DSL file. The process control 1121 of the extended UI engine 1120 instructs the parsing engine 1112 in the basic UI engine 1110 to process the basic interface description language file. The parsing engine 1112 reads and loads the basic interface description language file, and converts the basic interface description language file into a data format that matches the execution engine 1113. The execution engine 1113 builds a basic UI according to the content of the basic interface description language file and generates a native control 1130. The process control 1121 instructs the parsing engine 1122 in the extended UI engine 1120 to process the DSL file. The parsing engine 1122 reads and loads the DSL file, and converts the DSL file into a data format that matches the execution engine 1123. The execution engine 1123 applies custom UI programming capabilities in the native control 1130 according to the DSL file.

[0339] Please refer to Fig.13, which shows a flow diagram of OEM OS building custom controls. The App development engineering package of OEM OS 1101 includes a basic interface description language file and a DSL file. The process control 1121 of the extended UI engine 1120 instructs the parsing engine 1112 in the basic UI engine 1110 to process the basic interface description language file. The parsing engine 1112 reads and loads the basic interface description language file, and converts the basic interface description language file into a data format that matches the execution engine 1113. The execution engine 1113 builds a basic UI according to the content of the basic interface description language file and generates a native control 1130. The process control 1121 instructs the parsing engine 1122 in the extended UI engine 1120 to process the DSL file. The parsing engine 1122 reads and loads the DSL file, and converts the DSL file into a data format that matches the execution engine 1123. The execution engine 1123 generates a custom control 1140 on the basic UI according to the DSL file.

[0340] The OEM OS provided in the embodiment of the present application includes a basic UI engine and an extended UI engine. When an electronic device builds a UI, the basic UI engine is used to interpret and execute the basic interface description language to generate a basic UI (with basic UI programming capabilities); the extended UI engine is used to interpret and execute DSL, and superimpose custom UI programming capabilities on the basic UI. The user interface implementation method provided in the embodiment of the present application can adapt to a variety of OS platforms and provide rich UI programming capabilities; the technical implementation difficulty is low and it is convenient for developers to use.

[0341] The embodiment of the present application also provides a method for implementing a user interface, which is easy to implement and convenient for developers to use.

[0342] With the rapid development of the Internet of Things (IoT), the types and number of IoT devices are growing rapidly. Different IoT devices have different screen sizes and user interaction methods. For example, the screen size of mobile phones is mostly around 4-6 inches, and the user interaction method is mainly touching and clicking the display screen; the screen size of TVs can reach 50 inches or larger, and the user interaction method is usually remote control operation; devices such as car computers have a wider range of screen forms and user interaction methods. The current general OS platforms (such as ) supports a development method in which, for the same UI in the same App, developers design different interface description files for each type of electronic device. Obviously, using this method to develop differentiated UIs for various types of devices is labor-intensive and difficult to develop. The present application embodiment provides a user interface implementation method and device that can achieve one-time development and multi-device deployment; that is, develop a set of interface description files that are suitable for various types of electronic devices; and reduce the development difficulty for developers.

[0343] for example, As an open source OS, it is widely used in portable electronic devices. When developing the UI of an App, for the same UI in the same App, the developer designs different interface description files for each type of electronic device to develop differentiated UIs for different types of electronic devices. It supports setting up layout folders for each type of electronic device to achieve independent development of UI for multiple types of devices. For example, add a suffix to the name of the layout folder to distinguish different layout folders. In this way, for the same UI, different types of electronic devices read the interface description files in different layout folders to present different UI display effects. If the App needs to be installed on other types of electronic devices, it is necessary to develop corresponding interface description files for this type of electronic device separately (add a separate layout folder). In this development method, developers need to develop corresponding interface description files for different types of electronic devices separately, which requires a large amount of development work and is difficult.

[0344] Some vendors also provide complete, independent UI programming frameworks. For example, Flutter, ReacNative, Weex and other frameworks. This kind of UI programming framework contains UI interface description language and corresponding parsing execution engine, provides independent interface control library, layout engine and rendering engine, etc., can run across devices, but has poor compatibility.

[0345] The present application embodiment provides a method and device for implementing a user interface. Fig.14 , the developer opens a development tool (such as DevEco Studio) in the electronic device 200 (developer device), and generates an interface description file in the development interface of the development tool. Exemplarily, the developer opens the development interface of the development tool, and the development interface includes an initial version of the interface description file. The initial version of the interface description file can be a blank file or contain simple examples. It can be understood that the initial version of the interface description file can be preset in the development tool or added by the developer in the development tool. In some examples, the development tool may also include interface description language templates, interface description language grammar rule description files, interface description examples, etc., which will not be repeated in the embodiments of the present application.

[0346] Furthermore, the developer can use an interface description language to add interface description and interface behavior definition in the initial version of the interface description file to form an interface description file for release. In one implementation, the developer generates an interface description file for each UI in the App; for example, multiple interface description files can be generated in a folder, each interface description file corresponding to a UI.

[0347] Generate an installation package of the App on the developer's device, which includes an interface description file. The installation package of the App is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the above-mentioned electronic device 100) to download the installation package of the App in the application market. After the user-side electronic device runs the installation package of the App, it obtains the interface description file in the installation package; when the user-side electronic device runs the App, the UI matching the electronic device is displayed on the display screen according to the interface description file.

[0348] In one example, the interface description file is in json format. Fig.14 As shown, the installation package of App includes a folder "app" 410. The src\main\assets directory of the folder "app" 410 includes a layout folder "layout" 411, and the layout folder "layout" 411 includes interface description files layout1.json412, layout2.json 413 and layout3.json 414, etc. Each interface description file corresponds to a UI of App. Different types of user-side electronic devices such as mobile phone 420, car machine 430, TV 440, watch 450 all run the same interface description file in "layout" 411, and display different display effects of the same UI respectively. For example, mobile phone 420, car machine 430, TV 440 and watch 450 all parse and execute layout1.json 512, and display different display effects of the UI corresponding to layout1.json 512 respectively.

[0349] In the embodiment of the present application, for the same UI in the same App, the developer can develop a set of codes in an interface description file to develop differentiated UIs for different types of electronic devices. Different types of electronic devices can present different UI display effects by reading the same interface description file of the same UI. It is possible to develop a set of interface description files that are applicable to various types of electronic devices, reducing the development difficulty for developers.

[0350] The user interface implementation method provided in the embodiment of the present application supports the use of the interface description file Native UI programming capabilities, as well as operating system-customized UI programming capabilities. Native UI programming capabilities enable controls to have Native control properties and the custom UI programming capabilities of the operating system enable controls to have extended visual properties, layout properties, interaction properties, dynamic properties, and software and hardware dependency properties. After the electronic device runs the installation package of the App and obtains the interface description file in the installation package; when the user runs the App on the electronic device, the electronic device can present the corresponding UI on the display screen according to the interface description file. The controls in the UI can include Native control properties may also include extended control properties. The custom UI engine provided in the embodiment of the present application supports The native control properties and all control properties extended in the operating system are parsed and executed.

[0351] In one example, Fig.15 FIG. 1 shows a software architecture of the electronic device 100. Fig.15 As shown, the software system of the electronic device 100 may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer. In one example, the application layer, the Android runtime (Android runtime) and the system library, and the kernel layer can refer to Figure 3 middle The corresponding description in the software architecture is not repeated here. The software system of the electronic device 100 provided in the embodiment of the present application partially reuses the UI programming framework in the conventional technology, which is easy to learn and reduces the development difficulty of the developer.

[0352] The operating system of the application framework layer includes a custom UI engine 11. The custom UI engine 11 is used to parse and execute the interface description file of the App and generate the UI of the App. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, an MVVM (model-view-viewmodel) framework 11c, a grammatical semantic library 11d and a UI rendering engine 11e. It can be understood that the application framework layer may also include more modules, and conventional technologies may be referred to, and the embodiments of the present application are not limited to this.

[0353] The above modules are described in detail below with reference to the accompanying drawings.

[0354] The syntax and semantics library 11d includes a set of syntax and semantic specifications for all fields in the interface description file, such as field definitions and syntax for variable interfaces, public fields, visual attributes, layout attributes, interaction attributes, dynamic attributes, hardware and software dependency attributes, etc. Among them, layout attributes refer to the layout of each control in the UI; such as the shape, position, size, etc. of the control. Visual attributes refer to the visual effects of the control, such as color and grayscale. Interaction attributes are the ability to provide control responses based on user behavior; such as performing searches based on the user's "confirmation" behavior. Dynamic attributes refer to displaying animation effects on controls; such as displaying click rebound effects on controls. Hardware and software dependency attributes refer to the hardware and software parameters of the device on which the control depends.

[0355] Developers need to add code to the interface description file and develop the UI according to the syntax and semantics specifications defined in the syntax and semantics library 11d. The following introduces the syntax and semantics specifications defined in the syntax and semantics library 11d from the aspects of layout arrangement, data & interface binding, interactive behavior arrangement and differentiation description.

[0356] Taking the interface description file in json format as an example, the interface description file may include the following structure:

[0357]

[0358]

[0359] The meta-data includes information such as the version number. The following is an example:

[0360] "meta-data":{

[0361] "version":"10.0.1008"

[0362] }

[0363] version indicates the version number of the interface description file; for example, the version format is xyz, where x indicates the product, y indicates the subsystem of the product, and z indicates the number of developments. The version of the interface description file needs to match the version of the custom UI engine. For example, the version of the custom UI engine must be consistent with the version of the interface description file, or newer than the version of the interface description file, in order to successfully parse the interface description file.

[0364] Import is used to import objects, and model is used to declare objects. The following is an example:

[0365]

[0366] Import the full path com.myapp.UserInfo saved by UserInfo and the full path com.myapp.TestActivity saved by Context in import; declare the object user of the UserInfo type and the object context of the Context type in the model; in this way, user and context can be directly called in the interface description files (layout-data-common and layout-data-uimode). In this application, files such as UserInfo and TestActivity are called resource files; resource files include resources used to generate the UI of the application, which may include data structures, controls, control properties, etc. defined by the developer.

[0367] The specific usage of import and model will be introduced in detail later in conjunction with the grammatical and semantic rules in layout-data-common, layout-data-uimode, and styles.

[0368] layout-data-common is used to describe the common UI. All types of electronic devices parse the content in layout-data-common and lay out the common UI according to the content in layout-data-common. layout-data-uimode is used to describe the UI of a specified device. In one implementation, layout-data-uimode declares the difference between the UI of a specified device and the common UI. In another implementation, layout-data-uimode declares all conditions applicable to the UI of a specified device. The specified device may be a mobile phone, a watch, a car machine, a smart home device (e.g., a smart TV, a smart screen, a smart speaker, etc.), a large screen, a laptop computer, a desktop computer, etc. For example, the specific forms of layout-data-uimode may include layout-data-phone (for mobile phones), layout-data-watch (for watches), layout-data-television (for TVs), etc.

[0369] Styles are used to define custom parameters in the App. Developers can customize parameters in styles.

[0370] 1. Layout

[0371] All UI in the App is composed of controls. The layout of the UI is to arrange the properties of the controls in the UI.

[0372] 1. Controls

[0373] Custom UI Engine 11 supports all Native controls and controls extended in the operating system also support controls that developers customize in the App or integrate through static packages. Among them, controls can specifically include text controls, such as TextView controls, EditText controls, etc., and can also include button controls, such as Button controls, ImageButton controls, etc., and can also include picture controls, such as Image controls, etc., and the embodiments of the present application do not impose any restrictions on this.

[0374] for For native controls and controls extended in the operating system, you can directly call the control name in layout-data-common or layout-data-uimode. For example, Native controls may include TextView, EditText, etc.; controls extended in the operating system may include HwButton, etc. An example of declaring a control is as follows. In this example, Native controls TextView and EditText.

[0375]

[0376] For controls that developers customize in the App or integrate through static packages, you need to import the full package name of the resource package of the control in import. In this way, you can call it in layout-data-common or layout-data-uimode. An example of declaring a custom control in an App is as follows. In this example, first import the full package name com.myapp.widget.MyCircleView of the resource package of the control MyCircleView in import, and then directly call MyCircleView in layout-data-common.

[0377]

[0378] In one implementation, the custom UI engine 11 supports developers to specify aliases for controls. An example is as follows, in which when the full package name com.myapp.widget.MyCircleView of the resource package of MyCircleView is introduced in import, the name of MyCircleView is specified as AliasName.

[0379]

[0380] In one implementation, when calling a control in layout-data-common or layout-data-uimode, the control is declared in the form of ComponentName():{}. For example, TextView():{} indicates that a TextView is declared.

[0381] 2. Control properties

[0382] The control properties supported by Custom UI Engine 11 include: Native properties as well as extended visual properties, layout properties, interaction properties, motion properties, and software and hardware dependency properties in the operating system.

[0383] When describing a control using the ComponentName():{} method, in one implementation, you can pass in the control's properties and property values ​​in {}, in the format of "property 1:property value 1, property 2:property value 2". The following example declares the TextView control, and passes in the TextView's property textSize in {}, where the property value of textSize is @dimen / mySize.

[0384]

[0385] In another implementation, the properties and property values ​​of the control can be passed in (). The following example declares the control TextView, and passes the property text of TextView in (), and the property value of text is @string / text_name.

[0386] {

[0387] "TextView(text:@string / text_name)":{}

[0388] }

[0389] In one implementation, if for the same control, control properties and property values ​​are passed in both () and {}, the content in () is ignored.

[0390] The property value of the control property can be assigned in any of the following ways: directly specifying through a string value; accessing the resource value defined in the background data; accessing the classification parameter declared in the background data; or accessing the value in the control model (ViewModel) object.

[0391] The custom UI engine 11 supports specifying the namespace of a control property by using the namespace.propertyName method. In one implementation, not specifying a namespace means that the default is In one implementation, the custom UI engine 11 supports using the namespace androidhwext to point to the extended resource package in the operating system and the namespace app to point to the custom resource package in the App. The extended resource package in the operating system provides the custom UI programming capability in the operating system; the custom resource package in the App provides the custom control properties in the App.

[0392] In one implementation, the developer can also define other namespaces. The namespace defined by the developer is introduced through import, and the package name of the resource package that defines the control attributes is provided. The example is as follows. In this example, the namespace myspace defined by the developer is introduced in import, and the full package name of the resource package of myspace is com.myapp. After myspace is introduced in import, the attribute borderWidth in myspace can be called in layout-data-common.

[0393]

[0394] 2. Data & Interface Binding

[0395] The custom UI engine 11 supports two-way binding of UI elements with background data. The binding relationship between UI elements (such as controls, control groups) and background data can be declared and specified in the interface description file (layout-data-common or layout-data-uimode). The MVVM framework 11c in the custom UI engine 11 can refresh the background data according to UI changes, or automatically refresh the corresponding UI according to background data changes.

[0396] For example, you can bind elements in the UI to a ViewModel object. First, import the ViewModel in import, declare an object of the ViewModel type in model, and then call the ViewModel object in layout-data-common or layout-data-uimode.

[0397] In one example, the property value of the control property in the UI is bound to the value of the ViewModel object. The example is as follows. In this example, the full package name com.myapp.UserInfo of the resource package of UserInfo (UserInfo is a ViewModel) is introduced in import, and an object user of the UserInfo type is declared in the model; then the data in user is accessed in layout-data-common.

[0398]

[0399] In one implementation, the variable value (field) in the ViewModel object (model) is accessed through the $model.field method; for example, the above $user.photo is used to access the variable photo in user, and $user.name is used to access the variable name in user. In one implementation, the function (function) return value in the ViewModel object (model) is accessed through the $model::function method. For example, the above $user::hasName is used to access the return value of the function hasName in user.

[0400] In the above example, the imageUri (image) property of the ImageView control is bound to the background data user.photo, the text (text) property of a TextView control is bound to the background data user.name, the text property of a TextView control is bound to the background data user.age, the checked (confirmed) property of the CheckBox control is bound to the background data user.agreed, and the visible (visible) property of a TextView control is bound to the background data user::hasName. When the background data changes, the property value of the control property changes, and the display effect of the control in the UI changes.

[0401] In one implementation, the visibility of controls can be obtained based on background data to implement the function of hiding parts of the UI. When the variables in the background data change (visible to invisible, or invisible to visible), the controls in the UI can be hidden or displayed accordingly. An example is as follows, in which the visibility of a column of controls (Column) is determined by the value of the variable user.visible.

[0402]

[0403]

[0404] In another example, user input is received on the UI and bound to the value of the ViewModel object. The example is as follows. In this example, the full package name com.myapp.UserInfo of the resource package of UserInfo (UserInfo is a ViewModel) is introduced in import, and an object user of the UserInfo type is declared in model; then in layout-data-common, the user input value of the attribute text (text) of the control EditText is assigned to the variable name in user. The background data is assigned through "=".

[0405]

[0406] 3. Interaction Behavior Choreography

[0407] The custom UI engine 11 supports declaring execution actions corresponding to control response events in the interface description file. The event range supported by the control is determined by the event listening supported by the control. For example, the button control (Button) supports the listener setOnClickListener for click events, so the onClick event can be bound to the control in the interface description file. The custom UI engine 11 transmits the event parameters and the response function return value in the background data in both directions between the control and the background data. The following example shows that the control Button is declared in layout-data-common to execute the action defined in the background data context.buttonClick (response function return value) in response to the event onClick.

[0408]

[0409]

[0410] Custom UI Engine 11 supports the life cycle events of controls loaded by the UI execution engine, including onPreMount, onMount, onUnmount, onPreUpdate and onUpdate, etc.; among them, onPreMount means it is called before the control is mounted to the UI; onMount means it is called after the control is mounted to the UI; onUnmount means it is called after the control is removed from the UI; onPreUpdate means it is called before the UI is refreshed due to data changes; onUpdate means it is called after the UI is refreshed due to background data changes.

[0411] In one implementation, whether the event is consumed or not is determined by the return value of the response function. Native interface definition, transparently passing the processing results in the background data to the control.

[0412] 4. Differentiation Description

[0413] 1. The properties of the controls supported by the custom UI engine 11 depend on the configuration parameters of the electronic device.

[0414] The variables of the electronic device configuration parameters are defined in the operating system. The variables of the electronic device configuration parameters can be declared in the interface description file. When the electronic device runs the interface description file, the configuration parameters of the electronic device are accessed, and the electronic device obtains the value of the configuration parameters according to its hardware and software conditions. In this way, when different types of electronic devices run the same interface description file, due to their different hardware and software conditions, different configuration parameters, and different generated UIs.

[0415] In one implementation, the configuration parameters (config) of the current electronic device are accessed through the $env.config method.

[0416] Exemplarily, the configuration parameters of the electronic device may include the contents shown in Table 1:

[0417] Table 1

[0418]

[0419] The following example shows that the attribute value of the control's dependOn attribute can be assigned to a field in the configuration parameter to declare that the control's attribute depends on a certain configuration parameter. In this example, the visibility of the Scan control (TextView) depends on the electronic device's camera hardware (camera_sensor); this means that if the electronic device has a camera, the Scan control is displayed; if the electronic device does not have a camera, the Scan control is not displayed.

[0420]

[0421]

[0422] 2. layout-data-uimode is used to describe the UI of a specified device.

[0423] Developers can declare the UI for a specific device in layout-data-uimode. The display effects of the UI for a specific device are different from those of the general UI.

[0424] In one implementation, layout-data-uimode declares all conditions applicable to the UI of a specified device. For example, please refer to Fig.16The interface description file 710 includes code blocks such as layout-data-common 711 and layout-data-watch 712. Layout-data-common 711 is used to describe a common UI applicable to various types of electronic devices, and layout-data-watch 712 is used to describe a UI applicable to a watch. Fig.16 As shown, the properties and property values ​​of each control in the common UI are declared in layout-data-common 711. The mobile phone reads the interface description file 710, parses and executes the content in layout-data-common 711, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 711. Exemplarily, the mobile phone generates a picture control 721 according to the content block 7111 in layout-data-common 711, generates a control group 722 according to the content block 7112 in layout-data-common 711, generates a control group 723 according to the content block 7113 in layout-data-common 711, generates a button control 724 according to the content block 7114 in layout-data-common 711, and generates a control group 725 according to the content block 7115 in layout-data-common 711. In this way, UI 720 of the mobile phone is generated according to content block 7111 , content block 7112 , content block 7113 , content block 7114 , and content block 7115 .

[0425] The properties and property values ​​of the controls in the UI applicable to the watch are declared in layout-data-watch 712. The watch reads the interface description file 710, and determines that there is layout-data-watch 712 for the watch in the interface description file 710, then parses and executes the content in layout-data-watch 712, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-watch 712. Exemplarily, the watch generates a picture control 731 according to the content block 7121 in layout-data-watch 712, generates a control group 732 according to the content block 7122 in layout-data-watch 712, generates a control group 733 according to the content block 7123 in layout-data-watch 712, and generates a button control 734 according to the content block 7124 in layout-data-watch 712. In this way, the UI 730 of the watch is generated according to the content block 7121, the content block 7122, the content block 7123 and the content block 7124.

[0426] In this way, the watch, as a designated device, reads the content in the second code segment (layout-data-watch 712), and electronic devices other than the watch read the content in the first code segment (layout-data-common 711); different types of electronic devices read the same interface description file of the same UI to present different UI display effects; by developing a set of interface description files, it is possible to develop differentiated UIs for different types of electronic devices, reducing the development difficulty for developers.

[0427] In another implementation, the layout-data-uimode specifies the difference between the device UI and the general UI. For example, please refer to Fig.17 The interface description file 810 includes code blocks such as layout-data-common 811 and layout-data-watch 812. The layout-data-common 811 is used to describe a common UI applicable to various types of electronic devices, and the layout-data-watch 812 is used to describe the difference between the watch UI and the common UI. Fig.17 As shown, the properties and property values ​​of each control in the common UI are declared in layout-data-common 811. The mobile phone reads the interface description file 810, parses and executes the content in layout-data-common 811, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 811. The mobile phone generates a picture control 721 according to the content block 8111 in layout-data-common 811, a control group 722 according to the content block 8112 in layout-data-common 811, a control group 723 according to the content block 8113 in layout-data-common 811, a button control 724 according to the content block 8114 in layout-data-common 811, and a control group 725 according to the content block 8115 in layout-data-common 811. In this way, UI 720 of the mobile phone is generated according to content block 8111 , content block 8112 , content block 8113 , content block 8114 , and content block 8115 .

[0428] The layout-data-watch 812 declares the properties and property values ​​of the controls in the watch UI that are different from the general UI. The watch reads the interface description file 810, parses and executes the content in layout-data-common 811; the watch determines that there is layout-data-watch 812 for the watch in the interface description file 810, and parses and executes the content in layout-data-watch 812; the watch generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 811 and layout-data-watch 812. Fig.17 As shown, the watch generates a picture control 731 corresponding to content block 8111 in layout-data-common811, a control group 732 corresponding to content block 8112 in layout-data-common 811, a control group 733 corresponding to content block 8113 in layout-data-common 811, and a button control 734 corresponding to content block 8114 in layout-data-common 811. According to the description of layout-data-watch 812, the property value of the visible property (visible) of the control group generated corresponding to content block 8115 in layout-data-common 811 is set to invisible (gone). In other words, the control group corresponding to content block 8115 in layout-data-common 811 is not displayed on the watch. Fig.17 As shown, UI 730 of the watch includes controls generated based on content block 8111, content block 8112, content block 8113, and content block 8114.

[0429] In this way, all types of electronic devices read the content in the first code segment (layout-data-common 711), and the watch as a designated device also reads the content in the second code segment (layout-data-watch 712); different types of electronic devices read the same interface description file of the same UI to present different UI display effects; by developing a set of interface description files, it is possible to develop differentiated UIs for different types of electronic devices, reducing the development difficulty for developers.

[0430] 3. Style

[0431] Custom UI Engine 11 supports developers to customize parameters in style for the current interface description file. The example is as follows: the developer defines myTextStyle in style and can call the custom parameter in layout-data-common as $style.myTextStyle.

[0432]

[0433] The grammatical semantic rules provided in the embodiment of the present application are used to develop the UI, and the grammar is concise and efficient. A set of interface description files can be developed to be applicable to different types of electronic devices, avoiding the need to develop UIs separately for different types of electronic devices and reducing development costs.

[0434] The UI parsing engine 11a is used to parse the interface description file and convert the content in the interface description file into a data format that matches the UI execution engine 11b. In some examples, the UI parsing engine 11a can also perform a syntax check on the content in the interface description file. If the syntax check on the interface description file succeeds, the interface description file is parsed; if the syntax check on the interface description file fails, the interface description file is not parsed.

[0435] In one example, see Fig.18 , the UI parsing engine 11a reads the interface description file, parses the data in the fields such as declaration (model), style (style), layout (layout-data-common, layout-data-uimode) in the interface description file, and saves it to the database after preprocessing. Use the control parser to parse the data in the layout field, recursively call the UI execution engine 11b to instantiate the control according to the logical structure of the layout description, and form the control tree of the UI. Use the attribute parser to parse the attribute field of each control, call the UI execution engine 11b to set the attributes for each control, and complete the UI drawing.

[0436] The workflow of the control parser and attribute parser is as follows: Fig.19As shown. The control parser obtains the name of the control and obtains the property list of the control. If the control identity (ID) exists, add the control ID; if the control style (style) exists, add the control style; instantiate the control to form a control queue. If there is a sub-layout, recursively call the controls in the sub-layout. After parsing the controls in all layouts, add controls and return the generated controls. The property parser obtains the instantiated control from the control queue, reads the property name and property value corresponding to the control, and stores the property (including the property name and property value) in the hash table. If there is a sub-layout, recursively call the controls in the sub-layout. After the properties of all controls are parsed, the property values ​​of each control stored in the hash table are assigned to the corresponding controls.

[0437] The UI execution engine 11b is used to build UI controls (instantiate controls and set properties) based on the data parsed by the UI parsing engine 11a, layout the controls, and generate the interface declared in the interface description file; it can also implement the mapping between device events and user behaviors, and execute actions corresponding to user behaviors defined in the interface description file in response to user behaviors.

[0438] For example, please refer to Fig. 20A In the UI execution engine 11b, a Builder class is constructed for each control. The Builder class uses The same inheritance logic is used to achieve that child controls can inherit all the properties and setting methods of parent controls without repeated definition. The Builder class contains the entity construction method of the corresponding control and the setting method of the unique visual properties of the control to complete the control instantiation and property setting. For developer-defined controls, a Builder class for the custom control can be provided in the UI execution engine 11b, which has low access cost and is more developer-friendly.

[0439] In some embodiments, for the interface description file declared Native control properties, UI execution engine 11b can set properties according to the declaration in the interface description file, complete control instantiation, and construct controls with Native control properties.

[0440] Please refer to Fig. 20B For the extended attributes of the control declared in the interface description file, if it is determined that the operating system includes the custom UI programming capability, the UI execution engine 11b sets the attributes according to the declaration in the interface description file, completes the control instantiation, and constructs the control with the extended attributes; if it is determined that the operating system does not include the custom UI programming capability, the UI execution engine 11b maps the corresponding extended attributes declared in the interface description file to the corresponding Native control properties, according to the corresponding Set the properties of the native control, complete the control instantiation, and construct a A control with native control properties. This control does not have extended properties.

[0441] For example, please refer to Fig. 20C , an installation package of the App is generated on the developer's device, which includes an interface description file. The installation package of the App is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the above-mentioned electronic device 100) to download the installation package of the App in the application market. After the user-side electronic device runs the installation package of the App, it obtains the interface description file in the installation package; when the user-side electronic device runs the App, the UI matching the electronic device is displayed on the display screen according to the interface description file.

[0442] For example, the interface description file includes the following content:

[0443]

[0444] In one example, tablet 460 runs an App as a user-side electronic device. The operating system of tablet 460 includes custom UI programming capabilities. On the UI of tablet 460, the custom control group HwMagicLayout is successfully constructed, and the control group has layout properties extended in the operating system. Exemplarily, the layout properties extended in the operating system may include layout properties such as automatic stretching, hiding, equal distribution, proportion, extension or line break. Among them, automatic stretching means that the height or width of the control is automatically enlarged or reduced according to the window size to adapt to the window size. Hiding refers to the ability of the control to be visible or invisible in the layout. Equal distribution means that the content in the control is evenly distributed in the layout. Proportion means that the control occupies the total size of the layout in a specified direction according to a specified percentage. Extension means that the control is extended and displayed on the UI according to the size of the display. Line break means that the content in the control is displayed in one or more lines in the layout. As Fig. 20C As shown, the UI layout of tablet 460 has the layout attributes extended in the system. When tablet 460 is displayed in portrait mode, control 461, control group 462, control group 463, control 464, and control group 465 are arranged vertically in one column. When tablet 460 is displayed in landscape mode, control 461, control group 462, control group 463, and control 464 are arranged vertically in the first column, and control group 465 is arranged in the second column. The UI layout of tablet 460 is adaptively adjusted according to the size and shape of the display window.

[0445] On the UI of tablet 460, the interactive capability "zoomEnable" is effective on control 461 "imageview". When tablet 460 is connected to mouse 480, control 461 can be enlarged and displayed in response to the user's zoom operation on control 461 (for example, when the cursor corresponding to mouse 480 is placed on control 461, the scroll wheel of mouse 480 is turned upward).

[0446] In another example, tablet 470 is used as a user-side electronic device to run an App. The operating system of tablet 470 does not include a custom UI programming capability. The UI of tablet 470 does not support the custom control group HwMagicLayout. The controls in the UI have Native linear layout (LinerLayout) properties. When tablet 470 is displayed in portrait or landscape orientation, control 471, control group 472, control group 473, control 474, and control group 475 are arranged vertically in a column. The UI layout of tablet 470 is fixed and cannot be adaptively adjusted according to the size and shape of the display window.

[0447] In the UI of tablet 470, the interactive capability "zoomEnable" cannot be effective on control 471 "imageview". That is, when tablet 470 is connected to mouse 480, if the cursor corresponding to mouse 480 is placed on control 471, the size of control 471 remains unchanged when the scroll wheel of mouse 480 is turned upward.

[0448] In this way, the interface description file that conforms to the grammatical rules in the grammatical semantic library 11d can be successfully run in different operating systems, achieving cross-operating system platform operation and reducing the development difficulty for developers.

[0449] It should be noted that the UI execution engine 11b dynamically parses data when the electronic device runs the interface description file, and obtains the electronic device related parameters when the electronic device runs the interface description file; it avoids the developer from pre-compiling the interface description file in the development tool and generating preset data files; in this way, the UI development can be independent of the compilation environment and realize cross-development platform development and operation.

[0450] MVVM framework 11c is used to perform two-way binding between UI elements and background data. In the interface description file, declare and specify the binding relationship between UI elements (such as controls, control groups) and background data. Optionally, you can also perform simple data instance settings; MVVM framework 11c can refresh background data according to UI changes, and automatically refresh the corresponding UI according to background data changes. It helps developers focus on UI design and layout, simplifies the UI development process, and greatly reduces the development time developers invest in realizing front-end and back-end data interaction.

[0451] For example, please refer to Fig.21 , the developer declares the control fields in the interface description file, binds properties to the controls, and binds the background data objects. The UI parsing engine 11a parses the binding behavior in the interface description file to obtain the correspondence between the control properties and the background data objects. The MVVM framework 11c is used to implement two-way binding between controls and background data. When the background data changes, the MVVM framework 11c maps the background data with the data of the corresponding control properties; the UI execution engine 11b sets the property data of the control and refreshes the UI. When the property data of the control in the UI changes (for example, in response to user input, the shape, text and other data of the control changes), the MVVM framework 11c maps the data of the control properties with the background data; the background data is refreshed.

[0452] The UI rendering engine 11e is used to render and organize the interface generated by the UI execution engine 11b, and output the display content to the display screen.

[0453] The user interface implementation method provided in the embodiment of the present application enables different types of electronic devices to read the same interface description file of the same UI and present different UI layouts. It is possible to develop a set of interface description files suitable for various types of electronic devices, thereby reducing the development difficulty for developers.

[0454] In some embodiments, please refer to Fig. 22 The software system of the electronic device 100 may include an application layer, an application framework layer, an Android runtime and a system library, and a kernel layer.

[0455] The interface description file of the application layer App1 adopts the json format; the interface description file of App2 adopts the xml format.

[0456] The operating system of the application framework layer includes a control unit. When the electronic device 100 runs the App, the control unit obtains the interface description file of the App. Exemplarily, when the electronic device 100 runs App1, the control unit obtains the json format interface description file of App1; when the electronic device 100 runs App2, the control unit obtains the xml format interface description file of App2. The control unit distributes the interface description file to the basic UI engine 10 or the custom UI engine 11 for UI drawing according to the type of the interface description file. For example, the control unit obtains the json format interface description file of App1, and distributes the json format interface description file of App1 to the custom UI engine 11 for processing. For example, the control unit obtains the xml format interface description file of App2, and distributes the xml format interface description file of App2 to the basic UI engine 10 for processing. In one implementation, the specified paths of the json format interface description file and the xml format interface description file in the application installation package are different. The control unit obtains the json format interface description file at the first specified path of the App1 application installation package, and obtains the xml format interface description file at the second specified path of the App2 application installation package. In another implementation, different tags are preset in the json format interface description file and the xml format interface description file, and the control unit determines the type of the interface description file according to the preset tags of the interface description file.

[0457] The custom UI engine 11 parses, executes, and renders the json format interface description file of App1 to generate the UI of App1. The controls in the UI of App1 can support general OS (such as ) native UI programming capabilities, and can also support customized UI programming capabilities in the electronic device 100 operating system.

[0458] The basic UI engine 10 parses, executes, and renders the XML format interface description file of App2 to generate the UI of App2. The controls in the UI of App2 can support general OS (such as )Native UI programming capabilities.

[0459] In this way, the electronic device 100 can run both apps developed using the json format interface description language and apps developed using the xml format interface description language, thereby achieving forward compatibility with the operating system.

[0460] The embodiment of the present application also provides a method for implementing a user interface for a UI of an application widget.

[0461] Developers can develop widgets for apps. For example, mobile phones support displaying widgets of apps in the notification bar, on the desktop, and on the negative one screen. Usually, app widgets displayed in the notification bar are called custom notification bars, app widgets displayed on the desktop are called desktop widgets, and app widgets displayed on the negative one screen are called negative one screen cards. Custom notification bars, desktop widgets, and negative one screen cards can present information in apps to users more intuitively, and support performing operations on apps without opening the apps, making it easier for users to use apps. More and more apps provide widgets for users to use.

[0462] At present, there are relatively few layout modes and control types supported for display on the user interface (UI) of application widgets, which cannot meet the diverse needs of users. The embodiments of the present application provide a method and device for implementing a user interface, which supports displaying various layout modes and control types on the UI of application widgets, making it easier for users to use application widgets and improving user experience.

[0463] Typically, developers use an interface description language to develop the UI of an application (Application, App) in an application development tool. Developers also use the interface description language to develop the UI of an application widget in an application development tool.

[0464] Please refer to Fig.23 , an application development tool (such as Android Studio, DevEcoStudio, etc.) is installed on the electronic device 200. In this application, the electronic device 200 may also be referred to as a developer device. The developer develops the UI of the App in the application development tool to form an interface description file. In this application, the interface description file may also be referred to as a description file. The developer also develops the UI of the application widget in the application development tool to form a component interface description file. The developer packages the interface description file and the component interface description file into the installation package of the App, and publishes the App in the application market provided by the server 300. The application market may provide installation packages for each App for users to download. For example, the installation package may be Application package (Android application package, APK) file.

[0465] It should be noted that, in one implementation, the component interface description file is independent of the interface description file. In another implementation, the component interface description file may be a part of the interface description file (for example, a code segment in the interface description file is used as the component interface description file). The embodiments of the present application do not limit this. In the following embodiments, an exemplary description is given by taking the component interface description file as a separate file as an example.

[0466] Taking a mobile phone as an example of electronic device 100, a user can use a mobile phone to download an installation package of an App in an application market. The installation package of an App includes an interface description file and a component interface description file. Taking a music App as an example, after the mobile phone downloads the installation package of the music App, the music App can be installed in the mobile phone by running the installation package. In this way, the mobile phone also obtains the interface description file and the component interface description file in the installation package.

[0467] For example, Fig.23 As shown, after the mobile phone installs the music app, the mobile phone desktop includes a shortcut icon of the music app - "music" icon 103. The mobile phone can receive a user's click operation on the "music" icon 103. In response to the user's click operation on the "music" icon 103, the mobile phone generates the UI of the music app according to the interface description file and presents the UI of the music app on the display screen. The mobile phone can also display a small component (called a music widget) of the music app on the mobile phone desktop according to the user's settings. The mobile phone generates the UI of the music widget according to the component interface description file and displays the UI 104 of the music widget on the display screen.

[0468] It is understandable that in some embodiments, the developer can directly develop the UI of the App and the UI of the application widget on the electronic device 100, and run the App and the application widget on the electronic device 100; that is, the electronic device 200 and the electronic device 100 can be the same electronic device. The embodiments of the present application are not limited to this.

[0469] Generally, the elements presented in the UI are called controls (View), which can provide users with certain operation functions or display certain content. System native controls include text controls (TextView), input boxes (EditText), buttons (Button), image buttons (ImageButton), image controls (ImageView), etc.

[0470] Taking the Android system as an example, all the elements in the UI of the application are composed of controls (View) and control groups (ViewGroup). A UI can contain one or more Views or ViewGroups. View is an element displayed in the display interface; ViewGroup is a layout container for storing Views (or ViewGroups). New Views or ViewGroups can be added to ViewGroup so that each View is arranged according to a certain hierarchy and structural relationship. For example, developers can use linear layout (LinearLayout), table layout (TableLayout), relative layout (RelativeLayout), layer layout (FrameLayout), absolute layout (AbsoluteLayout) or grid layout (GridLayout) and other layout methods to design the View or ViewGroup in each UI in the App, thereby generating a layout file for each UI; such as an interface description file or a component interface description file.

[0471] at present, The system supports limited layout modes and control types in application widgets, which cannot meet the diverse needs of users. The embodiment of the present application provides a user interface implementation method, which can support the application of various layout modes and control types in application widgets.

[0472] The user interface implementation method provided in the embodiment of the present application not only supports The system's native linear layout (LinearLayout), layer layout (FrameLayout), relative layout (RelativeLayout) and grid layout (GridLayout) are applied to application widgets, and also support The system's native table layout (TableLayout), absolute layout (AbsoluteLayout) and other layout methods are applied to application widgets.

[0473] The user interface implementation method provided in the embodiment of the present application not only supports The system's native buttons (Button), image controls (ImageView), image buttons (ImageButton), progress bars (ProgressBar), text controls (TextView), list controls (ListView), grid controls (GridView), stack controls (StackView), dynamic loading of controls (ViewStub), adaptive controls (AdapterViewFlipper), switching controls (ViewFlipper), clocks (AnalogClock), timers (Chronometer) and other controls are applied to application widgets; it also supports System-native input boxes (EditText), check boxes (CheckBox), slide selectors (Picker), scroll views (ScrollView), radio buttons (RadioButton), rating bars (RatingBar), search boxes (SearchView), drag bars (SeekBar), switches (Switch) and other controls are applied to application widgets.

[0474] For example, a mobile phone is used as the electronic device 100, and a music app is used on the mobile phone. The mobile phone 100 can display a small component (called a music widget) of the music app on the desktop. Fig.24A As shown, the mobile phone 100 displays a UI 910 of a music widget. The UI 910 includes a picture control 911 for displaying a picture set by the App; a text control 912 for displaying the name of the track being played; a search box 913 for receiving user input and performing a search in response to the user's input text; a picture button 914 for switching the display style of the music widget; a drag bar 915 for adjusting the progress of the music being played according to user operations; and other controls.

[0475] For example, Fig. 24B As shown, the mobile phone 100 can receive a user's click operation on the search box 913, and in response to the click operation, an enlarged search box 913 and a soft keyboard 91a are displayed on the desktop. The user can use the soft keyboard 91a to input text for the search box 913. The mobile phone 100 searches according to the text input into the search box 913.

[0476] For example, Fig.24C As shown, the mobile phone 100 can receive the user's dragging operation on the drag bar 915 to adjust the progress of playing music.

[0477] The user interface implementation method provided in the embodiment of the present application also supports applying the customized UI programming capabilities in the operating system to the application widget, so that the controls in the application widget have the extended visual attributes, layout attributes, interaction attributes, dynamic attributes and hardware and software dependency attributes in the operating system. Among them, the layout attribute refers to the layout of each control in the UI; such as the shape, position, size, etc. of the control. The visual attribute refers to the visual effects of the control such as color and grayscale. The interactive attribute is the ability to provide control response based on user behavior; such as performing a search based on the user's "confirmation" behavior. The dynamic attribute refers to displaying animation effects on the control; such as displaying click rebound dynamic effects on the control. The hardware and software dependency attribute refers to the hardware and software parameters of the control dependent device. Exemplarily, the extended layout attributes in the operating system may include layout attributes such as automatic stretching, hiding, equal distribution, proportion, extension or line break. Among them, automatic stretching refers to the height or width of the control automatically enlarging or reducing according to the window size to adapt to the window size. Hiding refers to the ability of the control to be visible or invisible in the layout. Equal distribution refers to the uniform distribution of the content in the control in the layout. Proportion refers to the control occupying the total size of the layout according to the specified percentage in the specified direction. Stretching means that the control is stretched and displayed on the UI according to the size of the display screen. Wrapping means that the content in the control is displayed in one or more lines in the layout.

[0478] For example, Fig.24D As shown, the UI 910 of the music widget may include a picture button 916 for displaying the lyrics of the music. The mobile phone may receive a click operation of the user on the picture button 916, and in response to the user's click operation on the picture button 916, the mobile phone displays the lyrics of the currently playing music. The picture button 916 may be displayed or not displayed on the UI 910. Whether the picture button 916 is displayed depends on whether the currently playing music has lyrics. If the currently playing music has corresponding lyrics, the picture button 916 is displayed on the UI 910; if the currently playing music does not have corresponding lyrics, the picture button 916 is not displayed on the UI 910. For example, Fig.24D As shown, when the currently playing music is music 1, the UI 910 includes a picture button 916; when the currently playing music is music 2, the UI 910 does not include a picture button 916.

[0479] The user interface implementation method provided in the embodiment of the present application also supports applying the layout method and control type defined by the developer in the App to the application widget.

[0480] Developers can use Various system-native layout methods and control types, layout methods and control types defined in the operating system, and layout methods and control types defined in the App are applied to application widgets for user convenience.

[0481] In one implementation, see Fig.25 In the App development phase, the developer opens a development tool (e.g., DevEco Studio) in the electronic device 200 (developer device), uses an interface description language in the development tool, describes the interface and defines the interface behavior according to the syntax and semantic specifications of the interface description language, and performs UI development; and forms interface description files and component interface description files for release.

[0482] For example, the interface description file and the component interface description file adopt the json format. Developers can perform UI layout arrangement, data & interface binding, interactive behavior arrangement and differentiated description in the interface description file and the component interface description file respectively. All UIs in applications and application widgets are composed of controls. The layout arrangement of the UI is to arrange the properties of the controls in the UI. Data & interface binding is to declare and specify the binding relationship between the elements in the UI (such as controls, control groups) and the background data in the interface description file or the component interface description file. Interactive behavior arrangement is to declare the execution action corresponding to the control response event in the interface description file or the component interface description file. The event range supported by the control is determined by the event listening supported by the control. For example, if a button supports setOnClickListener for the click event, the onClick event can be bound to the control in the interface description file. Differential description includes arranging different code segments for different types of electronic devices, so that the UI of the application widget has different display effects on different types of electronic devices; obtaining the value of the configuration parameter according to the hardware and software conditions of the electronic device and applying it to the control; defining parameters applicable to the App, etc.

[0483] For example, users can declare in the component interface description file of the music app Figure 24A-Figure 24D The picture control 911, text control 912, search box 913, picture button 914, drag bar 915 and picture button 916 shown in the figure are attributed to these controls so that these controls are included in the UI 910 of the music widget. These controls can be System native controls can also be controls defined in the operating system or in the music app.

[0484] Developers can also apply control properties defined in the operating system to these controls in the component interface description file. For example, the software dependency properties defined in the operating system are applied to the picture button 916. It is stated that the display properties of the picture button 916 depend on the music currently played by the App, including lyrics.

[0485] Developers can also bind controls to background data in the component interface description file. For example, the drag bar 915 is bound to the background data; when the mobile phone receives a user's drag operation on the drag bar 915, the current music playing progress in the background data is updated according to the user's drag operation on the drag bar 915; if the current music playing progress in the background data changes, the drag bar 915 is updated.

[0486] Developers can also declare the execution actions corresponding to the control response events in the component interface description file, for example, declaring that the search box 913 executes a search action in response to a click event.

[0487] Afterwards, an installation package of the App is generated on the developer's device, which includes an interface description file and a component interface description file. The installation package of the App is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the electronic device 100 described above) to download the installation package of the App in the application market. After the user-side electronic device runs the installation package of the App, the interface description file and the component interface description file in the installation package are obtained. For example, Fig.25 As shown, after the mobile phone 100 runs the installation package of the music app, the icon 103 of the music app is displayed on the desktop. The mobile phone 100 can receive the user's click operation on the icon 103, run the music app, and display the UI of the music app on the display screen according to the interface description file.

[0488] Further, the user-side electronic device adds the application widget to the notification bar, desktop or negative one screen according to the user's settings. The user-side electronic device generates the UI of the application widget according to the component interface description file and displays the UI of the application widget in the notification bar, desktop or negative one screen. For example, Fig.25 As shown, the mobile phone 100 displays a UI 910 of a music widget on the desktop.

[0489] Please refer to Fig.26 , the application widget process in the electronic device 100 runs independently of the application process. The application installed in the electronic device 100 runs by calling the application process, and the application widget runs by calling the application widget process. For example, if the application widget is set on the desktop, the desktop process is the application widget process; if the application widget is set on the negative one screen, the negative one screen display process is the application widget process; if the application widget is set in a specified application, the process of the specified application is the application widget process.

[0490] The electronic device 100 also includes units such as a custom UI engine 11 and a widget framework 12 .

[0491] In some examples, the application process obtains the interface description file of the App, calls the custom UI engine 11 to parse and execute the interface description file of the App, and generates the UI of the App. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, an MVVM (model-view-viewmodel) framework 11c, etc. The UI parsing engine 11a is used to parse the interface description file and convert the content in the interface description file into a data format that matches the UI execution engine 11b. In some examples, the UI parsing engine 11a can also perform syntax verification on the content in the interface description file. If the syntax verification of the interface description file is successful, the interface description file is parsed; if the syntax verification of the interface description file is unsuccessful, the parsing of the interface description file is not performed. The UI execution engine 11b is used to build UI controls (instantiate controls and set properties) according to the data parsed by the UI parsing engine 11a, lay out the controls, and generate the interface declared in the interface description file; it can also realize the mapping between device events and user behaviors, and execute the actions corresponding to the user behaviors defined in the interface description file in response to the user behaviors. The MVVM framework 11c is used to perform two-way binding between the elements in the UI and the background data. In the interface description file, declare and specify the binding relationship between UI elements (such as controls, control groups) and background data. Optionally, you can also perform simple data instance settings. The MVVM framework 11c can refresh background data according to UI changes, and automatically refresh the corresponding UI according to background data changes. It helps developers focus on UI design and layout, simplifies the UI development process, and greatly reduces the development time invested by developers to achieve front-end and back-end data interaction.

[0492] In some examples, the application process obtains the component interface description file, calls the widget framework 12 to process the component interface description file, and forms widget UI data for displaying the application widget UI. Among them, the widget framework 12 includes modules such as virtual control construction 12a, data binding 12b, widget service 12c and event proxy 12d. Among them, the virtual control construction 12a parses the component interface description file by calling the UI parsing engine 11a, instantiates the parsed component interface description file, and calls the UI execution engine 11b to construct the interface, construct controls, control groups, etc., to form widget UI data. These widget UI data exist in the application process and are used to bind with background data (for example, control model (ViewModel)). Data binding 12b is used to bind the properties, interaction events, etc. of the control or control group constructed by the virtual control construction 12a with background data (for example, the control model (ViewModel) that processes business logic, which contains data processing logic related to virtual objects). The widget service 12c is used to track and process the currently processed object and the data (model) bound to the object during the generation of the application widget UI; it is also used to manage data transmission between the application process and the application widget process; it is also used to manage cross-process event agents and send and receive cross-process events. The event agent 12d is used to handle the return and response of events in the application widget process. Exemplarily, a dedicated event transmission class (such as HiAction) is defined in the event agent 12d. The event transmission class supports the implementation of the Parcelable interface and can be transmitted across processes (for example, calling Native cross-process Binder mechanism). A series of events are stored in the event transmission class, and each event includes layout identifier, control identifier, event type and other information. When the user operates the application widget, the application widget process receives the operation and triggers an interaction event, that is, a new event is added in HiAction. The application widget process transmits the newly added event to the application process. The application process responds to the event and performs corresponding actions. The application process also calls the MVVM framework for processing; if there is a change in data or control properties, the widget UI data is updated, the cross-process interface data and properties are updated, and the display interface of the application widget process is further updated. In one implementation method, the application process also sends the component interface description file to the application widget process. The application widget process calls the widget framework 12 to process the component interface description file, form the widget UI data, and display the widget UI data, that is, display the application widget IU.

[0493] When an application is installed on an electronic device, the user can add an application widget corresponding to the application in the notification bar, desktop or negative one screen.

[0494] In some embodiments, the user does not open the application, but adds the application widget corresponding to the application separately. Fig. 27 , the mobile phone 100 receives a two-finger pinch operation on the desktop by the user. In response to the two-finger pinch operation on the desktop, the mobile phone 100 displays a quick setting interface 1010. The quick setting interface 1010 includes a "widget" option 1011 for adding a desktop widget on the desktop. The mobile phone 100 can display a desktop widget adding interface 1020 in response to the user's click operation on the "widget" option 1011. The desktop widget adding interface 1020 includes a "music" option 1021 for adding a music widget on the desktop. The mobile phone 100 receives a user's click operation on the "music" option 1021. In response to the user's click operation on the "music" option 1021, the UI 910 of the "music widget" is displayed on the desktop of the mobile phone 100.

[0495] In other embodiments, the user adds an application widget corresponding to the application in the application. Fig.28 As shown, the user opens the "Music Settings" interface 1030 of the "Music" application on the mobile phone 100, and the "Music Settings" interface 1030 includes a "Desktop Widget" option 1031. The "Desktop Widget" option 1031 is used to add a music widget on the mobile phone desktop. The mobile phone 100 receives the user's click operation on the "Desktop Widget" option 1031, and in response to the user's click operation on the "Desktop Widget" option 1031, the mobile phone 100 displays the "Desktop Widget" interface 1040. The "Desktop Widget" interface 1040 includes a "Style 1" option 1041 and a "Style 2" option 1042; it also includes an "Add" button 1043 and a "Cancel" button 1044. The user can add the music widget to the desktop in the interface corresponding to the "Style 1" option 1041 or the "Style 2" option 1042 by clicking the "Add" button 1043; and can also exit the addition of the music widget by clicking the "Cancel" button 1044. Exemplarily, the mobile phone 100 receives a user click operation on the "add" button 1043, and according to the user's selection, adds the music widget to the desktop with the interface corresponding to the "style 2" option 1042. The UI 910 of the "music widget" is displayed on the desktop of the mobile phone 100.

[0496] In one implementation, see Fig.29A , the user performs an operation of adding an application widget. The application widget process of the electronic device 100 receives the user's operation of adding an application widget (for example, the user Fig. 27The application widget process notifies the application process of receiving the user's operation of adding an application widget; if the application process is not started, the electronic device 100 starts the application process so that the application process runs in the background. Alternatively, the application process of the electronic device 100 receives the user's operation of adding an application widget (for example, the user clicks on the "Music" option 1021 in the music menu). Fig.28 The application process notifies the application widget process that it has received the user's operation to add an application widget.

[0497] The application process obtains the component interface description file from the application installation package. The application process calls the custom UI engine 11 to parse and execute the component interface description file, and then calls the virtual control construction 12a to build the widget UI data control, and generates controls, control groups, etc. according to the layout arrangement in the component interface description file to form the widget UI data (including controls and control layout information).

[0498] It should be noted that the component interface description file can be a file independent of the interface description file, or it can be a code segment in the interface description file. Optionally, after the electronic device 100 obtains the component interface description file, it can only parse and execute part of the code segment therein. Fig.28 In the music widget interface corresponding to the "Style 1" option 1041, after receiving the user's click operation on the "Add" button 1043, the mobile phone 100 parses and executes the code segment corresponding to the "Style 1" in the component interface description file; the user selects Fig.28 In the music widget interface corresponding to the "Style 2" option 1042, after receiving the user's click operation on the "Add" button 1043, the mobile phone 100 parses and executes the code segment corresponding to the "Style 2" in the component interface description file.

[0499] Afterwards, the data binding 12b calls the MVVM framework 11c to data bind the widget UI data to the background data (eg, the control model). In this way, if the background data changes, the corresponding widget UI data can be refreshed; if the widget UI data changes, the corresponding background data can be refreshed.

[0500] Furthermore, the application process sends the component interface description file to the application widget process.

[0501] The application widget process calls the custom UI engine 11 to parse and execute the component interface description file, and then calls the virtual control construction 12a to construct the widget UI data control, and generates controls, control groups, etc. according to the layout arrangement in the component interface description file to form widget UI data (including controls and control layout information); and displays according to the widget UI data, that is, displays the application widget UI. Since the application process generates the widget UI data and the application widget process generates the application widget UI using the same code segment, the controls on the application widget UI correspond to the controls in the widget UI data.

[0502] Optional, such as Fig.29B As shown, after generating the widget UI data, the application process can also send the widget UI data to the application widget process, and the application widget process displays according to the widget UI data, that is, displays the application widget UI. In this way, the controls on the application widget UI also correspond one-to-one to the controls in the widget UI data.

[0503] After adding an app widget to the notification bar, desktop, or negative one screen, users can operate the app on the app widget UI. For example, users can drag Fig.24A The drag bar 915 on the music widget UI 910 is used to adjust the playing progress of the music currently being played by the music application.

[0504] In one implementation, see Fig.29C When the user operates on the UI of the application widget, the application widget process receives the user operation and transmits the user operation to the event agent 12d. A dedicated event transmission class is defined in the event agent 12d, and the event transmission class is used for cross-process transmission. The event transmission class stores multiple events, each of which includes information such as layout identifier, control identifier, event type, etc. After receiving the user operation, the event agent 12d generates an event corresponding to the operation in the event transmission class and sends the event to the application process (if the application process is not started, the application process is pulled up so that the application process runs in the background). After receiving the event, the application process obtains the corresponding control according to the layout identifier and the control identifier, and executes the corresponding business logic according to the event acting on the control. Since there is a one-to-one correspondence between the controls on the application widget UI and the controls in the widget UI data, the application process also refreshes the background data according to the received event. The change of background data triggers the update of the widget UI data, and the application process can also send the updated widget UI data to the application widget process, and the application widget process displays the updated application widget UI according to the refreshed widget UI data.

[0505] In some embodiments, please refer to Fig.29D, the application process starts, and the application process obtains the interface description file. The application process calls the custom UI engine 11 to parse and execute the interface description file, generate the application UI, and display the application UI. Afterwards, the MVVM framework 11c binds the application UI to the background data (such as the control model).

[0506] The application process receives the user's operation of adding an application widget, and the application process obtains the component interface description file from the application installation package. The application process calls the custom UI engine 11 to parse and execute the component interface description file, and then calls the virtual control construction 12a to build the widget UI data control, and generates controls, control groups, etc. according to the layout arrangement in the component interface description file to form the widget UI data (including controls and control layout information). Afterwards, the data binding 12b calls the MVVM framework 11c to data bind the widget UI data with the background data (for example, the control model). Further, the application widget process displays the UI of the application widget according to the widget UI data.

[0507] In this way, the UI of the application and the UI of the corresponding application widget are displayed on the display screen of the electronic device 100.

[0508] In one example, see Fig.30 , which shows an example process of a method for implementing a user interface provided in an embodiment of the present application.

[0509] The user performs an operation of adding an application widget. The application widget process of the electronic device 100 receives the user's operation of adding an application widget (for example, the user Fig. 27 The application widget process notifies the application process of receiving the user's operation of adding an application widget; if the application process is not started, the electronic device 100 starts the application process so that the application process runs in the background. Alternatively, the application process of the electronic device 100 receives the user's operation of adding an application widget (for example, the user clicks on the "Music" option 1021 in the music menu). Fig.28(click operation of the "Add" button 1043 in the dialog box), the application process notifies the application widget process that it has received the user's operation of adding an application widget. The widget framework, MVVM framework, background data and other modules are initialized. The widget framework obtains the component interface description file from the application installation package and sends it to the virtual control construction module. The virtual control construction module constructs the control according to the component interface description file to form the widget UI data. The data binding module calls the MVVM framework to bind the widget UI data and the background data. The application process calls the widget service for binding service, and the widget service binds the event proxy to the widget UI data; afterwards, the widget UI data and the event proxy of the widget UI data and other information are sent across processes. In this way, after the application widget process receives the widget UI data and the event proxy information of the widget UI data, it can display the application widget UI according to the widget UI data.

[0510] After the application widget is successfully added, the user can operate on the application widget UI. The application widget process receives the user's operation on the application widget UI, and the event agent adds the corresponding event of the operation and sends the event to the application process; the application process executes the business logic in response to the event and calls the MVVM framework to update the background data. When the business data of the application changes, the background data changes cause the MVVM framework to update the widget UI data. The application process sends the updated widget UI data across processes. After the application widget process receives the updated widget UI data, it can display the updated application widget UI according to the updated widget UI data.

[0511] The user interface implementation method provided in the embodiment of the present application is that the application process generates widget UI data according to the component interface description file, and sends the component interface description file or widget UI data to the application widget process; the application widget process generates the application widget UI according to the component interface description file or widget UI data. Developers can declare in the component interface description file The system's native layout methods and control types can also declare custom control types and UI programming capabilities in the operating system, as well as the layout methods and control types defined by the developer in the App. The operating system supports the application widget process to call the UI engine to parse and execute the component interface description file to generate the application widget UI. In this way, various layout methods and control types can be displayed on the UI of the application widget, making it easier for users to use the application widget and improving the user experience.

[0512] In some embodiments, after the application widget is added to the electronic device, the electronic device is turned off. After the electronic device is turned on again, the UI of the application widget is displayed. That is, after the application widget is added to the electronic device, the application widget UI is reloaded. Fig.31As shown, the mobile phone 100 is turned on, and the desktop of the mobile phone 100 displays a UI 910 of a music widget.

[0513] In one example, see Fig.32 , which shows an example process of a method for reloading an application widget UI in an electronic device.

[0514] After the electronic device is turned on, the application widget process starts. The application widget process obtains the component interface description file from the application installation package. The application widget process calls the custom UI engine to parse and execute the component interface description file, build the widget UI data, and display the application widget UI according to the widget UI data.

[0515] After the application widget is successfully reloaded, the user can operate on the application widget UI. The application widget process receives the user's operation on the application widget UI, and the event agent adds the event corresponding to the operation, pulls up the application process, makes the application process run in the system background, and sends the event to the application process; the application process executes the corresponding business logic in response to the event, and calls the MVVM framework to update the background data. The processing flow of event interaction and background data changes can be referred to Fig.30 The relevant description will not be repeated here.

[0516] In this scenario, the application widget process only needs to generate and draw the loaded application widget UI. The application process generates widget UI data, binds the widget UI data with the background data, and establishes the corresponding relationship between the widget UI data and the application widget UI.

[0517] In the user interface implementation method provided by the embodiment of the present application, the application process of the electronic device generates widget UI data according to the component interface description file, and binds the widget UI data with the background data; the application widget process also obtains the widget UI data according to the component interface description file, and displays the widget UI data as the UI of the application widget. In this way, the UI of the application widget establishes a corresponding relationship with the background data, which can support the display of various layout modes and control types on the UI of the application widget, making it convenient for users to use the application widget and improving the user experience.

[0518] An embodiment of the present application also provides a method for implementing a user interface, which is used to present a UI when an App on an electronic device is projected onto a playback device for playback.

[0519] With the rapid development of the Internet of things (IoT), the types and number of IoT devices are growing rapidly. Consumers can use mobile phones, tablets and other devices as control devices for IoT devices to control IoT devices, so that the control devices and IoT devices work together. For example, when a user uses an App on a control device, the App can be projected to an IoT device for playback (IoT devices are called playback devices). For example, since the screen size of a TV is larger, it can bring a better viewing experience to users, and users can project the App on their mobile phones to the TV for playback. Since the screen shapes and sizes of IoT devices vary greatly, how to project screens on IoT devices with various shapes and sizes of screens and obtain a projection interface that matches the screen shape and size of the IoT device is a problem that needs to be solved.

[0520] The present application provides a method and device for implementing a user interface, which supports projecting various UIs on a control device to an IoT device for playback, thereby improving user experience. The control device is the user-side electronic device (electronic device 100) in the above embodiments.

[0521] This application embodiment provides a method for implementing a user interface. Fig.33 , an application development tool (such as Android Studio, DevEco Studio, etc.) is installed on the electronic device 200. The electronic device 200 in this application may also be referred to as a developer device. Typically, the developer uses an interface description language to develop the UI of the App and the UI of the playback end in the application development tool. It is understandable that in some embodiments, the developer can directly develop the UI of the App and the UI of the playback end on the control device 100, and run the App on the control device 100; that is, the electronic device 200 and the control device 100 can be the same electronic device. The embodiments of the present application do not limit this.

[0522] In some embodiments, the developer develops the UI of the App (i.e., the UI displayed when the electronic device installs and runs the App) in the application development tool to form an interface description file. The interface description file in this application may also be referred to as a description file. The developer also develops the UI of the App for display at the playback end (i.e., the playback end UI) in the application development tool to form a playback end interface description file. The developer packages the interface description file and the playback end interface description file into the installation package of the App and publishes the App in the application market provided by the server 300. The application market may provide installation packages for various Apps for users to download. For example, the installation package may be Application package (Android application package, APK) file.

[0523] Taking a mobile phone as the control device 100 as an example, a user can use a mobile phone to download an installation package of an App in the application market. Taking a video App as an example, after the mobile phone downloads the installation package of the video App, the video App can be installed in the mobile phone by running the installation package. In this way, the mobile phone also obtains the interface description file and the playback end interface description file in the installation package.

[0524] Optionally, in some embodiments, the interface description file may also be used as the playback-end interface description file, that is, the interface description file and the playback-end interface description file are the same file.

[0525] The mobile phone can present the UI of the corresponding App on the display screen according to the interface description file. Exemplarily, after the mobile phone downloads the installation package of the video App, a "video" icon 101 is generated on the desktop. The user can click on the "video" icon 101 to open the video App. In response to the user's click operation on the "video" icon 101, the mobile phone runs the video App. The mobile phone is installed with an OS platform, and the custom UI engine of the OS platform reads the interface description file, parses and executes the interface description language, and renders the UI of the video App according to the interface description in the interface description file. The display device (such as a display screen) of the mobile phone presents the UI 105 of the video App. Furthermore, the interface description file may also include a definition of interface behavior. In response to the user's operation on UI 105, the mobile phone can execute corresponding interface actions and implement the interface behavior according to the interface behavior defined in the interface description file. Usually, the OS platform also has a corresponding program language for implementing interface behavior, realizing dynamic changes of UI 105 and responding to user operations on UI 105; for example Using JAVA, Use Swift programming language to implement interface behaviors.

[0526] The mobile phone can also project the various interfaces of the video app to the playback device 1000 for display. For example, the main interface or playback interface of the video app is projected to the playback device 1000. The playback device 1000 renders the corresponding playback UI according to the interface description matching its device type in the playback interface description file. For example, please continue to refer to Fig.33, the UI 105 of the video app includes a "cast screen" button 106; the "cast screen" button 106 is used to cast the interface of the App running on the mobile phone to the playback device for display. The mobile phone receives the user's click operation on the "cast screen" button 106, and in response to the user's click operation on the "cast screen" button 106, the mobile phone displays a device selection interface 107. The device selection interface 107 includes prompt information 108, which is used to prompt the user to select a playback device for screen casting. The device selection interface 107 also includes a "living room TV" option 109 and a "my tablet" option 10a. The user can click the "living room TV" option 109 to cast the UI of the video app to the smart TV for display. The mobile phone receives the user's click operation on the "living room TV" option 109, and in response to the user's click operation on the "living room TV" option 109, the mobile phone casts the UI of the video app to the smart TV. The smart TV displays the playback end UI 1001 corresponding to the UI 105. The user can also click on the "My Tablet" option 10a to project the UI of the video app to the tablet for display. The mobile phone receives the user's click operation on the "My Tablet" option 10a, and in response to the user's click operation on the "My Tablet" option 10a, the mobile phone projects the UI of the video app to the tablet. The tablet displays the playback end UI 1002 corresponding to UI 105. Smart TVs and tablets have different device types, screen sizes and shapes. The interface layout of the playback end UI 1001 on the smart TV and the playback end UI 1002 on the tablet are different; that is, the playback end UI is displayed differently on electronic devices of different device types.

[0527] The playback device 1000 may include a portable computer (such as a mobile phone, etc.), a smart home device (such as a smart TV, a smart screen, a large screen, a smart speaker, etc.), a handheld computer, a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), a tablet computer, a laptop computer, a netbook, an augmented reality (AR)\virtual reality (VR) device, a car computer, etc., and the present application embodiment does not impose any restrictions on this. In one example, the playback device 1000 may include Figure 2 The structure shown is not described here in detail. It can be understood that the embodiment of the present application Figure 2 The illustrated structure does not constitute a specific limitation on the playback device 1000. In other embodiments of the present application, the playback device 1000 may include Figure 2 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 2The components shown may be implemented in hardware, software or a combination of software and hardware.

[0528] Please refer to Fig.34 The control device 100 includes a custom UI engine 11, a screen projection framework 13, a transmission channel adapter 14 and other units. The custom UI engine 11 provides an IF1 interface, and the screen projection framework 13 provides IF2, IF3, IF4 and IF5 interfaces.

[0529] The description of interfaces IF1 to IF5 is shown in Table 2:

[0530] Table 2

[0531]

[0532] The custom UI engine 11 parses and executes the interface description file of the App to generate the UI of the App. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, an MVVM (model-view-viewmodel) framework 11c, etc. The UI parsing engine 11a is used to parse the interface description file and convert the content in the interface description file into a data format that matches the UI execution engine 11b. In some examples, the UI parsing engine 11a can also perform syntax verification on the content in the interface description file. If the syntax verification of the interface description file is successful, the interface description file is parsed; if the syntax verification of the interface description file is unsuccessful, the parsing of the interface description file is not performed. The UI execution engine 11b is used to build UI controls (instantiate controls and set properties) based on the data parsed by the UI parsing engine 11a, lay out the controls, and generate the interface declared in the interface description file; it can also realize the mapping between device events and user behaviors, and execute the actions corresponding to the user behaviors defined in the interface description file in response to the user behaviors. The MVVM framework 11c is used to perform two-way binding between the elements in the UI and the background data. In the interface description file, declare and specify the binding relationship between UI elements (such as controls, control groups) and background data. Optionally, you can also perform simple data instance settings. The MVVM framework 11c can refresh background data according to UI changes, and automatically refresh the corresponding UI according to background data changes. It helps developers focus on UI design and layout, simplifies the UI development process, and greatly reduces the development time invested by developers to achieve front-end and back-end data interaction.

[0533] The transmission channel adapter 14 is used to adapt the data transmission channel between the control device 100 and the playback device 1000. For example, the data of the control device 100 is converted into a format suitable for the data transmission channel, so that the control device 100 can send data to the playback device 1000 through the data transmission channel.

[0534] The screen projection framework 13 is used to process the playback end interface description file to form the playback end UI data for displaying the playback end UI. The screen projection framework 13 includes modules such as virtual control construction 13a, data binding 13b, screen projection service 13c, data transceiver 13d, resource transmission 13e, event agent 13f and life cycle 13g. Among them, the virtual control construction 13a calls the UI parsing engine 11a and the UI execution engine 11b to construct controls, control groups, etc. according to the playback end interface description file to form the playback end UI data. These playback end UI data exist in the App process and are used to bind with the background data. Data binding 13b is used to bind the properties, interactive events, etc. of the controls or control groups constructed by the virtual control construction 13a to the background data (for example, the control model (ViewModel) that processes business logic). The screen projection service 13c is used to track and process the currently processed object and the data (model) bound to the object during the screen projection process; it is also used to manage the data transmission channel between the control device 100 and the playback device 1000. Data transceiver 13d is used to control the sending and receiving of data between device 100 and playback device 1000. For example, the interface of the transceiver agent can be defined to implement the default transceiver built into the control device 100. For another example, the transceiver suitable for the App can be customized in accordance with the interface specification in the App. For example, if the information is transmitted through the ContentProvider method in the App, the ContentProvider is used in the send() function in the data transceiver 13d to implement data sending and receiving. Resource transmission 13e is used to transmit specific types of data resources (for example, data, pictures, videos, etc. whose data volume is greater than the set value). Resource transmission 13e is used to manage the specific type of data resources, such as sending, receiving, caching, identification, progress control, etc. Event agent 13f is a channel for transmitting events to avoid events being blocked by data transmission. Life cycle 13g is used to manage the life cycle of the joint operation entities of the control device 100 and the playback device 1000 during the screen projection process. Exemplarily, the life cycle of the control device 100 and the playback device 1000 is shown in Table 3:

[0535] Table 3

[0536]

[0537] For example, Fig.35AAs shown, the control device 100 is in a stand-alone running state, triggers screen projection, and waits for user authorization to project the screen to the playback device. If the user agrees to the authorization, the playback device sends an authorization instruction to the control device, and the control device 100 enters the server-side running state. If the user refuses the authorization, the playback device sends a refusal authorization instruction to the control device, and the control device 100 stops projecting the screen. When the control device 100 is in the server-side running state, the App switches to the background running, and then stops pushing data to the playback device; the App switches to the foreground running and starts pushing data to the playback device. When the control device 100 is in the server-side running state, the control device closes the App, or the playback device is turned off, then the screen projection stops. When the control device 100 is in the server-side running state, the App of the playback device switches to the background running, and the control device 100 enters the server-side paused state. When the control device 100 is in the server-side paused state, the App of the playback device switches to the foreground to play, and the control device 100 enters the server-side running state. When the control device 100 is in the server-side paused state and the playback device is turned off, the screen projection stops.

[0538] For example, Fig.35B As shown, the playback device 1000 receives the screen projection request initiated by the control device and waits for the user to confirm the authorization. If the user confirms the authorization, the playback device 1000 enters the screen projection running state; if the user refuses the authorization, the playback device 1000 stops running. When the playback device 1000 is in the screen projection running state and the App switches to the background running, it enters the background resident state. When the playback device 1000 is in the background resident state and the App switches to the foreground for playback, it enters the screen projection running state. When the playback device 1000 is in the screen projection running state or the background resident state, the playback device is turned off or the control device turns off the App, it stops running.

[0539] Please refer to Fig.36 In the App development phase, developers generate an App installation package on the developer device, which includes an interface description file and a player interface description file. Developers use the interface description language to develop the interface description file and the player interface description file on the developer device according to the syntax and semantics of the interface description language, add code to the interface description file and the player interface description file, and perform UI development.

[0540] Developers can perform UI layout arrangement, data & interface binding, interactive behavior arrangement and differentiated description in the interface description file and the player interface description file respectively.

[0541] All UI in the app is composed of controls. The layout of the UI is to arrange the properties of the controls in the UI. For example, the controls in the UI can include all Native controls and controls extended in the operating system also support controls that developers customize in the App or integrate through static packages. Among them, controls can specifically include text controls, such as TextView controls, EditText controls, etc., and can also include button controls, such as Button controls, ImageButton controls, etc., and can also include picture controls, such as Image controls, etc., and the present application embodiment does not impose any restrictions on this. Control properties include Native properties and extended visual properties, layout properties, interaction properties, motion properties, and hardware-dependent properties in the operating system. Visual properties refer to the visual effects of controls, such as color and grayscale. Interaction properties refer to the ability to provide control responses based on user behavior; for example, performing searches based on the user's "confirmation" behavior. Motion properties refer to displaying animation effects on controls; for example, displaying click rebound effects on controls. Hardware-dependent properties refer to the hardware and software parameters of the device that controls depend on.

[0542] Data & interface binding, that is, declaring and specifying the binding relationship between UI elements (such as controls, control groups) and background data in the interface description file or the player interface description file.

[0543] Interaction behavior arrangement means declaring the execution actions corresponding to the control response events in the interface description file or the player interface description file. The event range supported by the control is determined by the event listening supported by the control. For example, the button control supports the listener setOnClickListener for click events, so the onClick event can be bound to the control in the interface description file.

[0544] Differentiation description:

[0545] 1. Developers can declare variables of electronic device configuration parameters in the interface description file or the player interface description file. When the electronic device runs the interface description file or the player interface description file, the configuration parameters of the electronic device are accessed, and the electronic device obtains the value of the configuration parameter according to its hardware and software conditions. In this way, when different types of electronic devices run the interface description file or the player interface description file, due to their different hardware and software conditions, different configuration parameters, and different generated UIs.

[0546] 2. Developers can customize parameters in the interface description file or the player interface description file. Taking the interface description file and the player interface description file in json format as an example, the interface description file or the player interface description file can include style, layout-data-common and other parts. The developer defines myTextStyle in style and can call the customized parameter in layout-data-common as $style.myTextStyle. The example is as follows,

[0547]

[0548] 3. Developers can compile code for a specified device in the interface description file or the player interface description file. For example, taking the interface description file and the player interface description file in json format as an example, the interface description file or the player interface description file may include the following structure:

[0549]

[0550] Developers can declare a universal player UI in layout-data-common. All types of playback devices parse the content in layout-data-common and lay out the universal player UI according to the content in layout-data-common. layout-data-uimode is used to describe the player UI of a specified device. In one implementation, layout-data-uimode declares the difference between the player UI of a specified device and the universal player UI. The specified device parses and executes the content in layout-data-common and layout-data-uimode to generate the player UI of the specified device. In another implementation, layout-data-uimode declares all conditions applicable to the player UI of the specified device. The specified device lays out its player UI according to the content in layout-data-uimode. The specified device can be one of the types such as a mobile phone, a watch, a car machine, a smart home device (such as a smart TV, a smart screen, a smart speaker, etc.), a large screen, a tablet computer, a laptop computer, or a desktop computer. For example, the specific forms of layout-data-uimode may include layout-data-phone (for mobile phones), layout-data-watch (for watches), layout-data-television (for smart TVs), layout-data-pad (for tablets), layout-data-car (for car computers), etc. In this way, different types of playback devices can be parsed and executed according to their corresponding code segments and the playback end UI can be constructed; the playback end UI matching the type of playback device can be displayed on different types of playback devices.

[0551] The developer uploads the App installation package generated on the developer device to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the above-mentioned control device 100) to download the App installation package in the application market. After the control device runs the App installation package, it obtains the interface description file and the playback end interface description file in the installation package. When the control device runs the App, the UI matching the control device is displayed on the display screen according to the interface description file.

[0552] When the control device is running the App, the App interface can also be projected to the playback device for display. For example, the control device determines the playback device to be projected based on the user input, and sends a projection command to the playback device, where the projection command includes the projection interface identifier. The playback device receives the projection command, obtains the corresponding playback interface description file based on the projection interface identifier, and forms a playback UI that matches its device type based on the playback interface description file.

[0553] For example, please refer to Fig.37A , the control device 100 receives the user's screen projection operation (for example, the mobile phone receives the user's Fig.33 The control device 100 also determines the device type of the playback device 1000 for screen projection based on the user input (for example, the mobile phone receives the user's input). Fig.33 The mobile phone receives the user's click on the "living room TV" option 109, and determines that the playback device 1000 is a smart TV; Fig.33 By clicking the "My Tablet" option 10a in the display, it is determined that the playback device 1000 is a tablet computer).

[0554] The virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data. For example, if the playback device 1000 is a smart TV, the control device 100 parses and executes the code segment corresponding to the smart TV in the playback end interface description file to form the playback end UI data for casting the screen to the smart TV; if the playback device 1000 is a tablet computer, the control device 100 parses and executes the code segment corresponding to the tablet computer in the playback end interface description file to form the playback end UI data for casting the screen to the tablet computer. Afterwards, the data binding 13b calls the MVVM framework 11c to data bind the objects in the playback end UI data to the background data (for example, the control model). In this way, if the background data changes, the corresponding playback end UI data can be refreshed, and the playback end UI data changes and refreshes the corresponding background data. Further, the control device 100 sends the playback end interface description file and the resource file (including the data resources associated in the playback end interface description file) to the playback device 1000 through the data transceiver 13d. In one implementation, the control device 100 encodes the playback end interface description file, and after the layout information, resource value, data, response event definition and other data are encoded, they are transmitted to the playback device 1000 through the data transmission channel; specific types of data resources (for example, data, pictures, videos, etc. with a data volume greater than a set value) are transmitted to the playback device 1000 through a specific data transmission channel. Optionally, specific types of data resources can be transmitted to the playback device 1000 before sending the playback end interface description file, so that the rate of transmitting the playback end interface description file to the playback device 1000 is increased, and the delay of the playback device 1000 displaying the playback end UI is shortened. The control device 100 also initializes the event proxy 13f to establish an event transmission channel between the control device 100 and the playback device 1000 for transmitting event information.

[0555] The playback device 1000 receives the playback end interface description file and data resources through the data transceiver 13d, and the virtual control construction 13a in the OS of the playback device 1000 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file, and generates controls, control groups, etc. according to the layout arrangement in the code segment, forming the playback end UI data (including controls and control layout information); and displays according to the playback end UI data, that is, displays the playback end UI. Since the control device 100 generates the playback end UI data and the playback device 1000 generates the playback end UI data using the same code segment, the controls on the playback end UI generated by the playback device 1000 correspond one-to-one to the controls in the playback end UI data generated by the control device 100.

[0556] After the player UI is generated, the player device 1000 displays the player UI on the display screen in a shape and size that matches the player screen. Fig.36 As shown, when the smart TV is used as a playback device, the smart TV parses and executes the layout-data-television code segment in the playback interface description file to generate the corresponding playback UI of the smart TV; when the tablet computer is used as a playback device, the tablet computer parses and executes the layout-data-pad code segment in the playback interface description file to generate the corresponding playback UI of the tablet computer.

[0557] Optionally, in some embodiments, Fig.37B As shown, the control device 100 constructs a control according to the code segment corresponding to the device type of the playback device 1000 in the playback interface description file, generates playback UI data according to the layout in the code segment, and then sends the generated playback UI data to the playback device 1000. After receiving the playback UI data, the playback device 1000 displays the playback UI according to the playback UI data.

[0558] The user interface implementation method provided in the embodiment of the present application is that the playback device generates a playback UI corresponding to the playback device according to the code segment corresponding to the device type of the playback device in the playback interface description file. The playback UI displayed by different types of playback devices matches the shape and size of their screens. In addition, when developing an App, developers can easily develop the playback UI by defining various types of controls (including all The native controls and controls extended in the operating system also support controls customized by developers in the App or integrated through static packages), and the supported control types are diverse; various types of Apps can support the screen projection function, and the control types supported by the playback UI are richer and more diverse, which is convenient for users to use.

[0559] In one implementation, the virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback interface description file, and constructs the playback UI data according to the code segment corresponding to the device type of the playback device 1000 in the playback interface description file. The playback UI data is not displayed on the display screen of the control device 100 (that is, the playback UI data exists in the App process and is not sent to the display process). The control device 100 displays the UI generated according to the interface description file. After the interface of the control device 100 is projected to the playback device 1000, the user can perform other operations on the control device 100, and the playback device 1000 plays the projected content normally.

[0560] For example, Fig.38A As shown, the mobile phone displays the interface 1210 of the "Video" App, and the interface 1210 includes a "Cast Screen" button 1211. The mobile phone receives the user's click operation on the "Cast Screen" button 1211, and determines the playback device according to the user's input. The mobile phone casts the screen to the smart TV according to the user's input. Fig.37A or Fig.37B As shown in the figure, the mobile phone generates the player UI data according to the player interface description file, and sends the player interface description file to the smart TV. The smart TV generates the player UI data according to the player interface description file, and displays the player UI according to the player UI data. Please refer to Fig.38A , the smart TV displays the playback end UI 1220.

[0561] After that, the user can continue to perform other operations on the mobile phone. Fig.38A As shown, the mobile phone receives a click operation on the image 1212 by the user, and in response to the click operation on the image 1212 by the user, displays the interface 1230 of the “Video” App.

[0562] In this way, after the control device projects the screen to the playback device, it can continue to execute other functions, running independently from the playback device playing the projected content without affecting each other, thereby achieving better collaboration between devices.

[0563] For example, Fig.38BAs shown, the custom UI engine 11 in the OS of the control device 100 parses and executes the interface description file, generates the UI of the application, and displays the UI of the application. The MVVM framework 11c performs data binding of the UI of the application with the background data (eg, the control model).

[0564] The virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data. Afterwards, the data binding 13b calls the MVVM framework 11c to perform data binding on the objects in the playback end UI data and the background data (for example, the control model). Further, the control device 100 sends the playback end interface description file and the resource file (including the data resources associated in the playback end interface description file) to the playback device 1000 through the data transceiver 13d; or sends the playback end UI data to the playback device 1000, so that the playback device 1000 can display the playback end UI according to the playback end UI data.

[0565] In another implementation, the virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback interface description file, and constructs the playback UI data according to the code segment corresponding to the device type of the playback device 1000 in the playback interface description file. The App process sends the playback UI data to the display process, and displays the playback UI on the display screen of the control device 100. The control device 100 and the playback device 1000 display the playback UI generated according to the same code segment.

[0566] For example, Fig.39A As shown, the mobile phone displays the interface 1210 of the "Video" App, and the interface 1210 includes a "Cast Screen" button 1211. The mobile phone receives the user's click operation on the "Cast Screen" button 1211, and determines the playback device according to the user's input. The mobile phone casts the screen to the smart TV according to the user's input. Fig.39B As shown, the mobile phone generates the player UI data according to the player interface description file, and displays the player UI according to the player UI data. The mobile phone also sends the player interface description file to the smart TV. The smart TV generates the player UI data according to the player interface description file, and displays the player UI according to the player UI data.

[0567] like Fig.39A As shown, after the mobile phone casts the screen to the smart TV, the mobile phone also displays the playback end UI, and both the mobile phone and the smart TV display the playback end UI 1220.

[0568] In this way, the control device and the playback device play the playback end UI synchronously, mirroring screen projection can be achieved, and the control device and the playback device work together.

[0569] In some embodiments, when the control device 100 receives a user operation on the playback end UI or a change in business data, the data binding 13b calls the MVVM framework 11c to update the playback end UI data. Since there is a one-to-one correspondence between the controls of the playback end UI and the controls in the playback end UI data, the playback end UI data update triggers the playback end UI update. In this way, when the control device 100 receives a user operation or a change in business data, the playback end UI of the playback device 1000 can be updated synchronously.

[0570] For example, please refer to Fig.40A , the mobile phone displays the UI 1310 of the "Video" App. The mobile phone projects the UI 1310 of the "Video" App to the smart TV according to the user input. The smart TV displays the playback end UI 1320 of the "Video" App, and the playback end UI 1320 includes a "play" button 1321.

[0571] The user can start playing the video on the smart TV (for example, the user selects the "play" button 1321 through the remote control of the smart TV and clicks the "play" button 1321). The smart TV receives the user's click operation on the "play" button 1321, and in response to the user's click operation on the "play" button 1321, plays the video and displays the updated UI 1320. The updated playback UI 1320 includes a "pause" button 1322.

[0572] In one implementation, Fig.40B As shown, a dedicated event transmission class is defined in the event proxy 13f, and the event transmission class is used for cross-device transmission. Multiple events are stored in the event transmission class, wherein each event includes information such as layout identification, control identification, event type, etc. The playback device 1000 receives the operation of the user on the playback end UI, generates an event corresponding to the operation in the event transmission class, and transmits it to the control device 100 through the event transmission channel. After receiving the event, the control device 100 obtains the corresponding control according to the layout identification and the control identification, and executes the corresponding business logic according to the event acting on the control. Since there is a one-to-one correspondence between the control of the playback end UI and the control in the playback end UI data, the control device 100 also updates the background data, and the background data change triggers the playback end UI data update. The control device 100 sends the updated playback end UI data to the playback device 1000, and the playback device 1000 displays the updated playback end UI according to the updated playback end UI data.

[0573] In this way, the user can control the App on the playback device, and the control device executes the corresponding business logic and updates the playback UI on the playback device. In some examples, if the control device and the playback device mirror the playback UI, the UI on the control device can also be updated synchronously to facilitate user use. In addition, for the user's operations on the playback device, the control device executes the relevant business logic, and the control device uniformly controls the playback device for easy management; and avoids the playback device having low performance and not supporting complex business logic processing.

[0574] In some embodiments, the playback device 1000 receives a second operation of the user on the playback end UI. The playback device 1000 obtains an updated playback end interface description file from the control device 100 and generates an updated playback end UI.

[0575] For example, Fig.40C , the mobile phone displays UI 1330 of the "Video" App. The mobile phone casts UI 1330 of the "Video" App to the smart TV based on the user input. The smart TV displays the playback-end UI 1340 of the "Video" App. The smart TV receives the user's second operation on the playback-end UI 1340 (for example, the user moves the focus on the playback-end UI 1340 from "Movie" to "Variety Show" through the remote control). In response to the user's second operation on the playback-end UI 1340, the smart TV displays an updated playback-end UI, namely, playback-end UI 1350 of the "Video" App.

[0576] In one implementation, Fig.40DAs shown, the playback device 1000 receives the second operation of the user on the playback end UI, generates an event corresponding to the second operation in the event transmission class, and transmits it to the control device 100 through the event transmission channel. After receiving the event, the control device 100 obtains the corresponding control according to the layout identifier and the control identifier, and executes the corresponding business logic according to the event acting on the control. The control device 100 determines to update the playback end UI to the playback end UI with the focus on "Variety Show", and obtains the playback end interface description file 2 corresponding to the playback end UI with the focus on "Variety Show". The virtual control construction 13a in the OS of the control device 100 calls the UI, parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data 2. Afterwards, the data binding 13b calls the MVVM framework 11c to perform data binding on the objects in the playback end UI data 2 with the background data (for example, the control model). Further, the control device 100 sends the playback end interface description file 2 and the resource file (including the data resources associated in the playback end interface description file 2) to the playback device 1000 through the data transceiver 13d. In one implementation, the control device 100 encodes the playback end interface description file 2, and after the layout information, resource value, data, response event definition and other data are encoded, they are transmitted to the playback device 1000 through the data transmission channel; specific types of data resources (for example, data, pictures, videos, etc. with a data volume greater than a set value) are transmitted to the playback device 1000 through a specific data transmission channel. The playback device 1000 receives the playback end interface description file 2 and the data resource file through the data receiving and sending 13d. The virtual control construction 13a in the OS of the playback device 1000 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2 by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data 2 (including information such as controls and control layout); and displays according to the playback end UI data 2, that is, displays the updated playback end UI.

[0577] In this way, the user can directly operate the player UI on the playback device, control the device to execute the business logic corresponding to the operation, and send the player interface description file corresponding to the updated player UI to the playback device, and the playback device generates an updated player UI according to the updated player interface description file. This allows the user to directly operate the player UI on the playback device and successfully switch the player UI.

[0578] Please refer to Fig.41A , which shows an example of the processing flow of the control device in the user interface implementation method provided by the embodiment of the present application. After the control device installs the App, the screen projection framework, MVVM framework, background data, etc. are initialized. Then the resource transmission module transmits the data resources related to the App, and binds the data resources to the screen projection service. The screen projection framework obtains the playback end interface description file from the application installation package and sends it to the virtual control construction module. The virtual control construction module constructs the control according to the playback end interface description file to form the playback end UI data; and binds the playback end UI data to the screen projection service. Afterwards, the virtual control construction module notifies the data binding module to bind the playback end UI data with the background data, and the data binding module mobilizes the MVVM framework to bind the playback end UI data with the background data. The screen projection service is also bound to an event proxy. Furthermore, the playback end interface description file is encoded and sent to the playback device. In this way, after the playback device receives the encoded playback end interface description file, it can generate and display the playback end UI according to the playback end interface description file.

[0579] The screen projection framework receives the event sent by the playback device and sends the event to the MVVM framework; the MVVM framework updates the background data according to the event. When the business data of the App changes, the background data change causes the MVVM framework to update the playback UI data. The control device sends the updated playback UI data to the playback device. In this way, after the playback device receives the updated playback UI data, it can display the updated playback UI according to the updated playback UI data.

[0580] Please refer to Fig.41B , which shows an example of the processing flow of the playback device in the user interface implementation method provided in the embodiment of the present application. The playback device receives the encoded playback end interface description file sent by the control device through the transmission channel. The screen projection framework calls the custom UI engine to parse and execute the playback end interface description file, form the playback end UI data, and display the playback end UI according to the playback end UI data. The event proxy module receives the user's operation on the playback end UI, generates a corresponding event, and transmits the event to the control device so that the control device processes the event.

[0581] The embodiment of the present application provides a method for implementing a user interface. During the process of controlling a device to run an App, if a preset condition is met, the control device pushes preset information to a playback device for playback.

[0582] Take a mobile phone as the control device and a smart watch as the playback device as an example. Fig.42A, the user opens the "takeout" app on the phone to order food, place an order and pay. Optionally, the app switches to background operation. When the preset conditions are met (for example, the phone confirms that the takeout order is expected to be delivered in 20 minutes; for example, the user performs a query operation on the smart watch), the phone pushes the preset information to the smart watch for display. For example, Fig.42A As shown, the smart watch displays the playback end UI 1410 ; the playback end UI 1410 includes a “takeaway order progress” control 1411 and prompt information 1412 .

[0583] In one implementation, the developer defines a playback end interface description file (or a section of code in the playback end interface description file) that pushes information to the playback end when the preset conditions are met during the development phase. The playback end UI of the smart watch is defined in the playback end interface description file, including controls 1411 and prompt information 1412. During the process of the mobile phone running the "Takeaway" App (including the "Takeaway" App switching to background operation), the mobile phone determines that the preset conditions are met, reads the specified code segment, generates playback end UI data according to the specified code segment, and sends the specified code segment (or the generated playback end UI data) to the smart watch. The smart watch generates the playback end UI 1410 according to the specified code segment (or the generated playback end UI data).

[0584] For example, Fig.42B , the user uses the navigation app on the mobile phone to navigate. When the preset conditions are met (for example, the forward direction is changed), the mobile phone generates the playback end UI data according to the specified code segment, and sends the specified code segment (or the generated playback end UI data) to the smart watch. The smart watch generates the playback end UI 1420 according to the specified code segment (or the generated playback end UI data).

[0585] Furthermore, the smartwatch receives the operation performed by the user on the smartwatch, generates an event corresponding to the operation, and sends the event to the mobile phone for processing. The mobile phone performs business logic processing and executes corresponding actions; and updates the playback end UI data. The mobile phone also sends the updated playback end UI data to the smartwatch, and the smartwatch updates the playback end UI according to the updated playback end UI data.

[0586] The user interface implementation method provided in the embodiment of the present application is that when the preset conditions are met, the control device automatically pushes part of the information of the running App to the playback device for playback. Different types of playback devices can read the code segments corresponding to the device type, which can easily realize the differentiated layout of the playback end UI of the device. In addition, the user can control the App on the playback device, and the control device performs business logic processing; this can improve the user experience and avoid the low performance of the playback device and the failure to support complex business logic processing.

[0587] It is understandable that, in order to realize the above functions, the above electronic device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0588] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0589] like Fig.43 As shown, an embodiment of the present application discloses an electronic device 1500, which may be an electronic device that runs the above-mentioned development tool, or an electronic device that runs the App in the above-mentioned embodiment, or an electronic device that runs the application widget in the above-mentioned embodiment; the electronic device may be the above-mentioned control device or playback device. The electronic device may specifically include: a display screen 1501; an input device 1502 (such as a mouse, keyboard or touch screen, etc.); one or more processors 1503; a memory 1504; one or more application programs (not shown); and one or more computer programs 1505, and the above-mentioned devices may be connected via one or more communication buses 1506. Among them, the above-mentioned one or more computer programs 1505 are stored in the above-mentioned memory 1504 and are configured to be executed by the one or more processors 1503, and the one or more computer programs 1505 include instructions, which can be used to execute the relevant steps in the above-mentioned embodiments. In one example, the electronic device 1500 may be Figure 1 In one example, the electronic device 100 or the electronic device 200 may be Fig.14 In one example, the electronic device 1500 may be a developer device or a user-side electronic device. Fig.23 In one example, the electronic device 100 or the electronic device 200 may be Fig.33 The control device 100 or the electronic device 200 or the playback device 1000.

[0590] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program code is stored. When a processor executes the computer program code, the electronic device executes the method in the above embodiment.

[0591] The embodiments of the present application also provide a computer program product. When the computer program product is run on a computer, the computer executes the method in the above embodiments.

[0592] Among them, the electronic device 1500, computer-readable storage medium or computer program product provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0593] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0594] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0595] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated units may be implemented in the form of hardware or software functional units.

[0596] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROMs, magnetic disks, or optical disks.

[0597] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for implementing a user interface, It is characterized in that include: The electronic device installs an application installation package of a first application, wherein the application installation package includes a first description file and a second description file; The first description file and the second description file are used to perform interface description and interface behavior definition for a first user interface UI of the first application; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; The electronic device runs the first application; wherein the first UI engine of the electronic device reads the first description file, parses and executes the first description file, and generates the first part of the first UI; the second UI engine of the electronic device reads the second description file, parses and executes the second description file, and generates the second part of the first UI; The electronic device displays the first UI.

2. The method according to claim 1, It is characterized in that The first UI engine of the electronic device generates the first part of the first UI including: The first UI engine of the electronic device generates one or more first controls in the first UI according to the first description file; the one or more first controls have first UI programming capabilities.

3. The method according to claim 2, It is characterized in that The second UI engine of the electronic device generating the second part of the first UI includes: The second UI engine of the electronic device applies a second UI programming capability to one or more of the first controls according to the second description file.

4. The method according to claim 2 or 3, It is characterized in that The second UI engine of the electronic device generating the second part of the first UI includes: The second UI engine of the electronic device generates one or more second controls in the first UI according to the second description file; the one or more second controls have second UI programming capabilities.

5. The method according to claim 3, It is characterized in that The second UI programming capability includes: at least one of visual attribute capability, layout capability, unified interaction capability and motion effect capability.

6. The method according to claim 5, It is characterized in that The layout capability includes at least one of stretching, hiding, folding, dividing, proportioning and extending.

7. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: The electronic device determines that the second description file exists, the second UI engine triggers the first UI engine to read the first description file, and the second UI engine reads the second description file.

8. The method according to any one of claims 1 to 3, It is characterized in that The first description file and the second description file are located in different paths in the application installation package.

9. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: The second UI engine performs syntax checking on the second interface description language; If the syntax check passes, the second UI engine parses and executes the second description file.

10. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: The second UI engine of the electronic device implements mapping between device events and user behaviors in the second description file; In response to the device event, a control action corresponding to the user behavior in the second description file is executed.

11. The method according to any one of claims 1 to 3, It is characterized in that The second UI engine includes a set of syntax and semantic specifications of fields in the second description file.

12. A method for implementing a user interface, It is characterized in that include: Displaying a development interface of a first application; The development interface of the first application includes a first description file and a second description file; The first description file and the second description file are used to perform interface description and interface behavior definition for a first user interface UI of the first application; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; In response to a first operation input by a user, adding a description of a first part of the first UI to the first description file; In response to a second operation input by the user, adding a description of a second part of the first UI to the second description file; An application installation package of the first application is generated according to the first description file and the second description file.

13. The method according to claim 12, It is characterized in that The adding a description of the first part of the first UI in the first description file includes: Adding a description of one or more first controls in the first UI to the first description file; A first UI programming capability is applied to the one or more first controls.

14. The method according to claim 13, It is characterized in that The adding a description of the second part of the first UI in the second description file includes: Adding description of one or more of the first controls in the second description file; A second UI programming capability is applied to one or more of the first controls.

15. The method according to claim 13 or 14, It is characterized in that The adding a description of the second part of the first UI in the second description file includes: Adding description of one or more second controls in the second description file; A second UI programming capability is applied to the one or more second controls.

16. The method according to claim 14, It is characterized in that The second UI programming capability includes: at least one of visual attribute capability, layout capability, unified interaction capability and motion effect capability.

17. The method according to claim 16, It is characterized in that The layout capability includes at least one of stretching, hiding, folding, dividing, proportioning and extending.

18. The method according to any one of claims 12 to 14, It is characterized in that The first description file and the second description file are located in different paths in the application installation package.

19. An electronic device, It is characterized in that include: one or more processors; Display screen; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the method as described in any one of claims 1-18.

20. A computer-readable storage medium having stored therein instructions, It is characterized in that When the instructions are executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 12 to 18.

21. A computer-readable storage medium, It is characterized in that The method comprises computer instructions for performing an interface description and an interface behavior definition on a first user interface UI of a first application, wherein: The computer instructions include a first instruction stored in a first description file, and a second instruction stored in a second description file; The first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; The first instruction is used to describe a first part of the first UI, and the second instruction is used to describe a second part of the first UI.

22. The computer-readable storage medium of claim 21, It is characterized in that The first instruction is specifically used for: Describe one or more first controls in the first UI, and apply first UI programming capabilities to the one or more first controls.

23. The computer-readable storage medium of claim 22, It is characterized in that The second instruction is specifically used for: A second UI programming capability is applied to the one or more first controls.

24. The computer-readable storage medium according to claim 22 or 23, It is characterized in that The second instruction is specifically used for: Describe one or more second controls in the first UI, and apply second UI programming capabilities to the one or more second controls.

25. The computer-readable storage medium of claim 23, It is characterized in that The second UI programming capability includes: at least one of visual attribute capability, layout capability, unified interaction capability and motion effect capability.

26. The computer-readable storage medium of claim 25, It is characterized in that The layout capability includes at least one of stretching, hiding, folding, dividing, proportioning and extending.

27. The computer-readable storage medium according to any one of claims 21 to 23, It is characterized in that The first description file and the second description file are located in different paths in the computer-readable storage medium.

Images