User interface implementation method and device

Abstract

The embodiment of the invention provides a user interface implementation method and device, and relates to the technical field of terminals. The application installation package of the first application comprises a first description file and a second description file which are used for performing interface description and interface behavior definition on the UI of the first application; the first description file and the second description file adopt different interface description languages; wherein the first description file adopts a first interface description language, and the second description file adopts a second interface description language. The electronic equipment runs the first application; the first UI engine reads, analyzes and executes the first description file to generate a first part of the first UI; and the second UI engine reads, analyzes and executes the second description file to generate a second part of the first UI. The first UI engine can be a universal operating system engine, and the second UI engine is not related to an operating system platform, so that the method can adapt to various operating system platforms, and the technical implementation difficulty is low.

Description

[0001] This application is a divisional application. The application number of the original application is 202011475517.8, and the original application date is December 14, 2020. The entire content of the original application is incorporated into this application by reference.

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 25, 2020, with application number 202010862489.9 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application.

[0003] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 30, 2020, with application number 202011064544.6 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application.

[0004] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 22, 2020, with application number 202011141010.9 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application.

[0005] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 22, 2020, with application number 202011142718.6 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application.

[0006] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 30, 2020, with application number 202011381146.7 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application.

[0007] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 30, 2020, with application number 202011384490.1 and application name “A User Interface Implementation Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0008] The present application relates to the field of terminal technology, and in particular to a method and device for implementing a user interface. Background Art

[0009] Developers usually develop applications (Applications, Apps) based on a certain operating system (operator system, OS) platform. In the App development process, a very important task is to develop the App's user interface (UI). Usually, developers use the software development kit (SDK) provided by the OS platform to develop the App's UI. UI development mainly includes interface description and interface behavior definition. Interface description refers to the use of an interface description language to describe the layout of the UI, the controls used, and the visual style of the layout and controls. Interface behavior definition refers to the use of an interface description language to define interface behavior; interface behavior includes dynamic changes in the UI, and the response of electronic devices to dynamic changes in the UI (such as the response to user operations on the UI). Each OS platform has its corresponding interface description language; for example Using XML (extensible markup language) format, Use the embedded domain specific language (EDSL) built with Swift to describe the interface and define the interface behavior. The UI engine provided by the OS platform can interpret and execute the UI description language and render the UI to the user. In addition, each OS platform also has a corresponding programming language for implementing interface behavior, realizing dynamic changes in the UI, and responding to user operations on the UI; for example Using JAVA, Use Swift programming language to implement interface behavior.

[0010] How to provide convenience for developers so that they can easily develop a UI that is compatible with the operating system and rich in functions is a problem that needs to be solved. Summary of the Invention

[0011] The embodiments of the present application provide a method and device for implementing a user interface, which can provide rich UI programming capabilities and facilitate developers to develop a functionally rich UI that is compatible with the operating system. To achieve the above purpose, the present application adopts the following technical solutions:

[0012] In a first aspect, the present application provides a user interface implementation method, including: an application installation package for a first application of an electronic device, the application installation package including a first description file and a second description file; wherein the first description file and the second description file are used to describe the interface and define the interface behavior of a first user interface UI of the first application; the first description file uses a first interface description language, and the second description file uses a second interface description language; the first interface description language and the second interface description language are different. The electronic device runs the first application; wherein the first UI engine of the electronic device reads, parses, and executes the first description file to generate a first part of the first UI; the second UI engine of the electronic device reads, parses, and executes the second description file to generate a second part of the first UI; and the electronic device displays the first UI.

[0013] In this method, two different interface description languages ​​are supported for co-development of UI. The operating system of the electronic device includes two UI engines, which respectively parse and execute the two different interface description languages. One of the UI engines can be a general-purpose OS (e.g. ) UI engine, which can parse a common interface description language; another UI engine is an extended UI engine, independent of the OS platform, that can parse a DSL. This allows developers to use the basic UI description language to describe the UI layout, including controls, and so on; and selectively use the DSL to apply custom UI programming capabilities to certain controls, such as adding animation effects to the UI. The extended UI engine provided in the embodiments of this application is independent of the OS platform and can therefore adapt to multiple OS platforms, with low technical implementation difficulty and convenient use by developers.

[0014] In one possible implementation, the first UI engine of the electronic device generates the first part of the first UI, including: the first UI engine of the electronic device generates one or more first controls in the first UI according to the first description file; wherein the one or more first controls have first UI programming capabilities.

[0015] The first control is a general-purpose OS (such as ) generated controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of a control; selecting a control; and entering text on the control.

[0016] In one possible implementation, the second UI engine of the electronic device generating the second portion of the first UI includes: the second UI engine of the electronic device applying the second UI programming capability to one or more first controls according to the second description file. In other words, a developer can use a custom interface description language to apply custom UI programming capabilities to controls generated by a universal OS in the second description file, thereby expanding the capabilities of the universal OS controls and enriching the usability of the universal OS controls.

[0017] In one possible implementation, the second UI engine of the electronic device generates the second part of the first UI, including: the second UI engine of the electronic device generates one or more second controls in the first UI according to the second description file; wherein the one or more second controls have second UI programming capabilities.

[0018] The second control is a custom control provided by the OEM OS in the embodiment of the present application, which supports custom second UI programming capabilities and rich control effects.

[0019] In one possible implementation, the second UI programming capability includes at least one of: visual attribute capability, layout capability, unified interaction capability, and motion effect capability.

[0020] Layout capabilities are used to describe the layout of controls in the UI, such as their shape, position, and size. Visual attribute capabilities are used to describe the visual attributes of controls, such as their color, grayscale, and other visual effects. Unified interaction capabilities are used to provide control responses based on user behavior, such as performing a search based on a user's "confirm" behavior. Motion effect capabilities are used to display animation effects on controls, such as displaying a click-and-bounce effect on controls.

[0021] In one possible implementation, the layout capability includes at least one of: stretch, hide, wrap, evenly distribute, proportion, and extend. Stretching refers to the ability to enlarge or reduce the width and height of a control at different ratios; hiding refers to the ability to make a control visible or invisible on a display interface; wrapping refers to the ability to display the content of a control across one or more lines on a display interface; evenly distribute refers to the ability to evenly distribute controls on a display interface; proportion refers to the ability to occupy a specified percentage of the total layout in a specified direction; and extending refers to the ability to arrange controls in a single direction on a UI.

[0022] In one possible implementation, if the second UI engine determines that the second description file exists, it triggers the first UI engine to read the first description file, and the second UI engine to read the second description file. In other words, the second UI engine controls the distribution process, triggering the first UI engine and the second UI engine to parse and execute the description file.

[0023] In a possible implementation, the first description file and the second description file are located in different paths in the application installation package, and the first UI engine and the second UI engine read the description files in different paths according to a preset rule.

[0024] In a possible implementation, different tags are preset in the first description file and the second description file, and the first UI engine and the second UI engine read the corresponding description files respectively according to the preset tags.

[0025] In a possible implementation, the second UI engine further performs syntax verification on the second interface description language; if the syntax verification passes, the second UI engine parses and executes the second description file.

[0026] In one possible implementation, the electronic device's second UI engine maps device events to user behaviors in the second description file. In response to device events, the control action corresponding to the user behavior in the second description file is executed. In other words, the OEM OS can map events triggered by different electronic devices to the same user behavior (for example, mapping a double-click on a PC to a "confirm" behavior and a single-click on a phone to a "confirm" behavior). This avoids duplication of effort for developers in defining the mapping between device events and user behaviors for different electronic devices. This allows the same description file to be applied to multiple electronic devices, reducing development complexity and bringing convenience to developers.

[0027] In a possible implementation, the second UI engine includes a set of syntax and semantic specifications of the fields in the second description file, so that developers can develop UI on the OEM OS platform according to the syntax and semantic specifications of the OEM OS.

[0028] In a possible implementation, the first interface description language is the extensible markup language (XML), and the second interface description language is a domain-specific language (DSL).

[0029] In a second aspect, the present application provides a user interface implementation method, including: displaying a development interface of a first application; the development interface of the first application includes a first description file and a second description file; the first description file and the second description file are used to describe the interface and define the interface behavior of the first user interface UI of the first application; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; in response to a first operation input by the user, a description of the first part of the first UI is added to the first description file; in response to a second operation input by the user, a description of the second part of the first UI is added to the second description file; and an application installation package of the first application is generated according to the first description file and the second description file.

[0030] In this method, developers can use two different interface description languages ​​to jointly develop UI. One of the languages ​​is a general-purpose OS (such as ) and a custom UI description language. Developers can use the basic UI description language to describe UI layouts and included controls; they can also selectively use DSL to apply custom UI programming capabilities to certain controls, adding animations and other features. Because custom UI description languages ​​are independent of the operating system (OS), they can adapt to multiple platforms, simplify technical implementation, and facilitate developer use.

[0031] In a possible implementation, adding a description of the first part of the first UI in the first description file includes: adding a description of one or more first controls in the first UI in the first description file; and applying the first UI programming capability to the one or more first controls.

[0032] The first control is a general OS (such as ) supports controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of a control; selecting a control; and entering text on the control.

[0033] In a possible implementation, adding a description of the second part of the first UI in the second description file includes: adding a description of one or more first controls in the second description file; and applying the second UI programming capability to the one or more first controls.

[0034] That is, developers can use the custom interface description language to apply custom UI programming capabilities to the controls generated by the universal OS in the second description file, thereby expanding the capabilities of the universal OS controls and enriching the usage effects of the universal OS controls.

[0035] In one possible implementation, adding a description of the second portion of the first UI to the second description file includes: adding a description of one or more second controls to the second description file; and applying a second UI programming capability to the one or more second controls. The second controls are custom controls provided by the OEM OS in the embodiment of the present application, supporting custom second UI programming capabilities and rich control effects.

[0036] In one possible implementation, the second UI programming capability includes at least one of: visual attribute capability, layout capability, unified interaction capability, and motion effect capability.

[0037] In a possible implementation, the layout capability includes at least one of stretching, hiding, folding, dividing, proportioning, and extending.

[0038] In one possible implementation, the first description file and the second description file are located in different paths in the application installation package. In this way, the first UI engine and the second UI engine of the OEM OS can read files in different paths according to preset rules to obtain corresponding description files.

[0039] In a possible implementation, different tags are preset in the first description file and the second description file, so that the first UI engine and the second UI engine of the OEM OS can read the corresponding description files respectively according to the preset tags.

[0040] In a third aspect, the present application provides a computer-readable storage medium comprising computer instructions for performing an interface description and interface behavior definition for a first user interface UI of a first application, wherein the computer instructions comprise a first instruction stored in a first description file and a second instruction stored in a second description file; the first description file adopts a first interface description language, and the second description file adopts a second interface description language; the first interface description language is different from the second interface description language; the first instruction is used to describe a first part of the first UI, and the second instruction is used to describe a second part of the first UI.

[0041] In this method, developers use two different interface description languages ​​to jointly develop the UI. One of the interface description languages ​​is a general-purpose OS (e.g. ) supports a basic UI description language. Another UI description language is a custom UI description language. Developers use the basic UI description language to describe UI layouts and included controls. They can also selectively use DSL to apply custom UI programming capabilities to certain controls, adding animations and other features. Because custom UI description languages ​​are independent of the operating system (OS), they can adapt to multiple platforms, simplify technical implementation, and are convenient for developers.

[0042] In a possible implementation, the first instruction is specifically used to: describe one or more first controls in the first UI, and apply the first UI programming capability to the one or more first controls.

[0043] The first control is a general OS (such as ) supports controls. The first UI programming capability is the general OS (such as ) supports UI programming capabilities. For example, it includes setting the length, width, height, spacing, and color of a control; selecting a control; and entering text on the control.

[0044] In one possible implementation, the second instruction is specifically used to apply the second UI programming capability to one or more first controls. In another possible implementation, the second instruction is specifically used to describe one or more second controls in the first UI and apply the second UI programming capability to the one or more second controls.

[0045] In other words, developers can use a custom interface description language to apply custom UI programming capabilities to controls generated by the universal OS in the second description file, thereby expanding the capabilities of universal OS controls; they can also add custom controls with rich control effects.

[0046] In one possible implementation, the second UI programming capability includes at least one of visual attribute capability, layout capability, unified interaction capability, and motion effect capability. The layout capability includes at least one of stretching, hiding, line breaking, equal distribution, proportion, and extension.

[0047] In one possible implementation, the first description file and the second description file are located in different paths in the computer-readable storage medium. In this way, the first UI engine and the second UI engine of the OEM OS can read files in different paths according to preset rules to obtain corresponding description files.

[0048] In a possible implementation, different tags are preset in the first description file and the second description file, so that the first UI engine and the second UI engine of the OEM OS can read the corresponding description files respectively according to the preset tags.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned first aspect.

[0050] In a fifth aspect, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute the one or more computer programs stored in the memory so that the electronic device performs any one of the methods described in the first aspect above.

[0051] In the sixth aspect, the present application provides an electronic device comprising: a display screen, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the display screen and the memory, and the one or more computer programs are stored in the memory. When the electronic device runs the first application, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the second aspect.

[0052] The embodiments of this application provide a method and apparatus for implementing a user interface, which can achieve one-time development and multi-device deployment; that is, develop a set of interface description files that are applicable to various types of electronic devices, thereby reducing the development difficulty for developers. To achieve the above objectives, this application adopts the following technical solutions:

[0053] In the seventh aspect, the present application provides a method for implementing a user interface, including: a first electronic device and a second electronic device respectively download an application installation package of a first application from a server; and respectively install the application installation package. The application installation package includes a description file and a resource file; wherein the description file is used to perform an interface description and interface behavior definition for the first UI of the first application; the resource file includes resources used to generate the UI of the first application. The first electronic device reads a first code corresponding to the device type of the first electronic device in the description file, and uses the resources of the resource file according to the definition of the first code to generate the first UI of the first electronic device; the second electronic device reads a second code corresponding to the device type of the second electronic device in the description file, and uses the resources of the resource file according to the definition of the second code to generate the first UI of the second electronic device; the device type of the first electronic device is different from the device type of the second electronic device. Among them, the device types of electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screens, car computers, etc.

[0054] In this method, different types of electronic devices read the same description file for the same UI and present different UI layouts. This allows the development of a single set of description files that are applicable to various types of electronic devices, reducing the development difficulty for developers.

[0055] In combination with the seventh aspect, in a possible implementation, the method also includes: the first electronic device generates a first control in the first UI of the first electronic device according to the definition of the third code in the description file, and the first control in the first UI of the first electronic device has control properties customized by the operating system of the first electronic device; the third electronic device generates a first control in the first UI of the third electronic device according to the definition of the third code in the description file, and the first control in the first UI of the third electronic device has control properties of a general operating system; wherein the third code is part or all of the first code.

[0056] In this method, the description file defines that the first control supports the control properties customized by the operating system. The operating system of the first electronic device provides customized control properties, and the first control in the first UI of the first electronic device has the control properties customized by the operating system of the first electronic device; the third electronic device supports a general operating system (such as ), the first control in the first UI of the third electronic device has the control properties of a universal operating system. In this way, the same description file can be successfully run in different operating systems, achieving cross-operating system platform operation and reducing development difficulty for developers.

[0057] In an eighth aspect, the present application provides a user interface implementation method, comprising: a first electronic device downloading an application installation package of a first application; and installing the application installation package; wherein the application installation package includes a description file and a resource file; the description file is used to describe the interface and define the interface behavior of a first user interface (UI) of the first application; and the resource file includes resources used to generate the UI of the first application. The first electronic device reads a first code corresponding to the device type of the first electronic device in the description file, and uses resources in the resource file to generate the first UI of the first electronic device according to the definition of the first code.

[0058] In this method, an electronic device reads the code corresponding to its device type in a description file. This allows different electronic devices to present different UI layouts when reading the same description file. This allows a single set of description files to be developed for a variety of different electronic devices, reducing development complexity for developers.

[0059] In combination with the eighth aspect, in a possible implementation method, the first electronic device generates a first control in the first UI of the first electronic device according to the definition of the third code in the description file, and the first control in the first UI of the first electronic device has control properties customized by the operating system of the first electronic device; wherein the third code is part or all of the first code.

[0060] In the method, the description file defines that the first control supports control properties customized by the operating system. The operating system of the first electronic device provides customized control properties, and the first control in the first UI of the first electronic device has the control properties customized by the operating system of the first electronic device.

[0061] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the operating system of the first electronic device includes a custom UI programming capability, and the custom UI programming capability is used to provide custom control properties of the operating system of the first electronic device.

[0062] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the customized control properties include: at least one of: visual properties, layout properties, interaction properties, motion properties and software and hardware dependency properties.

[0063] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the layout attributes include: at least one of stretching, hiding, folding, dividing, proportioning, and extending.

[0064] In conjunction with the seventh or eighth aspects, in one possible implementation, a first UI of a first electronic device includes a second control, and the second control has control attributes of a general operating system. In other words, the first UI generated by the first electronic device based on the description file may include both controls with control attributes customized by the operating system of the first electronic device and controls with control attributes of a general operating system. This provides a richer range of controls.

[0065] In conjunction with the seventh or eighth aspects, in one possible implementation, a first UI of a first electronic device includes a third control, and the third control has control properties customized by a first application. In this method, a developer can customize the control properties belonging to the first application in a file of the installation package to enrich the UI.

[0066] In conjunction with the seventh or eighth aspect, in one possible implementation, the description file includes fourth code for defining a correspondence between a control attribute of a fourth control in a first UI of a first electronic device and first data in an operating system of the first electronic device. The method further includes: the first electronic device receiving a first user input on the fourth control; and modifying a value of the first data based on the first input.

[0067] In this method, developers define the correspondence between control properties and background data in the operating system in a description file. The electronic device's UI engine then implements the function of modifying background data based on user input. This eliminates the need for developers to describe how to modify background data based on user input in the description file, reducing development complexity.

[0068] In combination with the seventh aspect or the eighth aspect, in a possible implementation, the method further includes: the control property of the fourth control in the first UI of the first electronic device changes as the first data in the operating system of the first electronic device changes.

[0069] In this method, developers define the correspondence between control properties and operating system background data in a description file. The electronic device's UI engine then implements changes to the control properties as the background data in the electronic device's operating system changes. This allows controls in the UI to change as electronic device parameters change, eliminating the need for developers to specify in the description file how control properties change as electronic device parameters change, thus reducing development complexity.

[0070] In a ninth aspect, the present application provides a method for implementing a user interface, comprising: displaying a development interface of a first application; the development interface of the first application includes a description file for performing an interface description and interface behavior definition on a first user interface UI of the first application; in response to a first operation input by a user, adding a first code corresponding to a device type of a first electronic device to the description file; in response to a second operation input by a user, adding a second code corresponding to a device type of a second electronic device to the description file; and generating an application installation package for the first application according to the description file. The device type of the first electronic device is different from the device type of the second electronic device. The device types of electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screens, car computers, and the like.

[0071] In this method, a single description file contains code corresponding to different types of electronic devices. Different electronic devices can read the same description file for the same UI and present different UI layouts. This allows developers to develop a single set of description files that can be applied to a wide variety of electronic devices, reducing development complexity.

[0072] In conjunction with the ninth aspect, in a possible implementation, the application installation package of the first application also includes a resource file, and the resource file includes resources used to generate the UI of the first application.

[0073] In conjunction with the ninth aspect, in one possible implementation, the description file includes third code defining that the first control has control attributes customized by the operating system of the first electronic device, and fourth code defining that the second control has control attributes of a general operating system. In other words, the first UI generated by the electronic device based on the description file can include both controls with control attributes customized by the operating system of the first electronic device and controls with control attributes of a general operating system. This provides a richer range of controls.

[0074] In combination with the ninth aspect, in a possible implementation, the control attributes customized by the operating system of the first electronic device include: at least one of: visual attributes, layout attributes, interaction attributes, motion effect attributes and software and hardware dependency attributes.

[0075] In conjunction with the ninth aspect, in a possible implementation, the layout attributes include: at least one of stretching, hiding, folding, dividing, proportioning, and extending.

[0076] In the tenth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned ninth aspect.

[0077] In an eleventh aspect, the present application provides a computer-readable storage medium comprising computer instructions for describing and defining interface behaviors of a first user interface (UI) of a first application; wherein the computer instructions comprise a first code corresponding to a device type of a first electronic device and a second code corresponding to a device type of a second electronic device; wherein the device type of the first electronic device is different from the device type of the second electronic device. The device types of the electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptops, netbooks, large screen computers, and in-vehicle computers.

[0078] In conjunction with the eleventh aspect, in a possible implementation, the computer instructions further include generating resources used by the UI of the first application.

[0079] In combination with the eleventh aspect, in a possible implementation, the computer instructions also include a third code defining that the first control has control properties customized by the operating system of the first electronic device, and a fourth code defining that the second control has control properties of a general operating system.

[0080] In the twelfth aspect, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the above-mentioned eighth aspect.

[0081] In the thirteenth aspect, the present application provides an electronic device comprising: a display screen, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the display screen and the memory, and the one or more computer programs are stored in the memory. When the electronic device runs the first application, the processor can execute the one or more computer programs stored in the memory so that the electronic device executes any one of the methods described in the ninth aspect.

[0082] The embodiments of the present application provide a method and apparatus for implementing a user interface, which supports displaying various layout modes and control types on the UI of an application widget, making it easier for users to use the application widget and improving the user experience. To achieve the above objectives, the present application adopts the following technical solutions:

[0083] In the fourteenth aspect, the present application provides a user interface implementation method, including: the first application process of the electronic device reads the component interface description file, and generates first widget UI data based on the component interface description file, and binds the controls in the first widget UI data to the background data in the electronic device operating system; wherein, the component interface description file is used to perform interface description and interface behavior definition for the first UI of the application widget of the first application; then, the first application process sends the first data to the application widget process; the application widget process receives the first data, obtains the first widget UI data based on the first data, and displays the first UI of the application widget according to the first widget UI data.

[0084] In this method, both the application process and the widget process generate widget UI data based on the widget interface description file. The application process binds the controls in the widget UI data to the backend data, and the widget process displays the widget UI data as the widget UI. This allows developers to define various control types in the widget interface description file, allowing the widget UI to support a variety of control types. When a user operates on the widget UI, the application process executes the corresponding business logic based on the correspondence between the controls in the widget UI data and the backend data.

[0085] In one possible implementation, the first application process sends a component interface description file to the application widget process; the application widget process receives the component interface description file, generates first widget UI data based on the component interface description file, and displays the first UI of the application widget according to the first widget UI data.

[0086] In one possible implementation, the first application process sends first widget UI data to the application widget process; the application widget process receives the first widget UI data and displays the first UI of the application widget according to the first widget UI data.

[0087] In conjunction with aspect 14, in one possible implementation, the method further includes: generating, by the first application process, a first control in the first widget UI data according to a definition of the first code in the component interface description file, wherein the first control has control attributes native to the operating system of the electronic device. The native operating system controls include: an input box, a check box, a slide selector, a scroll view, a radio button, a rating bar, a search box, a drag bar, or a switch.

[0088] In other words, developers can define various native control properties of the operating system in the component interface description file, so that the UI of the application widget supports various native controls of the operating system.

[0089] In conjunction with the fourteenth aspect, in one possible implementation, the method further includes: the first application process generating a second control in the first widget UI data according to the definition of the second code in the component interface description file, wherein the second control has control properties customized by the operating system of the electronic device. The customized control properties include: at least one of visual properties, layout properties, interaction properties, motion properties, and software and hardware dependency properties. The layout properties include: at least one of stretch, hide, wrap, split evenly, proportion, and extend.

[0090] In other words, developers can define various custom control properties in the operating system in the component interface description file, so that the UI of the application widget supports various custom controls in the operating system.

[0091] In combination with the fourteenth aspect, in a possible implementation, the component interface description file includes a third code for defining the correspondence between the control properties of the third control in the first UI of the application widget and the first data in the operating system of the electronic device. The method also includes: the electronic device receives the user's first input on the third control; and modifies the value of the first data according to the first input.

[0092] In combination with the fourteenth aspect, in a possible implementation, the control properties of the third control in the first UI of the application widget change as the first data in the operating system of the electronic device changes.

[0093] In combination with the fourteenth aspect, in a possible implementation, the method further includes: the electronic device downloads an application installation package of the first application from the server; the application installation package includes a component interface description file; and the electronic device uses the application installation package to install the first application.

[0094] In this method, the component interface description file is obtained from the application installation package.

[0095] In a fifteenth aspect, the present application provides a method for implementing a user interface, comprising: displaying a development interface of a first application; the development interface of the first application includes a component interface description file; the component interface description file is used to describe the interface and define the interface behavior of a first UI of an application widget of the first application. In response to a first operation input by a user, first code defining a first control in the first widget UI is added to the component interface description file; wherein the first control has control properties native to the operating system; native operating system controls include: an input box, a check box, a slide selector, a scroll view, a radio button, a rating bar, a search box, a drag bar, or a switch, etc. An application installation package for the first application is generated based on the component interface description file.

[0096] In this method, the developer can define in the component interface description file that the control has the native control properties of the operating system, so that the UI of the application widget running on the electronic device includes various controls with the native control properties of the operating system.

[0097] In conjunction with the fifteenth aspect, in one possible implementation, the method further includes: in response to a second operation input by the user, adding, to the component interface description file, second code defining a second control in the first widget UI, wherein the second control has control properties customized by the operating system; the customized control properties include at least one of visual properties, layout properties, interaction properties, animation properties, and software and hardware dependency properties. The layout properties include at least one of stretch, hide, wrap, split evenly, proportion, and extend.

[0098] In this method, the developer can define in the component interface description file that the control has control properties customized by the operating system, so that the UI of the application widget running on the electronic device includes various controls with control properties customized by the operating system.

[0099] In the sixteenth aspect, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device can execute any one of the methods described in the above-mentioned fifteenth aspect.

[0100] In the seventeenth aspect, the present application provides a computer-readable storage medium, including computer instructions, which are used to describe the interface and define the interface behavior of a first user interface UI of an application widget of a first application; wherein the computer instructions include a first code for generating a first control in the first widget UI, and the first control has control properties native to the operating system; the native controls of the operating system include: input boxes, check boxes, sliding selectors, scroll views, radio buttons, rating bars, search boxes, drag bars, or switches, etc.

[0101] In conjunction with aspect 17, in one possible implementation, the computer instructions further include second code for generating a second control in the first widget UI, wherein the second control has control properties customized by the operating system; the customized control properties include at least one of visual properties, layout properties, interaction properties, motion properties, and software and hardware dependency properties. The layout properties include at least one of stretch, hide, wrap, split evenly, proportion, and extend.

[0102] In an eighteenth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes any one of the methods described in the fourteenth aspect.

[0103] In the nineteenth aspect, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute the one or more computer programs stored in the memory to enable the electronic device to execute the method described in any one of the above-mentioned fourteenth or fifteenth aspects.

[0104] The embodiments of this application provide a method and device for implementing a user interface, which supports projecting various UIs on a control device to an IoT device for playback, thereby improving the user experience. To achieve the above objectives, this application adopts the following technical solutions:

[0105] In the twentieth aspect, the present application provides a user interface implementation method, including: a first electronic device reads a first playback end interface description file of a first application, and generates first playback end UI data based on the first playback end interface description file, and binds the controls in the first playback end UI data to the background data in the operating system of the first electronic device; wherein the first playback end interface description file is used to perform interface description and interface behavior definition for the first playback end UI of the first application played on the second electronic device; the first electronic device sends the first data to the second electronic device; the second electronic device receives the first data, obtains the first playback end UI data based on the first data, and displays the first playback end UI according to the first playback end UI data.

[0106] In this method, both the control device and the player generate player UI data based on a player interface description file. The control device binds the controls in the player UI data to the background data, and the player displays the player UI data as the player UI. This allows developers to define various UIs within the player interface description file, enriching the player UI. Different UI layouts can also be defined for players of different device types, ensuring that the player UI matches the size and shape of the player screen. When a user operates on the player UI, the control device executes the corresponding business logic based on the correspondence between the controls in the player UI data and the background data.

[0107] In one possible implementation, the first electronic device sends a first playback end interface description file to the second electronic device. The second electronic device generates first playback end UI data based on the first playback end interface description file and displays the first playback end UI according to the first playback end UI data.

[0108] In a possible implementation, the first electronic device sends the first playback end UI data to the second electronic device. The second electronic device receives the first playback end UI data and displays the first playback end UI according to the first playback end UI data.

[0109] In combination with the twentieth aspect, in a possible implementation, the method also includes: the second electronic device receives the first operation of the user on the first playback end UI; in response to the first operation of the user on the first playback end UI, the second electronic device sends a first instruction to the first electronic device; the first electronic device receives the first instruction, reads the second playback end interface description file, generates second playback end UI data according to the second playback end interface description file, and binds the controls in the second playback end UI data with the background data in the operating system of the first electronic device; wherein the second playback end interface description file is used to perform interface description and interface behavior definition for the second playback end UI that plays the first application on the second electronic device; the first electronic device sends the second playback end interface description file to the second electronic device; the second electronic device receives the second playback end interface description file, generates second playback end UI data according to the second playback end interface description file, and displays the second playback end UI according to the second playback end UI data.

[0110] In this method, the user can directly operate the player UI on the playback device, controlling the device to execute the business logic corresponding to the operation and sending the playback device an updated playback UI description file corresponding to the playback UI. The playback device then generates an updated playback UI based on the updated playback UI description file. This allows users to directly operate the playback UI on the playback device and successfully switch the playback UI.

[0111] In combination with the twentieth aspect, in a possible implementation, the method also includes: the second electronic device receives the first operation of the user on the first playback end UI; in response to the first operation of the user on the first playback end UI, the second electronic device sends a first instruction to the first electronic device; the first electronic device receives the first instruction and reads the second playback end interface description file; the second playback end interface description file is used to perform interface description and interface behavior definition for the second playback end UI that plays the first application on the second electronic device; the first electronic device generates second playback end UI data based on the second playback end interface description file, and binds the controls in the second playback end UI data with the background data in the operating system of the first electronic device; the first electronic device sends the second playback end UI data to the second electronic device; the second electronic device receives the second playback end UI data and displays the second playback end UI according to the second playback end UI data.

[0112] In this method, users can directly operate the playback UI on the playback device, controlling the device to execute the business logic corresponding to the operation and send updated playback UI data to the playback device. The playback device then updates the playback UI based on the updated playback UI data. This allows users to directly operate the playback UI on the playback device and successfully switch the playback UI.

[0113] In combination with the twentieth aspect, in a possible implementation, the method also includes: the first electronic device downloads an application installation package of the first application from the server; the application installation package includes a first playback end interface description file and a resource file; the resource file includes resources used to generate the playback end UI of the first application; the first electronic device uses the application installation package to install the first application.

[0114] In conjunction with the twentieth aspect, in a possible implementation, the method further includes: the first electronic device reads a first code corresponding to the device type of the third electronic device in the first playback-end interface description file, and uses the resources of the resource file according to the definition of the first code to generate third playback-end UI data; the first electronic device reads a second code corresponding to the device type of the fourth electronic device in the first playback-end interface description file, and uses the resources of the resource file according to the definition of the second code to generate fourth playback-end UI data; wherein the device type of the fourth electronic device is different from the device type of the third electronic device. The first electronic device sends the first playback-end interface description file and the resource file to the third electronic device and the fourth electronic device respectively; the third electronic device uses the resources of the resource file according to the definition of the first code corresponding to the device type of the third electronic device in the first playback-end interface description file; the first playback-end UI is displayed according to the third playback-end UI data; the fourth electronic device uses the resources of the resource file according to the definition of the second code corresponding to the device type of the fourth electronic device in the first playback-end interface description file to generate fourth playback-end UI data; the first playback-end UI is displayed according to the fourth playback-end UI data.

[0115] In this method, different types of playback devices read the same playback interface description file for the same UI and present different playback UI layouts. This allows the development of a single set of playback interface description files that are applicable to various types of playback devices, reducing the development effort for developers.

[0116] In conjunction with the twentieth aspect, in one possible implementation, the method further includes: the first electronic device reading a first code corresponding to the device type of the third electronic device in the first playback-end interface description file, and using resources in the resource file as defined by the first code to generate third playback-end UI data; the first electronic device reading a second code corresponding to the device type of the fourth electronic device in the first playback-end interface description file, and using resources in the resource file as defined by the second code to generate fourth playback-end UI data; wherein the device type of the fourth electronic device is different from the device type of the third electronic device. The first electronic device sends the third playback-end UI data to the third electronic device; the third electronic device displays the first playback-end UI according to the third playback-end UI data; the first electronic device sends the fourth playback-end UI data to the fourth electronic device; and the fourth electronic device displays the first playback-end UI according to the fourth playback-end UI data.

[0117] In this method, different types of playback devices can present different playback UI layouts based on the same playback interface description file. This allows the development of a single set of playback interface description files that can be applied to various types of playback devices, reducing the development effort for developers.

[0118] In conjunction with the twentieth aspect, in one possible implementation, the method further includes: the first electronic device generating a first control in the first playback end UI according to the definition of the third code in the first playback end interface description file, the first control having control attributes customized by the operating system of the first electronic device; wherein the control attributes customized by the operating system of the first electronic device include: at least one of visual attributes, layout attributes, interaction attributes, motion effect attributes, and software and hardware dependency attributes. The layout attributes include: at least one of stretch, hide, wrap, evenly divide, proportion, and extend.

[0119] In conjunction with aspect 20, in one possible implementation, the method further includes: the first electronic device displaying the first playback UI according to the first playback UI data. In this method, the control device and the playback device synchronously play the playback UI, which can achieve mirroring and projection, and the control device and the playback device work together.

[0120] In the twenty-first aspect, the present application provides a user interface implementation method, including: displaying the development interface of a first application; the development interface of the first application includes a playback end interface description file; the playback end interface description file is used to perform interface description and interface behavior definition for the playback end UI of the first application played on the playback end; in response to the user's first input, adding a first code corresponding to the device type of the first electronic device to the playback end interface description file; in response to the user's second input, adding a second code corresponding to the device type of the second electronic device to the playback end interface description file; the device type of the first electronic device is different from the device type of the second electronic device; and generating an application installation package for the first application according to the playback end interface description file.

[0121] In this method, a single player interface description file includes code corresponding to different types of playback devices. Different playback devices can access the same player interface description file for the same UI and present different player UI layouts. This allows developers to develop a single set of player interface description files that can be used by a variety of different playback devices, reducing development complexity.

[0122] In combination with the twenty-first aspect, in a possible implementation, the application installation package of the first application also includes a resource file, and the resource file includes resources used to generate the playback end UI of the first application.

[0123] In conjunction with the twenty-first aspect, in one possible implementation, the playback end interface description file includes third code defining a first control in the first playback end UI as having control attributes customized by the operating system of the first electronic device; the control attributes customized by the operating system of the first electronic device include at least one of visual attributes, layout attributes, interaction attributes, motion attributes, and software and hardware dependency attributes. The layout attributes include at least one of stretch, hide, wrap, evenly divide, proportion, and extend.

[0124] In aspect 22, the present application provides a computer-readable storage medium, for example, an application development tool, which may specifically include computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device may execute any one of the methods described in aspect 21 above.

[0125] In a twenty-third aspect, the present application provides a computer-readable storage medium comprising computer instructions for describing an interface and defining interface behaviors for a first playback end UI of a first application; wherein the computer instructions comprise a first code corresponding to a device type of a first electronic device and a second code corresponding to a device type of a second electronic device; wherein the device type of the first electronic device is different from the device type of the second electronic device. The device types of the electronic devices may include mobile phones, smart TVs, smart watches, tablet computers, laptop computers, netbooks, large screen computers, and in-vehicle computers.

[0126] In combination with the twenty-third aspect, in one possible implementation, the computer instructions also include generating resources used by the playback end UI of the first application.

[0127] In conjunction with aspect 23, in one possible implementation, the computer instructions further include third code for defining that the first control in the UI of the first playback terminal has control attributes customized by the operating system of the first electronic device; the control attributes customized by the operating system of the first electronic device include at least one of visual attributes, layout attributes, interaction attributes, motion attributes, and software and hardware dependency attributes. The layout attributes include at least one of stretch, hide, wrap, evenly divide, proportion, and extend.

[0128] In a twenty-fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes any one of the methods described in the twentieth aspect.

[0129] In aspect 25, the present application provides an electronic device comprising: a display screen, an input device, one or more processors, one or more memories, and one or more computer programs; wherein the processor is coupled to the input device, the display screen, and the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute one or more computer programs stored in the memory to enable the electronic device to perform the method described in any one of aspect 20 or aspect 21 above.

[0130] It can be understood that the electronic devices and computer-readable storage media provided in the above aspects are all applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0131] Figure 1 A schematic diagram of a scenario for implementing a user interface method provided in an embodiment of the present application;

[0132] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0133] Figure 3 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0134] Figure 4 A schematic diagram of a method for implementing a user interface;

[0135] Figure 5 A schematic diagram of a method for implementing a user interface;

[0136] Figure 6 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0137] Figure 7 A schematic diagram of the architecture of a method for implementing a user interface provided in an embodiment of the present application;

[0138] Figure 8 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0139] Figure 9 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0140] Figure 10 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0141] Figure 11A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0142] Figure 12 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0143] Figure 13 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0144] Figure 14 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0145] Figure 15 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0146] Figure 16 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0147] Figure 17 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0148] Figure 18 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0149] Figure 19 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0150] Figure 20A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0151] Figure 20B A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0152] Figure 20C A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0153] Figure 21 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0154] Figure 22 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0155] Figure 23 A schematic diagram of a scenario for implementing a user interface method provided in an embodiment of the present application;

[0156] Figure 24AA schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0157] Figure 24B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0158] Figure 24C A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0159] Figure 24D A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0160] Figure 25 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0161] Figure 26 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0162] Figure 27 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0163] Figure 28 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0164] Figure 29A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0165] Figure 29B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0166] Figure 29C A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0167] Figure 29D A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0168] Figure 30 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0169] Figure 31 A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0170] Figure 32 A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0171] Figure 33A schematic diagram of a scenario for implementing a user interface method provided in an embodiment of the present application;

[0172] Figure 34 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0173] Figure 35A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0174] Figure 35B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0175] Figure 36 A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0176] Figure 37A A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0177] Figure 37B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0178] Figure 38A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0179] Figure 38B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0180] Figure 39A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0181] Figure 39B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0182] Figure 40A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0183] Figure 40B A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0184] Figure 40C A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0185] Figure 40D A schematic diagram of a method for implementing a user interface provided in an embodiment of the present application;

[0186] Figure 41AA flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0187] Figure 41B A flowchart of a method for implementing a user interface provided in an embodiment of the present application;

[0188] Figure 42A A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0189] Figure 42B A schematic diagram of a scenario example of a method for implementing a user interface provided in an embodiment of the present application;

[0190] Figure 43 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0191] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0192] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized. The term "connected" includes direct and indirect connections, unless otherwise stated.

[0193] Please refer to Figure 1, an application development tool (e.g., Android Studio, DevEcoStudio, etc.) is installed on the electronic device 200. Typically, developers use an interface description language to develop a UI in the application development tool to form an interface description file. In this application, the electronic device 200 may also be referred to as a developer device. The interface description file may also be referred to as a description file.

[0194] UI development mainly includes interface description and interface behavior definition. Interface description refers to the use of interface description language to describe the layout of the UI, the controls used, and the visual style of the layout and controls. Interface behavior definition refers to the use of interface description language to define interface behavior; interface behavior includes dynamic changes in the UI and the response of electronic devices to dynamic changes in the UI (such as the response to user operations on the UI). Each OS platform has its corresponding interface description language; for example Using XML (extensible markup language) format, Use the embedded domain specific language (EDSL) built with Swift to describe the interface and define the interface behavior.

[0195] The developer packages the interface description file into the installation package of the App and publishes the App in the application market provided by the server 300. The application market can provide the installation package of each App for users to download. For example, the installation package can be Android application package (APK) file.

[0196] Taking a mobile phone as an example of an electronic device 100, a user can use a mobile phone to download an installation package of an App in the application market. Taking a video App as an example, after the mobile phone downloads the installation package of the video App, the video App can be installed in the mobile phone by running the installation package. In this way, the mobile phone also obtains the interface description file in the installation package. The mobile phone can build a UI according to the interface description file. The UI engine provided by the mobile phone's OS platform interprets and executes the interface description language, and renders the UI to present it to the user. The constructed UI is presented on the display device of the mobile phone (such as a display screen). The mobile phone's OS platform also executes the programming language that implements the interface behavior, realizes the dynamic changes of the UI and responds to the user's operations on the UI.

[0197] Exemplarily, a developer develops the UI of a video app using an interface description language supported by an OS platform on an electronic device 200 and publishes the video app. The user uses the installation package of the "Video" app to install the video app on the mobile phone, and a "Video" icon 101 is generated on the mobile phone desktop. The user can click on the "Video" icon 101 to open the video app. In response to the user's click on the "Video" icon 101, the mobile phone runs the video app. The mobile phone is installed with an OS platform, which reads the interface description file, parses and executes the interface description language, renders the UI of the video app according to the interface description in the interface description file, and presents the UI 102 of the video app on the display screen. Furthermore, the interface description file may also include a definition of interface behavior. In response to the user's operation on the UI 102, the mobile phone can perform corresponding interface actions and implement interface behavior according to the interface behavior defined in the interface description file. Usually, the OS platform also has a corresponding programming language for implementing interface behavior, realizing dynamic changes of the UI 102, and responding to user operations on the UI 102; for example Using JAVA, Use Swift programming language to implement interface behavior.

[0198] It is understood that in some embodiments, the developer can directly develop the UI of the App on the electronic device 100 and run the App on the electronic device 100; that is, the electronic device 200 and the electronic device 100 can be the same electronic device. This embodiment of the present application is not limited to this.

[0199] The electronic device 100 may include a portable computer (such as a mobile phone, etc.), a handheld computer, a tablet computer, a laptop computer, a netbook, a personal computer (PC), a smart home device (such as a smart TV, a smart screen, a large screen, a smart speaker, etc.), a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), an augmented reality (AR)\virtual reality (VR) device, a car computer, etc., and the present application embodiment does not impose any restrictions on this. Exemplary embodiments of the electronic device 100 include but are not limited to a device equipped with Or a portable electronic device with other operating systems. It is understandable that, in some other embodiments, the electronic device 100 may not be a portable electronic device, but a desktop computer.

[0200] For example, please refer to Figure 2, which shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, an audio module 130, a speaker 130A, a microphone 130B, a display 140, a wireless communication module 150, a power module 160, and the like.

[0201] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0202] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110.

[0203] The controller is the nerve center and command center of the electronic device 100. It can generate operation control signals according to instruction operation codes and timing signals to complete the control of instruction fetching and execution.

[0204] The application processor can run the operating system of the electronic device 100, which is used to manage the hardware and software resources of the electronic device 100. For example, it manages and configures memory, determines the priority of system resource supply and demand, controls input and output devices, operates the network, manages the file system, and manages drivers. The operating system also provides an operating interface for users to interact with the system. Various software can be installed within the operating system, such as drivers and application programs (apps).

[0205] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0206] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface.

[0207] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0208] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0209] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can enable the electronic device 100 to execute the user interface implementation method provided in some embodiments of the present application, as well as various applications and data processing, etc. by running the above-mentioned instructions stored in the internal memory 121. The internal memory 121 may include a code storage area and a data storage area. Among them, the data storage area can store data created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, the processor 110 can enable the electronic device 100 to execute the user interface implementation method provided in the embodiments of the present application, as well as other applications and data processing by running the instructions stored in the internal memory 121, and / or the instructions stored in the memory provided in the processor 110.

[0210] The electronic device 100 can implement audio functions through the audio module 130, speaker 130A, microphone 130B, and application processor. For example, music playback, recording, etc. The audio module 130 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 130 can also be used to encode and decode audio signals. In some embodiments, the audio module 130 can be provided in the processor 110, or some functional modules of the audio module 130 can be provided in the processor 110.

[0211] The speaker 130A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0212] The microphone 130B, also called a "microphone" or "speaker", is used to convert sound signals into electrical signals. A user can speak by putting their mouth close to the microphone 130B to input the sound signal into the microphone 130B.

[0213] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2 and the wireless communication module 150.

[0214] The wireless communication module 150 can provide wireless communication solutions including Wi-Fi, Bluetooth (BT), wireless data transmission modules (e.g., 433MHz, 868MHz, 915MHz), etc. applied to the electronic device 100. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via antenna 1 or antenna 2, filters and frequency modulates the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 can also receive the signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through antenna 1 or antenna 2.

[0215] Electronic device 100 implements display functionality through a GPU, display screen 140, and an application processor. The GPU is a microprocessor for image processing that connects display screen 140 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0216] The display screen 140 is used to display images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 140, where N is a positive integer greater than 1. In the embodiment of the present application, the display screen 140 can be used to display a UI and receive user operations on the UI.

[0217] In some embodiments, the display screen 140 is provided with a pressure sensor 170A, a touch sensor 170B, etc. The pressure sensor 170A is used to sense pressure signals and can convert pressure signals into electrical signals. When a touch operation is applied to the display screen 140, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 170A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 170A. The touch sensor 170B, also known as a "touch panel", can form a touch screen, also known as a "touch screen", with the display screen 140. The touch sensor 170B is used to detect touch operations applied thereto or nearby. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can also be provided through the display screen 140.

[0218] The power module 160 can be used to supply power to various components included in the electronic device 100. In some embodiments, the power module 160 can be a battery, such as a rechargeable battery.

[0219] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

[0220] Figure 3 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0221] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the software system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0222] The application layer can include a series of application packages.

[0223] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, negative one screen, WLAN, desktop, music, video, short message, and other applications.

[0224] The application framework layer, including the OS, provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions that implement predefined functionality. For example, functions such as obtaining the display size, determining whether a status bar is present, locking the screen, and taking screenshots provide applications with access to data and various resources, such as localized strings, icons, images, interface description files, and video files. The OS's view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon can include a view for displaying text and a view for displaying images. The OS also enables applications to display notifications in the status bar. These can convey informational messages and can remain on the screen for a short time before disappearing automatically without user interaction. Notifications can also appear in the system's top status bar as icons or scrolling text, such as notifications from background applications. Notifications can also appear in dialog windows on the screen. Examples include text messages in the status bar, beeping, vibrating electronic devices, or flashing indicator lights.

[0225] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0226] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0227] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0228] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0229] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0230] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0231] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0232] A 2D graphics engine is a drawing engine for 2D drawings.

[0233] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0234] As an open source OS, it is widely used in portable electronic devices. At the same time, many manufacturers have also launched their own enhanced systems (OEM OS); for example, Huawei's EMUI is based on OEM OS can provide more than the basic OS (such as ) A more optimized and enhanced SDK, providing manufacturers with customized UI programming capabilities.

[0235] In one implementation method, the OEM OS released by the manufacturer supports Interface description language (such as XML) can provide Basic UI programming capabilities and manufacturer-defined UI programming capabilities. Please refer to Figure 4 , the UI development language of the App is Applicable interface description language (such as XML) for declaration Provides basic UI programming capabilities. Add a manufacturer-defined field to the interface description language (such as XML) to declare the manufacturer's customized UI programming capabilities. The basic UI engine provided interprets and executes the interface description language; the basic UI engine adds interpretation and execution of manufacturer-defined fields. OEM OS support Provides basic UI programming capabilities and provides manufacturers with customized UI programming capabilities. In this method, use Native development interface, interface description language and UI engine, using The native interface description language and the method of adding custom fields to the UI engine provide manufacturers with the ability to customize UI programming.

[0236] In another implementation method, the OEM OS released by the manufacturer is independent of the general OS platform (such as ), providing manufacturer-defined UI programming capabilities. Please refer to Figure 5 The app's UI development language is a manufacturer-defined interface description language. The OEM OS platform provides a custom UI engine that parses and executes the custom interface description language, providing manufacturer-defined UI programming capabilities. In this approach, the manufacturer customizes a complete UI programming framework independent of the general OS platform, meeting developers' needs for cross-platform app development and operation.

[0237] The embodiments of the present application provide a method and device for implementing a user interface, which can provide rich UI programming capabilities and can adapt to multiple OS platforms. The technical implementation is low in difficulty and is convenient for developers to use.

[0238] Please refer to Figure 6 The OEM OS platform provided in the embodiment of the present application supports a basic interface description language and a custom interface description language. The basic interface description language is an interface description language supported by a general OS platform; for example, xml, Swift, etc. In the embodiment of the present application, the custom interface description language is a domain-specific language (DSL), and the custom interface description language is not related to the general OS platform. In the following embodiments of the present application, the custom interface description language is called DSL. Developers can use the basic interface description language and DSL to jointly develop the UI of the App. For example, developers use the basic interface description language to describe the UI layout, the included controls, etc.; and selectively use DSL to apply custom UI programming capabilities to some controls, add some dynamic effects to the UI, etc. For example, custom UI programming capabilities may include layout capabilities, visual attribute capabilities, unified interaction capabilities, dynamic effect capabilities, etc. Layout capabilities are used to describe the layout of controls in the UI; such as the shape, position, size, etc. of controls. Visual attribute capabilities are used to describe the visual attributes of controls; such as the color, grayscale, and other visual effects of controls. Unified interaction capabilities are used to provide control responses based on user behavior; such as performing searches based on the user's "confirmation" behavior. Dynamic effect capabilities are used to display animation effects on controls; such as displaying click rebound dynamic effects on controls, etc.

[0239] The OEM OS provided in the embodiments of this application can implement both the basic UI programming capabilities provided by the general OS platform and the customized UI programming capabilities that are extended relative to the OS platform. The OEM OS platform includes a basic UI engine and an extended UI engine. When an electronic device builds a UI, the basic UI engine is used to interpret and execute the basic interface description language to generate a basic UI (with basic UI programming capabilities); the extended UI engine is used to interpret and execute the DSL, adding customized UI programming capabilities to the basic UI.

[0240] The user interface implementation method provided by the embodiment of the present application, the custom interface description language and the extended UI engine only need to cover the custom UI programming capabilities, so the manufacturer's release difficulty is low and easy to expand; and the developer access threshold is low. The custom interface description language and the extended UI engine are not related to the general OS platform, which can be It can also be other general-purpose OS platforms. Customized interface description language and extended UI engine can be easily applied to multiple general-purpose OS platforms.

[0241] Developers use basic interface description language and custom interface description language to develop apps. When the app runs on an electronic device, the UI engine of the OEM OS on the electronic device parses and executes the interface description language (basic interface description language and custom interface description language) to generate a UI. Among them, the basic UI engine is used to interpret and execute the basic interface description language, and the extended UI engine provided by the OEM OS in the embodiment of this application is used to parse and execute the custom interface description language. Please refer to Figure 7In one example, the extended UI engine 310 includes modules such as a process control 311, a DSL file loader 312, a parsing engine 313, and an execution engine 314. The process control 311 is used to control the execution flow of each module within the extended UI engine 310, as well as the interaction flow between the extended UI engine 310 and other modules in the OEM OS. The DSL file loader 312 is used to read DSL files. The parsing engine 313 includes submodules such as DSL syntax checking and DSL parsing. The DSL syntax checking submodule is used to perform syntax checking on the content in the DSL file. The DSL parsing submodule is used to parse the DSL file and convert the content into a data format compatible with the execution engine. In one implementation, if the DSL syntax checking submodule successfully verifies the DSL file syntax, the DSL parsing submodule parses the DSL file; if the DSL syntax checking submodule fails to verify the DSL file syntax, the DSL parsing submodule does not parse the DSL file. The parsing engine 313 may also include submodules such as DSL preprocessing. For example, the DSL preprocessing submodule is used to precompile the DSL file. The execution engine 314 includes sub-modules such as version management, control construction, event proxy, interpretation execution engine, and semantic support library. Among them, the version management sub-module is used to match the version of the extended UI engine 310 with the DSL file in the App. The version of the extended UI engine 310 needs to be consistent with the version of the DSL file or newer than the version of the DSL file in order to run normally. The control construction sub-module is used to build UI controls based on the content of the DSL file. The event proxy sub-module is used to implement the mapping between device events and user behaviors. For example, a mouse double-click event and a finger single-click event on the display can be mapped to the user's "confirmation" behavior on the electronic device. The interpretation execution engine is used to interpret and execute the code in the DSL file, and in response to user behavior, execute the action corresponding to the user behavior defined in the DSL file. The semantic support library includes a set of grammatical semantic specifications for all fields in the DSL file, such as field definitions and syntax such as environment variable interfaces, public fields, layout template properties, visual properties, layout properties, interaction properties, and animation properties.

[0242] Please continue to refer to Figure 7In the embodiment of the present application, the OEM OS also includes a custom UI programming capability 320. The custom UI programming capability 320 includes a DSL adaptation layer, which is used to provide an adaptation interface for the custom UI programming capability for the extended UI engine 310. The custom UI programming capability 320 also provides the implementation of custom UI programming capabilities such as visual attribute capabilities, layout capabilities, unified interaction capabilities, and dynamic effect capabilities. For example, a control is declared in the DSL file to enable vertical stretching capabilities, and the implementation of the custom UI programming capability (vertical stretching capability) is completed by the custom UI programming capability 320; that is, when the display window of the control changes, the custom UI programming capability 320 implements the vertical stretching of the control, and the developer does not need to implement the vertical stretching of the control in the DSL.

[0243] Developers can use the basic interface description language and DSL to jointly develop apps. The syntax rules and development tools of the basic interface description language can follow conventional technologies. The embodiments of the present application also provide the syntax rules and development tools of the DSL. In one example, the embodiments of the present application provide a development tool that supports the syntax rules of the basic interface description language and DSL, and provides an editing and compilation environment for the basic interface description language and DSL.

[0244] In some embodiments, embodiments of the present application provide a development tool, wherein the development interface of the development tool includes a basic interface description language file and a DSL file. Exemplarily, when a developer opens the development interface of the development tool, the development interface includes an initial version of the basic interface description language file and an initial version of the DSL file. Furthermore, the developer can use the basic interface description language to add control descriptions in the initial version of the basic interface description language file, and can also use DSL to add control descriptions in the initial version of the DSL file. It is understandable that the initial version of the DSL file can be pre-set in the development tool, or it can be added by the developer in the development tool. In some instances, the development tool can also include DSL templates, DSL syntax rule description files, interface description examples, etc.

[0245] In some examples, basic UI description language files are used to describe native controls and apply basic UI programming capabilities to them. DSL files are used to declare custom UI programming capabilities for controls. For example, you can apply custom UI programming capabilities to native controls in a DSL file; or, in another example, you can declare custom controls in a DSL file and apply custom UI programming capabilities to them.

[0246] In one implementation, the basic interface description language file and the DSL file are respectively set in different paths of the development tool folder. For example, the basic interface description language is carried by one or more xml format files, and the DSL is carried by one or more json format files. For example, Figure 8 As shown, Taking the platform as an example, developers create an app folder in the development tool. The res directory of the app folder contains an AndroidManifest.xml file. Developers can use this XML file to declare basic UI programming capabilities. The assets directory of the app folder contains a huawei_dsl.json file. Developers can use this JSON file to declare custom UI programming capabilities.

[0247] It is understandable that the above-mentioned placement of the basic interface description language file and the DSL file in different paths of the development tool folder is only to enable the UI engine of the OEM OS to distinguish between the basic interface description language file and the DSL file. In actual applications, other methods can also be used to distinguish between the basic interface description language file and the DSL file. For example, different tags are preset for the basic interface description language file and the DSL file, and the UI engine of the OEM OS can obtain the basic interface description language file and the DSL file respectively according to the preset tags. The embodiments of the present application are not limited to this.

[0248] Developers complete app development in the development tool, compile it, and generate the app installation package. The basic interface description language file and DSL file are integrated into the app installation package so that the OEM OS's UI engine can read them. In one implementation, the basic interface description language file and DSL file are stored in the app installation package in the same location as they are in the app's folder in the development tool.

[0249] In one example, the DSL file uses a standard JSON format; illustratively, the DSL file includes content such as version, app, and layout.

[0250] Wherein, version represents the version number of the DSL file. Exemplarily, the version format is xyz, where x indicates the product, y indicates the subsystem of the product, and z indicates the development number. For example, it can be 101.1.003.

[0251] The app content block is used to declare custom UI programming capabilities for the app global controls within the app installation package where the DSL file is located. For example, the format of the app content block is:

[0252]

[0253] feature_name is the attribute of the custom UI programming capability, and value is the attribute value of the custom UI programming capability.

[0254] The layout content block is used to declare custom UI programming capabilities that act on controls in a layout. For example, the layout content block format is:

[0255]

[0256] Among them, layoutId is used to indicate a layout; for example, layoutId is the identifier of a layout. widgetId is used to indicate a control in the layout; for example, widgetId is the identifier of the control. prop_name is the attribute of the custom UI programming capability, which represents a feature of the custom UI programming capability; for example, the custom UI programming capability is enabled, the priority of the custom UI programming capability, the parameters of the custom UI programming capability, etc. value is the attribute value of the custom UI programming capability, and the attribute value is used to specify the value of the attribute; for example, the attribute is the custom UI programming capability enabled, and accordingly, the attribute value is true, which indicates that the custom UI programming capability is enabled, and the attribute value is false, which indicates that the custom UI programming capability is not enabled.

[0257] For example, the DSL file includes the following code snippet:

[0258]

[0259] The version number is 101.1.003. The zoom attribute value for the custom UI programming capability is enabled, meaning that zoom is enabled for all controls in the app globally. The onSearch attribute value is com.app.Search$onSearchPrice, which specifies the search function's execution behavior as defined in com.app.Search$onSearchPrice.

[0260] It is understood that a DSL file may include fewer or more fields. For example, a DSL file may include version and layout content blocks, but not an app content block. For another example, a layout content block may include description fields for multiple controls. For another example, a control may enable multiple custom UI programming capabilities. This embodiment of the present application does not limit this.

[0261] The custom UI programming capabilities of the embodiment of the present application may include visual attribute capabilities, layout capabilities, unified interaction capabilities, motion effect capabilities, etc. Developers can declare custom UI programming capabilities in a DSL file to use the custom UI programming capabilities provided by the OEM OS.

[0262] For example, the visual attributes of the UI are reflected in the visual attributes of the control. The OEM OS defines a set of visual parameter variables for the control to describe the visual attributes of the control; this set of visual parameter variables can be used to switch the visual attributes of multiple brands or multiple devices. When developers describe the visual attributes of the UI, they can use visual parameter variables (dynamically obtain attribute values ​​that match the brand or the electronic device itself when the electronic device is running), and the developer does not need to specify specific variable values. Declaring visual parameter variables in the DSL file enables the control to have the corresponding visual attribute capabilities.

[0263] An example of using the visual attribute capability in a DSL file is as follows:

[0264]

[0265] In this example, the value of the textColor visual property of the R.id.textview control is emuiColor1; the value of the foreground visual property of the R.id.image control is emui_color_bg. emuiColor1 and emui_color_bg are visual parameter variables that map to different color values ​​on different brands or devices. The OEM OS pre-configures the mapping between visual parameter variables and color values ​​for different brands or devices, eliminating the need for developers to repetitively specify textColor and foreground property values ​​for different brands or devices.

[0266] OEM OSes provide adaptive layout capabilities to build responsive UIs, adapting UI layouts to different display sizes and shapes, eliminating the need for developers to perform different layouts for different devices. Exemplary adaptive layout capabilities include automatic stretching, hiding, line wrapping, equalizing, scaling, and extending. In one example, the adaptive layout capabilities provided by OEM OSes apply to LinearLayout layouts and the controls within them.

[0267] Several layout capabilities are exemplified below.

[0268] 1. The adaptive layout capability master switch is used to turn on or off the adaptive layout capability of the control. In one example, the adaptive layout capability master switch must be turned on to enable any of the layout capabilities.

[0269] 1. Field definition

[0270] ability property Field Definition attribute belongs to Main switch Enable adaptive layout capabilities use_HwConstraintLayout layout

[0271] "Capability" indicates the ability to customize UI programming. "Property" represents the characteristic parameters of this capability. "Property Belongs" indicates the classification of the property's function. For example, if the property belongs to "Layout", it means that the property is used for control layout; if the property belongs to "Sub-element", it means that the property is used to describe the control.

[0272] 2. Automatic stretching capability: If the control is enabled with automatic stretching capability, the control can be automatically stretched in the UI according to the window size to adapt to the window size.

[0273] 1. Field definition

[0274]

[0275] 2. Example of using automatic stretching capability in DSL files

[0276]

[0277]

[0278] In this example, the R.layout.linearlayout_vertical layout enables vertical stretching of controls. Controls in this layout automatically stretch vertically to adapt to the display window size when the display window changes.

[0279] 3. Hiding capability: If the control has the hiding capability enabled, it can be hidden in the UI.

[0280] 1. Field definition

[0281]

[0282] 2. Example of using hidden capabilities in DSL files

[0283]

[0284] In this example, the R.id.container control in the R.layout.mainpage layout has vertical hiding enabled. The vertical hiding priority of R.id.image1 in R.id.container is 2, and the vertical hiding priority of R.id.image2 is 1.

[0285] 4. Line Wrapping. If a control enables line wrapping, it can be displayed across multiple lines within the UI. In one example, the line wrapping width limit can be used to specify the maximum width of each line displayed by the control.

[0286] 1. Field definition

[0287]

[0288] 2. Example of using line break capability in DSL files

[0289]

[0290]

[0291] In this example, the controls in the R.layout.mainpage layout are enabled for line wrapping. The line wrapping width limit for R.id.image1 is 160dp, and the line wrapping width limit for R.id.image2 is also 160dp. This means that the maximum width of each line displayed for R.id.image1 is 160dp, and the maximum width of each line displayed for R.id.image2 is 160dp.

[0292] 5. Equal sharing capability: If the control is enabled with equal sharing capability, the control can be evenly displayed in the UI.

[0293] 1. Field definition

[0294]

[0295] 2. Example of using the balancing capability in a DSL file

[0296]

[0297] In this example, the control in the R.layout.mainpage layout has the sharding capability enabled. The sharding type of R.id.image1 is "spread."

[0298] 6. Proportion Capability: A control that enables proportion capability means that the control can occupy a specified percentage of the total layout size in a specified direction.

[0299] 1. Field definition

[0300]

[0301] 2. Example of using the proportion capability in DSL files

[0302]

[0303] In this example, the vertical scale is enabled for the controls in the R.layout.mainpage layout. The vertical scale of R.id.image1 is 33.33%.

[0304] 7. Stretchability. If a control has stretchability enabled, it can stretch and display on the UI based on the display size. In one example, the exposure value specifies the exposure characteristics of the last control that can be displayed on the UI.

[0305] 1. Field definition

[0306]

[0307] 2. Examples of using extension capabilities in DSL files

[0308]

[0309] In this example, the controls in the R.layout.mainpage layout have the extension capability enabled. Among them, R.id.image1 has the exposure feature enabled, with an exposure value of 40dp.

[0310] The OEM OS also provides unified interaction capabilities, allowing developers to define behavior-based control responses. In one example, unified interaction capabilities include search and zoom. Developers can declare unified interaction capabilities in a DSL file, enabling controls to have search, zoom, and other capabilities.

[0311] When developers develop UI based on a general OS platform, they define the behavior corresponding to the event. For example, define the "confirm" behavior for a mouse double-click event, define the "confirm" behavior for a finger single-click event on the display screen, and define the correspondence between other events and the "confirm" behavior. The developer's workload is large. The OEM OS provided in the embodiment of the present application supports developers to directly define responses to "confirm" behaviors (i.e., define unified interactive capabilities corresponding to behaviors) without defining events corresponding to "confirm" behaviors; the mapping relationship between events and behaviors is completed by the OEM OS. The OEM OS maps events triggered by electronic devices of different forms to the same behavior (for example, mapping a mouse double-click event on a PC to a "confirm" behavior, and mapping a finger single-click event on a mobile phone to a "confirm" behavior), avoiding developers defining the correspondence between events and behaviors for electronic devices of different forms, which brings duplication of work.

[0312] An example of using the search capability in a DSL file is as follows:

[0313]

[0314] In this example, the control R.id.sample_text has the onSearch capability. When the electronic device receives a user confirmation action on the control R.id.sample_text (for example, a mouse double-click or finger single-click event), it executes the search function defined in com.sample.SearchImplSample$onSearchSample.

[0315] An example of using the zoom capability in a DSL file is as follows:

[0316]

[0317] In this example, the control R.id.sample_text has the onZoom capability. When the electronic device receives a user confirmation action on the control R.id.sample_text (for example, a mouse double-click or finger single-click event), it executes the zoom function defined in com.sample.ZoomImplSample$onZoomSample.

[0318] OEM OS also provides enhanced motion effects, making control animations more expressive. The motion effects provided by OEM OS are applicable to Button and its subclasses and can be enabled globally within an app or for specific controls.

[0319] In one example, the animation capability includes a click rebound micro-animation of a Button control (field definition: reboundAnimation).

[0320] The following is an example of using the animation capability in a DSL file:

[0321]

[0322] Declare reboundAnimation in the layout content block, and enable the click rebound micro-animation effect on the target control.

[0323] For example, Figure 9 A schematic diagram showing a process of generating a UI when an App is running.

[0324] S401. The process control module of the extended UI engine reads the basic interface description language file, and calls the basic UI engine to parse and execute the basic interface description language to construct a basic UI.

[0325] Among them, the basic UI controls use basic UI programming capabilities.

[0326] S402 : The process control module of the extended UI engine calls the DSL file loading module to read and load the DSL file.

[0327] S403: The parsing engine performs syntax verification, parsing, and preprocessing on the content in the DSL file to obtain a data format that matches the execution engine.

[0328] In one example, the DSL syntax check submodule performs syntax check on the content in the DSL file. If the check passes, the DSL parsing submodule parses the fields in the DSL file. Furthermore, the DSL preprocessing submodule preprocesses the DSL file to obtain a data format that matches the execution engine.

[0329] S404: The execution engine builds an enhanced UI based on the basic UI built in S401 and in units of controls according to the DSL file content.

[0330] In one implementation, the control construction submodule sequentially retrieves the semantic processing components corresponding to the fields in the DSL file from the semantic support library. For example, it retrieves the semantic processing component SearchHandler for the "onSearch" field from the semantic support library. Furthermore, the control construction submodule applies custom UI programming capabilities to the control through the DSL adaptation layer, constructing an enhanced UI.

[0331] For example, Figure 10 A schematic diagram shows a flow chart of an electronic device responding to user operations on a UI.

[0332] S501: The execution engine creates an event proxy and registers the event proxy with the UI through the DSL adaptation layer.

[0333] S502: The OEM OS monitors user operation events of the UI and reports the user operation events to the event agent.

[0334] S503: The event proxy implements the mapping between events and behaviors.

[0335] S504: The interpretation and execution engine interprets and executes the code in the DSL file, implements the response specified in the DSL file according to the behavior, and completes the response to the user's operation in the UI.

[0336] The user interface implementation method provided in the embodiment of the present application supports native controls and custom controls in the App, and also supports the application of custom UI programming capabilities in native controls. Among them, native controls are general OS (such as ) supports the control, general OS provides basic UI programming capabilities for native controls; custom controls are controls that are not supported by general OS but supported by OEM OS, and OEM OS provides custom UI programming capabilities for custom controls.

[0337] Please refer to Figure 11 , which shows a schematic diagram of the process for building native controls on an OEM OS. The App development project package for OEM OS 1101 includes a basic UIDL file. Process control 1111 of basic UI engine 1110 instructs parsing engine 1112 to process the basic UIDL file. Parsing engine 1112 reads and loads the basic UIDL file, converting it into a data format compatible with execution engine 1113. Execution engine 1113 builds a basic UI based on the contents of the basic UIDL file, generating native controls 1130.

[0338] Please refer to Figure 12 , which shows a flow diagram of the OEM OS applying custom UI programming capabilities to native controls. The App development project package of OEM OS 1101 includes a basic interface description language file and a DSL file. The process control 1121 of the extended UI engine 1120 instructs the parsing engine 1112 in the basic UI engine 1110 to process the basic interface description language file. The parsing engine 1112 reads and loads the basic interface description language file and converts it into a data format that matches the execution engine 1113. The execution engine 1113 constructs a basic UI based on the contents of the basic interface description language file and generates a native control 1130. The process control 1121 instructs the parsing engine 1122 in the extended UI engine 1120 to process the DSL file. The parsing engine 1122 reads and loads the DSL file and converts it into a data format that matches the execution engine 1123. The execution engine 1123 applies custom UI programming capabilities to the native control 1130 based on the DSL file.

[0339] Please refer to Figure 13, which shows a flow diagram of building custom controls on an OEM OS. The App development project package for OEM OS 1101 includes a basic UI description language file and a DSL file. The process control 1121 of the extended UI engine 1120 instructs the parsing engine 1112 in the basic UI engine 1110 to process the basic UI description language file. The parsing engine 1112 reads and loads the basic UI description language file, converting it into a data format compatible with the execution engine 1113. The execution engine 1113 builds a basic UI based on the contents of the basic UI description language file and generates a native control 1130. The process control 1121 instructs the parsing engine 1122 in the extended UI engine 1120 to process the DSL file. The parsing engine 1122 reads and loads the DSL file, converting it into a data format compatible with the execution engine 1123. The execution engine 1123 generates a custom control 1140 on the basic UI based on the DSL file.

[0340] The OEM OS provided in the embodiments of the present application includes a basic UI engine and an extended UI engine. When an electronic device builds a UI, the basic UI engine is used to interpret and execute the basic interface description language to generate a basic UI (with basic UI programming capabilities); the extended UI engine is used to interpret and execute the DSL, adding customized UI programming capabilities to the basic UI. The user interface implementation method provided in the embodiments of the present application can adapt to multiple OS platforms and provide rich UI programming capabilities; the technical implementation is easy to implement and convenient for developers to use.

[0341] The embodiment of the present application also provides a method for implementing a user interface, which is easy to implement and convenient for developers to use.

[0342] With the rapid development of the Internet of Things (IoT), the types and number of IoT devices are growing rapidly. Different IoT devices have different screen sizes and user interaction methods. For example, the screen size of mobile phones is mostly around 4-6 inches, and the user interaction method is mainly touching and clicking the display screen; the screen size of TVs can reach 50 inches or larger, and the user interaction method is usually remote control operation; devices such as car computers have a wider range of screen forms and user interaction methods. The current general OS platforms (such as ) supports a development method in which, for the same UI in the same App, developers design different interface description files for each type of electronic device. Obviously, using this method to develop differentiated UIs for various types of devices is labor-intensive and difficult to develop. The embodiments of the present application provide a user interface implementation method and apparatus that can achieve one-time development and multi-device deployment; that is, develop a set of interface description files that are applicable to various types of electronic devices, reducing the development difficulty for developers.

[0343] for example, As an open source OS, it is widely used in portable electronic devices. When developing the UI of an App, for the same UI in the same App, developers design different interface description files for each type of electronic device to develop differentiated UIs for different types of electronic devices. Supports the independent development of UI for multiple types of devices by setting up separate layout folders for each type of electronic device. For example, add a suffix to the name of the layout folder to distinguish different layout folders. In this way, for the same UI, different types of electronic devices read the interface description files in different layout folders to present different UI display effects. If the App needs to be installed on other types of electronic devices, it is necessary to develop corresponding interface description files for this type of electronic device separately (add a separate layout folder). In this development method, developers need to develop corresponding interface description files for different types of electronic devices separately, which requires a large amount of development work and is difficult.

[0344] Some manufacturers also provide some complete, independent UI programming frameworks, such as Flutter, ReacNative, and Weex. These frameworks include a UI interface description language and a corresponding parsing and execution engine, providing an independent interface control library, layout engine, and rendering engine. They can run across multiple devices, but their compatibility is poor.

[0345] The present application embodiment provides a method and device for implementing a user interface. Figure 14 , the developer opens a development tool (such as DevEco Studio) in the electronic device 200 (developer device) and generates an interface description file in the development interface of the development tool. Exemplarily, the developer opens the development interface of the development tool, and the development interface includes an initial version of the interface description file. The initial version of the interface description file can be a blank file or contain simple examples. It can be understood that the initial version of the interface description file can be pre-set in the development tool or added by the developer in the development tool. In some examples, the development tool may also include interface description language templates, interface description language grammar rule description files, interface description examples, etc., which will not be repeated in the embodiments of the present application.

[0346] Furthermore, developers can use an interface description language to add interface descriptions and interface behavior definitions to the initial version of the interface description file to form an interface description file for release. In one implementation, developers generate an interface description file for each UI in the app; for example, multiple interface description files can be generated in a folder, each corresponding to a UI.

[0347] An installation package for the App is generated on the developer's device, which includes an interface description file. The App's installation package is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the electronic device 100 described above) to download the App's installation package from the application market. After the user-side electronic device runs the App's installation package, it obtains the interface description file in the installation package; when the user-side electronic device runs the App, it displays a UI that matches the electronic device on the display screen according to the interface description file.

[0348] In one example, the interface description file adopts json format. For example, Figure 14 As shown, the installation package of the App includes a folder "app" 410. The src\main\assets directory of the folder "app" 410 includes a layout folder "layout" 411. The layout folder "layout" 411 includes interface description files layout1.json412, layout2.json 413 and layout3.json 414, etc. Each interface description file corresponds to a UI of the App. Different types of user-side electronic devices such as mobile phones 420, car computers 430, TVs 440, and watches 450 all run the same interface description file in "layout" 411, and display different display effects of the same UI respectively. For example, mobile phones 420, car computers 430, TVs 440, and watches 450 all parse and execute layout1.json 512, and display different display effects of the UI corresponding to layout1.json 512 respectively.

[0349] In the embodiments of this application, for the same UI within the same app, developers can develop differentiated UIs for different types of electronic devices by developing a single set of code within a single interface description file. Different electronic devices can read the same interface description file for the same UI and present different UI display effects. This allows developers to develop a single set of interface description files that are applicable to a wide variety of electronic devices, reducing development complexity.

[0350] The user interface implementation method provided in the embodiment of the present application supports the use of Native UI programming capabilities, as well as operating system-customized UI programming capabilities. Native UI programming capabilities enable controls to have Native control properties and the operating system's customized UI programming capabilities enable controls to have extended visual properties, layout properties, interaction properties, dynamic properties, and hardware and software dependency properties. After the electronic device runs the App installation package and obtains the interface description file in the installation package; when the user runs the App on the electronic device, the electronic device can present the corresponding UI on the display screen according to the interface description file. The controls in the UI can include Native control properties can also include extended control properties. The custom UI engine provided by the embodiment of the present application supports The native control properties and all control properties extended in the operating system are parsed and executed.

[0351] In one example, Figure 15 FIG. 1 shows a software architecture of the electronic device 100. Figure 15 As shown, the electronic device 100 software system may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer. In one example, the application layer, the Android runtime (Android runtime) and the system library, and the kernel layer may refer to Figure 3 middle The corresponding description in the software architecture is not repeated here. The software system of the electronic device 100 provided in the embodiment of the present application partially reuses the UI programming framework in conventional technology, which is easy to learn and reduces the development difficulty for developers.

[0352] The operating system of the application framework layer includes a custom UI engine 11. The custom UI engine 11 is used to parse and execute the app's interface description file and generate the app's UI. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, an MVVM (model-view-viewmodel) framework 11c, a syntax and semantic library 11d, and a UI rendering engine 11e. It is understood that the application framework layer may also include more modules, and reference may be made to conventional technologies, which are not limited in this embodiment of the present application.

[0353] The above modules are described in detail below with reference to the accompanying drawings.

[0354] The syntax and semantics library 11d includes a set of syntax and semantic specifications for all fields in the interface description file, such as variable interfaces, public fields, visual attributes, layout attributes, interaction attributes, animation attributes, hardware and software dependency attributes and other field definitions and syntax. Among them, layout attributes refer to the layout of each control in the UI; such as the shape, position, size, etc. of the control. Visual attributes refer to the color, grayscale and other visual effects of the control. Interaction attributes are the ability to provide control responses based on user behavior; such as performing searches based on the user's "confirmation" behavior. Animation attributes refer to displaying animation effects on controls; such as displaying click rebound effects on controls. Hardware and software dependency attributes refer to the hardware and software parameters of the device on which the control depends.

[0355] Developers need to add code to the interface description file and develop the UI according to the syntax and semantics specifications defined in the syntax and semantics library 11d. The following describes the syntax and semantics specifications defined in the syntax and semantics library 11d from the aspects of layout arrangement, data and interface binding, interaction behavior arrangement, and differentiation description.

[0356] Taking the interface description file in json format as an example, the interface description file may include the following structure:

[0357]

[0358] The metadata includes information such as the version number. An example is as follows:

[0359] "meta-data":{

[0360] "version":"10.0.1008"

[0361] }

[0362] "version" represents the version number of the interface description file. For example, the format is xyz, where x indicates the product, y indicates the product subsystem, and z indicates the development cycle. The interface description file version must match the custom UI engine version. For example, the custom UI engine version must be the same as or newer than the interface description file version to successfully parse the interface description file.

[0363] Import is used to import objects, and model is used to declare objects. The following example:

[0364]

[0365]

[0366] Import the full path com.myapp.UserInfo stored in UserInfo and the full path com.myapp.TestActivity stored in Context in the import function. In the model, declare an object of type UserInfo, user, and an object of type Context, context. This allows you to directly call user and context in the interface description files (layout-data-common and layout-data-uimode). In this application, files such as UserInfo and TestActivity are referred to as resource files. Resource files include the resources used to generate the application's UI, which can include developer-defined data structures, controls, and control properties.

[0367] The specific usage of import and model will be introduced in detail later in conjunction with the grammatical and semantic rules in layout-data-common, layout-data-uimode, and styles.

[0368] layout-data-common is used to describe the common UI. All types of electronic devices parse the content in layout-data-common and layout the common UI according to the content in layout-data-common. layout-data-uimode is used to describe the UI of a specified device. In one implementation, layout-data-uimode declares the difference between the UI of a specified device and the common UI. In another implementation, layout-data-uimode declares all conditions that apply to the UI of a specified device. The specified device can be a mobile phone, watch, car machine, smart home device (such as smart TV, smart screen, smart speaker, etc.), large screen, laptop computer, desktop computer, etc. For example, the specific forms of layout-data-uimode may include layout-data-phone (for mobile phones), layout-data-watch (for watches), layout-data-television (for TVs), etc.

[0369] Styles are used to define custom parameters in the App. Developers can customize parameters in styles.

[0370] 1. Layout

[0371] All UI components in an app are composed of controls. UI layout involves arranging the properties of controls within the UI.

[0372] 1. Controls

[0373] Custom UI Engine 11 supports all Native controls and controls extended by the operating system are supported, as well as controls customized by developers in the app or integrated through static packages. Specifically, controls can include text controls, such as TextView controls and EditText controls, button controls, such as Button controls and ImageButton controls, and image controls, such as Image controls. This embodiment of the application does not impose any restrictions on this.

[0374] for For native controls and controls extended in the operating system, you can directly call the control name in layout-data-common or layout-data-uimode. For example, Native controls can include TextView, EditText, etc.; controls extended in the operating system can include HwButton, etc. An example of declaring a control is as follows. In this example, Native controls TextView and EditText.

[0375]

[0376] For controls that developers customize within their apps or integrate via static packages, they must include the full package name of the control's resource bundle in the import statement. This allows them to be called from layout-data-common or layout-data-uimode. An example of declaring a custom control within an app is as follows: In this example, the full package name of the MyCircleView control's resource bundle, com.myapp.widget.MyCircleView, is first included in the import statement. Then, MyCircleView is directly called from layout-data-common.

[0377]

[0378]

[0379] In one implementation, the custom UI engine 11 supports developers to specify aliases for controls. An example is as follows: in this example, when the full package name com.myapp.widget.MyCircleView of the resource package of MyCircleView is introduced in the import, the name of MyCircleView is specified as AliasName.

[0380]

[0381] In one implementation, when calling a control in layout-data-common or layout-data-uimode, the control is declared in the form of ComponentName():{}. For example, TextView():{} declares a TextView.

[0382] 2. Control properties

[0383] The control properties supported by Custom UI Engine 11 include: Native properties as well as extended visual properties, layout properties, interaction properties, motion properties, and software and hardware dependency properties in the operating system.

[0384] When describing a control using the ComponentName():{} method, in one implementation, you can pass in control properties and their values ​​within the {}, using the format "property 1:property value 1, property 2:property value 2". The following example declares a TextView control and passes in its textSize property within the {}. The value of textSize is @dimen / mySize.

[0385]

[0386] In another implementation, you can pass in control properties and property values ​​in (). The following example declares a TextView control and passes in its property text in (), where the property value is @string / text_name.

[0387] {

[0388] "TextView(text:@string / text_name)":{}

[0389] }

[0390] In one implementation, if for the same control, control properties and property values ​​are passed in both () and {}, the content in () is ignored.

[0391] The property values ​​of control properties can be assigned in any of the following ways: directly specifying a string value; accessing a resource value defined in the backend data; accessing a category parameter declared in the backend data; or accessing a value in the control model (ViewModel) object.

[0392] The custom UI engine 11 supports specifying the namespace of the control property through the namespace.propertyName method. In one implementation, not specifying the namespace means the default is In one implementation, the custom UI engine 11 supports using the androidhwext namespace to point to the operating system's extended resource packages and the app namespace to point to the app's custom resource packages. The operating system's extended resource packages provide custom UI programming capabilities within the operating system; the app's custom resource packages provide custom control properties within the app.

[0393] In one implementation, developers can also define other namespaces. These namespaces are imported through the import statement, along with the package name of the resource bundle that defines the control's properties. For example, the import statement imports the developer-defined namespace myspace, whose full package name is com.myapp. After importing myspace, the borderWidth property within myspace can be called in layout-data-common.

[0394]

[0395]

[0396] 2. Data & Interface Binding

[0397] The custom UI engine 11 supports two-way binding between UI elements and backend data. Binding relationships between UI elements (such as controls and control groups) and backend data can be declared and specified in interface description files (layout-data-common or layout-data-uimode). The MVVM framework 11c in the custom UI engine 11 allows for refreshing backend data based on UI changes, or automatically refreshing the corresponding UI based on backend data changes.

[0398] For example, you can bind UI elements to a ViewModel object. First, import the ViewModel in the import section, declare an object of the ViewModel type in the model section, and then call the ViewModel object in layout-data-common or layout-data-uimode.

[0399] In one example, the property values ​​of UI controls are bound to the values ​​of ViewModel objects. The following example imports the UserInfo resource package (UserInfo is a ViewModel) with the full package name com.myapp.UserInfo. A UserInfo object, user, of type UserInfo, is declared in the model. The data in user is then accessed in layout-data-common.

[0400]

[0401] In one implementation, the $model.field method is used to access the value of a variable (field) in the ViewModel object (model). For example, $user.photo accesses the variable photo in user, and $user.name accesses the variable name in user. In another implementation, the $model::function method is used to access the return value of a function (function) in the ViewModel object (model). For example, $user::hasName accesses the return value of the hasName function in user.

[0402] In the above example, the imageUri property of the ImageView control is bound to the background data user.photo, the text property of a TextView control is bound to the background data user.name, the text property of a TextView control is bound to the background data user.age, the checked property of the CheckBox control is bound to the background data user.agreed, and the visible property of the TextView control is bound to the background data user::hasName. When the background data changes, the property values ​​of the control properties change, and the display of the control in the UI changes.

[0403] In one implementation, the visibility of controls can be retrieved based on backend data, enabling the hiding of parts of the UI. When a variable in the backend data changes (from visible to invisible, or vice versa), controls in the UI can be hidden or shown accordingly. For example, the visibility of a column control (Column) is determined by the value of the variable user.visible.

[0404]

[0405]

[0406] In another example, user input is received on the UI and bound to the value of a ViewModel object. The following example imports the UserInfo resource package (UserInfo is a ViewModel) with the full package name com.myapp.UserInfo and declares an object of type UserInfo in the model. Then, in layout-data-common, the user-entered value of the text property of the EditText control is assigned to the variable name in the user property. The "=" operator assigns the value to the backend data.

[0407]

[0408] 3. Interaction Behavior Choreography

[0409] The custom UI engine 11 supports declaring the execution actions corresponding to the control response events in the interface description file. The range of events supported by the control is determined by the event listening supported by the control. For example, the button control (Button) supports the listener setOnClickListener for click events, and the onClick (click) event can be bound to the control in the interface description file. The custom UI engine 11 transmits the event parameters and the response function return value in the background data in both directions between the control and the background data. The following example shows that the control Button is declared in layout-data-common to execute the action defined in the background data context.buttonClick (response function return value) in response to the event onClick.

[0410]

[0411] The custom UI engine 11 supports the life cycle events of controls loaded by the UI execution engine, including onPreMount, onMount, onUnmount, onPreUpdate and onUpdate; among them, onPreMount means it is called before the control is mounted on the UI; onMount means it is called after the control is mounted on the UI; onUnmount means it is called after the control is removed from the UI; onPreUpdate means it is called before the UI is refreshed due to data changes; onUpdate means it is called after the UI is refreshed due to background data changes.

[0412] In one implementation, whether the event is consumed or not is determined by the return value of the response function. Native interface definition, transparently passing the processing results of the background data to the control.

[0413] 4. Differentiation Description

[0414] 1. The properties of the controls supported by the custom UI engine 11 depend on the configuration parameters of the electronic device.

[0415] The operating system defines variables for electronic device configuration parameters. These variables can be declared in interface description files. When an electronic device executes the interface description file, it accesses the configuration parameters and obtains the values ​​based on its hardware and software. This allows different electronic devices to generate different UIs when running the same interface description file, due to their varying hardware and software conditions and configuration parameters.

[0416] In one implementation, the configuration parameters (config) of the current electronic device are accessed through $env.config.

[0417] For example, the configuration parameters of the electronic device may include the contents shown in Table 1:

[0418] Table 1

[0419]

[0420] The following example shows how the control's dependOn attribute can be assigned a value to a field in a configuration parameter, declaring that the control's properties depend on that configuration parameter. In this example, the visibility of the Scan control (TextView) depends on the device's camera hardware (camera_sensor); this means that if the device has a camera, the Scan control is displayed; otherwise, the Scan control is not displayed.

[0421]

[0422] 2. layout-data-uimode is used to describe the UI of a specified device.

[0423] Developers can declare the UI for a specific device in layout-data-uimode. The display effects of the UI for a specific device are different from those of the general UI.

[0424] In one implementation, layout-data-uimode declares all conditions applicable to the UI of a specified device. For example, please refer to Figure 16The interface description file 710 includes code blocks such as layout-data-common 711 and layout-data-watch 712. Layout-data-common 711 is used to describe a common UI applicable to various types of electronic devices, and layout-data-watch 712 is used to describe a UI applicable to a watch. Figure 16 As shown, the properties and property values ​​of each control in the common UI are declared in layout-data-common 711. The mobile phone reads the interface description file 710, parses and executes the content in layout-data-common 711, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 711. For example, the mobile phone generates an image control 721 according to the content block 7111 in layout-data-common 711, generates a control group 722 according to the content block 7112 in layout-data-common 711, generates a control group 723 according to the content block 7113 in layout-data-common 711, generates a button control 724 according to the content block 7114 in layout-data-common 711, and generates a control group 725 according to the content block 7115 in layout-data-common 711. In this way, the UI 720 of the mobile phone is generated based on the content block 7111 , the content block 7112 , the content block 7113 , the content block 7114 , and the content block 7115 .

[0425] The layout-data-watch 712 declares the properties and property values ​​of the controls in the watch UI. The watch reads the interface description file 710 and determines that the layout-data-watch 712 for the watch exists in the interface description file 710. It then parses and executes the content in layout-data-watch 712, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-watch 712. For example, the watch generates an image control 731 according to the content block 7121 in layout-data-watch 712, a control group 732 according to the content block 7122 in layout-data-watch 712, a control group 733 according to the content block 7123 in layout-data-watch 712, and a button control 734 according to the content block 7124 in layout-data-watch 712. In this way, the watch UI 730 is generated according to the content blocks 7121, 7122, 7123, and 7124.

[0426] In this way, the watch, as a designated device, reads the content in the second code segment (layout-data-watch 712), and electronic devices other than the watch read the content in the first code segment (layout-data-common 711); different types of electronic devices can present different UI display effects by reading the same interface description file of the same UI; by developing a set of interface description files, differentiated UIs can be developed for different types of electronic devices, reducing the development difficulty for developers.

[0427] In another implementation, layout-data-uimode specifies the difference between the device UI and the general UI. For example, please refer to Figure 17 The interface description file 810 includes code blocks such as layout-data-common 811 and layout-data-watch 812. Layout-data-common 811 is used to describe a common UI applicable to various types of electronic devices, and layout-data-watch 812 is used to describe the difference between the watch UI and the common UI. Figure 17 As shown, layout-data-common 811 declares the properties and property values ​​of each control in the common UI. The mobile phone reads the interface description file 810, parses and executes the content in layout-data-common 811, and generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 811. The mobile phone generates an image control 721 according to content block 8111 in layout-data-common 811, a control group 722 according to content block 8112 in layout-data-common 811, a control group 723 according to content block 8113 in layout-data-common 811, a button control 724 according to content block 8114 in layout-data-common 811, and a control group 725 according to content block 8115 in layout-data-common 811. In this way, the UI 720 of the mobile phone is generated based on the content block 8111 , the content block 8112 , the content block 8113 , the content block 8114 , and the content block 8115 .

[0428] The layout-data-watch 812 declares the properties and property values ​​of the controls in the watch UI that are different from the general UI. The watch reads the interface description file 810, parses and executes the content in layout-data-common 811; the watch determines that there is layout-data-watch 812 for the watch in the interface description file 810, and parses and executes the content in layout-data-watch 812; the watch generates corresponding controls according to the properties and property values ​​of each control declared in layout-data-common 811 and layout-data-watch 812. Figure 17 As shown, the watch generates a picture control 731 corresponding to the content block 8111 in layout-data-common811, generates a control group 732 corresponding to the content block 8112 in layout-data-common 811, generates a control group 733 corresponding to the content block 8113 in layout-data-common 811, and generates a button control 734 corresponding to the content block 8114 in layout-data-common 811. According to the description of layout-data-watch 812, the attribute value of the visible property (visible) of the control group generated corresponding to the content block 8115 in layout-data-common 811 is set to invisible (gone). In other words, the control group corresponding to the content block 8115 in layout-data-common 811 is not displayed on the watch. As shown in FIG. Figure 17 As shown, UI 730 of the watch includes controls generated based on content block 8111 , content block 8112 , content block 8113 , and content block 8114 .

[0429] In this way, all types of electronic devices read the content in the first code segment (layout-data-common 711), and the watch, as a designated device, also reads the content in the second code segment (layout-data-watch 712); different types of electronic devices can present different UI display effects by reading the same interface description file of the same UI; by developing a set of interface description files, it is possible to develop differentiated UIs for different types of electronic devices, reducing the development difficulty for developers.

[0430] 3. Style

[0431] Custom UI Engine 11 supports custom parameters in style for the current interface description file. For example, the developer defines myTextStyle in style and can call this custom parameter in layout-data-common as $style.myTextStyle.

[0432]

[0433] The grammatical semantic rules provided in the embodiment of the present application are used to develop UI. The grammar is concise and efficient. A set of interface description files can be developed to be applicable to different types of electronic devices, avoiding the need to develop UIs separately for different types of electronic devices and reducing development costs.

[0434] The UI parsing engine 11a is used to parse the interface description file and convert the content in the interface description file into a data format that matches the UI execution engine 11b. In some examples, the UI parsing engine 11a can also perform a syntax check on the content in the interface description file. If the syntax check succeeds, the interface description file is parsed; if the syntax check fails, the interface description file is not parsed.

[0435] In one example, see Figure 18 The UI parsing engine 11a reads the interface description file, parses the data in the declaration (model), style (style), layout (layout-data-common, layout-data-uimode) and other fields in the interface description file, and saves it to the database after pre-processing. The control parser is used to parse the data in the layout field, and the UI execution engine 11b is recursively called to instantiate the control according to the logical structure of the layout description to form the UI control tree. The attribute parser is used to parse the attribute field of each control, and the UI execution engine 11b is called to set the attributes for each control to complete the UI drawing.

[0436] The workflow of the control parser and attribute parser is as follows: Figure 19As shown. The control parser obtains the name of the control and obtains the property list of the control. If the control identity (ID) exists, add the control ID; if the control style (style) exists, add the control style; instantiate the control to form a control queue. If there is a sub-layout, recursively call the controls in the sub-layout. After parsing the controls in all layouts, add the controls and return the generated controls. The property parser obtains the instantiated control from the control queue, reads the property name and property value corresponding to the control, and stores the properties (including property name and property value) in the hash table. If there is a sub-layout, recursively call the controls in the sub-layout. After the properties of all controls are parsed, the property values ​​of each control saved in the hash table are assigned to the corresponding controls.

[0437] The UI execution engine 11b is used to build UI controls (instantiate controls and set properties) based on the data parsed by the UI parsing engine 11a, layout the controls, and generate the interface declared in the interface description file; it can also realize the mapping between device events and user behaviors, and execute actions corresponding to user behaviors defined in the interface description file in response to user behaviors.

[0438] For example, please refer to Figure 20A In the UI execution engine 11b, a Builder class is constructed for each control. The Builder class uses The same inheritance logic is used to enable child controls to inherit all properties and setting methods of parent controls without the need for repeated definitions. The Builder class contains the entity construction method of the corresponding control and the setting method of the control's unique visual properties to complete control instantiation and property setting. For developer-defined controls, a Builder class for the custom control can be provided in the UI execution engine 11b, which has low access costs and is more developer-friendly.

[0439] In some embodiments, for the interface description file declared Native control properties, UI execution engine 11b can set properties according to the declaration in the interface description file, complete the control instantiation, and the constructed control has Native control properties.

[0440] Please refer to Figure 20B For the extended properties of the control declared in the interface description file, if it is determined that the operating system includes the custom UI programming capability, the UI execution engine 11b sets the properties according to the declaration in the interface description file, completes the control instantiation, and constructs a control with extended properties; if it is determined that the operating system does not include the custom UI programming capability, the UI execution engine 11b maps the corresponding extended properties according to the extended properties declared in the interface description file. Native control properties, according to the corresponding Set the properties of the native control, complete the control instantiation, and construct a A control with native control properties. This control does not have extended properties.

[0441] For example, please refer to Figure 20C , an installation package of the App is generated on the developer's device, which includes an interface description file. The App's installation package is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the above-mentioned electronic device 100) to download the App's installation package in the application market. After the user-side electronic device runs the App's installation package, it obtains the interface description file in the installation package; when the user-side electronic device runs the App, it displays a UI that matches the electronic device on the display screen according to the interface description file.

[0442] For example, the interface description file includes the following content:

[0443]

[0444] In one example, tablet 460 runs App as a user-side electronic device. The operating system of tablet 460 includes custom UI programming capabilities. On the UI of tablet 460, the custom control group HwMagicLayout is successfully constructed, and the control group has the layout properties extended in the operating system. Exemplarily, the layout properties extended in the operating system may include layout properties such as automatic stretching, hiding, equal distribution, proportion, extension or line break. Among them, automatic stretching means that the height or width of the control is automatically enlarged or reduced according to the window size to adapt to the window size. Hiding refers to the ability of the control to be visible or invisible in the layout. Equal distribution means that the content in the control is evenly distributed in the layout. Proportion means that the control occupies the total size of the layout according to the specified percentage in the specified direction. Extension means that the control is extended and displayed on the UI according to the size of the display screen. Line break means that the content in the control is displayed in one or more lines in the layout. As Figure 20C As shown, the UI layout of tablet 460 has the extended layout properties of the system. When tablet 460 is displayed in portrait mode, control 461, control group 462, control group 463, control 464, and control group 465 are arranged vertically in a single column. When tablet 460 is displayed in landscape mode, control 461, control group 462, control group 463, and control 464 are arranged vertically in the first column, and control group 465 is arranged in the second column. The UI layout of tablet 460 adapts to the size and shape of the display window.

[0445] In the UI of tablet 460, the interactive capability "zoomEnable" is enabled for control "imageview" 461. When tablet 460 is connected to mouse 480, control 461 can be magnified in response to a user's zoom operation on control 461 (for example, when the cursor of mouse 480 is positioned on control 461 and the mouse wheel of mouse 480 is rotated upward).

[0446] In another example, the tablet 470 is used as a user-side electronic device to run the App. The operating system of the tablet 470 does not include the ability to program a custom UI. The UI of the tablet 470 does not support the custom control group HwMagicLayout. The controls in the UI have Native LinearLayout properties. When tablet 470 is displayed in portrait or landscape orientation, control 471, control group 472, control group 473, control 474, and control group 475 are arranged vertically in a column. The UI layout of tablet 470 is fixed and cannot be adaptively adjusted based on the size or shape of the display window.

[0447] In the UI of tablet 470, the interactive capability "zoomEnable" does not take effect on control 471 "imageview". That is, when tablet 470 is connected to mouse 480, if the cursor of mouse 480 is placed on control 471 and the scroll wheel of mouse 480 is rotated upward, the size of control 471 remains unchanged.

[0448] In this way, the interface description file that conforms to the grammatical rules in the grammatical semantic library 11d can be successfully run in different operating systems, achieving cross-operating system platform operation and reducing the development difficulty for developers.

[0449] It should be noted that the UI execution engine 11b dynamically parses data when the electronic device runs the interface description file, and obtains the electronic device related parameters when the electronic device runs the interface description file; it avoids the developer from pre-compiling the interface description file in the development tool and generating preset data files; in this way, UI development does not rely on the compilation environment, and cross-development platform development and operation can be achieved.

[0450] The MVVM framework 11c is used to achieve two-way binding between UI elements and backend data. In the interface description file, you declare and specify the binding relationship between UI elements (such as controls and control groups) and backend data. Optionally, you can also perform simple data instance settings. The MVVM framework 11c can refresh backend data based on UI changes, and automatically refresh the corresponding UI based on backend data changes. This helps developers focus on UI design and layout, simplifies the UI development process, and significantly reduces the development time required to implement frontend and backend data interaction.

[0451] For example, please refer to Figure 21 , the developer declares the control fields in the interface description file, binds properties to the controls, and binds the background data objects. The UI parsing engine 11a parses the binding behavior in the interface description file to obtain the correspondence between the control properties and the background data objects. The MVVM framework 11c is used to implement two-way binding between controls and background data. When the background data changes, the MVVM framework 11c maps the background data with the data of the corresponding control properties; the UI execution engine 11b sets the property data of the control and refreshes the UI. When the property data of the control in the UI changes (for example, in response to user input, the shape, text and other data of the control change), the MVVM framework 11c maps the data of the control properties with the background data; the background data is refreshed.

[0452] The UI rendering engine 11e is used to render and organize the interface generated by the UI execution engine 11b and output the display content to the display screen.

[0453] The user interface implementation method provided in the embodiment of the present application allows different types of electronic devices to read the same interface description file for the same UI and present different UI layouts. This allows the development of a single set of interface description files that are applicable to various types of electronic devices, reducing the development difficulty for developers.

[0454] In some embodiments, please refer to Figure 22 The software system of the electronic device 100 may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.

[0455] The interface description file of the application layer App1 adopts the json format; the interface description file of App2 adopts the xml format.

[0456] The operating system of the application framework layer includes a control unit. When the electronic device 100 runs an App, the control unit obtains the interface description file of the App. Exemplarily, when the electronic device 100 runs App1, the control unit obtains the interface description file of App1 in json format; when the electronic device 100 runs App2, the control unit obtains the interface description file of App2 in xml format. The control unit distributes the interface description file to the basic UI engine 10 or the custom UI engine 11 for UI drawing according to the type of the interface description file. For example, the control unit obtains the interface description file of App1 in json format and distributes the interface description file of App1 in json format to the custom UI engine 11 for processing. For example, the control unit obtains the interface description file of App2 in xml format and distributes the interface description file of App2 in xml format to the basic UI engine 10 for processing. In one implementation, the specified paths of the interface description file in json format and the interface description file in xml format in the application installation package are different. The control unit obtains the interface description file in json format at the first specified path of the App1 application installation package and obtains the interface description file in xml format at the second specified path of the App2 application installation package. In another implementation, different tags are preset in the json format interface description file and the xml format interface description file, and the control unit determines the type of the interface description file according to the preset tags of the interface description file.

[0457] The custom UI engine 11 parses, executes, and renders the json format interface description file of App1 to generate the UI of App1. The controls in the UI of App1 can support general OS (such as ) native UI programming capabilities, and can also support customized UI programming capabilities in the operating system of the electronic device 100.

[0458] The basic UI engine 10 parses, executes, and renders the XML format interface description file of App2 to generate the UI of App2. The controls in the UI of App2 can support general OS (such as )Native UI programming capabilities.

[0459] In this way, the electronic device 100 can run both apps developed using the json format interface description language and apps developed using the xml format interface description language, thus achieving forward compatibility with the operating system.

[0460] The embodiment of the present application also provides a method for implementing a user interface for the UI of an application widget.

[0461] Developers can create widgets for apps. For example, mobile phones support displaying app widgets in the notification bar, on the desktop, and on the negative screen. Typically, app widgets displayed in the notification bar are called custom notification bars, those displayed on the desktop are called desktop widgets, and those displayed on the negative screen are called negative screen cards. Custom notification bars, desktop widgets, and negative screen cards can present app information more intuitively to users and allow them to perform actions on apps without opening them, making it easier for users to use them. An increasing number of apps are providing widgets for users to use.

[0462] Currently, the supported layouts and control types for displaying on the user interface (UI) of an application widget are relatively limited, failing to meet the diverse needs of users. Embodiments of the present application provide a method and apparatus for implementing a user interface that supports displaying various layouts and control types on the UI of an application widget, facilitating user use of the widget and improving the user experience.

[0463] Typically, developers use an interface description language to develop the UI of an application (Application, App) in an application development tool. Developers also use an interface description language to develop the UI of an application widget in an application development tool.

[0464] Please refer to Figure 23 , an application development tool (such as Android Studio, DevEcoStudio, etc.) is installed on the electronic device 200. In this application, the electronic device 200 may also be referred to as a developer device. The developer develops the UI of the App in the application development tool to form an interface description file. The interface description file in this application may also be referred to as a description file. The developer also develops the UI of the application widget in the application development tool to form a component interface description file. The developer packages the interface description file and the component interface description file into the installation package of the App and publishes the App in the application market provided by the server 300. The application market can provide installation packages for various Apps for users to download. For example, the installation package can be Application package (Android application package, APK) file.

[0465] It should be noted that, in one implementation, the component interface description file is independent of the interface description file. In another implementation, the component interface description file may be part of the interface description file (for example, a code segment in the interface description file is used as the component interface description file). The embodiments of the present application are not limited to this. In the following embodiments, an example is given in which the component interface description file is a separate file.

[0466] Taking a mobile phone as an example, a user can use the mobile phone to download an app's installation package from an application market. The app's installation package includes interface description files and component interface description files. For example, after downloading the music app's installation package, the mobile phone can install the music app by running the installation package. This also allows the mobile phone to obtain the interface description files and component interface description files contained in the installation package.

[0467] For example, Figure 23 As shown, after the mobile phone has installed the music app, the mobile phone desktop includes a shortcut icon for the music app - the "Music" icon 103. The mobile phone can receive a user click operation on the "Music" icon 103. In response to the user click operation on the "Music" icon 103, the mobile phone generates the UI of the music app according to the interface description file and presents the UI of the music app on the display. The mobile phone can also display a small component of the music app (called a music widget) on the mobile phone desktop according to user settings. The mobile phone generates the UI of the music widget according to the component interface description file and displays the UI 104 of the music widget on the display.

[0468] It is understood that in some embodiments, developers can develop the UI of the app and the UI of the application widget directly on the electronic device 100 and run the app and application widget on the electronic device 100; that is, the electronic device 200 and the electronic device 100 can be the same electronic device. This embodiment of the present application is not limited to this.

[0469] Typically, the elements presented in the UI are called controls (Views), which can provide users with certain operating functions or display certain content. For example, System native controls include text controls (TextView), input boxes (EditText), buttons (Button), image buttons (ImageButton), image controls (ImageView), etc.

[0470] Taking the Android system as an example, all elements in the UI of the application are composed of controls (View) and control groups (ViewGroup). A UI can contain one or more Views or ViewGroups. View is an element displayed in the display interface; ViewGroup is a layout container for storing Views (or ViewGroups). New Views or ViewGroups can be added to ViewGroup so that each View is arranged according to a certain hierarchy and structural relationship. For example, developers can use linear layout (LinearLayout), table layout (TableLayout), relative layout (RelativeLayout), layer layout (FrameLayout), absolute layout (AbsoluteLayout) or grid layout (GridLayout) to design the View or ViewGroup in each UI in the App, thereby generating a layout file for each UI; such as an interface description file or a component interface description file.

[0471] at present, The system supports a limited number of layout modes and control types in application widgets, which cannot meet the diverse needs of users. The embodiment of the present application provides a user interface implementation method that can support the application of diverse layout modes and control types in application widgets.

[0472] The user interface implementation method provided in the embodiment of the present application not only supports The system's native linear layout (LinearLayout), layer layout (FrameLayout), relative layout (RelativeLayout) and grid layout (GridLayout) are applied to application widgets, and also support System-native layout methods such as TableLayout and AbsoluteLayout are applied to application widgets.

[0473] The user interface implementation method provided in the embodiment of the present application not only supports System native buttons (Button), image controls (ImageView), image buttons (ImageButton), progress bars (ProgressBar), text controls (TextView), list controls (ListView), grid controls (GridView), stack controls (StackView), dynamic loading of controls (ViewStub), adaptive controls (AdapterViewFlipper), switch controls (ViewFlipper), clocks (AnalogClock), timers (Chronometer) and other controls are applied to application widgets; it also supports System-native controls such as the input box (EditText), check box (CheckBox), slide selector (Picker), scroll view (ScrollView), radio button (RadioButton), rating bar (RatingBar), search box (SearchView), drag bar (SeekBar), switch (Switch) are applied to application widgets.

[0474] For example, take a mobile phone as the electronic device 100 and use a music app on the mobile phone as an example. The mobile phone 100 can display a small component of the music app (called a music widget) on the desktop. Figure 24A As shown, mobile phone 100 displays UI 910 for a music widget. UI 910 includes an image control 911 for displaying an image set by the app; a text control 912 for displaying the name of the music track being played; a search box 913 for receiving user input and performing a search in response to the user's input text; an image button 914 for switching the display style of the music widget; a drag bar 915 for adjusting the progress of the music playback based on user operations; and other controls.

[0475] For example, Figure 24B As shown, the mobile phone 100 can receive a user's click operation on the search box 913. In response to the click operation, an enlarged search box 913 and a soft keyboard 91a are displayed on the desktop. The user can use the soft keyboard 91a to enter text for the search box 913. The mobile phone 100 searches based on the text entered in the search box 913.

[0476] For example, Figure 24C As shown, the mobile phone 100 can receive the user's drag operation on the drag bar 915 to adjust the progress of playing music.

[0477] The user interface implementation method provided in the embodiments of this application also supports applying the customized UI programming capabilities of the operating system to application widgets, enabling the widgets to possess the visual attributes, layout attributes, interaction attributes, animation attributes, and hardware-dependency attributes extended by the operating system. Layout attributes refer to the layout of individual controls in the UI, such as their shape, position, and size. Visual attributes refer to visual effects such as color and grayscale. Interaction attributes provide control responses based on user actions, such as performing a search based on a user's "confirm" action. Animation attributes provide animation effects on controls, such as displaying a click-and-rebound animation on controls. Hardware-dependency attributes refer to the hardware and software parameters of the device to which a control depends. Exemplarily, the extended operating system layout attributes may include automatic stretching, hiding, equal distribution, proportion, extension, or line wrapping. Automatic stretching refers to automatically expanding or contracting the height or width of a control to fit the window size. Hiding refers to the ability to make a control visible or invisible within the layout. Equal distribution refers to evenly distributing the content within a control within the layout. Proportion refers to a control occupying a specified percentage of the total layout size in a specified direction. Stretching means that the control is stretched on the UI according to the display size. Wrapping means that the content in the control is displayed in one or more lines in the layout.

[0478] For example, Figure 24D As shown, the UI 910 of the music widget may include a picture button 916 for displaying the lyrics of the music. The mobile phone may receive a click operation on the picture button 916 by the user, and in response to the click operation on the picture button 916 by the user, the mobile phone displays the lyrics of the currently playing music. The picture button 916 may be displayed or not displayed on the UI 910. Whether the picture button 916 is displayed depends on whether the currently playing music has lyrics. If the currently playing music has corresponding lyrics, the picture button 916 is displayed on the UI 910; if the currently playing music does not have corresponding lyrics, the picture button 916 is not displayed on the UI 910. For example, Figure 24D As shown, when the currently playing music is Music 1, the UI 910 includes a picture button 916; when the currently playing music is Music 2, the UI 910 does not include a picture button 916.

[0479] The user interface implementation method provided in the embodiment of the present application also supports applying the layout method and control types defined by the developer in the App to the application widgets.

[0480] Developers can use Various system-native layout methods and control types, layout methods and control types defined in the operating system, and layout methods and control types defined in the App are applied to application widgets for user convenience.

[0481] In one implementation, see Figure 25 In the App development phase, the developer opens a development tool (e.g., DevEco Studio) in the electronic device 200 (developer device), uses an interface description language in the development tool, describes the interface and defines the interface behavior according to the syntax and semantic specifications of the interface description language, and develops the UI; forms interface description files and component interface description files for release.

[0482] For example, consider the use of JSON format for interface description files and component interface description files. Developers can separately design UI layout, data and interface binding, interactive behavior orchestration, and differentiated descriptions in these files. All UIs in applications and widgets are composed of controls. UI layout orchestration involves arranging the properties of controls within the UI. Data and interface binding involves declaring and specifying the binding relationships between UI elements (such as controls and control groups) and backend data within the interface description file or component interface description file. Interactive behavior orchestration involves declaring the actions that controls execute in response to events in the interface description file or component interface description file. The range of events supported by a control is determined by the event listeners it supports. For example, if a button supports the setOnClickListener listener for click events, the onClick event can be bound to the control in the interface description file. Differentiated descriptions include designating different code segments for different electronic devices, enabling the widget's UI to display differently on these devices; obtaining configuration parameter values ​​based on the hardware and software conditions of the electronic device and applying them to controls; and defining parameters applicable within the app.

[0483] For example, users can declare in the component interface description file of the music app Figures 24A-24D The picture control 911, text control 912, search box 913, picture button 914, drag bar 915 and picture button 916 shown in the figure are attributed to these controls so that these controls are included in the UI 910 of the music widget. These controls can be System-native controls can also be controls defined in the operating system or in a music app.

[0484] Developers can also apply control properties defined in the operating system to these controls in the component interface description file. For example, the software dependency properties defined in the operating system can be applied to the image button 916. This declares that the display properties of the image button 916 depend on the music currently playing in the app, including lyrics.

[0485] Developers can also bind controls to background data in the component interface description file. For example, a drag bar 915 can be bound to background data. When the phone receives a user drag operation on the drag bar 915, it updates the current music playback progress in the background data based on the user drag operation on the drag bar 915. If the current music playback progress in the background data changes, the drag bar 915 is updated.

[0486] Developers can also declare the execution actions corresponding to the control response events in the component interface description file. For example, the developer can declare that the search box 913 executes a search action in response to a click event.

[0487] Afterwards, an installation package of the App is generated on the developer's device, which contains the interface description file and the component interface description file. The App's installation package is uploaded to the server, and the App is published in the application market provided by the server. The user can use the user-side electronic device (the above-mentioned electronic device 100) to download the App's installation package in the application market. After the user-side electronic device runs the App's installation package, it obtains the interface description file and the component interface description file in the installation package. For example, Figure 25 As shown, after the mobile phone 100 runs the installation package of the music app, the music app icon 103 is displayed on the desktop. The mobile phone 100 can receive the user's click operation on the icon 103, run the music app, and display the music app UI on the display screen according to the interface description file.

[0488] Furthermore, the user-side electronic device adds an application widget to the notification bar, desktop, or negative one screen according to the user's settings. The user-side electronic device generates the UI of the application widget according to the component interface description file and displays the UI of the application widget in the notification bar, desktop, or negative one screen. For example, Figure 25 As shown, the mobile phone 100 displays a UI 910 of a music widget on the desktop.

[0489] Please refer to Figure 26 The application widget process in electronic device 100 runs independently of the application process. Applications installed on electronic device 100 run by calling the application process, and application widgets run by calling the application widget process. For example, if an application widget is set on the desktop, the desktop process is the application widget process; if an application widget is set on the negative one screen, the process displayed on the negative one screen is the application widget process; if an application widget is set in a specified application, the process of the specified application is the application widget process.

[0490] The electronic device 100 also includes units such as a custom UI engine 11 and a widget framework 12.

[0491] In some examples, an application process obtains an app's interface description file and invokes a custom UI engine 11 to parse and execute the file, generating the app's UI. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, and an MVVM (model-view-viewmodel) framework 11c. The UI parsing engine 11a parses the interface description file and converts its contents into a data format compatible with the UI execution engine 11b. In some examples, the UI parsing engine 11a also performs syntax validation on the contents of the interface description file. If the syntax validation succeeds, the file is parsed; if not, the parsing is not performed. The UI execution engine 11b constructs UI controls (instantiates controls and sets properties) based on the data parsed by the UI parsing engine 11a, lays out the controls, and generates the interface declared in the interface description file. It also maps device events to user actions, executing actions corresponding to user actions defined in the interface description file in response to user actions. The MVVM framework 11c performs bidirectional binding between UI elements and backend data. In the interface description file, you declare and specify the binding relationships between UI elements (such as controls and control groups) and backend data. Optionally, you can also perform simple data instance settings. The MVVM framework 11c can refresh backend data based on UI changes, and automatically refresh the corresponding UI based on backend data changes. This helps developers focus on UI design and layout, simplifies the UI development process, and significantly reduces the development time required to implement frontend and backend data interaction.

[0492] In some examples, the application process obtains a component interface description file, calls the widget framework 12 to process the component interface description file, and forms widget UI data for displaying the application widget UI. Among them, the widget framework 12 includes modules such as virtual control construction 12a, data binding 12b, widget service 12c, and event proxy 12d. Among them, the virtual control construction 12a parses the component interface description file by calling the UI parsing engine 11a, instantiates the parsed component interface description file, and calls the UI execution engine 11b to construct the interface, construct controls, control groups, etc., to form widget UI data. These widget UI data exist in the application process and are used to bind to background data (such as the control model (ViewModel)). Data binding 12b is used to bind the properties, interaction events, etc. of the control or control group constructed by the virtual control construction 12a to background data (such as the control model (ViewModel) that processes business logic, which contains data processing logic related to virtual objects). The widget service 12c is used to track and process the currently processed object and the data (model) bound to the object during the generation of the application widget UI; it is also used to manage data transmission between the application process and the application widget process; it is also used to manage cross-process event agents and send and receive cross-process events. The event agent 12d is used to handle the return and response of events in the application widget process. For example, a dedicated event transmission class (such as HiAction) is defined in the event agent 12d. The event transmission class supports the implementation of the Parcelable interface and can be transmitted across processes (for example, calling Native cross-process Binder mechanism). A series of events are stored in the event transmission class, and each event includes information such as layout identifier, control identifier, event type, etc. When the user operates the application widget, the application widget process receives the operation and triggers an interaction event, that is, a new event is added in HiAction. The application widget process transmits the newly added event to the application process. The application process responds to the event and performs corresponding actions. The application process also calls the MVVM framework for processing; if there is a change in data or control properties, the widget UI data is updated, the cross-process interface data and properties are updated, and the display interface of the application widget process is further updated. In one implementation method, the application process also sends the component interface description file to the application widget process. The application widget process calls the widget framework 12 to process the component interface description file, forms the widget UI data, and displays the widget UI data, that is, displays the application widget IU.

[0493] After an application is installed on an electronic device, the user can add an application widget corresponding to the application in the notification bar, desktop or negative one screen.

[0494] In some embodiments, the user does not open the application, but adds the application widget corresponding to the application separately. Figure 27 , the mobile phone 100 receives a two-finger pinch operation on the desktop by the user. In response to the two-finger pinch operation on the desktop, the mobile phone 100 displays a quick setting interface 1010. The quick setting interface 1010 includes a "Widget" option 1011 for adding a desktop widget to the desktop. In response to the user clicking the "Widget" option 1011, the mobile phone 100 may display a desktop widget adding interface 1020. The desktop widget adding interface 1020 includes a "Music" option 1021 for adding a music widget to the desktop. The mobile phone 100 receives a user clicking the "Music" option 1021. In response to the user clicking the "Music" option 1021, the "Music Widget" UI 910 is displayed on the desktop of the mobile phone 100.

[0495] In other embodiments, the user adds an application widget corresponding to the application in the application. For example, Figure 28 As shown, a user opens the "Music Settings" interface 1030 of the "Music" application on mobile phone 100. "Music Settings" interface 1030 includes a "Widget" option 1031. "Widget" option 1031 is used to add a music widget to the mobile phone's home screen. Mobile phone 100 receives a user click on "Widget" option 1031 and, in response to the user click on "Widget" option 1031, displays "Widget" interface 1040. "Widget" interface 1040 includes a "Style 1" option 1041 and a "Style 2" option 1042; it also includes an "Add" button 1043 and a "Cancel" button 1044. The user can click "Add" button 1043 to add the music widget to the home screen using the interface corresponding to "Style 1" option 1041 or "Style 2" option 1042; the user can also click "Cancel" button 1044 to exit the music widget addition process. For example, the mobile phone 100 receives a user click on the "Add" button 1043, and according to the user's selection, adds the music widget to the desktop with the interface corresponding to the "Style 2" option 1042. The UI 910 of the "Music Widget" is displayed on the desktop of the mobile phone 100.

[0496] In one implementation, see Figure 29A , the user performs an operation to add an application widget. The application widget process of the electronic device 100 receives the user's operation to add an application widget (for example, the user Figure 27The application widget process notifies the application process of receiving the user's operation to add the application widget; if the application process is not started, the electronic device 100 pulls up the application process and makes the application process run in the background. Alternatively, the application process of the electronic device 100 receives the user's operation to add the application widget (for example, the user clicks on the "Music" option 1021). Figure 28 (The "Add" button 1043 is clicked in the dialog box), and the application process notifies the application widget process that it has received the user's operation to add the application widget.

[0497] The application process obtains the component interface description file from the application installation package. The application process calls the custom UI engine 11 to parse and execute the component interface description file. Then, it calls the virtual control builder 12a to build the widget UI data control. It generates controls and control groups according to the layout in the component interface description file, forming the widget UI data (including information such as controls and their layout).

[0498] It should be noted that the component interface description file can be a file independent of the interface description file, or it can be a code segment in the interface description file. Optionally, after the electronic device 100 obtains the component interface description file, it can only parse and execute part of the code segment therein. For example, the user selects Figure 28 In the music widget interface corresponding to the "Style 1" option 1041, the mobile phone 100 parses and executes the code segment corresponding to "Style 1" in the component interface description file after receiving the user's click operation on the "Add" button 1043; the user selects Figure 28 In the music widget interface corresponding to the "Style 2" option 1042, after receiving the user's click operation on the "Add" button 1043, the mobile phone 100 parses and executes the code segment corresponding to "Style 2" in the component interface description file.

[0499] Afterwards, data binding 12b calls MVVM framework 11c to data bind the widget UI data to the backend data (e.g., the control model). This way, if the backend data changes, the corresponding widget UI data can be refreshed; if the widget UI data changes, the corresponding backend data can be refreshed.

[0500] Furthermore, the application process sends the component interface description file to the application widget process.

[0501] The application widget process calls the custom UI engine 11 to parse and execute the component interface description file, then calls the virtual control builder 12a to construct the widget UI data controls. It generates controls and control groups according to the layout in the component interface description file, forming the widget UI data (including information such as controls and their layout). This widget UI data is then displayed, i.e., the application widget UI is displayed. Because the application process uses the same code segment to generate the widget UI data and the application widget process uses to generate the application widget UI, the controls on the application widget UI correspond one-to-one to the controls in the widget UI data.

[0502] Optional, such as Figure 29B As shown, after generating the widget UI data, the application process can also send the widget UI data to the application widget process, and the application widget process displays the widget UI according to the widget UI data, that is, displays the application widget UI. In this way, the controls on the application widget UI also correspond one-to-one with the controls in the widget UI data.

[0503] After adding an application widget to the notification bar, desktop, or negative one screen, users can operate the application on the application widget UI. For example, users can drag Figure 24A The drag bar 915 on the music widget UI 910 is used to adjust the playback progress of the music currently being played by the music application.

[0504] In one implementation, see Figure 29C When a user performs an operation on the app widget's UI, the app widget process receives the user operation and transmits it to the event broker 12d. Event broker 12d defines a dedicated event transmission class for cross-process transmission. This event transmission class stores multiple events, each of which includes information such as the layout identifier, control identifier, and event type. After receiving the user operation, event broker 12d generates an event corresponding to the operation in the event transmission class and sends the event to the app process (if the app process is not already started, it starts the app process so that it runs in the background). After receiving the event, the app process retrieves the corresponding control based on the layout identifier and control identifier and executes the corresponding business logic based on the event acting on the control. Because there is a one-to-one correspondence between the controls on the app widget UI and the controls in the widget UI data, the app process also refreshes the background data based on the received events. Changes in the background data trigger updates to the widget UI data. The app process can also send the updated widget UI data to the app widget process, which then displays the updated app widget UI based on the refreshed widget UI data.

[0505] In some embodiments, please refer to Figure 29DThe application process starts and retrieves the interface description file. The application process calls the custom UI engine 11 to parse and execute the interface description file, generate the application UI, and display the application UI. The MVVM framework 11c then binds the application UI to backend data (e.g., the control model).

[0506] The application process receives the user's operation to add an application widget, and the application process obtains the component interface description file from the application installation package. The application process calls the custom UI engine 11 to parse and execute the component interface description file, and then calls the virtual control construction 12a to build the widget UI data control, and generates controls, control groups, etc. according to the layout arrangement in the component interface description file to form the widget UI data (including information such as controls and control layout). Afterwards, the data binding 12b calls the MVVM framework 11c to data bind the widget UI data with the background data (for example, the control model). Furthermore, the application widget process displays the UI of the application widget based on the widget UI data.

[0507] In this way, the UI of the application and the UI of the corresponding application widget are displayed on the display screen of the electronic device 100.

[0508] In one example, see Figure 30 , which shows a process example of the user interface implementation method provided in an embodiment of the present application.

[0509] The user performs an operation to add an application widget. The application widget process of the electronic device 100 receives the user's operation to add an application widget (for example, the user Figure 27 The application widget process notifies the application process of receiving the user's operation to add the application widget; if the application process is not started, the electronic device 100 pulls up the application process and makes the application process run in the background. Alternatively, the application process of the electronic device 100 receives the user's operation to add the application widget (for example, the user clicks on the "Music" option 1021). Figure 28(click operation of the "Add" button 1043 in the dialog box), the application process notifies the application widget process that it has received the user's operation of adding an application widget. The widget framework, MVVM framework, background data and other modules are initialized. The widget framework obtains the component interface description file from the application installation package and sends it to the virtual control construction module. The virtual control construction module constructs the control according to the component interface description file to form the widget UI data. The data binding module calls the MVVM framework to bind the widget UI data and the background data. The application process calls the widget service for binding service, and the widget service binds the event proxy to the widget UI data; after that, the widget UI data and the event proxy of the widget UI data and other information are sent across processes. In this way, after the application widget process receives the widget UI data and the event proxy information of the widget UI data, it can display the application widget UI according to the widget UI data.

[0510] After successfully adding an app widget, users can perform operations on the widget UI. When the widget process receives the user's operation on the widget UI, the event broker adds an event corresponding to the operation and sends the event to the app process. The app process executes business logic in response to the event and calls the MVVM framework to update backend data. When the app's business data changes, the backend data changes, causing the MVVM framework to update the widget UI data. The app process sends the updated widget UI data across processes. Upon receiving the updated widget UI data, the app process can display the updated widget UI based on the updated widget UI data.

[0511] The user interface implementation method provided by the embodiment of the present application is that the application process generates widget UI data according to the component interface description file and sends the component interface description file or widget UI data to the application widget process; the application widget process generates the application widget UI according to the component interface description file or widget UI data. Developers can declare The system supports native layout methods and control types, as well as custom control types and UI programming capabilities within the operating system, as well as layout methods and control types defined by developers within the app. The operating system supports the widget process invoking the UI engine to parse and execute the component interface description file, generating the widget UI. This allows the widget UI to display a variety of layout methods and control types, making it easier for users to use the widget and improving the user experience.

[0512] In some embodiments, after the electronic device adds the application widget, the electronic device is turned off. After the electronic device is turned on again, the UI of the application widget is displayed. That is, after the electronic device adds the application widget, the application widget UI is reloaded. Figure 31As shown, the mobile phone 100 is turned on, and the desktop of the mobile phone 100 displays a UI 910 of a music widget.

[0513] In one example, see Figure 32 , which shows an example process of a method for reloading an application widget UI in an electronic device.

[0514] When the electronic device is powered on, the widget process starts. It retrieves the widget interface description file from the application installation package. The widget process then calls the custom UI engine to parse and execute the component interface description file, construct the widget UI data, and display the widget UI based on the widget UI data.

[0515] After the application widget is successfully reloaded, the user can perform operations on the application widget UI. The application widget process receives the user's operation on the application widget UI, and the event agent adds the event corresponding to the operation, pulls up the application process, makes the application process run in the system background, and sends the event to the application process; the application process executes the corresponding business logic in response to the event and calls the MVVM framework to update the background data. The process of event interaction and background data change can be referred to Figure 30 The relevant description will not be repeated here.

[0516] In this scenario, the application widget process only needs to generate and draw the loaded application widget UI. The process of generating widget UI data, binding the widget UI data with the background data, and establishing the corresponding relationship between the widget UI data and the application widget UI can be omitted.

[0517] In the user interface implementation method provided by the embodiments of the present application, the application process of an electronic device generates widget UI data based on a widget interface description file and binds the widget UI data to background data. The application widget process also obtains the widget UI data based on the component interface description file and displays the widget UI data as the UI of the application widget. This establishes a correspondence between the application widget UI and the background data, allowing the display of various layouts and control types on the application widget UI, making it easier for users to use the application widget and improving the user experience.

[0518] An embodiment of the present application also provides a user interface implementation method for presenting a UI when an App on an electronic device is projected onto a playback device for playback.

[0519] With the rapid development of the Internet of Things (IoT), the types and number of IoT devices are growing rapidly. Consumers can use mobile phones, tablets and other devices as control devices for IoT devices to control IoT devices, so that the control devices and IoT devices work together. For example, when a user uses an App on a control device, the App can be cast to an IoT device for playback (the IoT device is called a playback device). For example, since the screen size of a TV is larger, it can bring a better viewing experience to users. Users can cast the App on their mobile phone to the TV for playback. Since the screen shapes and sizes of IoT devices vary greatly, how to cast screens on IoT devices with screens of various shapes and sizes and obtain a casting interface that matches the screen shape and size of the IoT device is a problem that needs to be solved.

[0520] The present application provides a user interface implementation method and apparatus that supports projecting various UIs on a control device to an IoT device for playback, thereby improving user experience. The control device is the user-side electronic device (electronic device 100) in each of the above embodiments.

[0521] This application embodiment provides a user interface implementation method, please refer to Figure 33 , an application development tool (such as Android Studio, DevEco Studio, etc.) is installed on the electronic device 200. The electronic device 200 in this application may also be referred to as a developer device. Typically, developers use an interface description language to develop the UI of the App and the UI of the playback end in the application development tool. It is understandable that in some embodiments, the developer can directly develop the UI of the App and the UI of the playback end on the control device 100, and run the App on the control device 100; that is, the electronic device 200 and the control device 100 can be the same electronic device. The embodiments of this application are not limited to this.

[0522] In some embodiments, the developer develops the UI of the App (i.e., the UI displayed when the electronic device installs and runs the App) in the application development tool to form an interface description file. The interface description file in this application may also be referred to as a description file. The developer also develops the UI of the App for display on the playback end (i.e., the playback end UI) in the application development tool to form a playback end interface description file. The developer packages the interface description file and the playback end interface description file into the installation package of the App and publishes the App in the application market provided by the server 300. The application market can provide installation packages for various Apps for users to download. For example, the installation package can be Android application package (APK) file.

[0523] For example, using a mobile phone as the control device 100, a user can use the phone to download an app's installation package from an app store. For example, after downloading the video app's installation package, the phone can install the video app by running the package. This also allows the phone to obtain the interface description file and the playback interface description file contained in the installation package.

[0524] Optionally, in some embodiments, the interface description file may also be used as the playback-end interface description file, that is, the interface description file and the playback-end interface description file are the same file.

[0525] The mobile phone can present the UI of the corresponding App on the display screen according to the interface description file. For example, after the mobile phone downloads the installation package of the video App, a "video" icon 101 is generated on the desktop. The user can click on the "video" icon 101 to open the video App. In response to the user's click operation on the "video" icon 101, the mobile phone runs the video App. The mobile phone is installed with an OS platform, and the custom UI engine of the OS platform reads the interface description file, parses and executes the interface description language, and renders the UI of the video App according to the interface description in the interface description file. The display device of the mobile phone (such as a display screen) presents the UI 105 of the video App. Furthermore, the interface description file can also include a definition of interface behavior. In response to the user's operation on UI 105, the mobile phone can perform corresponding interface actions and implement interface behavior according to the interface behavior defined in the interface description file. Usually, the OS platform also has a corresponding programming language for implementing interface behavior, realizing dynamic changes of UI 105 and responding to user operations on UI 105; for example Using JAVA, Use Swift programming language to implement interface behavior.

[0526] The mobile phone can also project the various interfaces of the video app to the playback device 1000 for display. For example, the main interface or playback interface of the video app is projected to the playback device 1000. The playback device 1000 renders the corresponding playback UI according to the interface description in the playback interface description file that matches its device type. For example, please continue to refer to Figure 33The video app's UI 105 includes a "Cast" button 106; "Cast" button 106 is used to cast the app's interface running on the phone to a playback device for display. The phone receives a user click on "Cast" button 106. In response to the user's click, the phone displays a device selection interface 107. Device selection interface 107 includes a prompt 108, prompting the user to select a playback device for casting. Device selection interface 107 also includes a "Living Room TV" option 109 and a "My Tablet" option 10a. The user can click "Living Room TV" option 109 to cast the video app's UI to a smart TV for display. The phone receives a user click on "Living Room TV" option 109. In response, the phone casts the video app's UI to the smart TV. The smart TV displays the playback UI 1001 corresponding to UI 105. The user can also click on the "My Tablet" option 10a to cast the UI of the video app to the tablet for display. The mobile phone receives the user's click operation on the "My Tablet" option 10a, and in response to the user's click operation on the "My Tablet" option 10a, the mobile phone casts the UI of the video app to the tablet. The tablet displays the playback end UI 1002 corresponding to the UI 105. Smart TVs and tablets are different in device types, screen sizes and shapes. The interface layout of the playback end UI 1001 on the smart TV and the playback end UI 1002 on the tablet are different; that is, the playback end UI is displayed differently on electronic devices of different device types.

[0527] The playback device 1000 may include a portable computer (such as a mobile phone), a smart home device (such as a smart TV, a smart screen, a large screen, a smart speaker, etc.), a handheld computer, a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), a tablet computer, a laptop computer, a netbook, an augmented reality (AR)\virtual reality (VR) device, a car computer, etc., and the present application embodiment does not impose any restrictions on this. In one example, the playback device 1000 may include Figure 2 The structure shown is not described here in detail. It is understood that the embodiment of the present application Figure 2 The illustrated structure does not constitute a specific limitation on the playback device 1000. In other embodiments of the present application, the playback device 1000 may include Figure 2 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 2The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0528] Please refer to Figure 34 The control device 100 includes a custom UI engine 11, a screen projection framework 13, a transmission channel adapter 14 and other units. The custom UI engine 11 provides an IF1 interface, and the screen projection framework 13 provides IF2, IF3, IF4 and IF5 interfaces.

[0529] Table 2 shows the description of interfaces IF1 to IF5.

[0530] Table 2

[0531]

[0532] The custom UI engine 11 parses and executes the app's interface description file to generate the app's UI. The custom UI engine 11 may include a UI parsing engine 11a, a UI execution engine 11b, and an MVVM (model-view-viewmodel) framework 11c. The UI parsing engine 11a is used to parse the interface description file and convert its content into a data format compatible with the UI execution engine 11b. In some examples, the UI parsing engine 11a may also perform syntax validation on the content in the interface description file. If the syntax validation succeeds, the interface description file is parsed; if the syntax validation fails, the interface description file is not parsed. The UI execution engine 11b is used to construct UI controls (instantiate controls and set properties) based on the data parsed by the UI parsing engine 11a, lay out the controls, and generate the interface declared in the interface description file. It may also map device events to user behaviors and execute actions corresponding to user behaviors defined in the interface description file in response to user behaviors. The MVVM framework 11c is used to bidirectionally bind UI elements to backend data. In the interface description file, you declare and specify the binding relationships between UI elements (such as controls and control groups) and backend data. Optionally, you can also perform simple data instance settings. The MVVM framework 11c can refresh backend data based on UI changes, and automatically refresh the corresponding UI based on backend data changes. This helps developers focus on UI design and layout, simplifies the UI development process, and significantly reduces the development time required to implement frontend and backend data interaction.

[0533] The transmission channel adapter 14 is used to adapt the data transmission channel between the control device 100 and the playback device 1000. For example, the data of the control device 100 is converted into a format suitable for the data transmission channel so that the control device 100 can send data to the playback device 1000 through the data transmission channel.

[0534] The screen projection framework 13 is used to process the playback end interface description file to form the playback end UI data for displaying the playback end UI. The screen projection framework 13 includes modules such as virtual control construction 13a, data binding 13b, screen projection service 13c, data transceiver 13d, resource transmission 13e, event agent 13f and life cycle 13g. Among them, the virtual control construction 13a calls the UI parsing engine 11a and the UI execution engine 11b to construct controls, control groups, etc. according to the playback end interface description file to form the playback end UI data. These playback end UI data exist in the App process and are used to bind with the background data. The data binding 13b is used to bind the properties, interactive events, etc. of the controls or control groups constructed by the virtual control construction 13a with the background data (for example, the control model (ViewModel) that processes business logic). The screen projection service 13c is used to track and process the currently processed object and the data (model) bound to the object during the screen projection process; it is also used to manage the data transmission channel between the control device 100 and the playback device 1000. Data transceiver 13d is used to control the sending and receiving of data between device 100 and playback device 1000. For example, the interface of the transceiver agent can be defined to implement the default transceiver built into the control device 100. For another example, the transceiver suitable for the App can be customized according to the interface specification in the App. For example, if information is transmitted through ContentProvider in the App, ContentProvider is used in the send() function in data transceiver 13d to implement data sending and receiving. Resource transmission 13e is used to transmit specific types of data resources (for example, data, pictures, videos, etc. whose data volume is greater than the set value). Resource transmission 13e is used to manage data resources of this specific type, such as sending, receiving, caching, identification, progress control, etc. Event agent 13f is a channel for transmitting events to avoid events being blocked by data transmission. Life cycle 13g is used to manage the life cycle of the joint operation entities of the control device 100 and the playback device 1000 during the screen projection process. Exemplarily, the life cycle of the control device 100 and the playback device 1000 is shown in Table 3:

[0535] Table 3

[0536]

[0537] For example, Figure 35AAs shown, the control device 100 is in a stand-alone running state, triggers screen projection, and waits for user authorization to project the screen to the playback device. If the user agrees to the authorization, the playback device sends an authorization instruction to the control device, and the control device 100 enters the server-side running state. If the user refuses the authorization, the playback device sends a refusal authorization instruction to the control device, and the control device 100 stops projecting the screen. When the control device 100 is in the server-side running state, the App switches to the background running, and then the data push to the playback device is stopped; the App switches to the foreground running and starts pushing data to the playback device. When the control device 100 is in the server-side running state, the control device closes the App, or the playback device is turned off, then the screen projection is stopped. When the control device 100 is in the server-side running state, the App of the playback device switches to the background running, and the control device 100 enters the server-side paused state. When the control device 100 is in the server-side paused state, the App of the playback device switches to the foreground playing, and the control device 100 enters the server-side running state. When the control device 100 is in the server-side paused state and the playback device is turned off, the screen projection is stopped.

[0538] For example, Figure 35B As shown, the playback device 1000 receives the screen projection request initiated by the control device and waits for the user to confirm the authorization. If the user confirms the authorization, the playback device 1000 enters the screen projection running state; if the user refuses the authorization, the playback device 1000 stops running. When the playback device 1000 is in the screen projection running state and the App switches to the background running, it enters the background resident state. When the playback device 1000 is in the background resident state and the App switches to the foreground to play, it enters the screen projection running state. When the playback device 1000 is in the screen projection running state or the background resident state, the playback device is turned off or the control device closes the App, it stops running.

[0539] Please refer to Figure 36 During the app development phase, developers generate an app installation package on their device, which includes an interface description file and a playback interface description file. Developers use an interface description language and follow the language's syntax and semantics to develop the interface description file and playback interface description file on their device. They then add code to these files to begin UI development.

[0540] Developers can perform UI layout arrangement, data & interface binding, interactive behavior arrangement, and differentiated description in the interface description file and the playback interface description file respectively.

[0541] All UI in the App is composed of controls. The layout of the UI is to arrange the properties of the controls in the UI. For example, the controls in the UI can include all Native controls and controls extended in the operating system also support controls that developers customize in the App or integrate through static packages. Among them, controls can specifically include text controls, such as TextView controls, EditText controls, etc., and can also include button controls, such as Button controls, ImageButton controls, etc., and can also include picture controls, such as Image controls, etc., and this embodiment of the application does not impose any restrictions on this. Control properties include Native properties as well as extended visual properties, layout properties, interaction properties, motion properties, and hardware-dependency properties in the operating system. Visual properties refer to visual effects such as color and grayscale of a control. Interaction properties are the ability to provide control responses based on user behavior; for example, performing a search based on a user's "confirmation" behavior. Motion properties refer to the animation effects displayed on a control; for example, displaying a click-and-bounce effect on a control. Hardware-dependency properties refer to the hardware and software parameters of the device on which a control depends.

[0542] Data & interface binding, that is, declaring and specifying the binding relationship between UI elements (such as controls, control groups) and background data in the interface description file or the player interface description file.

[0543] Interaction behavior orchestration involves declaring the actions that a control will perform in response to events in the interface description file or the player interface description file. The range of events supported by a control is determined by the event listeners it supports. For example, if a Button control supports the listener setOnClickListener for click events, you can bind the onClick event to the control in the interface description file.

[0544] Differentiation description:

[0545] 1. Developers can declare variables for electronic device configuration parameters in an interface description file or a player interface description file. When an electronic device runs the interface description file or a player interface description file, it accesses the electronic device's configuration parameters and obtains the values ​​of the configuration parameters based on its hardware and software conditions. This allows different electronic devices to generate different UIs when running the interface description file or the player interface description file, due to their different hardware and software conditions and different configuration parameters.

[0546] 2. Developers can customize parameters in the interface description file or the player interface description file. Taking the interface description file and the player interface description file in json format as an example, the interface description file or the player interface description file can include style, layout-data-common and other parts. Developers define myTextStyle in style and can call this customized parameter in layout-data-common as $style.myTextStyle. The example is as follows,

[0547]

[0548] 3. Developers can compile code for a specific device in the interface description file or the player interface description file. For example, taking the interface description file and the player interface description file in JSON format as an example, the interface description file or the player interface description file can include the following structure:

[0549]

[0550] Developers can declare a universal player UI in layout-data-common. All types of playback devices parse the content in layout-data-common and layout the universal player UI according to the content in layout-data-common. layout-data-uimode is used to describe the player UI of a specified device. In one implementation, layout-data-uimode declares the difference between the player UI of a specified device and the universal player UI. The specified device parses and executes the content in layout-data-common and layout-data-uimode to generate the player UI of the specified device. In another implementation, layout-data-uimode declares all conditions applicable to the player UI of a specified device. The specified device layouts its player UI according to the content in layout-data-uimode. The specified device can be one of the types such as mobile phones, watches, car machines, smart home devices (such as smart TVs, smart screens, smart speakers, etc.), large screens, tablets, laptops or desktop computers. For example, the specific forms of layout-data-uimode can include layout-data-phone (for mobile phones), layout-data-watch (for watches), layout-data-television (for smart TVs), layout-data-pad (for tablets), layout-data-car (for car computers), etc. In this way, different types of playback devices can parse and execute the corresponding code segments to build the playback UI; and the playback UI that matches the type of playback device can be displayed on different types of playback devices.

[0551] The developer uploads the app installation package generated on the developer device to the server, and the app is published on the app market provided by the server. The user can use the user-side electronic device (the control device 100 described above) to download the app installation package from the app market. After the control device runs the app installation package, it obtains the interface description file and the playback interface description file contained in the installation package. When the control device runs the app, it displays the UI that matches the control device on the display according to the interface description file.

[0552] While the control device is running an app, it can also project the app's interface to a playback device for display. For example, the control device determines the playback device to project the screen based on user input and sends a projection command to the playback device, which includes the projection interface identifier. The playback device receives the projection command, obtains the corresponding playback interface description file based on the projection interface identifier, and forms a playback UI that matches its device type according to the playback interface description file.

[0553] For example, please refer to Figure 37A , the control device 100 receives the user's screen projection operation (for example, the mobile phone receives the user's Figure 33 The control device 100 also determines the device type of the playback device 1000 for screen projection based on the user input (for example, the mobile phone receives the user's input). Figure 33 The user clicks on the "TV in the living room" option 109, and determines that the playback device 1000 is a smart TV; the mobile phone receives the user's Figure 33 The "My Tablet" option 10a is clicked to determine that the playback device 1000 is a tablet computer).

[0554] The virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11, constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data. For example, if the playback device 1000 is a smart TV, the control device 100 parses and executes the code segment corresponding to the smart TV in the playback end interface description file to form the playback end UI data for projecting the screen to the smart TV; if the playback device 1000 is a tablet computer, the control device 100 parses and executes the code segment corresponding to the tablet computer in the playback end interface description file to form the playback end UI data for projecting the screen to the tablet computer. Afterwards, the data binding 13b calls the MVVM framework 11c to data bind the objects in the playback end UI data to the background data (for example, the control model). In this way, if the background data changes, the corresponding playback end UI data can be refreshed, and if the playback end UI data changes, the corresponding background data can be refreshed. Furthermore, the control device 100 sends the playback end interface description file and the resource file (including the data resources associated with the playback end interface description file) to the playback device 1000 via the data transceiver 13d. In one implementation, the control device 100 encodes the playback end interface description file, and after encoding the layout information, resource values, data, response event definitions and other data, transmits them to the playback device 1000 via a data transmission channel; specific types of data resources (for example, data, pictures, videos, etc. with a data volume greater than a set value) are transmitted to the playback device 1000 via a specific data transmission channel. Optionally, specific types of data resources can be transmitted to the playback device 1000 before sending the playback end interface description file. In this way, the rate of transmitting the playback end interface description file to the playback device 1000 is increased, and the delay of the playback device 1000 displaying the playback end UI is shortened. The control device 100 also initializes the event agent 13f to establish an event transmission channel between the control device 100 and the playback device 1000 for transmitting event information.

[0555] The playback device 1000 receives the playback end interface description file and data resources through the data transceiver 13d. The virtual control construction 13a in the OS of the playback device 1000 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11. The control is constructed according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file. Controls and control groups are generated according to the layout arrangement in the code segment to form playback end UI data (including information such as controls and control layout); and the playback end UI data is displayed according to the playback end UI data, that is, the playback end UI is displayed. Because the control device 100 generates the playback end UI data using the same code segment as the playback device 1000 generates the playback end UI data, the controls on the playback end UI generated by the playback device 1000 correspond one-to-one to the controls in the playback end UI data generated by the control device 100.

[0556] After the player UI is generated, the player device 1000 displays the player UI on the display screen in a shape and size that matches the player device screen. Figure 36 As shown, when the smart TV is used as a playback device, the smart TV parses and executes the layout-data-television code segment in the playback interface description file to generate the corresponding playback UI of the smart TV; when the tablet is used as the playback device, the tablet parses and executes the layout-data-pad code segment in the playback interface description file to generate the corresponding playback UI of the tablet.

[0557] Optionally, in some embodiments, Figure 37B As shown, the control device 100 constructs a control according to the code segment corresponding to the device type of the playback device 1000 in the playback interface description file, generates playback UI data according to the layout in the code segment, and then sends the generated playback UI data to the playback device 1000. After receiving the playback UI data, the playback device 1000 displays the playback UI according to the playback UI data.

[0558] The user interface implementation method provided by the embodiment of the present application is that the playback device generates a playback end UI corresponding to the playback device according to the code segment corresponding to the device type of the playback device in the playback end interface description file. The playback end UI displayed by different types of playback devices matches the shape and size of their screens. In addition, when developing an App, developers can easily develop the playback end UI and define various types of controls (including all It supports native controls and controls extended in the operating system, as well as controls customized by developers in the app or integrated through static packages. It supports a variety of control types; this allows various types of apps to support the screen projection function, and the playback UI supports richer and more diverse control types, making it easier for users to use.

[0559] In one implementation, the virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback-end interface description file, and constructs the playback-end UI data according to the code segment corresponding to the device type of the playback device 1000 in the playback-end interface description file. The playback-end UI data is not displayed on the display screen of the control device 100 (that is, the playback-end UI data exists in the App process and is not sent to the display process). The control device 100 displays the UI generated according to the interface description file. After the interface of the control device 100 is projected to the playback device 1000, the user can perform other operations on the control device 100, and the playback device 1000 plays the projected content normally.

[0560] For example, Figure 38A As shown, the mobile phone displays the interface 1210 of the "Video" App, and the interface 1210 includes a "Cast Screen" button 1211. The mobile phone receives the user's click operation on the "Cast Screen" button 1211 and determines the playback device according to the user's input. The mobile phone casts the screen to the smart TV according to the user input. Figure 37A or Figure 37B As shown, the mobile phone generates the player UI data according to the player interface description file and sends the player UI description file to the smart TV. The smart TV generates the player UI data according to the player interface description file and displays the player UI according to the player UI data. Figure 38A , the smart TV displays the playback end UI 1220.

[0561] After that, the user can continue to perform other operations on the phone. For example, Figure 38A As shown, the mobile phone receives a user click operation on the picture 1212, and in response to the user click operation on the picture 1212, displays the interface 1230 of the "Video" App.

[0562] In this way, after the control device projects the screen to the playback device, it can continue to perform other functions, and run independently from the playback device playing the projected content without affecting each other, thereby achieving better collaboration between devices.

[0563] For example, Figure 38BAs shown, the custom UI engine 11 in the OS of the control device 100 parses and executes the interface description file, generates the application UI, and displays the application UI. The MVVM framework 11c performs data binding between the application UI and the background data (eg, the control model).

[0564] The virtual control construction 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file, and generates controls, control groups, etc. according to the layout arrangement in the code segment to form the playback end UI data. Afterwards, the data binding 13b calls the MVVM framework 11c to data bind the objects in the playback end UI data with the background data (for example, the control model). Further, the control device 100 sends the playback end interface description file and the resource file (including the data resources associated with the playback end interface description file) to the playback device 1000 through the data transceiver 13d; or sends the playback end UI data to the playback device 1000, so that the playback device 1000 can display the playback end UI according to the playback end UI data.

[0565] In another implementation, the virtual control implementation 13a in the OS of the control device 100 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback interface description file, and constructs the playback UI data based on the code segment corresponding to the device type of the playback device 1000 in the playback interface description file. The App process sends the playback UI data to the display process, which displays the playback UI on the display screen of the control device 100. The control device 100 and the playback device 1000 display the playback UI generated based on the same code segment.

[0566] For example, Figure 39A As shown, the mobile phone displays the interface 1210 of the "Video" App, and the interface 1210 includes a "Cast Screen" button 1211. The mobile phone receives the user's click operation on the "Cast Screen" button 1211 and determines the playback device according to the user's input. The mobile phone casts the screen to the smart TV according to the user input. Figure 39B As shown, the mobile phone generates player UI data based on the player UI description file and displays the player UI according to the player UI data. The mobile phone also sends the player UI description file to the smart TV. The smart TV generates player UI data based on the player UI description file and displays the player UI according to the player UI data.

[0567] like Figure 39A As shown, after the mobile phone casts the screen to the smart TV, the mobile phone also displays the playback end UI, and both the mobile phone and the smart TV display the playback end UI 1220.

[0568] In this way, the control device and the playback device play the playback end UI synchronously, which can realize mirroring screen projection and the control device and the playback device work together.

[0569] In some embodiments, when the control device 100 receives a user action on the playback UI or a change in service data, the data binding 13b invokes the MVVM framework 11c to update the playback UI data. Because there is a one-to-one correspondence between controls in the playback UI and controls in the playback UI data, an update to the playback UI data triggers an update to the playback UI. This allows the playback UI of the playback device 1000 to be updated synchronously with user actions or service data changes received by the control device 100.

[0570] For example, please refer to Figure 40A , the mobile phone displays the UI 1310 of the "Video" app. Based on the user input, the mobile phone projects the UI 1310 of the "Video" app to the smart TV. The smart TV displays the player UI 1320 of the "Video" app, which includes a "Play" button 1321.

[0571] The user can start playing a video on the smart TV (for example, the user selects the "Play" button 1321 using the smart TV remote control and clicks the "Play" button 1321). The smart TV receives the user's click operation on the "Play" button 1321, and in response to the user's click operation on the "Play" button 1321, plays the video and displays an updated UI 1320. The updated playback UI 1320 includes a "Pause" button 1322.

[0572] In one implementation, Figure 40B As shown, a dedicated event transmission class is defined in event proxy 13f, which is used for cross-device transmission. This event transmission class stores multiple events, each of which includes information such as a layout identifier, a control identifier, and an event type. When the playback device 1000 receives a user operation on the playback UI, it generates an event corresponding to the operation in the event transmission class and transmits it to the control device 100 via the event transmission channel. After receiving the event, the control device 100 retrieves the corresponding control based on the layout identifier and the control identifier, and executes the corresponding business logic based on the event acting on the control. Because there is a one-to-one correspondence between the controls in the playback UI and the controls in the playback UI data, the control device 100 also updates the background data, and changes in the background data trigger updates to the playback UI data. The control device 100 sends the updated playback UI data to the playback device 1000, which then displays the updated playback UI according to the updated playback UI data.

[0573] This allows users to control the app on the playback device, with the control device executing the corresponding business logic and updating the playback UI on the device. In some examples, if the control device and the playback device mirror the playback UI, the UI on the control device can also be updated synchronously, facilitating user experience. Furthermore, the control device executes the relevant business logic for user actions on the playback device, providing unified control of the playback device for easier management and preventing low-performance playback devices from failing to support complex business logic processing.

[0574] In some embodiments, the playback device 1000 receives a second operation performed by the user on the playback UI. The playback device 1000 obtains an updated playback interface description file from the control device 100 and generates an updated playback UI.

[0575] For example, Figure 40C , the mobile phone displays the UI 1330 of the "Video" app. Based on the user input, the mobile phone casts the UI 1330 of the "Video" app to the smart TV. The smart TV displays the playback-end UI 1340 of the "Video" app. The smart TV receives the user's second operation on the playback-end UI 1340 (for example, the user moves the focus on the playback-end UI 1340 from "Movies" to "Variety Shows" through the remote control). In response to the user's second operation on the playback-end UI 1340, the smart TV displays the updated playback-end UI, namely, the playback-end UI 1350 of the "Video" app.

[0576] In one implementation, Figure 40DAs shown, the playback device 1000 receives the user's second operation on the playback UI, generates an event corresponding to the second operation in the event transmission class, and transmits it to the control device 100 via the event transmission channel. After receiving the event, the control device 100 obtains the corresponding control based on the layout identifier and the control identifier, and executes the corresponding business logic based on the event acting on the control. The control device 100 determines to update the playback UI to the playback UI with the focus on "Variety Show" and obtains the playback UI description file 2 corresponding to the playback UI with the focus on "Variety Show". The virtual control construction 13a in the OS of the control device 100 calls the UI, parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback interface description file 2, constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback interface description file 2, and generates controls, control groups, etc. according to the layout arrangement in the code segment, forming the playback UI data 2. Afterwards, the data binding 13b calls the MVVM framework 11c to data bind the objects in the playback UI data 2 to the background data (e.g., the control model). Furthermore, the control device 100 transmits the second player interface description file and the resource file (including the data resources associated with the second player interface description file) to the playback device 1000 via the data transceiver 13d. In one implementation, the control device 100 encodes the second player interface description file, including layout information, resource values, data, and response event definitions, and transmits the encoded data to the playback device 1000 via a data transmission channel. Specific types of data resources (e.g., data with a data volume greater than a set value, images, videos, etc.) are transmitted to the playback device 1000 via a specific data transmission channel. The playback device 1000 receives the playback end interface description file 2 and the data resource file through the data receiving and sending 13d. The virtual control construction 13a in the OS of the playback device 1000 parses and executes the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2 by calling the custom UI engine 11, and constructs the control according to the code segment corresponding to the device type of the playback device 1000 in the playback end interface description file 2. Controls, control groups, etc. are generated according to the layout arrangement in the code segment to form playback end UI data 2 (including information such as controls and control layout); and is displayed according to the playback end UI data 2, that is, the updated playback end UI is displayed.

[0577] In this way, users can directly operate the player UI on the playback device, control the device to execute the business logic corresponding to the operation, and send the playback device the corresponding updated player UI description file. The playback device generates an updated player UI based on the updated player UI description file. This allows users to directly operate the player UI on the playback device and successfully switch the player UI.

[0578] Please refer to Figure 41A , which shows an example of the processing flow of the control device in the user interface implementation method provided by the embodiment of the present application. After the control device installs the App, the screen projection framework, MVVM framework, background data, etc. are initialized. Then the resource transmission module transmits the data resources related to the App and binds the data resources to the screen projection service. The screen projection framework obtains the playback end interface description file from the application installation package and sends it to the virtual control construction module. The virtual control construction module constructs the control according to the playback end interface description file to form the playback end UI data; and binds the playback end UI data to the screen projection service. Afterwards, the virtual control construction module notifies the data binding module to bind the playback end UI data with the background data, and the data binding module mobilizes the MVVM framework to bind the playback end UI data with the background data. The screen projection service is also bound to the event agent. Furthermore, the playback end interface description file is encoded and sent to the playback device. In this way, after the playback device receives the encoded playback end interface description file, it can generate the playback end UI according to the playback end interface description file and display it.

[0579] The screen projection framework receives events from the playback device and sends them to the MVVM framework. The MVVM framework updates backend data based on these events. When the app's business data changes, these backend data changes trigger the MVVM framework to update the playback UI data. The control device sends the updated playback UI data to the playback device. Upon receiving the updated playback UI data, the playback device can display the updated playback UI based on the updated playback UI data.

[0580] Please refer to Figure 41B , which shows an example of the processing flow of the playback device in the user interface implementation method provided by the embodiment of the present application. The playback device receives the encoded playback end interface description file sent by the control device through the transmission channel. The screen projection framework calls the custom UI engine to parse and execute the playback end interface description file, forms the playback end UI data, and displays the playback end UI according to the playback end UI data. The event proxy module receives the user's operation on the playback end UI, generates a corresponding event, and transmits the event to the control device so that the control device processes the event.

[0581] An embodiment of the present application provides a method for implementing a user interface. When a control device runs an App, if a preset condition is met, the control device pushes preset information to a playback device for playback.

[0582] Take a mobile phone as the control device and a smart watch as the playback device as an example. Figure 42A, the user opens the "takeaway" app on the phone to order food and pay. Optionally, the app switches to background operation. When the preset conditions are met (for example, the phone confirms that the takeaway order is expected to be delivered in 20 minutes; for example, the user performs a query operation on the smart watch), the phone pushes the preset information to the smart watch for display. For example, Figure 42A As shown, the smart watch displays the playback end UI 1410 ; the playback end UI 1410 includes a “takeaway order progress” control 1411 and prompt information 1412 .

[0583] In one implementation, the developer defines a playback end interface description file (or a section of code in the playback end interface description file) during the development phase that pushes information to the playback end when preset conditions are met. The playback end UI of the smart watch is defined in the playback end interface description file, including controls 1411 and prompt information 1412. During the process of the mobile phone running the "takeout" app (including the "takeout" app switching to background operation), the mobile phone determines that the preset conditions are met, reads the specified code segment, generates playback end UI data based on the specified code segment, and sends the specified code segment (or the generated playback end UI data) to the smart watch. The smart watch generates the playback end UI 1410 based on the specified code segment (or the generated playback end UI data).

[0584] For example, Figure 42B The user uses a navigation app on the phone to navigate. When a preset condition is met (e.g., a change in direction), the phone generates playback UI data based on a specified code segment and sends the specified code segment (or generated playback UI data) to the smartwatch. The smartwatch generates playback UI 1420 based on the specified code segment (or generated playback UI data).

[0585] Furthermore, the smartwatch receives the user's actions, generates an event corresponding to the action, and sends the event to the phone for processing. The phone then performs business logic processing, executes the corresponding action, and updates the playback UI data. The phone also sends the updated playback UI data to the smartwatch, which then updates the playback UI based on the updated playback UI data.

[0586] The user interface implementation method provided in the embodiments of this application enables a control device to automatically push certain information about a running app to a playback device for playback when preset conditions are met. Different types of playback devices can access the corresponding code segments, facilitating the implementation of differentiated playback UI layouts. Furthermore, users can control the app on the playback device, while the control device performs business logic processing. This improves the user experience and avoids the problem of low-performance playback devices that lack support for complex business logic processing.

[0587] It is understandable that, in order to realize the above functions, the above electronic device includes a hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0588] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0589] like Figure 43 As shown, an embodiment of the present application discloses an electronic device 1500, which may be an electronic device that runs the above-mentioned development tool, or an electronic device that runs the App in the above-mentioned embodiment, or an electronic device that runs the application widget in the above-mentioned embodiment; the electronic device may be the above-mentioned control device or playback device. The electronic device may specifically include: a display screen 1501; an input device 1502 (such as a mouse, keyboard or touch screen, etc.); one or more processors 1503; a memory 1504; one or more applications (not shown); and one or more computer programs 1505, and the above-mentioned devices may be connected via one or more communication buses 1506. The one or more computer programs 1505 are stored in the above-mentioned memory 1504 and are configured to be executed by the one or more processors 1503, and the one or more computer programs 1505 include instructions, which can be used to execute the relevant steps in the above-mentioned embodiments. In one example, the electronic device 1500 may be Figure 1 In one example, the electronic device 100 or the electronic device 200 may be Figure 14 In one example, the electronic device 1500 may be a developer device or a user side electronic device. Figure 23 In one example, the electronic device 100 or the electronic device 200 may be Figure 33 The control device 100 or the electronic device 200 or the playback device 1000.

[0590] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program code is stored. When a processor executes the computer program code, the electronic device executes the method in the above embodiment.

[0591] The embodiments of the present application also provide a computer program product, which, when executed on a computer, enables the computer to execute the method in the above embodiments.

[0592] Among them, the electronic device 1500, computer-readable storage medium or computer program product provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0593] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0594] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0595] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0596] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a magnetic disk, or an optical disk.

[0597] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for implementing a user interface, characterized in that: include: The first application process of the electronic device reads a component interface description file, where the component interface description file is used to perform an interface description and an interface behavior definition on a first user interface UI of an application widget of the first application; The first application process generates first widget UI data according to the widget interface description file, and binds controls in the first widget UI data to background data in the electronic device operating system; The first application process sends first data to the application widget process; The application widget process receives the first data, obtains the first widget UI data according to the first data, and displays the first UI of the application widget according to the first widget UI data.

2. The method according to claim 1, characterized in that The first data is the component interface description file, and the application widget process obtains the first widget UI data according to the first data, including: The application widget process generates the first widget UI data according to the component interface description file.

3. The method according to claim 1, characterized in that The first data is the first widget UI data.

4. The method according to claim 2 or 3, characterized in that The method further comprises: The first application process generates a first control in the first widget UI data according to the definition of the first code in the component interface description file, and the first control has control properties native to the operating system of the electronic device.

5. The method according to claim 4, characterized in that The native controls of the operating system include: Input field, checkbox, slide selector, scroll view, radio button, rating bar, search box, drag bar, or switch.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The first application process generates a second control in the first widget UI data according to the definition of the second code in the component interface description file, and the second control has control properties customized in the operating system of the electronic device.

7. The method according to claim 6, characterized in that The customized control attributes include: at least one of visual attributes, layout attributes, interaction attributes, motion effect attributes and software and hardware dependency attributes.

8. The method according to claim 7, characterized in that The layout attributes include at least one of stretch, hide, fold, divide, proportion and extend.

9. The method according to any one of claims 1 to 8, characterized in that The component interface description file includes a third code, and the third code is used to define a corresponding relationship between a control attribute of a third control in the first UI of the application widget and first data in the operating system of the electronic device. The method further comprises: The electronic device receives a first input from a user on the third control; The value of the first data is modified according to the first input.

10. The method according to claim 9, characterized in that The method further comprises: The control properties of the third control in the first UI of the application widget change as the first data in the operating system of the electronic device changes.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: The electronic device downloads the application installation package of the first application from the server; the application installation package includes the component interface description file; The electronic device installs the first application using the application installation package.

12. A method for implementing a user interface, characterized in that: include: Displaying a development interface of the first application; The development interface of the first application includes a component interface description file; The component interface description file is used to describe the first user interface UI of the application widget of the first application and define the interface behavior; In response to a first operation input by a user, adding a first code defining a first control in the first widget UI to the component interface description file; The first control has native control properties of the operating system; wherein the native controls of the operating system include: an input box, a check box, a slide selector, a scroll view, a radio button, a rating bar, a search box, a drag bar, or a switch; Generate an application installation package for the first application according to the component interface description file.

13. The method according to claim 12, characterized in that The method further comprises: In response to a second operation input by the user, a second code defining a second control in the first widget UI is added to the component interface description file, and the second control has customized control properties in the operating system; the customized control properties include: visual properties, layout properties, interaction properties, motion properties and at least one of software and hardware dependency properties.

14. The method according to claim 13, characterized in that The layout attributes include at least one of stretch, hide, fold, divide, proportion and extend.

15. An electronic device, characterized in that: include: one or more processors; Display screen; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the method according to any one of claims 1 to 14.

16. A computer-readable storage medium storing instructions, characterized in that: When the instruction is executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 12 to 14.

17. A computer-readable storage medium, characterized in that The computer instructions are used to describe and define the interface behavior of a first user interface UI of an application widget of a first application; The computer instructions include generating a first code for a first control in the first widget UI, wherein the first control has native control properties of an operating system; The native controls of the operating system include: input boxes, check boxes, slide selectors, scroll views, radio buttons, rating bars, search boxes, drag bars, or switches.

18. The computer-readable storage medium according to claim 17, wherein: The computer instructions further include second code for generating a second control in the first widget UI, the second control having control properties customized in the operating system; The customized control attributes include: at least one of visual attributes, layout attributes, interaction attributes, motion effect attributes and software and hardware dependency attributes.

19. The computer-readable storage medium according to claim 18, wherein The layout attributes include at least one of stretch, hide, fold, divide, proportion and extend.

Images