Display devices, interface display methods, storage media and software products

Abstract

This application relates to a display device, an interface display method, a storage medium, and a program product, and is applied in the field of display technology. The display device includes: a user input interface configured to receive user input text; a controller configured to: parse the input text to obtain multiple entity data included in the input text; determine multiple target components matching the multiple entity data from a pre-stored component library based on the multiple entity data; generate interface description data based on the multiple target components and interface display data constraints, the interface description data including at least configuration information and layout information corresponding to each target component, and the interface display data constraints including predefined constraint information for each component when displayed on the screen; perform interface rendering based on the interface description data to generate a rendered image to be displayed; and a display screen configured to display the image to be displayed.

Description

Technical Field

[0001] This application relates to the field of display technology. More specifically, it relates to a display device, an interface display method, a storage medium, and a program product. Background Technology

[0002] Large-screen visualization is an important form of enterprise-level data display, widely used in smart cities, operational monitoring, command and dispatch, and other scenarios. Currently, the development of large-screen visualization interfaces mainly requires developers to manually write a large amount of specialized code. Various component configuration items are numerous, and component layout requires drag-and-drop configuration. This not only demands sufficient technical knowledge from developers to complete the entire development process for visualization large screens, but also leads to low efficiency, unstable final visualization quality, certain security risks, and difficulty in ensuring the normal operation of the production environment. Summary of the Invention

[0003] To address, or at least partially address, the aforementioned technical problems, this application provides a display device, an interface display method, a storage medium, and a program product to solve the problems of low development efficiency, unstable final visualization quality, certain security risks, and difficulty in ensuring the normal operation of the production environment in the prior art for large-screen visualization.

[0004] To achieve the above objectives, the technical solutions provided in this application are as follows: In a first aspect, embodiments of this application provide a display device, the display device comprising: The user input interface is configured to receive text input from the user. The controller is configured to parse the input text to obtain multiple entity data included in the input text; Based on the multiple entity data, determine multiple target components that match the multiple entity data from a pre-stored component library; Based on the multiple target components and interface display data constraints, interface description data is generated. The interface description data includes at least the configuration information and layout information corresponding to each target component. The interface display data constraints include predefined constraint information for each component when it is displayed on the screen. Based on the interface description data, the interface is rendered to generate a rendered screen to be displayed. The display is configured to display the image to be displayed.

[0005] Secondly, embodiments of this application provide an interface display method, the interface display method comprising: Receive text input from the user; The input text is parsed to obtain multiple entity data included in the input text; Based on the multiple entity data, determine multiple target components that match the multiple entity data from a pre-stored component library; Based on the multiple target components and interface display data constraints, interface description data is generated. The interface description data includes at least the configuration information and layout information corresponding to each target component. The interface display data constraints include predefined constraint information for each component when it is displayed on the screen. Based on the interface description data, the interface is rendered to generate a rendered screen to be displayed. Display the screen to be displayed.

[0006] Thirdly, embodiments of this application provide a computer-readable storage medium, including: storing a computer program on the computer-readable storage medium, wherein when the computer program is executed by a processor, it implements the interface display method as described in the second aspect.

[0007] Fourthly, embodiments of this application provide a computer program product, including: when the computer program product is run on a computer, causing the computer to implement the interface display method as described in the second aspect.

[0008] Compared with the prior art, the technical solution provided in this application has the following advantages: receiving user input text; parsing the input text to obtain multiple entity data included in the input text; determining multiple target components matching the multiple entity data from a pre-stored component library based on the multiple entity data; generating interface description data based on the multiple target components and interface display data constraints, wherein the interface description data includes at least the configuration information and layout information corresponding to each target component, and the interface display data constraints include predefined constraint information for each component when displayed on the screen; rendering the interface based on the interface description data to generate a rendered screen to be displayed; and displaying the screen to be displayed. The interface display method implemented through this display device only requires the user to input natural language, which can be automatically parsed and matched with the corresponding components. Then, based on the predefined constraint information for each component when displayed on the screen, standardized interface description data can be generated for interface rendering. This eliminates the need for professional developers to develop the interface, as well as the need to manually select components and lay out layouts, enabling fully automated large-screen development and greatly improving development efficiency. Furthermore, the predefined component constraints can effectively improve the stability and accuracy of the final generated interface and eliminate code execution risks. Attached Figure Description

[0009] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0010] Figure 1 This illustration shows a scenario diagram between a display device and a control device provided in an embodiment of this application; Figure 2 A configuration block diagram of the control device provided in an embodiment of this application is shown; Figure 3 A hardware configuration block diagram of a display device provided in an embodiment of this application is shown; Figure 4 The flowchart of an interface display method provided in an embodiment of this application is shown. Figure 1 ; Figure 5 The flowchart of an interface display method provided in an embodiment of this application is shown. Figure 2 ; Figure 6 A schematic diagram of the interface display system provided in an embodiment of this application is shown. Detailed Implementation

[0011] To better understand the above-mentioned objectives, features, and advantages of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0013] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0014] The terms “comprising” and “having”, and any variations thereof, in this application are intended to cover but not exclude inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0015] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0016] Large-screen visualization is an important form of enterprise-level data display, widely used in smart cities, operational monitoring, command and dispatch, and other scenarios. Currently, the development of large-screen visualization interfaces mainly requires developers to manually write a large amount of HTML / CSS / JavaScript code. For the chart content, separate integration of visualization libraries such as ECharts and AntV is necessary. Each component has numerous configuration options, requiring developers to understand a large number of parameters to complete basic configurations, which demands high technical skills. The overall layout of the large-screen display components requires drag-and-drop configuration; developers drag the corresponding components from the component library to the interface and then configure the parameters. This leads to inefficiency in the current development model. Furthermore, current low-code platforms can introduce AI agents to directly generate HTML / JS code from models, but the generated code quality is unstable, posing certain security risks and making it difficult to guarantee normal operation in production environments. Finally, current development can only be done separately for each platform, failing to achieve cross-platform development and operation.

[0017] To address some or all of the aforementioned technical problems, embodiments of this application provide a display device, an interface display method, a storage medium, and a program product. The method receives user input text; parses the input text to obtain multiple entity data items; determines multiple target components matching the entity data items from a pre-stored component library; generates interface description data based on the target components and interface display data constraints. The interface description data includes at least configuration and layout information corresponding to each target component, and the interface display data constraints include predefined constraints for each component when displayed on the screen; renders the interface based on the interface description data to generate a rendered screen to be displayed; and displays the screen to be displayed. This interface display method, implemented through this display device, only requires user input of natural language, which is automatically parsed and matched with corresponding components. Then, based on predefined constraints for each component when displayed on the screen, standardized interface description data can be generated for interface rendering. This eliminates the need for professional developers to perform interface development or manually select and lay out components, enabling fully automated large-screen development and significantly improving development efficiency. Furthermore, the predefined component constraints effectively improve the stability and accuracy of the final generated interface, eliminating code execution risks.

[0018] The display device provided in this application can have various implementation forms, such as a light-emitting diode (LED) display screen, a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, etc.

[0019] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device in an embodiment of this application, wherein the control device includes a smart device or a control apparatus. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0020] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0021] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, an application running on the smart device can be used to control the display device 200.

[0022] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0023] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0024] Figure 2 An exemplary configuration block diagram of the control device 100 is shown. For example... Figure 2 As shown, the control device 100 may include a controller 110, a communication interface 130, a user input / output interface 140, an external memory, and a power supply. The control device 100 can receive user operation commands and convert them into commands that the display device 200 can recognize and respond to, thus acting as an interactive intermediary between the user and the display device 200.

[0025] Figure 3 A hardware configuration block diagram of display device 200 is shown. (For example...) Figure 3 As shown, the display device 200 may include at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a user interface 280, an external memory, and a power supply.

[0026] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0027] The display 260 can be a liquid crystal display, an OLED display, a touch display, or a projection display, and can also be a projection device and a projection screen.

[0028] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0029] User interface 280 can be used to receive control signals from control device 100 (such as infrared remote control), or to directly receive user input and convert the input into instructions that display device 200 can recognize and respond to. In this case, it can be called user input interface.

[0030] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds. The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0031] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory (internal or external memory). The controller 250 controls the overall operation of the display device 200. For example, in this embodiment, in response to received user input text, the controller 250 can further parse the input text and perform a series of subsequent processing operations.

[0032] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), and random access memory (RAM), read-only memory (ROM), a first to an nth interface for input / output, a communication bus, etc.

[0033] RAM, also known as main memory, is an internal memory that directly exchanges data with the controller. It can be read and written at any time (except during refresh) and is very fast, typically serving as temporary data storage for the operating system or other running programs. Its biggest difference from ROM is data volatility; data stored in RAM is lost when power is off. RAM is used in computers and digital systems to temporarily store programs, data, and intermediate results. ROM operates in a non-destructive read-only manner, allowing only reading and not writing. Once information is written, it is fixed and will not be lost even if power is cut off; therefore, it is also called fixed-function memory.

[0034] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0035] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, realizing the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the GUI, which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0036] This application provides a display device, an interface display method, a storage medium, and a program product. The display device can implement the interface display method provided in this application embodiment, or the functional modules or entities within the display device can implement the interface display method provided in this application embodiment. The display device includes: a user input interface, a controller, and a display, corresponding to the above-described... Figure 3 The user interface 280, controller 250, and display 260 are included.

[0037] In this embodiment of the application, the user interface 280 can be a user input interface, which is configured to receive user input text. Controller 250 is configured to parse the input text to obtain multiple entity data included in the input text; Based on multiple entity data, determine multiple target components that match the multiple entity data from a pre-stored component library; Based on multiple target components and interface display data constraints, interface description data is generated. The interface description data includes at least the configuration information and layout information corresponding to each target component. The interface display data constraints include predefined constraint information for each component when it is displayed on the screen. Render the interface based on the interface description data to generate the rendered screen to be displayed. Monitor 260 is configured to display the screen to be displayed.

[0038] In some embodiments, the controller 250 is specifically configured to: Based on multiple entity data, determine at least one component requirement corresponding to each entity data; Based on at least one component requirement, identify multiple target components from the component library.

[0039] In some embodiments, the controller 250 is specifically configured to: Based on the component configuration constraints included in the data constraints displayed on the interface, determine the configuration information of each target component; Based on the component layout constraints included in the data constraints displayed on the interface, determine the layout information of each target component.

[0040] In some embodiments, the controller 250 is specifically configured to: Determine the initial display position of the target component from the component layout constraints; Check if the initial display position is occupied; If the initial display position is not occupied, generate the layout information of the target component based on the initial display position; If the initial display position is occupied, the system iterates through other free areas on the screen until it detects a free area that satisfies the component layout constraints of the target component. The free area is then marked as occupied, and the layout information of the target component is generated based on the free area.

[0041] In some embodiments, the controller 250 is specifically configured to: Based on the interface description data, construct an initial interface component tree that includes multiple target components; Based on the configuration information corresponding to each target component, determine the display content corresponding to each target component, and update the initial interface component tree based on the display content to obtain the target interface component tree; In response to the rendering start command, the target interface component tree is rendered to generate the screen to be displayed.

[0042] In some embodiments, the controller 250 is specifically configured to: Perform security verification on the interface description data and obtain the verification result; If the verification result indicates that the verification passed, the interface is rendered based on the interface description data to generate the screen to be displayed.

[0043] To illustrate this solution in more detail, the following will use examples to illustrate it. Figure 4 and Figure 5 To explain, it is understandable that Figure 4 and Figure 5 The steps involved may include more or fewer steps in actual implementation, and the order of these steps may also be different, as long as the interface display method provided in the embodiments of this application can be achieved. The interface display method is applied to a display device controlled by a control device. The executing entity of the interface display method can be the display device, or it can be a functional module or functional entity in the display device that can implement the interface display method, which is not limited here.

[0044] like Figure 4 As shown, Figure 4 A flowchart of an interface display method provided in this application embodiment, the method may include the following steps: 401. Receive user input text.

[0045] In this embodiment of the application, the user's input text can be natural language, that is, it can be a sentence used to express the user's needs for the interface display, such as: "Please use a bar chart to display the sales data of each region in the past week". The input text can be entered directly by the user in the form of text, or it can be entered in the form of voice and then converted into text, or it can be content converted into text form through other input forms.

[0046] 402. Parse the input text to obtain the multiple entity data included in the input text.

[0047] In this embodiment, the user's input text includes the user's desired content to be displayed on the interface. Therefore, the input text can be parsed to obtain multiple entity data. These entity data can be understood as keywords in the input text that express the user's interface display requirements. For example, if the input text is "Please display the sales data of each region in the past week using a bar chart," then parsing the input text reveals that the user wants to display sales data on the interface, that the sales data corresponds to each region, that the time span of the sales data is the past week, and that the display format is a bar chart. Therefore, the entity data included in the input text can at least include: bar chart, past week, each, region, sales data, etc.

[0048] 403. Based on multiple entity data, determine multiple target components that match the multiple entity data from the pre-stored component library.

[0049] In this embodiment of the application, after determining multiple entity data, it is necessary to display the content corresponding to each entity data separately. That is, it is necessary to display the content of each entity data through corresponding components. Different entity data may require different components. Therefore, multiple target components that match multiple entity data can be determined. These target components can be components that match the entity data and are selected from a pre-stored component library.

[0050] It should be noted that the pre-stored component library can store various components, each of which supports the display of different content. These components can be divided into multiple categories according to the different content they support. Each category can include at least one specific component. To facilitate viewing and filtering of components, a component directory can be formed for the components in the component library, as shown in Table 1 below.

[0051] Table 1. Component catalog of the pre-stored component library

[0052] For example, for the entity data "bar chart", it can be seen that the entity data specifies a chart format. Therefore, the target component that matches "bar chart" can be selected from the data chart category of the component library. By rendering the target component, a bar chart can be displayed on the interface.

[0053] 404. Generate interface description data based on multiple target components and interface display data constraints.

[0054] In this embodiment, when displaying the content of each component in the interface, it is necessary to ensure that these components meet certain constraints. These constraints can be restrictions on the configuration, content, layout, etc. of the components. Therefore, after determining multiple target components, interface description data can be generated for these target components based on the interface display data constraints. The interface display data constraints can include predefined constraint information for each component when displayed on the screen. In other words, the interface display data constraints must be met when displaying target components. Therefore, the generated interface description data can include the configuration information and layout information corresponding to each target component. The interface description data can be in JSON language form. The interface description data can be understood as a description of each target component displayed in the interface using a standard language. When rendering the interface, it can be rendered directly based on the interface description data.

[0055] In some embodiments, the interface display data constraints can be a JSON Schema specification specifically for large-screen visualization, which is further extended based on the existing A2UI standard protocol. The interface display data constraints uniformly define the JSON format of the large-screen page for database storage, AI generation, and rendering engine parsing, and enforce constraints on the field types, required fields, and nested structures of pages, canvases, components, data, events, and animations to achieve automatic verification, front-end and back-end unification, and AI-generated compliant large-screen data.

[0056] In some embodiments, the interface display data constraints may include: basic identifier fields, root object constraints, global canvas constraints (defining the basic parameters of the entire large-screen canvas, corresponding to the platform canvas settings panel), and complete descriptions of each visual component. Specifically, the complete descriptions of each visual component may include: basic category and identifier fields, canvas layout coordinates, layer and group control, business rendering, core data interaction configuration, nested containers, and animation binding. These interface display data constraints are equivalent to pre-defining the constraint information that each component must adhere to when displayed on the large screen. The generated interface description data thus describes the large-screen display data of each target component while satisfying its corresponding constraints.

[0057] In some embodiments, the A2UI standard protocol is an existing open-source protocol used for declarative UI descriptions of agent-driven interfaces. Its features include: its format is JSON, not executable code; the client maintains a predefined component catalog; agents can only request existing components, eliminating the risk of code execution; in addition, components are stored in a flat list, supporting streaming and progressive rendering; and the protocol defines a three-layer decoupled architecture, including a UI structure (surfaceUpdate) to describe the interface component tree, a data model (dataModelUpdate) to describe the application state, and a component catalog to define available components and constraints.

[0058] 405. Render the interface based on the interface description data to generate the rendered screen to be displayed.

[0059] In this embodiment of the application, after generating the interface description data, since the interface data description is equivalent to describing the displayed interface in a standardized language, the interface can be rendered according to the interface description data. In this way, the rendered screen to be displayed is a screen that completely conforms to the interface description data.

[0060] In this embodiment of the application, if there are erroneous languages ​​or security risks in the interface description data, it may fail to render normally or render an incorrect screen when rendering the interface description data, or even cause security impact on the display device. Therefore, before rendering the interface based on the interface description data, the interface description data can be verified for security and the verification result can be obtained. If the verification result indicates that the verification is passed, it means that the interface description data is safe and can be rendered normally. Then, the interface can be rendered based on the interface description data to generate the rendered screen to be displayed.

[0061] It should be noted that the interface description data can be validated using a JSON Schema validator. This interface description data must be a JSON structure that conforms to the constraints of the interface display data and cannot include HTML / JS executable code. In addition, the JSON Schema validator will restrict component types and parameter formats, and will directly intercept illegal fields, malicious scripts, and undefined components, thereby ensuring the security of the interface description data and the security of the interface rendering.

[0062] 406. Display the screen to be displayed.

[0063] In this embodiment of the application, after obtaining the rendered image to be displayed, the image to be displayed can be displayed on the display device's screen.

[0064] This application discloses a user interface display method, which involves receiving user input text; parsing the input text to obtain multiple entity data; determining multiple target components matching the multiple entity data from a pre-stored component library; generating interface description data based on the multiple target components and interface display data constraints, wherein the interface description data includes at least configuration and layout information corresponding to each target component, and the interface display data constraints include predefined constraint information for each component when displayed on the screen; rendering the interface based on the interface description data to generate a rendered screen to be displayed; and displaying the screen to be displayed. This user interface display method, implemented through a display device, only requires user input of natural language, which is automatically parsed and matched with the corresponding components. Then, based on the predefined constraint information for each component when displayed on the screen, standardized interface description data can be generated for interface rendering. This eliminates the need for professional developers to perform interface development, as well as manual component selection and layout, enabling fully automated large-screen development and significantly improving development efficiency. Furthermore, the predefined component constraints effectively improve the stability and accuracy of the final generated interface, eliminating code execution risks.

[0065] like Figure 5 As shown, Figure 5Another flowchart of an interface display method provided in this application embodiment, the method may include the following steps: 501. Receive user input text.

[0066] 502. Perform intent recognition on the input text to determine the user's intent for displaying the interface.

[0067] In this embodiment of the application, when parsing the user's input text, the user's intent can be analyzed first to see what content the user wants to display on the screen. Here, a trained large language model can be used to perform semantic analysis on the input text to understand the meaning of the user's input text, thereby obtaining the user's interface display intent.

[0068] 503. Based on the interface display intent, extract multiple entity data from the input text.

[0069] In this embodiment of the application, after obtaining the user's interface display intention, it is already known what content the user wants to display on the screen. Then, based on the interface display intention, multiple entity data included in the input text can be extracted.

[0070] It's important to note that the interface display intent can be understood as the extraction of the core content of the input text, expressing what the user most wants to see. The various entity data can be understood as supplementary descriptions of this interface display intent. For example, if the input text is "Please use a bar chart to display sales data for each region over the past week," then after extracting the input text, we can see that the user most wants to see sales data on the interface. Therefore, the user's interface display intent can be: display sales data. Further analysis of the input text based on this interface display intent reveals that the time dimension description of the sales data includes: the past week; the regional dimension description includes: each region; and the display format dimension description includes: bar chart. Therefore, the multiple entity data can include: the past week, each region, bar chart, and sales data.

[0071] 504. Based on multiple entity data, determine at least one component requirement corresponding to each entity data.

[0072] In this embodiment, after determining multiple entity data, to display this entity data on the interface, component support is required. Each entity data can be matched with at least one component. That is, to display an entity data, more than one component may be needed. For example, for the entity data "bar chart," to display the bar chart, not only is the "bar chart" component in the chart needed, but also text describing the content indicated by each bar, so a text box component is also needed. At the same time, to support the bar chart, a specific data source is also needed, so a data source address component is also needed, etc. In other words, for each entity data, it is necessary to determine which components are needed to display the entity data on the interface, thereby determining the at least one component requirement corresponding to each entity data.

[0073] 505. Based on at least one component requirement, identify multiple target components from the component library.

[0074] In this embodiment of the application, after determining at least one component requirement, a corresponding target component can be determined from the component library for each component requirement, thereby obtaining multiple target components.

[0075] 506. Determine the configuration information of each target component based on the component configuration constraints included in the data constraints displayed on the interface.

[0076] In this embodiment of the application, when generating interface description data, each target component can be described based on the constraints defined in the interface display data constraints, so that the components displayed on the final display interface satisfy the interface display data constraints. The interface display data constraints can at least constrain the basic configuration and interface layout of the components. Therefore, the configuration information of each target component can be standardized and constrained according to the component configuration constraints included in the interface display data constraints, thereby obtaining the configuration information corresponding to each target component.

[0077] In some embodiments, the configuration information of the target component may include: component type name, component category (including main category, sub-category, and other multi-level categories), the title displayed by the component in the layer panel, the component's unique identifier, whether the component is displayed, the component's function, and a list of component event configurations. Additionally, if the target component has sub-components, the configuration information may also include: sub-component configurations, a list of sub-components, a list of animation IDs, etc. It can be seen that for each target component, the above configuration information needs to be set and described based on the component configuration constraints included in the interface display data constraints to obtain the configuration information for each target component.

[0078] 507. Determine the layout information of each target component based on the component layout constraints included in the data constraints displayed on the interface.

[0079] In this embodiment of the application, in addition to describing the configuration information of each target component, since each target component has a certain size and needs to be displayed on the screen, it cannot be displayed arbitrarily in any position on the screen. The layout of each target component needs to be constrained in advance. Therefore, the layout information of each target component can be standardized and constrained according to the component layout constraints included in the interface display data constraints, thereby obtaining the layout information corresponding to each target component.

[0080] In some embodiments, the layout information of the target component may include: canvas width, canvas height, component width, component height, component left margin relative to the canvas, component top margin relative to the canvas, whether the component is locked (cannot be dragged / scaled in the canvas when locked), component stacking order, parent container stacking order, etc. It can be seen that for each target component, the above layout information needs to be set and described based on the component layout constraints included in the interface display data constraints to obtain the layout information of each target component.

[0081] In some embodiments, when determining the layout information corresponding to each target component, the initial display position of the target component can be determined from the component layout constraints; it can be detected whether the initial display position is occupied; if the initial display position is not occupied, the layout information of the target component can be generated based on the initial display position; if the initial display position is occupied, other free areas in the screen can be traversed in turn until a free area that satisfies the component layout constraints of the target component is detected, the free area is marked as occupied, and the layout information of the target component is generated based on the free area.

[0082] It should be noted that while the interface display data constraints define an initial display position for each component, the overall screen size is limited, and there are many components. Since not all components are typically displayed on the screen, some components may share the same initial display position. Therefore, for any target component, we first determine its initial display position and then check if that position is already occupied. It's possible that other screens have already been generated before the input text's corresponding display screen is generated, meaning some components may already be present in the display, occupying space. To avoid overlap, we check if the target component's initial display position is occupied. If not, the target component can be displayed at that position. If the initial display position is detected as occupied (including both full and partial occupancy), since the target component has a certain width and height... Therefore, the initial display position can be considered as an area. If any other component exists in this area, the initial display position of the target component is considered to be occupied. At this time, the target component cannot continue to be displayed in the initial display position. It can only be determined from other places that there is enough area to display the target component. Here, we can traverse other free areas in the screen in turn. Specifically, we can start from a corner of the interface and traverse the free areas in a certain order, such as from left to right and from top to bottom. We can determine whether each free area is enough to display the target component and satisfy the component layout constraints of the target component. If a free area that satisfies the component layout constraints of the target component is detected and is sufficient to display the target component, then we can determine to display the target component in the free area. It is equivalent to the target component occupying the free area at this time. Then we can mark the free area as occupied and generate the layout information of the target component.

[0083] In some embodiments, during the process of traversing other free areas in the screen, if no other free area sufficient to display the target component or a free area that satisfies the component layout constraints of the target component is detected, then it can be considered that the target component cannot be displayed in the current interface. At this time, a layout error message for the target component can be output. Alternatively, it can be determined whether there are other replaceable or overridable components in the current interface. Some components may not need to be displayed on the current interface or may not affect their display effect even if they are covered by other components. If they exist, then the target component can be displayed at the position of the other replaceable or overridable component, thereby obtaining the layout information of the target component.

[0084] In some embodiments, the process of generating interface description data described above can be implemented by an AI model. That is, multiple target components and interface display data constraints can be input into the AI ​​model so that the AI ​​model can generate and output interface description data corresponding to multiple target components under the specification of interface display data constraints, and then directly render the interface through the rendering engine.

[0085] 508. Based on the interface description data, construct an initial interface component tree that includes multiple target components.

[0086] In this embodiment of the application, when rendering the interface, the rendering engine can first determine which components need to be displayed on the interface, what each component looks like, where it needs to be placed, and what the relationships are between these components based on the interface description data, thereby constructing an initial interface component tree of multiple target components.

[0087] 509. Based on the configuration information corresponding to each target component, determine the display content corresponding to each target component, and update the initial interface component tree based on the display content to obtain the target interface component tree.

[0088] In this embodiment of the application, after constructing the initial interface component tree, it is also necessary to know the specific configuration information of each target component, that is, what content these target components display, where the displayed data is obtained from, how these data fields are mapped to the component chart, etc., so as to update the initial interface component tree based on these displayed contents and obtain the target interface component tree.

[0089] It should be noted that the initial interface component tree is equivalent to a framework of multiple target components. After the component framework is built, the framework is further enriched by filling it with specific content, thereby obtaining a target interface component tree with the content filled in.

[0090] 510. In response to the rendering start command, render the target interface component tree and generate the screen to be displayed.

[0091] In this embodiment of the application, after obtaining the target interface component tree, the target interface component tree can be rendered. During rendering, after receiving the rendering start command, the interface rendering operation is triggered in response to the rendering start command to render the target interface component tree, thereby generating the screen to be displayed.

[0092] In some embodiments, the interface description data generated based on the interface display data constraints may include a three-part message, wherein the surfaceUpdate message is used to describe the page component structure, which can be understood as generating the initial interface component tree; the dataModelUpdate message is used to describe the data binding, which can be understood as generating the target interface component tree; and the beginRendering message is used to trigger the rendering start instruction.

[0093] In some embodiments, such as Figure 6 The diagram shows the structure of the interface display system, which can include: a user interaction layer, an AI processing layer, a large-screen adaptation layer, and a rendering output layer. The user interaction layer can obtain the user's natural language description, i.e., the input text, and then retrieve the interface display data constraints based on the A2UI-JSONSchema generation engine, and input them into the AI ​​processing layer. This AI processing layer can be understood as an AI model. Based on the input text, the AI ​​model matches components from the component directory stored in the large-screen adaptation layer, and performs grid layout of the components in conjunction with the layout engine. Then, it generates interface description data in conjunction with the interface display data constraints, outputs it after it passes the JSON Schema validator, and finally renders and displays it through the renderer in the rendering output layer.

[0094] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described interface display method and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0095] This application provides a computer program product, including: when the computer program product is run on a computer, causing the computer to implement the above-described interface display method.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0097] It should be understood, in the several embodiments provided in this application, that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0098] In this application, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0099] In this application, memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0100] In this application, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, including permanent and non-permanent, removable and non-removable storage media. The storage medium can implement information storage by any method or technology, and the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), other types of random access memory (RAM), read-only memory (ROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by a computing device. As defined in this document, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0102] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application. The above-mentioned multiple embodiments are not necessarily multiple independent embodiments; they are divided into multiple embodiments only to highlight different technical features in different embodiments. Those skilled in the art should understand that the above-mentioned multiple embodiments can also be combined arbitrarily.

[0103] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0104] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0107] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A display device, characterized in that, include: The user input interface is configured to receive text input from the user. The controller is configured to parse the input text to obtain multiple entity data included in the input text; Based on the multiple entity data, determine multiple target components that match the multiple entity data from a pre-stored component library; Based on the multiple target components and interface display data constraints, interface description data is generated. The interface description data includes at least the configuration information and layout information corresponding to each target component. The interface display data constraints include predefined constraint information for each component when it is displayed on the screen. Based on the interface description data, the interface is rendered to generate a rendered screen to be displayed. The display is configured to display the image to be displayed.

2. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: The input text is subjected to intent recognition to determine the user's interface display intent; Based on the interface display intent, the multiple entity data are extracted from the input text.

3. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: Based on the multiple entity data, determine at least one component requirement corresponding to each entity data; Based on the requirement of at least one component, the plurality of target components are determined from the component library.

4. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: Based on the component configuration constraints included in the data constraints displayed on the interface, determine the configuration information of each target component; Based on the component layout constraints included in the interface display data constraints, the layout information of each target component is determined.

5. The display device according to claim 4, characterized in that, The controller is specifically configured as follows: The initial display position of the target component is determined from the component layout constraints; Detect whether the initial display position is occupied; If the initial display position is not occupied, the layout information of the target component is generated based on the initial display position; If the initial display position is occupied, other free areas on the screen are traversed sequentially until a free area that satisfies the component layout constraints of the target component is detected. The free area is marked as occupied, and the layout information of the target component is generated based on the free area.

6. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: Based on the interface description data, an initial interface component tree including the multiple target components is constructed; Based on the configuration information corresponding to each target component, the display content corresponding to each target component is determined, and the initial interface component tree is updated based on the display content to obtain the target interface component tree; In response to the rendering start command, the target interface component tree is rendered to generate the screen to be displayed.

7. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: The security of the interface description data is verified, and the verification result is obtained. If the verification result indicates that the verification is successful, the interface is rendered based on the interface description data to generate the screen to be displayed.

8. A method for displaying an interface, characterized in that, include: Receive text input from the user; The input text is parsed to obtain multiple entity data included in the input text; Based on the multiple entity data, determine multiple target components that match the multiple entity data from a pre-stored component library; Based on the multiple target components and interface display data constraints, interface description data is generated. The interface description data includes at least the configuration information and layout information corresponding to each target component. The interface display data constraints include predefined constraint information for each component when it is displayed on the screen. Based on the interface description data, the interface is rendered to generate a rendered screen to be displayed. Display the screen to be displayed.

9. A computer-readable storage medium, characterized in that, include: A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the interface display method as described in claim 8.

10. A computer program product, characterized in that, include: When the computer program product is run on a computer, the computer implements the interface display method of claim 8.

Images