Graphical interface generation method and system, electronic equipment and storage medium

Abstract

The invention provides a graphical interface generation method and system, electronic equipment and a storage medium, and the method comprises the steps: defining a graphical interface description grammar rule, employing a unified grammar rule to represent the attribute and layout of an interface control, and representing an interface hierarchical relationship through a tree structure; receiving natural language description input by a user, extracting interface description information through natural language processing, and generating a corresponding structured interface description file; analyzing the interface description file, and rendering the graphical interface based on the control attributes and the layout information; obtaining an interface image through the AI agent, performing visual analysis on the interface image, and adjusting control attributes and layout in the interface description file or a regenerated interface description file according to a visual analysis result; and updating the graphical interface in real time based on the adjusted interface description file or the regenerated interface description file. Through the scheme, the interface development efficiency can be improved, and the accuracy and reliability of the interface can be guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of application development, and particularly relates to a method, a system, an electronic device and a storage medium for generating a graphical interface. Background Art

[0002] HMI (Human-Machine Interface) refers to an interface system designed and implemented for interaction between humans and machines, which restores the designed interface layout and visual effects in a programmatic way. In the process of HMI development, the method of restoring the interface layout by program usually goes through three processes: code implementation, manual arrangement and combination of user interface design tools, and generation of a description file in a fixed format.

[0003] Currently, Unreal Motion Graphics (UMG) is a powerful user interface design tool in Unreal Engine. It allows developers to create complex user interfaces by drag-and-drop. The core of UMG is controls, which are prefabricated functional elements that make up the interface, such as buttons, check boxes, sliders, progress bars, etc. Developers can add and configure various UI elements in the UMG editor. Although UMG provides a very high degree of freedom in interface editing, for some complex interfaces or interfaces with frequent requirement changes, using the method of dragging and dropping controls and editing and configuring the layout will result in low interface generation efficiency due to a large number of repetitive and time-consuming operations. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, a system, an electronic device and a storage medium for generating a graphical interface, which are used to solve the problem of low efficiency in generating the current graphical interface.

[0005] In the first aspect of the embodiments of the present invention, a method for generating a graphical interface is provided, including: Defining a syntax rule for graphical interface description, representing the attributes and layout of interface controls with a unified syntax rule, and representing the interface hierarchy with a tree structure; Receiving a natural language description input by a user, extracting interface description information through natural language processing, and generating a corresponding structured interface description file; Parsing the interface description file to obtain control attribute and layout information, and rendering the graphical interface based on the control attribute and layout information; Obtaining the rendered interface image through an AI agent, performing visual analysis on the interface image, and adjusting the interface description file or regenerating the interface description file according to the visual analysis result; Updating the graphical interface in real time based on the adjusted interface description file or the regenerated interface description file.

[0006] In the second aspect of the embodiments of the present invention, a graphical interface generation system is provided, including: A syntax definition module, configured to define the syntax rules for graphical interface description, represent the attributes and layouts of interface controls using unified syntax rules, and represent the interface hierarchy through a tree structure; An interface description module, configured to receive the natural language description input by the user, extract interface description information through natural language processing, and generate a corresponding structured interface description file; An interface rendering module, configured to parse the interface description file to obtain control attribute and layout information, and render the graphical interface based on the control attribute and layout information; An interface optimization module, configured to obtain the rendered interface image through an AI agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis result; An interface update module, configured to update the graphical interface in real time based on the adjusted interface description file or the regenerated interface description file.

[0007] In the third aspect of the embodiments of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect of the embodiments of the present invention are implemented.

[0008] In the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method provided in the first aspect of the embodiments of the present invention are implemented.

[0009] In the embodiments of the present invention, by receiving the natural language description input by the user, generating a corresponding interface description file based on natural language processing, rendering the graphical interface, and checking and optimizing the interface through an AI agent, hot update of the graphical interface is realized. Thereby, not only can the update efficiency of the graphical interface be improved and the update operation be simplified, but also the accuracy and reliability of the generated interface can be ensured. Description of the Drawings

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1A schematic flowchart of a graphical interface generation method provided by an embodiment of the present invention; Figure 2 A schematic structural diagram of a graphical interface generation system provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0012] To make the objectives, features, and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the embodiments described below are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0013] It should be understood that the term "including" and other similar expressions in the specification or claims of the present invention and the above-mentioned drawings mean non-exclusive inclusion. For example, a process, method, system, or device including a series of steps or units is not limited to the listed steps or units. In addition, "first" and "second" are used to distinguish different objects and are not used to describe a specific order.

[0014] Please refer to Figure 1 , a schematic flowchart of a graphical interface generation method provided by an embodiment of the present invention, including: S101. Define the syntax rules for describing the graphical interface, use a unified syntax rule to represent the attributes and layout of the interface controls, and represent the interface hierarchy relationship through a tree structure; The graphical interface is a computer user interface displayed in a graphical manner, which is an important part of front-end development. Based on the graphical interface, it is convenient for users to send operation instructions to the computer and display the operation results.

[0015] The syntax rules refer to the structure and rules in interface design, including interface layout, interaction design, and information hierarchy, etc., such as the organization and arrangement of controls. Using a unified syntax rule to describe the control attributes and layout, control attributes such as size, position, color, etc., and layout methods such as linear layout and relative layout can facilitate the unified design of the interface.

[0016] Among them, the interface is used as the root node, and the control or control layout is used as the child node, and the interface hierarchy relationship is represented through a tree structure. Each node contains multiple child nodes, and the entire interface can form a hierarchical structure.

[0017] In some embodiments, a complex interface can be split into multiple sub-interfaces or sub-modules, and each sub-interface or sub-module can be managed and reused separately.

[0018] Among them, resources such as interface fonts, pictures, materials, textures, and reusable interface description files are used as separate configuration files for interface resource configuration.

[0019] S102. Receive the natural language description input by the user, extract the interface description information through natural language processing, and generate a corresponding structured interface description file; The natural language description refers to the user's natural language description of the interface. The user can input the natural language in ways such as text and voice. For the text description input by the user, relevant content of the interface description can be obtained through natural language processing (NLP), such as the color and size of the control, the control layout method, etc.; for the voice input by the user, the corresponding natural language can be obtained through voice recognition, and then relevant content of the interface description can be obtained through natural language processing.

[0020] Extract key information through natural language processing, such as the control type, layout method, etc., and automatically generate a structured interface description file based on unified syntax rules.

[0021] Among them, the interface description file is associated with the resource configuration file based on the file name. The resource configuration file at least includes fonts, pictures, textures, and reusable interface description files.

[0022] For a complex interface, the interface is split into multiple sub-interfaces or sub-modules, and natural language descriptions are input separately and multiple interface description files are generated, and are configured in the resource configuration file and associated by name. After that, the name can be used as the natural language description to generate the interface of the reusable sub-module.

[0023] S103. Parse the interface description file to obtain the control attributes and layout information, and render the graphical interface based on the control attributes and layout information; Parse the intermediate format interface description file through the program to obtain control attributes, layout information, etc., and apply them to the actual program to render the interface effect.

[0024] The parsing process includes control type, layout, attribute conversion, etc., and can also include restoring the used configuration resources to corresponding attributes and adding them to the parent control in a way specific to interface reuse on the target platform.

[0025] S104. Obtain the rendered interface image through the AI agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis result; The described AI agent perceives changes in the environment or receives data inputs, makes judgments and decisions based on the knowledge and algorithms it has learned, and then executes actions to achieve a predetermined goal. It can be software, hardware, or a system. In this embodiment, an AI agent is introduced to use its visual ability to check the generated interface effect and feedback the inspection result to the agent that generates the AI, which can facilitate improvement and optimization.

[0026] It can be understood that visual analysis can achieve visual analysis of the interface through methods such as image recognition and object detection.

[0027] Among them, at least check the spacing, alignment, and hierarchical relationship between controls; at least check whether the colors, fonts, pictures, and textures of the controls conform to the preset style.

[0028] In layout inspection, it can be checked whether the spacing, alignment, hierarchical relationship, etc. between controls conform to the design specifications. For example, it can be identified whether the spacing between a button and a text box is too large or too small, and whether there is an alignment deviation; in appearance inspection, it can be checked whether the appearance, color, font, pictures, etc. of the controls conform to the design style. For example, it can be identified whether the color of the button is consistent with the design draft and whether the font size conforms to the specification.

[0029] S105. Update the graphical interface in real time based on the adjusted interface description file or the newly generated interface description file.

[0030] Based on the interface description file that can be adjusted by the AI agent or the newly generated interface description file, the interface is updated in real time. In this embodiment, by introducing an AI agent, hot update of the interface can be performed, and the interface content can be dynamically updated without restarting the application.

[0031] Among them, the AI agent dynamically manages the current interface and adds, deletes, or modifies sub-interfaces on the basis of the current interface. The sub-interfaces are different modules obtained after the interface is split. The AI agent supports dynamic management of interface modules and allows users to add, delete, or modify each sub-interface module during runtime.

[0032] In this embodiment, by introducing natural language processing during the development process to generate a corresponding structured interface description file for interface rendering, the development process of the graphical interface can be simplified and the development efficiency can be improved; by using the AI agent to check the generated interface, the accuracy and reliability of the interface can be guaranteed. At the same time, supporting hot update of the interface can significantly enhance the flexibility and real-time performance of interface generation and meet the personalized needs of users.

[0033] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0034] Figure 2 FIG. 4 is a schematic structural diagram of a graphical interface generation system provided by an embodiment of the present invention. The system includes: A syntax definition module 210, configured to define syntax rules for graphical interface description, represent the attributes and layouts of interface controls using a unified syntax rule, and represent the interface hierarchy relationship through a tree structure; Among them, representing the interface hierarchy relationship through a tree structure includes: Regarding the interface as the root node and regarding the control or control layout as the child node.

[0035] An interface description module 220, configured to receive a natural language description input by a user, extract interface description information through natural language processing, and generate a corresponding structured interface description file; Optionally, the interface description module 220 includes: A configuration association unit, configured to associate the interface description file with a resource configuration file based on the file name. The resource configuration file at least includes fonts, pictures, textures, and reusable interface description files.

[0036] An interface rendering module 230, configured to parse the interface description file to obtain control attribute and layout information, and render the graphical interface based on the control attribute and layout information; An interface optimization module 240, configured to obtain the rendered interface image through an AI agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis result; Among them, performing visual analysis on the interface image includes: At least checking the spacing, alignment, and hierarchy relationship between controls; At least checking whether the colors, fonts, pictures, and textures of the controls conform to a preset style.

[0037] An interface update module 250, configured to update the graphical interface in real time based on the adjusted interface description file or the regenerated interface description file.

[0038] Optionally, the interface update module 250 includes: A hot update unit, configured to dynamically manage the current interface through an AI agent, and add, delete, or modify sub-interfaces on the basis of the current interface. The sub-interfaces are different modules obtained after the interface is split.

[0039] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and modules described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0040] Figure 3 FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device is used for generating a graphical interface. As Figure 3 shown, the electronic device 3 in this embodiment includes: a memory 310, a processor 320, and a system bus 330. The memory 310 includes a runnable program 3101 stored thereon. Those skilled in the art can understand that Figure 3 the shown structure of the electronic device does not limit the electronic device, and it may include more or fewer components than shown, or combine some components, or have different component arrangements.

[0041] The following Figure 3 will specifically introduce each component of the electronic device: The memory 310 can be used to store software programs and modules. The processor 320 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 310. The memory 310 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the electronic device (such as cached data). In addition, the memory 310 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0042] The runnable program 3101 including the interface generation method is stored on the memory 310. The runnable program 3101 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 310 and executed by the processor 320 to implement automatic generation of a graphical interface, etc. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 3101 in the electronic device 3. For example, the runnable program 3101 can be divided into functional modules such as a syntax definition module, an interface description module, an interface rendering module, an interface optimization module, and an interface update module.

[0043] The processor 320 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 310, and by calling the data stored in the memory 310, it executes various functions of the electronic device and processes data, thereby monitoring the overall state of the electronic device. Optionally, the processor 320 may include one or more processing units; preferably, the processor 320 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 320 either.

[0044] The system bus 330 is used to connect the internal functional components of the computer, and can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, CAN bus, etc. The instructions of the processor 320 are transmitted to the memory 310 through the bus, and the memory 310 feeds back data to the processor 320. The system bus 330 is responsible for the data and instruction interaction between the processor 320 and the memory 310. Of course, the system bus 330 can also be connected to other devices, such as network interfaces, display devices, etc.

[0045] In the embodiments of the present invention, the executable programs executed by the processor 320 included in the electronic device include: Define the syntax rules for graphical interface description, represent the attributes and layouts of interface controls using unified syntax rules, and represent the interface hierarchical relationship through a tree structure; Receive the natural language description input by the user, extract the interface description information through natural language processing, and generate a corresponding structured interface description file; Parse the interface description file to obtain the control attributes and layout information, and render the graphical interface based on the control attributes and layout information; Obtain the rendered interface image through the AI agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis results; Based on the adjusted interface description file or the regenerated interface description file, update the graphical interface in real time.

[0046] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0047] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0048] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A graphical interface generation method, characterized in that: include: Define the grammatical rules for describing graphical interfaces, use unified grammatical rules to represent the properties and layout of interface controls, and use a tree structure to represent the interface hierarchy; Receive the natural language description input by the user, extract the interface description information through natural language processing, and generate the corresponding structured interface description file; Parsing the interface description file to obtain control attributes and layout information, and rendering the graphical interface based on the control attributes and layout information; Obtain the rendered interface image through the AI ​​agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis results; The graphical interface is updated in real time based on the adjusted interface description file or the regenerated interface description file.

2. The method according to claim 1, characterized in that The method of expressing the interface hierarchy relationship through a tree structure includes: Take the interface as the root node and the control or control layout as the child node.

3. The method according to claim 1, characterized in that The extracting the interface description information by natural language processing and generating a corresponding structured interface description file also includes: The interface description file is associated with a resource configuration file based on the file name, wherein the resource configuration file at least includes fonts, pictures, textures and a reusable interface description file.

4. The method according to claim 1, characterized in that: The visual analysis of the interface image comprises: At least check the spacing, alignment, and hierarchical relationships between controls; At least check that the colors, fonts, images, and textures of the controls match the preset style.

5. The method according to claim 1, characterized in that The real-time updating of the graphical interface based on the adjusted interface description file or the regenerated interface description file also includes: The current interface is dynamically managed through the AI ​​agent, and sub-interfaces are added, deleted or modified based on the current interface. The sub-interfaces are different modules obtained after the interface is split.

6. A graphical interface generation system, characterized in that: include: The grammar definition module is used to define the grammar rules for describing the graphical interface, use unified grammar rules to represent the properties and layout of the interface controls, and represent the interface hierarchy through a tree structure; The interface description module is used to receive the natural language description input by the user, extract the interface description information through natural language processing, and generate a corresponding structured interface description file; An interface rendering module, used to parse the interface description file, obtain control properties and layout information, and render the graphical interface based on the control properties and layout information; The interface optimization module is used to obtain the rendered interface image through the AI ​​agent, perform visual analysis on the interface image, and adjust the interface description file or regenerate the interface description file according to the visual analysis results; The interface update module is used to update the graphical interface in real time based on the adjusted interface description file or the regenerated interface description file.

7. The system according to claim 6, characterized in that The method of expressing the interface hierarchy relationship through a tree structure includes: Take the interface as the root node and the control or control layout as the child node.

8. The system according to claim 6, characterized in that The interface description module includes: The configuration association unit is used to associate the interface description file with the resource configuration file based on the file name, and the resource configuration file at least includes fonts, pictures, textures and reusable interface description files.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a graphical interface generation method as described in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the steps of a graphical interface generation method as claimed in any one of claims 1 to 5 are implemented.

Images