A method and system for automatically optimizing a user interface

Abstract

The present invention relates to a method and system for automatically optimizing a user interface. The method includes: for the user interface to be optimized, obtaining dictionary data of the user interface, where the dictionary data includes: basic information and first attribute information of UI components in the user interface; generating one or more first-type containers according to the basic information of the UI components, and updating the dictionary data structure according to the first-type containers and the dictionary data to obtain an updated dictionary; according to the updated dictionary, using a projection algorithm and a large language model to obtain the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout in the order from outside to inside; according to the layout and positional relationship of the UI components at each level in the current dictionary, adjusting the position coordinates of each UI component from outside to inside to generate an optimized-layout user interface. The above method solves the technical problems of unaesthetic manual layout, low efficiency, and repetitive labor in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of interface layout optimization, and in particular, to a method and system for automatically optimizing a user interface. Background Art

[0002] With the increasing complexity of modern software development and the improvement of user requirements, the design of the user interface (UI) has become one of the key factors for the success of software products. A good UI design not only affects the user's operation experience but also directly impacts the market competitiveness of the product. In the traditional UI design process, developers usually need to manually set the size, spacing, and alignment of each component. Especially for pages with a large number of components and complex nested layouts, manual adjustment becomes particularly cumbersome. The layout of complex pages not only requires beauty and simplicity but also needs to ensure consistency across different devices, making UI design one of the time-consuming and laborious parts of the development process.

[0003] Some existing UI design tools provide partial automatic layout functions, but these functions are often relatively basic and cannot comprehensively solve problems such as component alignment, container nesting, and layout optimization in complex layouts. For example, in a multi-row and multi-column form layout, the spacing and alignment between different components are prone to inconsistency, affecting the overall beauty of the page. Especially in the face of rapid iterative development, frequent adjustment of the page layout by developers leads to a large amount of repetitive labor and is prone to introducing new design non-conformities or visual misalignment problems.

[0004] To address these pain points, there is an urgent need in the industry for an automated UI beautification solution that can automatically adjust the coordinates, spacing, and alignment of components based on the existing layout structure, reduce manual intervention, and improve development efficiency while ensuring beauty and consistency. Summary of the Invention

[0005] In view of the above-mentioned drawbacks and deficiencies of the prior art, the present invention provides a method and system for automatically optimizing a user interface, which solves the technical problems of manual layout being unaesthetic, inefficient, and involving repetitive labor in the prior art.

[0006] To achieve the above object, the main technical solutions adopted by the present invention include:

[0007] In a first aspect, an embodiment of the present invention provides a method for automatically optimizing a user interface, including:

[0008] S100. For the user interface to be optimized, obtain the dictionary data of the user interface, where the dictionary data includes: basic information and first attribute information of UI components in the user interface;

[0009] S200. Generate more than one container of the first type according to the basic information of the UI component, where the container of the first type includes: UI components with at least one layer of parent-child hierarchical relationship;

[0010] S300. Update the dictionary data structure according to the container of the first type and the dictionary data to obtain an updated dictionary;

[0011] S400. According to the updated dictionary, use the projection algorithm and the large language model to obtain the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout in the order from outside to inside;

[0012] S500. Adjust the position coordinates of each UI component from outside to inside according to the layout and positional relationship of each UI component at each level in the current dictionary to generate an optimized user interface layout.

[0013] Optionally, the basic information of the UI component includes: component name, component type, component position, component width, and / or component height;

[0014] The attribute information of the UI component includes: component color, component font size, component position attribute, and / or flex layout attribute; the flex layout attribute is the attribute identified by the projection algorithm in S400;

[0015] The data structure of the UI components in the container of the first type, the container of the second type, and the container of the third type includes: a hierarchical relationship structure.

[0016] Optionally, S400. According to the updated dictionary, use the projection algorithm and the large language model to obtain the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout in the order from outside to inside, including:

[0017] S401. According to the updated dictionary, use the projection algorithm to judge the layout of each UI component at each level in the current dictionary and the first positional relationship between the UI components in the layout in the order from outside to inside; and generate a container of the second type according to the first positional relationship between the UI components in the layout;

[0018] S402. Optimize the container of the second type using the large language model to generate a container of the third type and the second positional relationship between the UI components in the container of the third type;

[0019] S403. Use the second positional relationship as the positional relationship indicating whether the UI components belong to a nested structure, and obtain the flex layout attribute of each UI component at each level according to the identified layout of the UI components at each level and add it to the first attribute information;

[0020] The layout includes a vertical layout or a horizontal layout.

[0021] Optionally, in S401, according to the updated dictionary, use the projection algorithm to judge the layout of each UI component at each level in the current dictionary and the first positional relationship of each UI component in the layout from the outside to the inside; and generate a second type of container according to the first positional relationship of each UI component in the layout, including:

[0022] For each level of UI components, according to the beam from left to right of the projection algorithm, judge whether there is a penetration line in the vertical layout. If there is a penetration line, determine that the UI components at the current level are in a vertical layout;

[0023] If there is no penetration line in the vertical layout, then according to the beam from top to bottom of the projection algorithm, determine whether there is a penetration line in the horizontal layout. If there is, determine that the UI components at the current level are in a horizontal layout;

[0024] Meanwhile, generate a second type of container according to the penetration line in the vertical layout / penetration line in the horizontal layout.

[0025] Optionally, in S402, use the large language model to optimize the second type of container to generate a third type of container and the second positional relationship of each UI component in the third type of container, including:

[0026] For each level of UI components, input the basic information and first attribute information of the UI components at the current level and the basic information and first attribute information of the UI components in the next level of the current level into the prompt words of the large language model to obtain the third type of container output by the large language model;

[0027] The prompt words are preset based on the semantic information or semantic structure of the UI components at the current level and the next level.

[0028] Optionally, according to the layout and positional relationship of each UI component at each level in the current dictionary, adjust the position coordinates of each UI component from the outside to the inside, including:

[0029] If the layout is a vertical layout, for each level of UI components, the UI components at the current level are the parent container, and the UI components in the next level are the child components / child containers;

[0030] Adjust the width of all child components within the current level to the width of the parent container minus the side margins on both sides. The side margins are the preset side margins of the current parent container;

[0031] Determine whether there are immutable components marked with immutable component marks within the parent container. If so, modify the interface XY coordinates of the immutable components; the height of the immutable components remains unchanged;

[0032] For other variable components in the parent container, the remaining parent container height is divided according to the original height numerical ratio; the original height numerical ratio is the height numerical ratio of the corresponding UI component in the dictionary data;

[0033] Arrange other variable components in the container in order, and adjust the interface XY coordinates of each variable component;

[0034] and / or,

[0035] If the layout is horizontal, then for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers;

[0036] Divide the width of all subcomponents in the parent container of the current level by the width of the parent container according to the original width numerical ratio; the original width numerical ratio is the width numerical ratio of the corresponding UI component in the dictionary data;

[0037] Determine whether there is an immutable component marked as an immutable component in the parent container, and if so, modify the interface XY coordinates of the immutable component; the width of the immutable component remains unchanged;

[0038] For other variable components in the parent container, set the height of other variable components to the maximum value of the heights of all immutable components;

[0039] Arrange other variable components in the parent container in order, and adjust the interface XY coordinates of each variable component.

[0040] Optionally, for each level, the parent container of the current level and the subcomponents of the next level in the parent container are adjusted in linkage;

[0041] When both the parent container and the child components are variable components, the internal coordinates of the child components are adjusted adaptively as the coordinates of the parent container are adjusted.

[0042] Optionally, according to the layout and position relationship of each level of UI components in the current dictionary, the position coordinates of each UI component are adjusted from outside to inside, further comprising:

[0043] According to the flex layout properties of each UI component at each level, the alignment of each UI component is obtained with the help of the trained GBDT model, and the alignment mode of each UI component at the current level is adjusted;

[0044] The alignment modes include: left alignment, right alignment, and center alignment.

[0045] Optionally, for an equal-division layout in a vertical layout or an equal-division layout in a horizontal layout, i.e., middle alignment, the number of rows and columns of child components in the parent container of the current layer is identified through a projection algorithm, and the XY coordinates, width, and height of each child component are recalculated; the adjusted component positions can maintain consistent layout logic;

[0046] Adaptive line spacing :

[0047]

[0048] Container coordinates , :

[0049] ;

[0050] ;

[0051] ;

[0052] Wherein: is the total height of the parent container of the current layer, is the number of vertical segmentation gaps, and len represents finding the number, represents the interval of vertical distribution, d h represents the height of the internal components of the container, represents the number of vertical components;

[0053] n represents the number of all components in the container, % represents the remainder flag, and dw represents the width of the internal components of the container;

[0054] is the total width of the parent container of the current layer; 12 is the margin on both sides inside the parent container of the current layer; 48 is the fixed interval between components; is the number of horizontal segmentation gaps;

[0055] , is the coordinate of the parent container of the current layer.

[0056] In a second aspect, an embodiment of the present invention provides a system for automatically optimizing a user interface, including:

[0057] A dictionary data acquisition unit for acquiring dictionary data of the user interface to be optimized, where the dictionary data includes: basic information and first attribute information of UI components in the user interface;

[0058] An optimization layout unit for generating one or more first-type containers according to the basic information of the UI interface, where the first-type containers include: UI components having at least one layer of parent-child hierarchical relationship;

[0059] Updating the dictionary data structure according to the first-type container and the dictionary data to obtain an updated dictionary;

[0060] According to the updated dictionary, use the projection algorithm and the large language model to obtain the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout from the outside to the inside;

[0061] According to the layout and positional relationship of each UI component at each level in the current dictionary, adjust the position coordinates of each UI component from the outside to the inside to generate an optimized layout of the user interface.

[0062] A method and system for automatically optimizing a user interface according to the present invention can automatically adjust the coordinates, spacing, and alignment of components according to the existing layout structure, reduce manual intervention, and improve development efficiency while ensuring aesthetics and consistency.

[0063] In the embodiments of the present invention, by parsing dictionary data, automatically identifying component relationships and layout patterns, and intelligently optimizing the overall structure of the page, it can not only significantly reduce the workload of developers, but also improve the accuracy and standardization of UI design, thereby enhancing the user experience and product quality. Brief Description of the Drawings

[0064] Figure 1 It is a schematic flowchart of a method for automatically optimizing a user interface provided by an embodiment of the present invention;

[0065] Figure 2 It is a schematic flowchart of a method for automatically optimizing a user interface provided by another embodiment of the present invention;

[0066] Figure 3 It is a schematic diagram of determining the layout by the projection algorithm of an embodiment of the present invention;

[0067] Figure 4 It is a schematic diagram of the large language model of an embodiment of the present invention identifying and generating a third type of container;

[0068] Figure 5 It is a schematic diagram of the processes of step S400 and step S500 of an embodiment of the present invention;

[0069] Figure 6 For Figure 4 It is a schematic diagram of the semantic information prompt words using the large language model in;

[0070] Figure 7 It is a schematic diagram of the nested structure of UI components shown in an embodiment of the present invention. Detailed Embodiments

[0071] In order to better explain the present invention and facilitate understanding, the present invention will be described in detail below with reference to the accompanying drawings through specific embodiments.

[0072] A method and system for automatically optimizing a user interface, by parsing and analyzing the input dictionary data, automatically identifying the layout relationships of various components in the UI page, performing alignment, merging, adjustment, and layout optimization of the containers and internal components, and finally generating a beautified page with clear structure, strong visual consistency, and compliance with design specifications.

[0073] In the user's use, the system of this embodiment can be understood as a plug-in, which calls the plug-in to input dictionary data: representing the detailed information of each component in the UI page, including the type of the component (such as button, text box, container, etc.), the component name (the unique ID identifying the component), the coordinate information of the component (x, y positions, width, height), and the style attributes of the component (such as color, font, background, etc.). The plug-in outputs the beautified page dictionary data structure: after the automatic optimization of the embodiment of the present invention, the output dictionary data will include the optimized component layout, the adjusted component coordinates, and the container structure.

[0074] To better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to be able to convey the scope of the present invention completely to those skilled in the art.

[0075] As Figure 1 and Figure 2 shown, the embodiment of the present invention provides a method for automatically optimizing a user interface. The steps of this method may include:

[0076] S100. For the user interface to be optimized, obtain the dictionary data of the user interface, where the dictionary data includes: the basic information and the first attribute information of the UI components in the user interface;

[0077] For example, the basic information of the UI components in this step includes: component name, component type, component position, component width, and component height;

[0078] Correspondingly, the attribute information of the UI components obtained in this step S100 includes: component color, component font size, and component position attribute.

[0079] S200. Generate one or more first-type containers according to the basic information of the UI components, where the first-type containers include: UI components having at least one layer of parent-child hierarchical relationship.

[0080] In this step, the information of the flattened structure of the UI components is parsed to identify the parent-child hierarchical relationship, and the first type of container is obtained. The first type of container is a UI component with a nested structure. When the first type of container has a two-layer structure, the parent container is the outer-layer UI component, and the child components of the parent container are the UI components inside the first type of container.

[0081] S300. Update the dictionary data structure according to the first type of container and the dictionary data to obtain an updated dictionary. The updated dictionary has a hierarchical relationship structure.

[0082] S400. According to the updated dictionary, use the projection algorithm and the large language model to obtain the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout in the order from outside to inside.

[0083] After step S400, the attribute information of the above UI components may include: flex layout attributes.

[0084] The flex layout attribute in this embodiment may be a horizontal layout attribute or a vertical layout attribute, that is, an attribute of arranging in the horizontal direction or the vertical direction.

[0085] S500. According to the layout and positional relationship of each UI component at each level in the current dictionary, adjust the position coordinates of each UI component from outside to inside to generate an optimized-layout user interface.

[0086] The method in this embodiment can achieve page margin normalization. By adjusting the page margins and component spacing, it ensures that the page layout is neat and consistent and meets the design standards.

[0087] At the same time, it can provide component spacing and alignment optimization. By automatically identifying the arrangement method of components (such as horizontal alignment, vertical alignment, etc.), it optimizes the spacing between different components, making the page layout more tidy and beautiful.

[0088] In practical applications, it supports flexible container flex layout and automatically adjusts the arrangement method and alignment mode of the components inside the container.

[0089] To better understand the above steps S400 and S500, the following provides a detailed description of these two steps.

[0090] S401. According to the updated dictionary, use the projection algorithm to judge the layout of each UI component at each level in the current dictionary and the first positional relationship between the UI components in the layout in the order from outside to inside; and generate a second type of container according to the first positional relationship between the UI components in the layout.

[0091] For example, for each level of UI components, according to the left-to-right light beam of the projection algorithm, determine whether there is a penetration line in the vertical layout. If there is a penetration line, determine that the UI components at the current level are in a vertical layout;

[0092] If there is no penetration line in the vertical layout, then according to the top-to-bottom light beam of the projection algorithm, determine whether there is a penetration line in the horizontal layout. If there is, determine that the UI components at the current level are in a horizontal layout; meanwhile, generate a second type of container according to the penetration line in the vertical layout / penetration line in the horizontal layout.

[0093] S402. Optimize the second type of container using a large language model to generate a third type of container and the second positional relationship of each UI component in the third type of container.

[0094] It can be understood that for each level of UI components, the basic information and first attribute information of the UI components at the current level, and the basic information and first attribute information of the UI components in the next level of the current level are input into the prompt words of the large language model to obtain the third type of container output by the large language model;

[0095] The prompt words are preset based on the semantic information or semantic structure of the UI components in the current level and the next level.

[0096] S403. Use the second positional relationship as the positional relationship of each UI component, and obtain the flex layout attributes of the UI components at each level according to the recognized layout of the UI components at each level and add them to the first attribute information;

[0097] The layout includes a vertical layout or a horizontal layout. The positional relationship in S403 can be the positional relationship of whether it belongs to a nested structure.

[0098] The above-mentioned step S500 may include the following sub-steps:

[0099] S501A. If the layout is a vertical layout, for each level of UI components, the UI components at the current level are the parent container, and the UI components at the next level are the child components / child containers;

[0100] S502A. Adjust the width of all child components within the current level to the width of the parent container minus the side margins on both sides, and the side margins are the side margins of the current parent container preset in advance;

[0101] S503A. Determine whether there are immutable components marked with an immutable component mark within the parent container. If so, modify the interface XY coordinates of the immutable components; the height of the immutable components remains unchanged;

[0102] S504A: for other variable components in the parent container, divide the remaining parent container height according to the original height numerical ratio; the original height numerical ratio is the height numerical ratio of the corresponding UI component in the dictionary data;

[0103] S505A, arrange the other variable components in the container in order, and adjust the interface XY coordinates of each variable component;

[0104] In addition, if the layout is horizontal,

[0105] S501B, for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers;

[0106] S502B, dividing the width of all child components in the parent container of the current level by the width of the parent container according to the original width numerical ratio; the original width numerical ratio is the width numerical ratio of the corresponding UI component in the dictionary data;

[0107] S503B, determining whether there is an immutable component marked as an immutable component in the parent container, and if so, modifying the interface XY coordinates of the immutable component; the width of the immutable component remains unchanged;

[0108] S504B, for other variable components in the parent container, set the height of other variable components to the maximum value among the heights of all immutable components;

[0109] S505B, arranging other variable components in the parent container in order, and adjusting the interface XY coordinates of each variable component.

[0110] S506 , according to the flex layout attributes of each UI component at each level, the alignment mode of each UI component is obtained with the help of the trained GBDT model, and the alignment mode of each UI component at the current level is adjusted.

[0111] The alignment methods in this embodiment may include: left alignment, right alignment, and center alignment.

[0112] In particular, in this embodiment, for each level, the parent container of the current level and the subcomponents of the next level in the parent container are adjusted in linkage;

[0113] That is to say, when both the parent container and the child component are variable components, the internal coordinates of the child component are adjusted adaptively as the coordinates of the parent container are adjusted.

[0114] Of course, the data structure of the UI components in the first type container, the second type container and the third type container includes: a hierarchical relationship structure. Each type of container is a UI component with a nested structure.

[0115] For a better understanding of nested structures or hierarchical structures, see Figure 7As shown, if the current level is the outermost layer, then a1 is the parent container, and the child components are a2 and b2; if the current level is a2, then the parent container is a2, and the child component is a3;

[0116] From the outside to the inside as described above, each container is traversed in sequence until the final innermost components a4 and b3.

[0117] To better understand the method of the above-mentioned Embodiment 1, with the help of Figures 3 to 7 , the content of the above method is described in detail as follows:

[0118] The first step is to parse the user interface to be optimized, that is, UI page parsing, to obtain the dictionary data of the original dictionary.

[0119] The UI page beautification system (that is, the system for automatically optimizing the user interface in this embodiment) first receives the input dictionary. This dictionary data is a flat structure describing the page layout and component information of the user interface to be optimized, and can be a dictionary obtained by identifying and acquiring through existing methods.

[0120] The dictionary data at this time may include: basic information such as the name, type, position (such as coordinates x, y), size (such as width, height) of the component, and attribute information of the UI component.

[0121] It can be understood that the system extracts the relevant attributes of each UI component by parsing this dictionary data to obtain the attribute information of the UI component.

[0122] The second step is to generate one or more containers of the first type according to the basic information of the UI component. The container of the first type includes: UI components with at least one layer of parent-child hierarchical relationship.

[0123] That is to say, based on the coordinate information (such as x, y) and hierarchical structure of the components, the parent-child relationship between the components is inferred. The parent container can contain child components. For example, a form container on a page may contain multiple input boxes, buttons, etc.

[0124] The third step is to update the dictionary data structure according to the container of the first type and the dictionary data to obtain the updated dictionary.

[0125] The parent-child relationship is reflected in the nested dictionary data structure. Each parent component contains a "children" key (hierarchical structure) representing the set of its internal child components, which exists in the form of a sub-dictionary. This nested structure facilitates subsequent identification of the association relationship and layout structure between components, ensuring accuracy when dealing with complex nested layouts.

[0126] In the fourth step, based on the updated dictionary, the projection algorithm and the large language model are used to obtain the layout of each UI component at each level in the current dictionary and the positional relationship of each UI component in the layout from the outside to the inside.

[0127] For example, using the projection algorithm, by analyzing the XY coordinates of the UI component and its width and height, the arrangement of the subcomponents can be determined. The subcomponents may be arranged in rows (horizontally) or columns (vertically). Figure 3 As shown, in this embodiment, components are preferentially distributed vertically, and thus vertical layout detection and horizontal layout detection are recursively performed on each layer of the dictionary data in turn, and the presence of penetration lines indicates that the layout mode exists.

[0128] The penetration line of the projection algorithm divides the parent container into child containers. If a child container contains multiple components, it is necessary to construct an encapsulated container for it. The size of the container can be understood as a rectangle covering all internal components.

[0129] The containers and subcomponents of this embodiment are adjusted in a two-way linkage during layout and adjustment. In order to better implement this linkage adjustment, the system will mark the type of the container to distinguish between variable and immutable components. Variable components refer to containers that allow users to adjust their external dimensions and internal layouts, such as retractable card-type containers such as content areas and navigation menus. Immutable components are containers with fixed sizes or positions, such as form controls. The subcomponents inside immutable components are adjustable horizontally, and the fixed component spacing and fixed component width are not adjustable vertically. The basis for this marking is mainly the function and usage scenario of the container. For some containers that need to maintain consistency and invariance (such as navigation bars and fixed toolbars), they are usually marked as immutable. Some content display containers and interactive containers often allow users to flexibly adjust the size and layout, so they are marked as variable.

[0130] The fifth step is to adjust the position coordinates of each UI component from the outside to the inside according to the layout and position relationship of each level of UI components in the current dictionary, and generate a user interface with optimized layout.

[0131] In this embodiment, the linkage adjustment between each container and the internal components is bidirectional. When the coordinates and size of the internal components are adjusted, the size of the container may need to change accordingly; similarly, if the size of the container changes, the layout and coordinates of the internal components also need to be recalculated to maintain the beauty and logical consistency of the overall design. For example, assuming that the components in a container are adjusted from a two-column layout to a three-column layout, the width of the container may increase. Conversely, when the container is reduced, the internal components may need to be rearranged to adapt to the new size.

[0132] The fourth and fifth steps above can be explained as follows:

[0133] M01. If the layout is a vertical layout, then for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers;

[0134] Adjust the width of all child components in the current hierarchy to the width of the parent container minus the side margins, which are the preset side margins of the current parent container;

[0135] Determine whether there is an immutable component marked as an immutable component in the parent container, and if so, modify the interface XY coordinates of the immutable component; the width of the immutable component remains unchanged;

[0136] For other variable components in the parent container, the remaining parent container height is divided according to the original height numerical ratio; the original height numerical ratio is the height numerical ratio of the corresponding UI component in the dictionary data;

[0137] Arrange other variable components in the container in order, and adjust the interface XY coordinates of each variable component;

[0138] M02. If the layout is a vertical layout, then for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers;

[0139] Divide the width of all subcomponents in the parent container of the current level by the width of the parent container according to the original width numerical ratio; the original width numerical ratio is the width numerical ratio of the corresponding UI component in the dictionary data;

[0140] Determine whether there is an immutable component marked as an immutable component in the parent container, and if so, modify the interface XY coordinates of the immutable component; the height of the immutable component remains unchanged;

[0141] For other variable components in the parent container, set the height of other variable components to the maximum value of the heights of all immutable components;

[0142] Arrange other variable components in the parent container in order, and adjust the interface XY coordinates of each variable component.

[0143] M03. According to the flex layout attributes of each UI component at each level, the alignment of each UI component is obtained with the help of the trained GBDT model, and the alignment mode of each UI component at the current level is adjusted;

[0144] The alignment modes include: left alignment, right alignment, and center alignment.

[0145] In this embodiment, for each level, the parent container of the current level and the subcomponents of the next level in the parent container are adjusted in linkage;

[0146] When both the parent container and the child component are variable components, the internal coordinates of the child component are adaptively adjusted as the coordinates of the parent container are adjusted.

[0147] In this embodiment, only the internal layout of the immutable component is adjusted. The internal coordinates of the variable component are adaptively scaled as the coordinates of the external container are adjusted. However, the method for adjusting the internal component layout can be applied to any component, not limited to the immutable component.

[0148] In the fourth step of the above steps, while generating the layout using the projection algorithm, a second type of container is also generated, and the large language model is used for optimization to generate a third type of container, which is specifically described as follows:

[0149] See Figure 4 , in the original UI page, some components are a whole in function, such as the title text and the input box behind it. However, in terms of data format, the two are not in a single container. The system needs to generate containers based on the relative position, size, and function of the components. This process is completed by the large language model.

[0150] Specifically: The dictionary file of the parent container is put into the preset prompt words, and then input into the large language model. The large language model outputs a list of components that can be encapsulated into a container. In this embodiment, the setting of the prompt words is shown in Figure 6 .

[0151] Generally, the projection algorithm tends to overgenerate containers. For example, each line under the query box will be regarded as a separate container. To avoid this problem, the large language model analyzes the function, structure, and semantic understanding of the UI components to merge the containers.

[0152] For example, for components with the same layout characteristics in multiple consecutive lines, they can be merged into a parent container to reduce unnecessary separation. The large language model determines whether they can be merged based on the function, relative position, and spacing of the components. By merging adjacent containers, a third type of container is obtained, improving the overall compactness and coordination of the UI page (i.e., the UI interface).

[0153] The system identifies the flex layout attributes of the UI components based on the arrangement of the internal components in the container (the first type of container, the second type of container, and the third type of container), such as horizontal or vertical arrangement.

[0154] For different alignment methods in the flex layout (such as left alignment, center alignment, right alignment), the system will automatically adjust the alignment mode of each component to ensure a more reasonable layout within the container.

[0155] In this embodiment, the arrangement direction is determined according to the projection algorithm, and the pre-trained machine learning model GBDT is used to identify the alignment method.

[0156] For the left-aligned and right-aligned layout methods. In this embodiment, the width and height of the components are set to the average values of the width and height of the components in the original data, and the component spacing is a fixed value. Of course, the above methods are not limited, and adjustments can be made according to the application scenario. For example, in the scenario where beautification needs to be close to the user's intention, the component spacing can be changed to be adaptive. In this embodiment, no special calculation is required for left alignment and right alignment, and the UI components can be arranged in sequence.

[0157] For the equal distribution layout (i.e., the centered alignment method), this embodiment is illustrated as follows: The number of rows and columns of the internal components of the container is identified through a projection algorithm, and the XY coordinates, width, and height of each internal component are recalculated. The adjusted positions of the components can maintain a consistent layout logic. The specific calculation logic in this embodiment takes the query box as an example:

[0158] Adaptive line spacing :

[0159]

[0160] Container coordinates , :

[0161] ;

[0162] ;

[0163] ;

[0164] Where: is the total height of the current layer's parent container, is the number of vertical segmentation gaps, len represents finding the number, represents the interval of vertical distribution, d h represents the height of the internal components of the container, represents the number of vertical components;

[0165] n represents the total number of all components in the container, % represents the remainder flag, d w represents the width of the internal components of the container;

[0166] is the total width of the current layer's parent container; 12 is the margin on both sides inside the current layer's parent container; 48 is the fixed interval between components; is the number of horizontal segmentation gaps; represents the interval of horizontal distribution, represents the number of horizontal components; , is the coordinate of the current layer's parent container.

[0167] In a specific scenario, the title and the input box will fill the entire container in a fixed ratio, specifically the average of the ratios of the two in the UI page. The component spacing and the side margins can also be adjusted accordingly according to the specific scenario.

[0168] In another alternative implementation, the training process of the GBDT model mentioned above can be described as follows:

[0169] First, collect the component data in the historical UI pages and use a large language model to generate alignment tags.

[0170] Second, manually verify and remove some incorrect tags.

[0171] Third, use dynamic time warping to align the input sequence (i.e., all components in the current layer of the historical UI interface); output the alignment of these components. Thus, the GBDT model is trained.

[0172] In practice, the number of components in different containers is different, and the lengths of all components in each current layer of the above-mentioned historical UI pages as the input sequence are inconsistent. Therefore, dynamic time warping is performed to normalize the sequence length (i.e., normalize the number of components, similar to retaining some key components).

[0173] The system of this embodiment combines a projection algorithm and a large language model, and can more accurately identify the arrangement of components and the container relationship in a complex layout. Compared with existing UI design tools, the present invention can not only automatically optimize the component spacing and alignment, but also reduce layout chaos and improve development efficiency by intelligently generating containers and merging components with similar layouts. At the same time, the linkage adjustment mechanism of the present invention ensures that the adjustments of the container and the components affect each other, further enhancing the overall aesthetics and consistency of the page.

[0174] In addition, the embodiment of the present invention also provides a system for automatically optimizing a user interface, including:

[0175] A dictionary data acquisition unit, configured to acquire the dictionary data of the user interface to be optimized for the user interface, where the dictionary data includes: the basic information and the first attribute information of the UI components in the user interface;

[0176] An optimized layout unit, configured to generate one or more first-type containers according to the basic information of the UI interface, where the first-type containers include: UI components having at least one layer of parent-child hierarchical relationship;

[0177] Update the dictionary data structure according to the first-type containers and the dictionary data to obtain an updated dictionary;

[0178] According to the updated dictionary, the layout of each UI component at each level in the current dictionary and the positional relationship between the UI components in the layout are obtained in the order from the outside to the inside by using the projection algorithm and the large language model.

[0179] According to the layout and positional relationship of each UI component at each level in the current dictionary, the position coordinates of each UI component are adjusted from the outside to the inside to generate an optimized-layout user interface.

[0180] The system in this embodiment corresponds to the above method embodiment. For details, refer to the above method embodiment, and the functions of each unit of the system will not be elaborated here. For the method embodiment, for simplicity of description, it is expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0181] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions.

[0183] It should be noted that the words "a" or "an" before a component do not exclude the existence of multiple such components. The present invention can be implemented by means of hardware including several different components and by means of a properly programmed computer. Among the several devices listed, several of these devices can be embodied by the same hardware. The use of the words first, second, third, etc. is only for the convenience of expression and does not represent any order. These words can be understood as part of the component name.

[0184] In addition, it should be noted that in the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0185] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications after learning the basic creative concepts. Therefore, the technical solutions should be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0186] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the technical solutions of the present invention and their equivalent technologies, the present invention should also include these modifications and variations.

Claims

1. A method for automatically optimizing a user interface, characterized in that: include: S100: for a user interface to be optimized, obtaining dictionary data of the user interface, the dictionary data including: basic information and first attribute information of UI components in the user interface; S200, generating one or more first-type containers according to basic information of the UI component, wherein the first-type containers include: UI components having at least one parent-child hierarchical relationship; S300, updating the dictionary data structure according to the first type container and the dictionary data to obtain an updated dictionary; S400, according to the updated dictionary, using the projection algorithm and the large language model to obtain the layout of each UI component at each level in the current dictionary and the position relationship of each UI component in the layout from the outside to the inside; specifically, S400, including: S401, according to the updated dictionary, using a projection algorithm to determine the layout of each UI component at each level in the current dictionary and the first position relationship of each UI component in the layout from the outside to the inside; and generating a second type of container according to the first position relationship of each UI component in the layout; Specifically, for each level of UI components, determine whether there is a penetration line of the vertical layout according to the beam from left to right of the projection algorithm. If there is a penetration line, determine that the UI components of the current level are in the vertical layout; if there is no penetration line of the vertical layout, determine whether there is a penetration line of the horizontal layout according to the beam from top to bottom of the projection algorithm. If so, determine that the UI components of the current level are in the horizontal layout; at the same time, generate a second type of container according to the penetration line of the vertical layout / the penetration line of the horizontal layout; S402: Optimize the second type container by using a large language model to generate a third type container and a second position relationship of each UI component in the third type container; Specifically, for each level of UI components, the basic information and first attribute information of the UI components of the current level, and the basic information and first attribute information of the UI components in the next level of the current level are input into the prompt words of the large language model to obtain a third type of container output by the large language model; the prompt words are pre-set based on the semantic information or semantic structure of the UI components in the current level and the next level; S403, taking the second positional relationship as the positional relationship between the UI components as to whether they belong to a nested structure, and according to the layout of the UI components at each level identified, obtaining the flex layout attributes of the UI components at each level and adding them to the first attribute information; the layout includes a vertical layout or a horizontal layout; S500: According to the layout and position relationship of each level of UI components in the current dictionary, adjust the position coordinates of each UI component from the outside to the inside to generate a user interface with optimized layout.

2. The method according to claim 1, characterized in that The basic information of a UI component includes: component name, component type, component position, component width, and / or component height; The attribute information of the UI component includes: component color, component font size, component position attribute, and / or flex layout attribute; the flex layout attribute is the attribute identified by the projection algorithm in S400; The data structure of the UI components in the first type container, the second type container and the third type container includes: a hierarchical relationship structure.

3. The method according to claim 1, characterized in that According to the layout and position relationship of each level of UI components in the current dictionary, adjust the position coordinates of each UI component from the outside to the inside, including: If the layout is a vertical layout, for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers; Adjust the width of all child components in the current hierarchy to the width of the parent container minus the side margins, which are the preset side margins of the current parent container; Determine whether there is an immutable component marked as an immutable component in the parent container, and if so, modify the interface XY coordinates of the immutable component; the height of the immutable component remains unchanged; For other variable components in the parent container, the remaining parent container height is divided according to the original height numerical ratio; the original height numerical ratio is the height numerical ratio of the corresponding UI component in the dictionary data; Arrange other variable components in the container in order, and adjust the interface XY coordinates of each variable component; and / or, If the layout is horizontal, then for each level of UI components, the UI components of the current level are parent containers, and the UI components of the next level are child components / child containers; Divide the width of all subcomponents in the parent container of the current level by the width of the parent container according to the original width numerical ratio; the original width numerical ratio is the width numerical ratio of the corresponding UI component in the dictionary data; Determine whether there is an immutable component marked as an immutable component in the parent container, and if so, modify the interface XY coordinates of the immutable component; the width of the immutable component remains unchanged; For other variable components in the parent container, set the height of other variable components to the maximum value of the heights of all immutable components; Arrange other variable components in the parent container in order, and adjust the interface XY coordinates of each variable component.

4. The method according to claim 3, characterized in that For each level, the parent container of the current level and the subcomponents of the next level in the parent container are adjusted in linkage; When both the parent container and the child components are variable components, the internal coordinates of the child components are adjusted adaptively as the coordinates of the parent container are adjusted.

5. The method according to claim 3, characterized in that: According to the layout and position relationship of each level of UI components in the current dictionary, adjust the position coordinates of each UI component from the outside to the inside, including: According to the flex layout properties of each UI component at each level, the alignment of each UI component is obtained with the help of the trained GBDT model, and the alignment mode of each UI component at the current level is adjusted; The alignment modes include: left alignment, right alignment, and center alignment.

6. The method according to claim 5, characterized in that For an equal-division layout in a vertical layout or an equal-division layout in a horizontal layout (i.e., center alignment), the number of rows and columns of child components in the parent container of the current layer is identified through a projection algorithm, and the XY coordinates, width, and height of each child component are recalculated; the adjusted component positions can maintain consistent layout logic; Adaptive line spacing : ； Container coordinates , : ； ； ； in: is the total height of the parent container of the current layer. is the number of vertical segmentation gaps, len indicates the number of gaps to be found, represents the interval of vertical distribution, d h Indicates the height of the components inside the container. Indicates the number of vertical components; n represents the number of all components in the container, % represents the remainder sign, and dw represents the width of the components inside the container; is the total width of the parent container of the current layer; 12 is the margins on both sides of the parent container of the current layer; 48 is the fixed spacing between components; is the number of lateral separation gaps; , The coordinates of the parent container of the current layer.

7. A system for automatically optimizing a user interface, characterized in that: include: A dictionary data acquisition unit, used for acquiring dictionary data of a user interface to be optimized, wherein the dictionary data includes basic information and first attribute information of a UI component in the user interface; The layout optimization unit is used to generate one or more first-type containers according to the basic information of the UI interface, wherein the first-type containers include: UI components having at least one parent-child hierarchical relationship; According to the first type of container and the dictionary data, updating the dictionary data structure to obtain an updated dictionary; According to the updated dictionary, the layout of each UI component at each level in the current dictionary and the positional relationship of each UI component in the layout are obtained from the outside to the inside using the projection algorithm and the large language model; specifically, according to the updated dictionary, the layout of each UI component at each level in the current dictionary and the first positional relationship of each UI component in the layout are determined from the outside to the inside using the projection algorithm; and, a second type of container is generated according to the first positional relationship of each UI component in the layout; specifically, for each level of UI components, according to the left-to-right beam of the projection algorithm, it is determined whether there is a penetration line of the vertical layout, and if there is a penetration line, it is determined that the UI components at the current level are in the vertical layout; if there is no penetration line of the vertical layout, according to the top-to-bottom beam of the projection algorithm, it is determined whether there is a penetration line of the horizontal layout, and if so, it is determined that the UI components at the current level are in the horizontal layout; at the same time, a second type of container is generated according to the penetration line of the vertical layout / the penetration line of the horizontal layout; The second type container is optimized by using a large language model to generate a third type container and a second position relationship of each UI component in the third type container; specifically, for each level of UI components, basic information and first attribute information of the UI components of the current level, and basic information and first attribute information of the UI components in the level below the current level are input into the prompt words of the large language model to obtain the third type container output by the large language model; the prompt words are pre-set based on the semantic information or semantic structure of the UI components in the current level and the level below; Taking the second positional relationship as the positional relationship between the UI components as to whether they belong to a nested structure, according to the layout of the UI components at each level identified, obtaining the flex layout attribute of the UI components at each level and adding it to the first attribute information; the layout includes a vertical layout or a horizontal layout; According to the layout and position relationship of each level of UI components in the current dictionary, the position coordinates of each UI component are adjusted from the outside to the inside to generate a user interface with optimized layout.

Images