Component library-based user interface generation method, device, medium and product

Abstract

The application relates to the technical field of game development, and discloses a user interface generation method and device based on a component library, a medium and a product. An original design draft is acquired; visual analysis is performed on the original design draft to determine element attribute information of interface elements; target interface components matched with the interface elements are determined according to the element attribute information of the interface elements; and each target interface component is laid out according to a layout rule corresponding to a layout type to generate a target user interface. Through visual analysis of the original design draft, each interface element is automatically identified, the labeling time of the interface elements is reduced, and the interface generation efficiency is improved. By matching the element attribute information of the interface elements with component information of each interface component in the component library, the interface components can be automatically searched, and the time cost is reduced. By matching the target interface components in the component library, the interface components in the component library can be used when designing the user interface, and the consistency of interface design in the same project is ensured.

Description

Technical Field

[0001] This application relates to the field of game development technology, specifically to user interface generation methods, devices, media, and products based on component libraries. Background Technology

[0002] In the current game development workflow, UI (User Interface) creation primarily relies on designers manually building components using general design tools. Designers must retrieve and drag components one by one from a large component library, set parameters to achieve layout and arrangement, and finally transform the interaction draft into a final version that conforms to visual specifications. However, the component library of large game projects typically contains hundreds of component variations. When building functional interfaces (such as inventory, shop, email, settings, pop-ups, etc.), designers must manually retrieve, drag, and configure components one by one, which is not only time-consuming and laborious but also prone to omissions and errors. In addition, the layout rules in related technologies (such as automatic layout plugins) cannot cover the multiple layout paradigms unique to game UIs (such as full-screen interfaces, side drawers, multi-level pop-ups, etc.), meaning they cannot automatically generate a final version that meets visual requirements according to game UI specifications. Summary of the Invention

[0003] This application provides a method, device, medium, and product for generating user interfaces based on a component library, in order to solve the problems of high labor costs, low efficiency, and error-prone operation caused by manual component retrieval, dragging and dropping, and configuration in related technologies.

[0004] Firstly, this application provides a user interface generation method based on a component library, including: Obtain the original design drafts; Visual analysis is performed on the original design draft to determine the element attribute information of each interface element in the original design draft; For any of the interface elements, the element attribute information of the interface element is matched with the component information of each interface component in the preset component library to determine the target interface component that matches the interface element. Obtain the layout type, and lay out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface.

[0005] The component library-based user interface generation method provided in this embodiment, through visual analysis of the original design draft, can convert the original design draft image into a set of interface element data. It automatically identifies each interface element and its attribute information without manual annotation, reducing annotation time, improving interface generation efficiency, and avoiding annotation deviations caused by differences in designer experience. By automatically comparing and matching the element attribute information of interface elements with the component information of each interface component in the component library, interface components can be automatically found, reducing time costs and lowering the threshold for interface design. Furthermore, by matching the target interface component corresponding to the interface element in the preset component library, it ensures that when multiple designers design interfaces based on different design drafts, the interface components in the component library are used, guaranteeing consistency in interface design within the same project. By obtaining the layout type and laying out each target interface component according to the layout rules corresponding to the layout type, the effect of automatically generating the target user interface is achieved, eliminating the need for manual dragging and alignment, and improving interface production efficiency.

[0006] Secondly, this application provides a user interface generation apparatus based on a component library, comprising: The design draft module is used to obtain the original design draft.

[0007] The visual analysis module is used to perform visual analysis on the original design draft and determine the element attribute information of each interface element in the original design draft.

[0008] The component matching module is used to match any interface element with the component information of each interface component in the preset component library based on the element attribute information of the interface element, and to determine the target interface component that matches the interface element.

[0009] The layout module is used to obtain the layout type, and lay out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface.

[0010] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the component library-based user interface generation method described in the first aspect or any corresponding embodiment.

[0011] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the component library-based user interface generation method described in the first aspect or any corresponding embodiment thereof.

[0012] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the component library-based user interface generation method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application; Figure 2 This is a schematic diagram of a first process of a component library-based user interface generation method according to an embodiment of this application; Figure 3 This is a schematic diagram of a second process for a component library-based user interface generation method according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating a scenario selection according to an embodiment of this application; Figure 5 This is a schematic diagram of a layout type selection according to an embodiment of this application; Figure 6 This is a schematic diagram illustrating a layout rule modification according to an embodiment of this application; Figure 7 This is a schematic diagram illustrating the modification of scenario configuration information according to an embodiment of this application; Figure 8 This is a schematic diagram of a target user interface according to an embodiment of this application; Figure 9 This is a structural block diagram of a component library-based user interface generation apparatus according to an embodiment of this application; Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0017] As one optional application scenario in the embodiments of this application, such as Figure 1 As shown, application 101 is installed in terminal device 110, and user 130 can interact with application 101 through terminal device 110 and / or access device of terminal device 110.

[0018] For example, application 101 can be any application that provides user interface generation services. For instance, application 101 could be a UI design application, etc. Figure 1 In the application scenario shown, if application 101 is active, the terminal device 110 can display the interface 102 of application 101. The interface 102 may include various pages that application 101 can provide, such as interactive pages, reference upload pages, settings pages, etc.

[0019] In some embodiments, terminal device 110 is communicatively connected to server 120 to provide services to application 101. Terminal device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to desktop computers, laptop computers, multimedia tablets, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, terminal device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0020] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this application.

[0021] The embodiments of this application will be described below with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations, one or more elements may be omitted or replaced, and one or more other elements may also be present; no limitations are imposed on the embodiments of this application. Furthermore, the embodiments are primarily described below with reference to terminal device 110. It should be understood that the actions described relative to terminal device 110 can be performed by application 101 on terminal device 110, or can be performed by application 101 in conjunction with its server (e.g., server 120).

[0022] This application provides a component library-based user interface generation method. By visually analyzing the original design draft, it automatically identifies each interface element and its attribute information without manual annotation, reducing the annotation time for interface elements. By obtaining the layout type, it lays out each target interface component according to the layout rules corresponding to the layout type, achieving the effect of automatically generating the target user interface without manual dragging and alignment, thus improving the efficiency of interface production.

[0023] According to an embodiment of this application, a user interface generation method based on a component library is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] This embodiment provides a user interface generation method based on a component library, which can be used in the aforementioned terminal devices, such as desktop computers and laptop computers. Figure 2 This is a flowchart of a component library-based user interface generation method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the original design draft.

[0025] When generating the user interface, the original design draft uploaded by the user can be obtained first. This original design draft can be a reference image for the interface design, such as an interactive wireframe, a scanned copy of a hand-drawn sketch, or a screenshot of the interface of another application or game. In this embodiment, an upload interface can be provided for users to upload their original design drafts. After obtaining the uploaded original design draft, consistency processing can be performed on it, for example, scaling the image to a preset standard resolution (e.g., 1920×1080) to facilitate subsequent visual analysis.

[0026] Step S202: Perform visual analysis on the original design draft to determine the element attribute information of each interface element in the original design draft.

[0027] After obtaining the original design draft, visual analysis can be performed to automatically identify the interface elements within it. Interface elements are the basic units that constitute the user interface in the original design draft. These elements may include buttons, icons, text labels, list items, grid cells, and other interface components with independent functions or visual significance. After identifying each interface element, its attribute information can be further determined.

[0028] The element attribute information is a collection of attribute information for each interface element. For example, element attribute information may include the element's position, size, and function. The function information refers to the function corresponding to that interface element. For instance, for a "Confirm Button" interface element, its function information could be "Button," "Confirm Button," etc. After determining each interface element and its corresponding attribute information, it can be stored in a corresponding data structure. For example, it can be stored as an array or list; it can also be stored using a tree-like data structure to represent the hierarchical relationship between interface elements.

[0029] By visually analyzing the original design draft, the original design draft image can be converted into a set of interface element data. It can automatically identify each interface element and its attribute information without manual annotation, reducing the annotation time of interface elements, improving the efficiency of interface generation, and avoiding annotation deviations caused by differences in the designer's personal experience.

[0030] Step S203: For any interface element, match the element attribute information of the interface element with the component information of each interface component in the preset component library to determine the target interface component that matches the interface element.

[0031] After identifying each interface element, for any given interface element, a target interface component matching that element can be determined from a pre-built component library. This pre-built component library is a collection of pre-built resources containing multiple reusable interface components. Each interface component stored in the library has its corresponding component information. This information may include functional information or a description of the component's purpose, such as "button," "input box," or "navigation menu." Component information may also include structural information such as size, style, and position. Specifically, for any given interface element, its element attribute information can be matched with the corresponding component information of various interface components in the component library to determine the target interface component corresponding to that element.

[0032] By automatically comparing and matching the element attribute information of interface elements with the component information of each interface component in the component library, interface components can be automatically found, reducing time costs and lowering the threshold for interface design. In addition, by matching the target interface component corresponding to the interface element in the preset component library, it can be ensured that when multiple designers design interfaces or based on different design drafts, the interface components in the component library are used, ensuring the consistency of interface design in the same project.

[0033] Step S204: Obtain the layout type, and lay out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface.

[0034] After determining the target interface components corresponding to each interface element, the layout type can be obtained. The layout type is the category information of the overall presentation form of the user interface; for example, the layout type can include full-screen layout, drawer layout, pop-up layout, etc. In this embodiment, a layout type selection interface can be provided to the user to obtain the selected layout type. Alternatively, the layout type corresponding to the original design draft can be automatically determined based on the original design draft. For example, if the interface elements cover the entire canvas and occupy a large area, it can be determined as a full-screen layout; if the interface elements are concentrated in the center of the canvas or occupy a relatively small area, it can be determined as a pop-up layout, etc. Both methods can be used in parallel. For example, after automatic recognition, a default layout type option can be provided, while simultaneously providing a layout type selection interface for the user to modify and confirm.

[0035] After determining the layout type, the target interface components are laid out according to the layout rules corresponding to that layout type. These layout rules can include a set of arrangement logic associated with the layout type, as well as the positions of each interface element in the original design draft. For example, if a target interface component has corresponding arrangement logic under this layout type, it can be laid out according to that logic. If a target interface component does not have corresponding arrangement logic under this layout type, the target interface component corresponding to that element can be laid out according to the position information in the element's attribute information. After all target interface components have been laid out, the target user interface is generated, which is the final version conforming to the game UI specifications.

[0036] The component library-based user interface generation method provided in this embodiment, through visual analysis of the original design draft, can convert the original design draft image into a set of interface element data. It automatically identifies each interface element and its attribute information without manual annotation, reducing annotation time, improving interface generation efficiency, and avoiding annotation deviations caused by differences in designer experience. By automatically comparing and matching the element attribute information of interface elements with the component information of each interface component in the component library, interface components can be automatically found, reducing time costs and lowering the threshold for interface design. Furthermore, by matching the target interface component corresponding to the interface element in the preset component library, it ensures that when multiple designers design interfaces based on different design drafts, the interface components in the component library are used, guaranteeing consistency in interface design within the same project. By obtaining the layout type and laying out each target interface component according to the layout rules corresponding to the layout type, the effect of automatically generating the target user interface is achieved, eliminating the need for manual dragging and alignment, and improving interface production efficiency.

[0037] This embodiment provides a user interface generation method based on a component library, which can be used in the aforementioned terminal devices, such as desktop computers and laptop computers. Figure 3 This is a flowchart of a component library-based user interface generation method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the original design draft.

[0038] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0039] In some alternative implementations, the component library-based user interface generation method further includes step a1 before step S302 "visual analysis of the original design".

[0040] Step a1 involves scanning the component library to obtain component information and identifier information for each UI component. The identifier information is used to call the target UI component when generating the target UI.

[0041] Before visual analysis of the original design draft, a scan operation can be performed on the preset component library to obtain the component information and identification information of each interface component in the component library. In this embodiment, all interface components in the component library can be traversed to read the component information and identification information corresponding to each interface component. The component information represents the function and structure of the interface component, such as the component function information and component structure information. The component function information represents the function of the interface component, such as "button" or "input box"; the component structure information represents the structure of the interface component, such as size and position. The identification information is a unique identifier for each interface component, such as the unique node ID of the interface component in Figma or the storage path of the interface component. This identification information is used to call the target interface component when generating the target user interface later.

[0042] To further improve scanning efficiency during the scanning operation, an incremental scanning strategy can be adopted. This involves performing a full scan on the first run and caching the scan results along with the version identifiers of the component library. Subsequent runs first check if the current version identifier of the component library matches the cached version identifier. If they match, the cached data is loaded directly, skipping the scanning process. If they don't match, only the changed UI components are incrementally updated, while unchanged UI components in the cache continue to be reused. Furthermore, intelligent filtering measures can be implemented during the scanning operation to exclude non-UI component noise nodes from the component library.

[0043] Step S302: Perform visual analysis on the original design draft to determine the element attribute information of each interface element in the original design draft.

[0044] Specifically, step S302, "perform visual analysis on the original design draft to determine the element attribute information of each interface element in the original design draft", includes steps S3021 to S3023.

[0045] Step S3021: Perform image segmentation on the original design draft, extract the element regions of each interface element, and use the geometric features of the element regions as the geometric features of the interface elements.

[0046] In determining the element attribute information of each interface element, image segmentation can be performed on the original design draft first. This involves dividing the UI elements in the original design draft into different regions to determine the element regions of each interface element. For example, connectivity analysis can be used to detect the connectivity between pixels in the original design draft, and connected pixel regions can be considered as one element region. Alternatively, edge detection can be used to divide different regions by identifying the boundary lines and contour lines of the interface elements, with the closed region enclosed by the boundary lines considered as one element region. This element region is the region corresponding to the interface element in the original design draft. After determining the element regions of each interface element, each element region has a corresponding boundary. This boundary can be described using a rectangular bounding box (e.g., top-left corner x-coordinate, top-left corner y-coordinate, width w, height h), and the geometric features of this element region are used as the geometric features of the interface element. These geometric features can include spatial attributes such as the size, aspect ratio, and area of ​​the element region (which is also the interface element).

[0047] Image segmentation technology is used to divide the original design draft from a single image into a set of element regions, which automatically separates and locates each interface element in the original design draft. At the same time, geometric features are extracted, and spatial attributes such as position, size, and proportion of each interface element can be recorded. Subsequently, component structure matching can be performed based on geometric features to ensure that the size of the matched target interface component is relatively close to that of the interface element, thereby improving the accuracy of matching.

[0048] Step S3022: Perform function identification on the element areas of each interface element to determine the element function information of each interface element.

[0049] After determining the geometric features of the interface elements, the function of the element areas can be identified to determine the element function information. As mentioned earlier, element function information refers to the function corresponding to the interface element, such as "button," "input box," and "confirm." In this embodiment, the function of the interface element can be determined by function identification for the element areas corresponding to the interface element, thereby determining the element function information corresponding to the interface element.

[0050] By using function recognition technology, each interface element is assigned a function label to determine its function within the interface. Subsequently, component functions can be matched based on the functions of each interface element, improving matching accuracy.

[0051] In some optional implementations, step S3022, "to identify the function of the element area of ​​each interface element and determine the element function information of each interface element", includes steps b1 and b2.

[0052] Step b1: Perform text recognition on the element areas of each interface element and extract the text content within each element area.

[0053] Step b2: For any interface element, determine the element function information of the interface element based on the text content and geometric features of the interface element.

[0054] When performing functional recognition on the element regions corresponding to interface elements, text recognition can be performed on these regions. For example, OCR (Optical Character Recognition) technology can be used to extract the text content within the element region, or a deep learning model can be used to recognize the text within the element region. If the element region contains text, the text recognition operation can output a string containing all the text content within that element region; if the region does not contain text, the text recognition operation returns an empty result or an indication that there is no text.

[0055] After determining the text content within each element area, the element function information of the interface element can be determined based on the text content and geometric features of the interface element. In practical applications, the text content of an interface element usually reflects its element function information. For example, if the text content of an interface element is identified as "Confirm," "OK," or similar text, the element function of that interface element is usually a "Confirm Button." In this case, the element function information can be determined through the text content. Further confirmation can be made based on the geometric features of the interface element. For example, if the interface element is small and located at the bottom of the screen, its element function information can be further confirmed as a "Confirm Button." In this embodiment, even if the text content cannot be directly mapped to a specific function, the geometric features themselves can provide clues for function determination. For example, a horizontal strip area located at the top of the screen with a width close to the screen width, combined with the title text it may contain, can be determined to be a "title bar" or "top navigation bar," etc.

[0056] By combining textual content and geometric features to determine function, the accuracy of function recognition can be improved.

[0057] Step S3023: Generate element attribute information based on geometric features and element functional information.

[0058] After determining the geometric features and functional information of the interface elements, the corresponding element attribute information can be generated. This element attribute information includes the geometric features and functional information of the interface elements, and may also include other auxiliary information, such as the hierarchical relationship of the interface elements in the original design (e.g., whether they contain other interface elements, or are contained by other interface elements). For example, the hierarchical relationship can be stored using a DOM tree, which will not be elaborated upon here.

[0059] The geometric features generated by image segmentation and the element function information generated by function recognition are integrated into unified element attribute information. A two-dimensional description information with morphological and semantic dimensions is established for each interface element. Subsequently, matching is performed simultaneously through the two dimensions to improve matching accuracy.

[0060] Step S303: For any interface element, match the element attribute information of the interface element with the component information of each interface component in the preset component library to determine the target interface component that matches the interface element.

[0061] Please see details Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0062] In some optional implementations, the component information includes component function information and component structure information. Step S303, "matching the element attribute information of the interface element with the component information of each interface component in the preset component library to determine the target interface component that matches the interface element", includes steps c1 to c3.

[0063] Step c1: Based on the element function information of the interface elements, traverse the component function information of each interface component in the preset component library and perform semantic matching to obtain the semantic matching result.

[0064] Step c2: Perform structural matching based on the geometric features of the interface elements and the component structure information of each interface component to obtain the structural matching result.

[0065] Step c3: Based on the semantic matching results and the structural matching results, determine the target interface component from multiple interface components.

[0066] As mentioned earlier, component information refers to data representing the function and structure of interface components. This can include component function information and component structure information. In determining the target interface component that matches an interface element, the element function information corresponding to the interface element can be compared and matched with the component function information of each interface component. Specifically, for any given interface element, the component function information of all interface components in the component library can be traversed, and semantic similarity calculations can be performed with the element function information of that interface element. For example, string-based similarity calculation methods or keyword-based matching methods can be used, which will not be elaborated further here. After completing the similarity calculation, the semantic matching result of each interface component relative to the interface element can be obtained. In this embodiment, the semantic matching result can be semantic similarity.

[0067] In game interface design, UI elements that perform the same or similar functions often have similar structural features. For example, while the "confirm button" in different games may have significant differences in visual style (e.g., different colors, fonts), their structural features are usually similar (e.g., they are all rectangular, have a specific aspect ratio, and are located at the bottom of the interface or the bottom area of ​​a pop-up). In this case, for any interface element, its geometric features can be structurally matched with the component structure information of various interface components. Similar to semantic matching, the component structure information of all interface components can be traversed in the component library, and structural similarity calculations can be performed with the geometric features of the interface element. For example, the geometric features of the interface element can be converted into feature vectors, and the component structure information of the interface component can be converted into feature vectors of the same dimension. Then, the distance or directional similarity between the two feature vectors can be calculated. After the calculation is completed, the structural matching results of each interface component relative to the interface element can be obtained. In this embodiment, the structural matching result can be structural similarity (e.g., directional similarity between two feature vectors).

[0068] After determining the semantic and structural matching results for each interface component, the matching degree of each component relative to the current interface element can be obtained based on these results. For example, the matching degree can be calculated by weighted summation of the semantic and structural matching results for each component, or by using a weighted average, etc., which will not be elaborated further here. After determining the matching degree, the interface component with the highest matching degree can be selected as the target interface component corresponding to the current interface element.

[0069] By comprehensively evaluating semantic and structural matching results, the degree of matching between interface elements and interface components is assessed from two dimensions: functional semantics and structural form. This approach offers higher matching accuracy compared to single-dimensional matching.

[0070] In some optional implementations, step c3, “determining the target interface component from multiple interface components based on semantic matching results and structural matching results,” includes steps c31 to c34.

[0071] Step c31: Determine candidate interface components based on the semantic matching results and structural matching results of each interface component.

[0072] Step c32: Obtain the matching degree of the candidate interface components. The matching degree is calculated based on the semantic matching results and structural matching results of the candidate interface components.

[0073] Step c33: When the matching degree is greater than the preset first threshold, determine the target interface component based on the candidate interface components.

[0074] Step c34: When the matching degree is less than the first threshold and greater than the preset second threshold, output a list of pending interface components to the user, and in response to the selection operation of the pending interface components, use the selected pending interface components as the target interface components.

[0075] In this embodiment, candidate interface components can be determined first based on the semantic and structural matching results of each interface component. For example, the three interface components with the highest semantic matching results can be used as initial screening interface components, and the interface component with the highest structural matching result can be selected as a candidate interface component from the initial screening interface components. After determining the candidate interface components, the matching degree between the candidate interface components and the interface elements can be calculated according to step c3. When the matching degree is greater than a preset first threshold, it can be determined that the matching degree between the candidate interface component and the interface element is high, and the candidate interface component can be applied to the corresponding interface element during the interface generation process. At this time, the target interface component can be determined based on the candidate interface component, for example, the candidate interface component can be directly used as the target interface component. In this embodiment, the preset first threshold can be 0.8.

[0076] When the matching degree is less than the first threshold but greater than the preset second threshold, it indicates that there is a certain difference between the currently matched candidate interface components and the interface elements. Directly using the candidate interface components as the target interface components may result in unexpected outcomes. In this case, a list of pending interface components can be output to the user. This list can include multiple pending interface components with high matching degrees. After generating the list, the user selects which pending interface component to use as the target interface component. It can be understood that this list of pending interface components can include the aforementioned candidate interface components. The second threshold can be 0.

[0077] If the second threshold is not 0 and the matching degree is less than the second threshold, it means that there are significant differences between the existing pending interface components and the interface elements. In this case, recommended interface components that are similar to the interface elements can be determined based on the large model, and the user can choose whether to use the recommended interface components as the target interface components.

[0078] Step S304: Obtain the layout type, and lay out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface.

[0079] In some alternative implementations, step S304, "obtaining the layout type," includes either step d1 or step d2.

[0080] Step d1: Identify the layout type of the original design draft based on the page layout analysis results corresponding to the visual analysis.

[0081] Step d2, in response to the selection operation for the layout type, determines the layout type.

[0082] In determining the layout type, the layout type of the original design draft can be automatically identified based on the page layout analysis results. For example, analysis can be performed based on information such as the boundary coordinates, coverage area, distribution position relative to the canvas, and relative positional relationship between each interface element. If the interface elements are spread throughout the entire canvas and the coverage area accounts for a high proportion of the total canvas area, it can be determined as a "full-screen layout"; if the interface elements are concentrated in the central area of ​​the canvas and the coverage area accounts for a small proportion, it can be determined as a "pop-up layout"; if the interface elements are mainly distributed in the left or right area of ​​the canvas and the overall area proportion is between that of a full-screen layout and a pop-up layout, it can be determined as a "drawer layout", etc.

[0083] In some cases, the automatically identified layout type may not match the requirements. In such cases, the automatically identified layout type can be set as the default option, and a list of layout types can be displayed to the user, allowing the user to manually select the layout type of the original design.

[0084] In some optional implementations, step S304, "laying out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface," includes steps e1 and e2.

[0085] Step e1: If the target interface component has a corresponding rule in the layout rules, then the target interface component is laid out according to the layout rules.

[0086] Step e2: If the target interface component does not have a corresponding rule in the layout rules, then the target interface component is laid out according to the geometric characteristics of the interface elements corresponding to the target interface component.

[0087] As mentioned earlier, layout rules are a set of arrangement logic associated with layout types. However, layout rules may not cover the arrangement logic of all target interface components in the original design. When a target interface component (such as a button or input box) has a corresponding rule (i.e., arrangement logic) in the layout rules, the target interface component needs to be laid out according to the arrangement logic corresponding to that target interface component in the layout rules. When a target interface component does not have a corresponding rule in the layout rules, the target interface component needs to be laid out according to the geometric characteristics of the interface element corresponding to the target interface component (such as the position and size of the interface element in the original design).

[0088] Within the same interface, some standard components (i.e., target interface components with corresponding rules in the layout rules) can be arranged according to the layout rules through sub-step e1, while some custom components (i.e., target interface components without corresponding rules in the layout rules) can be arranged according to geometric features through sub-step e2. Together, they constitute a complete layout scheme. In scenarios where they are used in combination, sub-step e1 can be executed first, and then sub-step e2 can be executed for the remaining custom components.

[0089] Through the layout mechanism of rule-based layout and geometric feature layout, adaptive layout for different types of target interface components is achieved. That is, target interface components with corresponding rules ensure positional consistency and visual standardization through preset arrangement rules, while target interface components without corresponding rules maintain their correspondence with the original design draft through geometric feature mapping.

[0090] In some alternative implementations, the component library-based user interface generation method further includes step f1.

[0091] Step f1: In response to the modification operation of the layout rules, determine the modified layout rules, and lay out each target interface component according to the modified layout rules. The modification operation includes the component spacing modification operation and / or the component arrangement modification operation.

[0092] In some cases, the target user interface generated according to the layout rules may not meet the user's needs, such as when the component spacing is too narrow. The user can modify the layout rules, for example, by manually inputting changes to component spacing and / or component arrangement. This user-initiated modification operation can include changes to component spacing, component arrangement, etc. Modification operations can be implemented through various interactive methods. For example, users can directly input specific values ​​for spacing parameters through numeric input boxes, continuously adjust the spacing using slider controls, or select arrangement direction or alignment from multiple preset options through drop-down menus or toggle buttons. Regardless of the interaction method used, after the user executes the modification operation, the modified layout rules are generated based on the user's changes, and the target interface components are laid out according to the modified layout rules. In some cases, user modification operations may also include anchor point modification, etc., which will not be elaborated upon here.

[0093] In some alternative implementations, the component library-based user interface generation method further includes steps g1 and g2.

[0094] Step g1: Obtain scene configuration information, which includes scene background type and / or mask layer parameters.

[0095] Step g2: Render the scene background and / or mask layer in the canvas corresponding to the target interface according to the scene configuration information.

[0096] In game UI design, scene backgrounds and overlays are crucial visual elements affecting the visual atmosphere and readability of the interface. Different scene types may require different scene backgrounds (for example, a battle interface might need an immersive background image, while a backpack interface might only need a simple solid color background or a blurred overlay). Related technologies lack parametric scene configuration capabilities, requiring designers to manually search, crop, place, and adjust transparency—repetitive and error-prone operations. By obtaining scene configuration information, the generated interface can be made to meet scene requirements. This scene configuration information includes the scene background type and / or overlay parameters (such as overlay transparency). Users can select the corresponding scene in the scene selection interface and manually input the overlay parameters. After determining the scene configuration information, the corresponding scene background can be confirmed based on the scene background type. The corresponding overlay is generated based on the overlay parameters and rendered in the canvas corresponding to the target interface (i.e., the canvas used for component layout when generating the target user interface). This overlay is used to obscure the background, ensuring that important information in the target user interface (such as pop-ups) is highlighted.

[0097] After users upload a reference draft (such as the original design draft), they can select a scene. Figure 4 This is a schematic diagram illustrating a scenario selection according to an embodiment of this application. Figure 4 In the displayed graphical user interface, users can select the scene (system) corresponding to the current original design draft to generate scene configuration information. Figure 5 This is a schematic diagram illustrating a layout type selection according to an embodiment of this application. Figure 5 In the displayed graphical user interface, the user can select the layout (function) corresponding to the current original design draft. After the user makes a selection, the layout type is determined. The layout type automatically identified in step d1 can be selected by default in this user interface. Figure 5 The "Full Screen" layout type is selected by default. After selecting the function, users can also select components, such as adding interface components that do not exist in the reference draft (for example, the original reference draft may not have an "icon" interface component, but the actual needs require the "icon" interface component, in which case the interface component can be added through component selection). Figure 6 This is a schematic diagram illustrating a layout rule modification according to an embodiment of this application. Figure 6 In the displayed graphical user interface, users can modify the current layout rules, such as adjusting component spacing, selecting component arrangement methods, or modifying alignment anchor points. Figure 7 This is a schematic diagram illustrating the modification of scenario configuration information according to an embodiment of this application. Figure 7 In the displayed graphical user interface, if the scene selected by the user corresponds to multiple background images, the user can select a background image from the scene configuration information (e.g., in...). Figure 4 If "Manor" is selected, the available background images are "Manor Background - Daytime", "Manor Background - Nighttime" and "Hall Background". At the same time, the mask layer parameters can be modified, and scene configuration information including scene background type and / or mask layer parameters can be generated.

[0098] Based on the above configuration, a target user interface can be generated, which can be as follows: Figure 8 As shown. Figure 8 This is a schematic diagram of a target user interface according to an embodiment of this application, wherein each target interface component is arranged according to a layout rule, with the first-level label being the label name and the second-level label being the label text. For example, the layout rule corresponding to "label text" is: "second-level label module - left 250 - list", that is, the target interface component corresponding to "label text" is a second-level label module, the distance between it and the left anchor point is 250 pixels, and it is presented in list form, wherein the specific parameters of the list can be based on... Figure 6 Configure the interface shown. Similarly, the character list section on the right also follows certain layout rules: Character List - Top 50, Left 20 - Grid. This means the target interface component is the character list, with a vertical distance of 50 pixels and a horizontal distance of 20 pixels between it and the top-left anchor point, presented in a grid format. The specific parameters of the grid can also be based on... Figure 6 Configure the interface shown.

[0099] The component-based user interface generation method provided in this embodiment uses image segmentation technology to divide the original design draft from a single image into a set of element regions, automatically separating and locating each interface element in the original design draft. Through function recognition technology, each interface element is assigned a function label to determine its function within the interface. The geometric features produced by image segmentation and the element function information produced by function recognition are integrated into unified element attribute information. A dual-dimensional description of each interface element, encompassing both morphological and semantic dimensions, is established. By evaluating the matching degree between interface elements and interface components from both functional semantic and structural morphological dimensions, the matching accuracy is higher than that of a single-dimensional evaluation. Furthermore, by matching target interface components corresponding to interface elements from a pre-set component library, it is ensured that multiple designers use interface components from the library when designing interfaces, guaranteeing consistency in interface design within the same project. By obtaining the layout type and laying out each target interface component according to the corresponding layout rules, the effect of automatically generating the target user interface is achieved, eliminating the need for manual dragging and alignment, thus improving interface production efficiency.

[0100] This embodiment also provides a user interface generation apparatus based on a component library, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0101] This embodiment provides a user interface generation device based on a component library, such as... Figure 9 As shown, it includes: Design Draft Module 901 is used to obtain the original design draft.

[0102] The visual analysis module 902 is used to perform visual analysis on the original design draft and determine the element attribute information of each interface element in the original design draft.

[0103] The component matching module 903 is used to match any interface element with the component information of each interface component in the preset component library based on the element attribute information of the interface element, and to determine the target interface component that matches the interface element.

[0104] The layout module 904 is used to obtain the layout type, and to lay out each target interface component according to the layout rules corresponding to the layout type to generate the target user interface.

[0105] In some alternative implementations, the visual resolution module 902 includes: The image segmentation submodule is used to segment the original design draft, extract the element regions of each interface element, and use the geometric features of the element regions as the geometric features of the interface elements.

[0106] The function identification submodule is used to identify the function of the element area of ​​each interface element and determine the element function information of each interface element.

[0107] The element attributes submodule is used to generate element attribute information based on geometric features and element functional information.

[0108] In some optional implementations, the function identification submodule includes: The text recognition unit is used to perform text recognition on the element areas of each interface element and extract the text content within each element area.

[0109] The function recognition unit is used to determine the element function information of any interface element based on the text content and geometric features of the interface element.

[0110] In some optional implementations, the component information includes component functional information and component structural information, and the component matching module 903 includes: The semantic matching submodule is used to traverse the component function information of each interface component in the preset component library based on the element function information of the interface elements, and perform semantic matching to obtain the semantic matching result.

[0111] The structure matching submodule is used to perform structure matching based on the geometric features of interface elements and the component structure information of each interface component to obtain the structure matching result.

[0112] The component matching submodule is used to determine the target interface component from multiple interface components based on semantic matching results and structural matching results.

[0113] In some alternative implementations, the component matching submodule includes: The candidate component unit is used to determine candidate interface components based on the semantic matching results and structural matching results of each interface component.

[0114] The matching degree unit is used to obtain the matching degree of candidate interface components. The matching degree is calculated based on the semantic matching result and the structural matching result of the candidate interface components.

[0115] The component matching unit is used to determine the target interface component based on the candidate interface components when the matching degree is greater than a preset first threshold.

[0116] The pending component unit is used to output a list of pending interface components to the user when the matching degree is less than a first threshold and greater than a preset second threshold, and in response to the selection operation of the pending interface components, to use the selected pending interface component as the target interface component.

[0117] In some alternative implementations, the layout module 904 includes: The layout parsing submodule is used to identify the layout type of the original design draft based on the page layout parsing results corresponding to the visual parsing.

[0118] or, The Select Layout submodule is used to determine the layout type in response to a selection operation for the layout type.

[0119] In some alternative implementations, the component library-based user interface generation apparatus further includes: The scene configuration module is used to obtain scene configuration information, which includes scene background type and / or mask layer parameters.

[0120] The scene rendering module is used to render the scene background and / or mask layer in the canvas corresponding to the target user interface based on the scene configuration information.

[0121] In some alternative implementations, the component library-based user interface generation apparatus further includes: The component library scanning module scans the component library to obtain component information and identification information for each UI component. The identification information is used to call the target UI components when generating the target user interface.

[0122] In some alternative implementations, the layout module 904 includes: The rule-based layout submodule is used to lay out the target UI component according to the layout rules if the target UI component has a corresponding rule in the layout rules.

[0123] The position layout submodule is used to lay out the target interface component according to the geometric characteristics of the corresponding interface element if there is no corresponding rule for the target interface component in the layout rules.

[0124] In some alternative implementations, the component library-based user interface generation apparatus further includes: The layout modification module is used to respond to modifications to the layout rules, determine the modified layout rules, and then lay out each target interface component according to the modified layout rules. Modification operations include modifying component spacing and / or modifying component arrangement.

[0125] The component library-based user interface generation apparatus provided in this application can execute the component library-based user interface generation method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0126] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0127] The following is a detailed reference. Figure 10 The diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from memory 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0128] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006, including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007, including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 1008, including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0129] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from memory 1008, or installed from ROM 1002. When the computer program is executed by processor 1001, it performs the functions defined in the component library-based user interface generation method of embodiments of this application.

[0130] Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0131] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc. Further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the component library-based user interface generation method shown in the above embodiments is implemented.

[0132] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0133] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for generating user interfaces based on a component library, characterized in that, The method includes: Obtain the original design drafts; Visual analysis is performed on the original design draft to determine the element attribute information of each interface element in the original design draft; For any of the interface elements, the element attribute information of the interface element is matched with the component information of each interface component in the preset component library to determine the target interface component that matches the interface element. Obtain the layout type, and lay out each of the target interface components according to the layout rules corresponding to the layout type to generate the target user interface.

2. The method according to claim 1, characterized in that, The step of visually analyzing the original design draft to determine the element attribute information of each interface element in the original design draft includes: The original design draft is segmented to extract the element regions of each interface element, and the geometric features of the element regions are used as the geometric features of the interface elements. Functional identification is performed on the element areas of each interface element to determine the element function information of each interface element. The element attribute information is generated based on the geometric features and the element functional information.

3. The method according to claim 2, characterized in that, The step of identifying the function of each interface element's element region and determining the element function information of each interface element includes: Text recognition is performed on the element areas of each of the interface elements to extract the text content within each element area; For any of the interface elements, the element function information of the interface element is determined based on the text content of the interface element and the geometric features of the interface element.

4. The method according to claim 2, characterized in that, The component information includes component function information and component structure information; the step of matching the element attribute information of the interface element with the component information of each interface component in a preset component library to determine the target interface component that matches the interface element includes: Based on the element function information of the interface elements, the component function information of each interface component is traversed in the preset component library, and semantic matching is performed to obtain the semantic matching result. The structure matching result is obtained by performing structural matching based on the geometric features of the interface elements and the component structure information of each interface component. Based on the semantic matching result and the structural matching result, the target interface component is determined from the plurality of interface components.

5. The method according to claim 4, characterized in that, The step of determining the target interface component from a plurality of interface components based on the semantic matching result and the structural matching result includes: Candidate interface components are determined based on the semantic matching results and the structural matching results of each interface component. Obtain the matching degree of the candidate interface component; the matching degree is calculated based on the semantic matching result and the structural matching result of the candidate interface component; When the matching degree is greater than a preset first threshold, the target interface component is determined based on the candidate interface components; When the matching degree is less than the first threshold and greater than the preset second threshold, a list of pending interface components is output to the user, and in response to the selection operation of the pending interface components, the selected pending interface component is used as the target interface component.

6. The method according to claim 1, characterized in that, The obtained layout type includes: Based on the page layout analysis results corresponding to the visual analysis, identify the layout type of the original design draft; or, In response to the selection operation for the layout type, determine the layout type.

7. The method according to claim 1, characterized in that, The method further includes: Obtain scene configuration information, which includes scene background type and / or masking layer parameters; Render the scene background and / or mask layer in the canvas corresponding to the target user interface according to the scene configuration information.

8. The method according to claim 1, characterized in that, Before performing visual analysis on the original design draft, the method further includes: The component library is scanned to obtain component information and identification information of each interface component in the component library; the identification information is used to call the target interface component when generating the target user interface.

9. The method according to claim 1, characterized in that, The step of laying out each of the target interface components according to the layout rules corresponding to the layout type to generate the target user interface includes: If the target interface component has a corresponding rule in the layout rules, then the target interface component is laid out according to the layout rules; If the target interface component has no corresponding rule in the layout rules, then the target interface component is laid out according to the geometric features of the interface element corresponding to the target interface component.

10. The method according to claim 9, characterized in that, The method further includes: In response to the modification operation of the layout rule, a modified layout rule is determined, and the target interface components are laid out according to the modified layout rule; the modification operation includes a component spacing modification operation and / or a component arrangement modification operation.

11. A user interface generation device based on a component library, characterized in that, The device includes: The design draft module is used to obtain the original design draft; The visual analysis module is used to perform visual analysis on the original design draft and determine the element attribute information of each interface element in the original design draft. The component matching module is used to match any interface element with the component information of each interface component in a preset component library based on the element attribute information of the interface element, and to determine the target interface component that matches the interface element. The layout module is used to obtain the layout type, and lay out each of the target interface components according to the layout rules corresponding to the layout type to generate the target user interface.

12. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the user interface generation method based on a component library as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the user interface generation method based on a component library as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes computer instructions for causing a computer to perform the component library-based user interface generation method according to any one of claims 1 to 10.

Images