User interface code generation method and device, equipment, medium and product
By identifying, parsing, and componentizing user interface design files, high-quality, maintainable user interface code is generated, solving the problems of low development efficiency and poor code quality in existing technologies, and enabling fast and efficient interface development.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 中移信息技术有限公司
- Filing Date
- 2026-01-21
- Publication Date
- 2026-05-12
AI Technical Summary
In current front-end application development, manual coding is inefficient and depends on the developer's skills, while automatically generated code by tools is of poor quality and cannot handle complex logic, making it difficult to achieve fast and efficient interface development.
By acquiring the user interface design file, identifying and parsing the user interface elements, performing componentization, and using a code generator to generate user interface code, the layout relationships and interaction logic of the elements are comprehensively considered.
It improves the efficiency and quality of user interface code generation, reduces reliance on developer skills, and generates highly maintainable code, enabling fast and efficient interface development.
Smart Images
Figure CN122018903A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of front-end application development technology, and in particular to a method, apparatus, device, medium and product for generating user interface code. Background Technology
[0002] In the field of front-end application development, how to efficiently and effectively transform visual design drafts into usable code interfaces has long been a core challenge. Currently, industry solutions mainly fall into two categories: developers coding entirely manually and using automated tools to generate code. However, both approaches have significant limitations.
[0003] Manual coding requires developers to write all the code from scratch, which is slow and labor-intensive, especially unfavorable for rapid progress in the early stages of a project. Furthermore, its output quality is highly dependent on the individual developer's ability, easily leading to visual discrepancies and inconsistent coding styles, requiring additional management costs to maintain consistency in team collaborations.
[0004] While automated code generation tools can quickly output interface structures, the generated code is often redundant, messy, and poorly maintainable, leading to high technical debt over time. Furthermore, these tools are limited in their capabilities, only able to handle static appearances and unable to implement dynamic interactions and complex layouts, ultimately still requiring significant manual intervention and rewriting.
[0005] Therefore, existing technologies are either inefficient and require high levels of skill from developers, or they generate poor-quality code that cannot handle complex logic. Both make it difficult to achieve fast and efficient interface development while ensuring high-quality code and good maintainability. Summary of the Invention
[0006] This invention provides a user interface code generation method, apparatus, device, medium, and product to solve the problem that existing user interface code generation methods are unable to achieve fast and efficient interface development while ensuring high-quality code and good maintainability.
[0007] In a first aspect, embodiments of the present invention provide a user interface code generation method, including: Obtain the user interface design file and identify the set of user interface elements in the user interface design file; The element parsing result is obtained by parsing each user interface element in the set of user interface elements. Based on the element parsing results, the user interface elements in the user interface element set are componentized to obtain at least one target component; The code generator generates user interface code based on the element parsing results and the target component.
[0008] Secondly, embodiments of the present invention provide a user interface code generation apparatus, comprising: The set determination module is used to obtain the user interface design file and identify the set of user interface elements in the user interface design file. The element parsing module is used to parse each user interface element in the user interface element set to obtain the element parsing result; The element componentization module is used to componentize the user interface elements in the user interface element set based on the element parsing result to obtain multiple target components; The interface code generation module is used to generate user interface code based on the element parsing results and the target component through a code generator.
[0009] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the user interface code generation method according to any embodiment of the present invention.
[0010] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the user interface code generation method described in any embodiment of the present invention.
[0011] Fifthly, embodiments of the present invention provide a computer program product including a computer program, which, when executed by a processor, implements the user interface code generation method described in any embodiment of the present invention.
[0012] The technical solution of this invention involves acquiring a user interface design file and identifying a set of user interface elements within it; parsing each user interface element in the set to obtain element parsing results; componentizing the user interface elements based on the parsing results to obtain multiple target components; and generating user interface code based on the parsing results and target components using a code generator. By identifying, parsing, componentizing, and generating code from the user interface design file, user interface code can be automatically generated. Compared to manually writing front-end code, this invention improves the efficiency of user interface code generation and does not overly rely on the capabilities of front-end developers, thus freeing up their productivity. Furthermore, after componentizing the user interface elements in the set, generating interface code based on the parsing results and target components, compared to methods that directly generate code using tools, resulting in redundant and chaotic code, this invention comprehensively considers the layout relationships and interaction logic of elements within or between components during code generation, improving the quality and maintainability of the generated front-end code. It solves the problems of either low development efficiency and high skill requirements, or poor code quality and inability to handle complex logic, achieving the beneficial effect of fast and efficient interface development while ensuring high-quality code and good maintainability.
[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 A flowchart of a user interface code generation method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a user interface code generation method provided in Embodiment 2 of the present invention; Figure 3 A diagram illustrating the user interface elements included in the login page; Figure 4 A diagram illustrating an abstract layout tree of a collection of user interface elements; Figure 5 A diagram illustrating the grouping of user interface elements; Figure 6 This is a schematic diagram of a user interface code generation device provided in Embodiment 3 of the present invention; Figure 7 A schematic diagram of the structure of an electronic device for implementing the user interface code generation method of this invention. Detailed Implementation
[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0018] Currently, the industry's solutions mainly fall into two categories: developers coding entirely manually and using automated tools to generate code. However, both of these approaches have significant limitations.
[0019] The purely manual coding method has the following disadvantages: (1) Low development efficiency and high cost: This solution requires developers to build all the code from scratch, which is a relatively slow and labor-intensive process. For large projects or complex interfaces, writing the structure, style and basic interaction will take a lot of time, especially in the early stages of the project, the progress of visual reconstruction will be significantly slower than that of the automatically generated solution. This means higher manpower and time costs, and in a market environment that pursues rapid iteration and launch, this disadvantage is sometimes particularly prominent.
[0020] (2) It places high demands on developers' skills and is prone to inconsistencies. Perfectly reproducing the design drafts depends heavily on the developers' technical level and attention to detail. Developers must have a strong foundation in CSS layout and a keen insight into design details; otherwise, layout deviations, style omissions, or improper responsiveness are very likely to occur. In team collaboration, differences in coding habits among different developers may also lead to inconsistencies in the overall style and code quality of the project, requiring strict coding standards and frequent reviews to maintain consistency.
[0021] The method of automatically generating code using tools has the following disadvantages: (1) Poor code quality and high hidden maintenance costs: The code generated by the tool is usually "disposable" code optimized for visual reproduction, and generally suffers from problems such as structural redundancy, chaotic class names, excessive nesting, and non-compliance with team coding standards. This leads to a situation of "fast generation, slow optimization", and developers must invest a lot of time in refactoring and cleaning up, otherwise it will become "garbage code" that is difficult to maintain. In the long run, this hidden technical debt and maintenance cost may completely offset its initial efficiency advantage.
[0022] (2) Limitations: Unable to handle dynamic logic and complex layouts. The core flaw of this method lies in its limitations. It can only generate a static UI shell, and all dynamic interactions, state management, and business logic must be manually injected by the developer from scratch. Furthermore, for complex or intricate layouts that deviate from the norm, the tool's parsing capabilities are often insufficient, resulting in fragile code structures that are difficult to adapt to responsive design requirements, ultimately requiring the developer to completely rewrite the code. This makes it essentially just an advanced auxiliary tool, rather than a true automated solution.
[0023] Example 1 Figure 1 This is a flowchart of a user interface code generation method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where user interface code is automatically generated based on user interface design documents. This method can be executed by a user interface code generation device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110. Obtain the user interface design file and identify the set of user interface elements in the user interface design file.
[0024] User interface (UI) design documents can be considered professional documents that systematically describe the appearance, interaction logic, and design specifications of a digital product's interface. UI design documents can be in image format, such as PNG or JPEG; or they can be the source format of various other files. The set of user interface elements can include the bounding boxes and category labels of the UI elements. Category labels for UI elements can include buttons, input boxes, images, text, and icons, etc.
[0025] In this embodiment, a user interface design file is obtained, and object detection is performed on the user interface design file to obtain a set of user interface elements composed of user interface elements in the user interface design file. For example, the object detection method can utilize an object detection algorithm or a well-trained object detection model.
[0026] S120. Parse each user interface element in the user interface element set to obtain the element parsing result.
[0027] The element parsing result can be considered as a collection of results obtained by parsing user interface elements. The element parsing result can include the style attribute information and layout relationship of user elements, as well as the corresponding text content of the elements.
[0028] In this embodiment, for each user interface element in the identified set of user interface elements, the user interface element is parsed in terms of style, layout, and content to obtain the element parsing result. For example, the parsing of user interface elements can be obtained through a neural network model, such as a neural network or a Transformer model.
[0029] S130. Based on the element parsing results, the user interface elements in the user interface element set are componentized to obtain at least one target component.
[0030] The target component can be considered as a combination of one or more user interface elements.
[0031] In this embodiment, the user interface elements are componentized based on the style attribute information, layout relationships, and text content contained in the element parsing results. This involves grouping the user interface elements to obtain at least one (or more) target components, each of which may contain one or more user interface elements. For example, the componentization of user interface elements can be achieved using a clustering algorithm.
[0032] S140. Generate user interface code based on element parsing results and target components using a code generator.
[0033] User interface code is the programming implementation that translates UI design into machine-executable instructions; it serves as the engineering layer connecting visual design and functional logic. Code generators are tools that automatically generate runnable source code using abstract rules, templates, or artificial intelligence models.
[0034] In this embodiment, user interface code is generated according to the configured framework based on the parsing results of the user interface elements and the generated target components.
[0035] The technical solution of this invention involves acquiring a user interface design file and identifying a set of user interface elements within it; parsing each user interface element in the set to obtain element parsing results; componentizing the user interface elements based on the element parsing results to obtain multiple target components; and generating user interface code based on the element parsing results and target components using a code generator. By identifying, parsing, componentizing, and generating code from the user interface design file, user interface code can be automatically generated. Compared to manually writing front-end code, this invention improves the efficiency of user interface code generation and does not overly rely on the capabilities of front-end developers, thus freeing up their productivity. Furthermore, after componentizing the user interface elements in the set, generating interface code based on the element parsing results and target components, compared to methods that directly generate code using tools, which can lead to redundant and chaotic code, this invention comprehensively considers the layout relationships and interaction logic of elements within or between components during code generation, improving the quality and maintainability of the generated front-end code. While ensuring high-quality code and good maintainability, it achieves rapid and efficient interface development.
[0036] Example 2 Figure 2 This is a flowchart of a user interface code generation method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further refines S110 to S140.
[0037] like Figure 2 As shown, the method includes: S210. Obtain the user interface design file, and identify the bounding boxes and category labels of the user interface elements in the user interface design file based on the object detection model; construct a set of user interface elements based on the bounding boxes and category labels of each user interface element.
[0038] In this embodiment, the object detection model can be, for example, a YOLO model or a Faster R-CNN model. The user interface design file is input into the object detection model to obtain the bounding boxes and category labels of the user interface elements detected by the model. For example, the category labels can be placed within or around the bounding boxes of the user interface elements.
[0039] For example, Figure 3 This is a diagram illustrating the user interface elements included in the login page. For example... Figure 3 As shown, the user interface design file is a login page. The user interface elements on the login page include: a title "User Login", a "Username" label, a "Username Input Box", a "Password" label, a "Password Input Box", and a "Login" button. The type label for the "User Login" title is "Text(H2)"; the type label for the "Username" label is "Text(Label)"; the type label for the "Username Input Box" is "Input"; the type label for the "Password" label is "Text(Label)"; the type label for the "Password Input Box" is "Input"; and the type label for the "Login" button is "Button".
[0040] S220. Perform layout parsing on each user interface element in the user interface element set to obtain an abstract layout tree.
[0041] In this embodiment, the spatial relationships between user interface elements in the user interface element set are analyzed, such as alignment, spacing, and nesting, and an abstract layout tree is constructed based on these spatial relationships. The abstract layout tree provides a clear visual representation of the hierarchical relationships between user interface elements, such as multiple elements residing within the same flexbox or grid container.
[0042] For example, Figure 4 This is a diagram illustrating an abstract layout tree of a collection of user interface elements. (Example) Figure 4 As shown, by analyzing Figure 3 The set of user interface elements in the login page shown is parsed to obtain an abstract layout tree. The abstract layout tree contains the hierarchical relationship between the title "User Login", the "Username" label, the "Username Input Box", the "Password" label, the "Password Input Box", and the "Login" button.
[0043] S230. Perform style parsing on each user interface element in the user interface element set to obtain style attribute information.
[0044] In this embodiment, the visual style of each user interface element in the user interface element set is parsed to obtain style attribute information, such as coordinates, size, background color, border, font size, font color, padding and margin.
[0045] S240. Perform semantic understanding on the text corresponding to each user interface element in the user interface element set to obtain the interaction intent of the user interface element; construct the element parsing result based on the abstract layout tree, style attribute information and interaction intent.
[0046] In this embodiment, the text corresponding to each user interface element in the set of user interface elements is determined, such as the names of buttons and input boxes. Semantic understanding is then performed on the text to obtain the interaction intent of the user interface elements. The element parsing result is constructed based on the parsed abstract layout tree, style attribute information, and interaction intent.
[0047] For example, a parsing model can be used to parse the layout and style of each user interface element in the set of user interface elements; a semantic understanding model can be used to analyze the semantics of the text corresponding to the user interface elements.
[0048] S250. Based on the element parsing results, cluster each user interface element in the user interface element set to obtain at least one target component; the target component includes a logical component and / or a text component.
[0049] The target components include logic components and / or text components. Logic components can be considered UI components that contain interaction logic, data binding, state management, or complex business rules. Text components can be considered UI components that focus on content display, do not contain complex interaction logic, and are mainly used to present static or dynamic text information.
[0050] In this embodiment, each user interface element is clustered based on the abstract layout tree, style attribute information of user interface elements, and interaction intent in the element parsing results, and a cluster obtained from the clustering is determined as a target component.
[0051] For example, Figure 5 A diagram illustrating the grouping of user interface element sets, such as... Figure 5 As shown, by analyzing Figure 3 The user interface elements on the login page shown are grouped to obtain a logical component and a text component.
[0052] This embodiment focuses on layout, style, and interaction. Figure 3 It can group user interface elements and accurately restore and identify the relative positional relationships between elements according to the design layout intent, providing a basis for generating adaptive layout code; it can also automatically infer a reasonable DOM structure based on the layout tree.
[0053] S260. Determine a predefined component that matches the target component from the predefined component library, and generate an event handling function based on the element parsing result and the predefined component.
[0054] Among them, predefined components can be considered as standardized UI units that are pre-designed, developed, and encapsulated, with clear API interfaces and reusability, and are the modular cornerstone for building user interfaces.
[0055] In this embodiment, a predefined component library is constructed, which can store historically used components and / or independently defined components. The target component, obtained through grouping, is matched against the predefined components stored in the predefined component library to obtain predefined components that match the target component. Then, the event type is determined based on the interaction intent in the element parsing results, and an event handling function is generated based on the event type and the API specification of the predefined component. The API specification of the predefined component may include: event interface definition, event triggering conditions, and return value format, etc.
[0056] For example, determining a predefined component matching the target component from a predefined component library can be achieved by calculating the similarity between the target component and the attribute information of each predefined component in the library, and identifying the predefined component with the highest similarity as the matching component. Similarity can be based on layout, style, and interaction. Figure 3 The degree of overall similarity in all aspects.
[0057] S270. Generate user interface code based on event handling functions and predefined components using a code generator.
[0058] As an optional implementation of this embodiment, the step of generating user interface code based on the event handling function and the predefined component using a code generator includes: A1. Using a code generator, parse the event handling logic in the event handling function and the relationship between the event handling logic and the predefined components.
[0059] In this embodiment, the code generator analyzes the event handling function, extracting its core logic structure. This core logic structure may include identifying the types of parameters received by the function, the sequence of operations executed internally, and the function's output or side effects. Simultaneously, it analyzes the binding relationship between the function and predefined components, determining which component's event triggered it, and clarifying how data flows between components and functions, including how component states are read and updated.
[0060] A2. Map the event handling logic and the attribute information of the associated predefined components to standardized configuration data.
[0061] In this embodiment, the code generator maps the parsed logic and relationships into standardized intermediate configuration data. This process abstracts event handling steps into describable action nodes and transforms component properties and event interface specifications into unified configuration objects. Ultimately, this information is structured into a standardized configuration set containing a component tree, event mapping, and state flow description.
[0062] A3. Generate the interface layout and interaction logic code by calling the code template based on the standardized configuration data.
[0063] In this embodiment, the code generator calls a pre-written code template based on the component type and layout information in the standardized configuration data. Specifically, it can select the corresponding UI structure template based on the component type and layout information in the configuration to generate "interface layout code" that describes the static structure of the interface. Based on the event logic and interaction flow described in the configuration, it selects the corresponding logic template to generate "interaction logic code" that includes state variables and event handling functions.
[0064] A4. Integrate the code of the interface layout and interaction logic, and output the user interface code that conforms to the target framework specification.
[0065] In this embodiment, the code generator correctly embeds event handlers into the properties of the corresponding components and ensures that state variables are properly imported and initialized. During integration, the generator strictly follows the syntax and best practices of the target front-end framework (such as React or Vue), formats the code, and adds necessary import statements. Finally, it outputs a complete user interface code file that can be run directly or used as the basis for development.
[0066] The technical solution of this invention involves: acquiring a user interface design file; identifying the bounding boxes and category labels of user interface elements in the design file based on an object detection model; constructing a user interface element set based on the bounding boxes and category labels of each user interface element; performing layout parsing on each user interface element in the user interface element set to obtain an abstract layout tree; performing style parsing on each user interface element in the user interface element set to obtain style attribute information; performing semantic understanding on the text corresponding to each user interface element in the user interface element set to obtain the interaction intent of the user interface element; constructing an element parsing result based on the abstract layout tree, style attribute information, and interaction intent; clustering each user interface element in the user interface element set based on the element parsing result to obtain at least one target component; the target component includes logical components and / or text components; determining a predefined component matching the target component from a predefined component library; generating an event handling function based on the element parsing result and the predefined component; and generating user interface code based on the event handling function and the predefined component using a code generator. By identifying, parsing, componentizing, and generating code from user interface design files, this invention can automatically generate user interface code. Compared to manually writing front-end code, this invention improves the efficiency of user interface code generation and does not rely excessively on the capabilities of front-end developers, thus freeing up their productivity. Furthermore, after componentizing the user interface elements in the user interface element set, the invention generates interface code based on the element parsing results and target components. Compared to methods that directly generate code using tools, which can lead to redundant and chaotic code, this invention comprehensively considers the layout relationships and interaction logic of elements within or between components during code generation, improving the quality and maintainability of the generated code. While ensuring high-quality code and good maintainability, this invention achieves rapid and efficient interface development.
[0067] In a specific example, the user interface design file is a webpage design draft, containing UI images of a top navigation bar, search box, and user avatar list. User interface elements are identified as one navigation bar container, one logo image, one search input box, one search button, ten user avatar images, and ten username texts. Layout analysis reveals that the navigation bar is a horizontally arranged Flex container; the user avatar list is a Grid layout. Style analysis obtains the background color of the navigation bar, the size of the logo, and the color and font of the text. Semantic understanding identifies the placeholder text of the search box as "Searching for users." The search input box and search button are grouped and identified as one... <searchbar>Component. Group each "avatar + username" combination and identify it as a single component. <usercard>Components. <searchbar>Components are mapped to SearchComponent in the predefined component library. <usercard>Go to UserProfileComponent. Generate a React event handler function using the code generator. In the returned JSX, use it directly. <searchcomponent / > And render 10 in a loop using the map function.<UserProfileComponentuser={userData} / > ,for <searchcomponent / > Generate a placeholder for the onSearch function. Implement the overall layout using CSSGrid or Flexbox and generate responsive CSS code.
[0068] Example 3 Figure 6 This is a schematic diagram of a user interface code generation device provided in Embodiment 3 of the present invention. Figure 6 As shown, the device includes: a set determination module 310, an element parsing module 320, an element componentization module 330, and an interface code generation module 340; wherein: The set determination module 310 is used to obtain the user interface design file and identify the set of user interface elements in the user interface design file. The element parsing module 320 is used to parse each user interface element in the user interface element set to obtain the element parsing result; The element componentization module 330 is used to componentize the user interface elements in the user interface element set based on the element parsing result to obtain multiple target components. The interface code generation module 340 is used to generate user interface code based on the element parsing results and the target component through a code generator.
[0069] This invention provides a user interface code generation device. It acquires a user interface design file and identifies a set of user interface elements within the file. Each user interface element in the set is parsed to obtain element parsing results. Based on the parsing results, the user interface elements are componentized to obtain multiple target components. A code generator then generates user interface code based on the parsing results and the target components. By identifying, parsing, componentizing, and generating code from the user interface design file, user interface code can be automatically generated. Compared to manually writing front-end code, this invention improves the efficiency of user interface code generation and does not overly rely on the capabilities of front-end developers, thus freeing up their productivity. Furthermore, after componentizing the user interface elements in the set, the device generates interface code based on the parsing results and target components. Compared to methods that directly generate code using tools, which often result in redundant and chaotic code, this invention comprehensively considers the layout relationships and interaction logic of elements within or between components during code generation, improving code generation quality and maintainability. While ensuring high-quality code and good maintainability, it achieves rapid and efficient interface development.
[0070] Optionally, the element parsing module 320 is specifically used for: Layout parsing is performed on each user interface element in the set of user interface elements to obtain an abstract layout tree; The style attribute information of each user interface element in the set of user interface elements is obtained by parsing the style. Semantic understanding is performed on the text corresponding to each user interface element in the set of user interface elements to obtain the interaction intent of the user interface elements; The element parsing results are based on the abstract layout tree, the style attribute information, and the interaction intent.
[0071] Optionally, the element componentization module 330 is specifically used for: Based on the element parsing results, cluster each user interface element in the user interface element set to obtain at least one target component; the target component includes a logical component and / or a text component.
[0072] Optionally, the interface code generation module 340 includes: A component matching unit is used to determine predefined components that match the target component from a predefined component library; A function generation unit is used to generate event handling functions based on the element parsing results and the predefined components; The code generation unit is used to generate user interface code based on the event handling function and the predefined components through a code generator.
[0073] Optionally, the code generation unit is specifically used for: The code generator is used to parse the event handling logic in the event handling function and the relationship between the event handling logic and the predefined components. Map the event handling logic and the attribute information of the associated predefined components to standardized configuration data; Based on the standardized configuration data, the code template is called to generate the code for the interface layout and interaction logic. Integrate the code for the interface layout and interaction logic, and output user interface code that conforms to the target framework specifications.
[0074] Optionally, the set determination module 310 is specifically used for: The bounding boxes and category labels of user interface elements in the user interface design file are identified based on the object detection model. The user interface element set is constructed based on the bounding box and category label of each user interface element.
[0075] The user interface code generation apparatus provided in this embodiment of the invention can execute the user interface code generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0076] Example 4 Figure 7 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0077] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0078] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0079] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as user interface code generation methods.
[0080] In some embodiments, the user interface code generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the user interface code generation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the user interface code generation method by any other suitable means (e.g., by means of firmware).
[0081] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0082] In some embodiments, the user interface code generation method may be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the user interface code generation method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program may be executed entirely on a machine, partially on a machine, partially on a remote machine as a standalone software package, or entirely on a remote machine or server.
[0083] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0085] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0086] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0087] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / usercard> < / searchbar> < / usercard> < / searchbar>
Claims
1. A method for generating user interface code, characterized in that, include: Obtain the user interface design file and identify the set of user interface elements in the user interface design file; The element parsing result is obtained by parsing each user interface element in the set of user interface elements. Based on the element parsing results, the user interface elements in the user interface element set are componentized to obtain at least one target component; The code generator generates user interface code based on the element parsing results and the target component.
2. The method according to claim 1, characterized in that, The step of parsing each user interface element in the set of user interface elements to obtain the element parsing result includes: Layout parsing is performed on each user interface element in the set of user interface elements to obtain an abstract layout tree; The style attribute information of each user interface element in the set of user interface elements is obtained by parsing the style. Semantic understanding is performed on the text corresponding to each user interface element in the set of user interface elements to obtain the interaction intent of the user interface elements; The element parsing results are based on the abstract layout tree, the style attribute information, and the interaction intent.
3. The method according to claim 1, characterized in that, The step of componentizing the user interface elements in the user interface element set based on the element parsing results to obtain at least one target component includes: Based on the element parsing results, cluster each user interface element in the user interface element set to obtain at least one target component; the target component includes a logical component and / or a text component.
4. The method according to claim 1, characterized in that, The step of generating user interface code based on the element parsing results and the target component using a code generator includes: Determine predefined components that match the target component from the predefined component library; An event handling function is generated based on the element parsing result and the predefined component; The code generator generates user interface code based on the event handling function and the predefined components.
5. The method according to claim 4, characterized in that, The step of generating user interface code using a code generator based on the event handling function and the predefined components includes: The code generator is used to parse the event handling logic in the event handling function and the relationship between the event handling logic and the predefined components. Map the event handling logic and the attribute information of the associated predefined components to standardized configuration data; Based on the standardized configuration data, the code template is called to generate the code for the interface layout and interaction logic. Integrate the code for the interface layout and interaction logic, and output user interface code that conforms to the target framework specifications.
6. The method according to claim 1, characterized in that, The identification of the set of user interface elements in the user interface design file includes: The bounding boxes and category labels of user interface elements in the user interface design file are identified based on the object detection model. The user interface element set is constructed based on the bounding box and category label of each user interface element.
7. A user interface code generation device, characterized in that, include: The set determination module is used to obtain the user interface design file and identify the set of user interface elements in the user interface design file. The element parsing module is used to parse each user interface element in the user interface element set to obtain the element parsing result; The element componentization module is used to componentize the user interface elements in the user interface element set based on the element parsing result to obtain multiple target components; The interface code generation module is used to generate user interface code based on the element parsing results and the target component through a code generator.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the user interface code generation method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the user interface code generation method according to any one of claims 1-6.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the user interface code generation method according to any one of claims 1-6.