Method, device, equipment, medium and program product for generating front-end code

Abstract

This application discloses a method, apparatus, device, medium, and program product for generating front-end code, belonging to the field of software development technology. The method includes: obtaining a design draft file, the design draft file indicating at least one UI element and attribute information of each of the at least one UI element; converting the design draft file into an intermediate representation stored in a tree structure, each node in the intermediate representation corresponding to one of the at least one UI elements, the intermediate representation being used to standardize the representation of different types of design draft files; and inputting the intermediate representation into a generative language model to obtain the front-end code corresponding to the design draft file. This achieves rapid generation of front-end code for different types of design draft files, improving the efficiency of front-end code generation.

Description

Technical Field

[0001] This application relates to the field of software development technology, and in particular to a method, apparatus, device, medium, and program product for generating front-end code. Background Technology

[0002] The software development process includes: product designers creating design drafts in UI design tools, and front-end developers writing front-end code to transform the design drafts into a user interface.

[0003] However, when front-end developers write front-end code, they need to refer to the design draft file to determine the corresponding HTML tags and attributes for each UI element, and determine the CSS properties for each UI element with different styles, and so on. This causes front-end developers to spend a lot of time on this repetitive and low-creativity coding work, reducing their development efficiency and even slowing down the overall software development process.

[0004] Therefore, how to free front-end developers from this coding work and thus improve development efficiency in the software development process is a problem that needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and program product for generating front-end code, the technical solution of which is as follows:

[0006] According to one aspect of this application, a method for generating front-end code is provided, the method comprising:

[0007] Obtain a design file, which indicates at least one UI element and attribute information of each UI element.

[0008] The design file is converted into an intermediate representation stored in a tree structure. Each node in the intermediate representation corresponds to one UI element in the at least one UI element. The intermediate representation is used to standardize the representation of different types of design files.

[0009] The intermediate representation is input into the generative language model to obtain the front-end code corresponding to the design file.

[0010] According to one aspect of this application, a front-end code generation apparatus is provided, the apparatus comprising:

[0011] Obtain a design file, which indicates at least one UI element and attribute information of each UI element.

[0012] The design file is converted into an intermediate representation stored in a tree structure. Each node in the intermediate representation corresponds to one UI element in the at least one UI element. The intermediate representation is used to standardize the representation of different types of design files.

[0013] The intermediate representation is input into the generative language model to obtain the front-end code corresponding to the design file.

[0014] According to one aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement a method for generating front-end code.

[0015] According to one aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer program, which is loaded and executed by a processor to implement a method for generating front-end code.

[0016] According to one aspect of this application, a computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor reads from the computer-readable storage medium and executes the computer program to implement a method for generating front-end code.

[0017] The beneficial effects of the technical solution provided in this application include at least the following:

[0018] On the one hand, this method enables the rapid conversion of design files into front-end code, eliminating the need for front-end developers to write code step-by-step based on product designers' designs. This improves front-end code development efficiency, shortens the software development cycle, and reduces software development costs. Furthermore, for front-end developers, the process from design to front-end code is simple yet tedious, often leading to syntax errors, naming errors, and other minor programming mistakes that are difficult to debug. Therefore, having a computer generate front-end code from design files improves the reliability and quality of the front-end code. In addition, quickly converting design files into runnable front-end code facilitates rapid software iteration and design verification.

[0019] On the other hand, in the process of generating front-end code, the front-end code is not directly generated based on the design draft file. Instead, the design draft file is first converted into an intermediate representation with a tree structure, and then the front-end code is generated based on this intermediate representation. By converting the design draft file into an intermediate representation, the design draft file is standardized, ensuring the consistency of the input to the generative language model. This allows the generative language model to better understand the input information, reducing ambiguity introduced by different types of design draft files and improving the reliability and accuracy of the front-end code generated by the generative language model. Furthermore, converting the design draft file into an intermediate representation before inputting it into the generative language model also makes the method provided in this application applicable to design draft files from different UI design tools. This ensures that the corresponding front-end code can be obtained for design draft files from different UI design tools, improving the versatility of the front-end code generation process. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown;

[0022] Figure 2 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0023] Figure 3 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0024] Figure 4 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0025] Figure 5 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0026] Figure 6 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0027] Figure 7 A schematic diagram illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0028] Figure 8A schematic diagram illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0029] Figure 9 A schematic diagram illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0030] Figure 10 A schematic diagram illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0031] Figure 11 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown;

[0032] Figure 12 A structural block diagram of a front-end code generation apparatus provided in an exemplary embodiment of this application is shown;

[0033] Figure 13 This invention provides a schematic diagram of the structure of a server according to an exemplary embodiment of the present application.

[0034] Figure 14 A schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the settings and operation information involved in this application were obtained with full authorization.

[0039] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0040] First, let me introduce the relevant terms used in this application:

[0041] AST (Abstract Syntax Tree): A tree-like data structure where each node represents a construct of source code, such as an expression, statement, or declaration. In this embodiment, the intermediate representation can be an AST, where each node represents a UI element in the design file. This UI element can be text, an image, a container, a layer, etc.

[0042] Front-end code: This refers to the code that provides the user interface (UI) and user experience (UX). This code primarily runs in the user's browser and is responsible for presenting data and server-side responses to the user in a visual and interactive format. Front-end code includes at least one of the following: HTML, CSS, or JavaScript. HTML (Hypertext Markup Language) is used to build the structure and content of web pages. HTML defines the structure of elements such as text, links, images, tables, and forms in a web page. CSS (Cascading Style Sheets) is used to set the visual style and layout of a web page. CSS controls the appearance attributes of a web page, such as font, color, spacing, and layout. JavaScript (JS) is a scripting programming language used to implement the interactivity and dynamic functionality of web pages. JavaScript allows developers to create dynamically updated content, control multimedia, animation effects, and more.

[0043] UI elements are the fundamental components of a user interface. They serve as bridges between users and software applications or websites. The design and implementation of UI elements directly impact user experience, enabling users to communicate and interact effectively with the system.

[0044] Best practices in coding refer to widely recognized and recommended programming methods and techniques used in software development to improve code quality, readability, maintainability, and efficiency. Following best practices helps developers reduce errors, avoid repetitive work, and ensure the long-term success of software projects. These best practices can be considered from the perspectives of code readability, reusability, maintainability, efficiency, and security. For example, regarding readability, this includes using clear variable and function names; keeping code concise and avoiding complex nested structures; and using appropriate comments to explain the code's intent and complex logic. Regarding reusability, this involves encapsulating repetitive code blocks through functions, classes, and modules; and using design patterns to solve common design problems. Regarding maintainability, this involves maintaining modularity, making each part relatively independent; and adhering to the single responsibility principle, ensuring that each class or function does only one thing. And so on.

[0045] BEM (Block Element Modifier) ​​naming convention is a popular front-end CSS naming methodology that represents Block, Element, and Modifier. BEM naming conventions improve code readability and maintainability by making code easier to read and understand through clear naming rules. It also enhances code scalability and reusability through modularization. Furthermore, it simplifies team collaboration by providing a unified naming standard, enabling different developers to better understand and maintain the code. Ultimately, this improves code maintainability; well-structured and named code is easier to maintain and update.

[0046] • Block: A block is an independent component on a page, such as a button, form, or navigation element. Each block is independent, has clearly defined responsibilities and functions, and is reusable. Blocks are named using a single word, such as .button or .header.

[0047] • Element: An element is a sub-part of a block and cannot exist independently of a block. They are smaller units that make up a block and are typically used to define the appearance and layout of a block. Element names use double underscores to connect the block name and the element name, such as .button__text and .header__logo.

[0048] • Modifier: Modifiers are used to represent different states or variations of a block or element, such as size, color, disabled state, etc. Modifiers are named using double hyphens connecting the block or element name and the modifier name, for example, .button--large, .button--disabled.

[0049] Responsive layout is a web design and development methodology that allows web pages to dynamically adjust their layout based on the screen size and orientation of different devices (such as desktop computers, tablets, and mobile phones) to ensure the best user experience and visual effects on all devices. The core of responsive layout lies in using fluid grids and flexible images, as well as CSS media queries to adjust styles for different screen sizes.

[0050] Generative language models are large language models designed to understand and generate human languages, including natural languages ​​and programming languages. For example, models trained on GPT-3, GPT-4, GPT-5 and other model architectures are used for code generation.

[0051] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system includes a computer device 110.

[0052] Alternatively, the computer device 110 may be a terminal or a server.

[0053] Optionally, the terminal may include, but is not limited to, mobile phones, tablets, smart voice interaction devices, game consoles, wearable devices, multimedia playback devices, PCs (Personal Computers), in-vehicle terminals, smart home appliances, and other electronic devices. Optionally, the terminal may have a UI design tool installed and running, allowing users to design front-end pages and obtain design drafts through interactive operations. For example, users can quickly build front-end pages by dragging and dropping and editing UI elements. UI design tools typically support previewing the front-end page, enabling users to more easily adjust the attribute information of UI elements to adjust the display effect of the front-end page. After designing the front-end page, users can export the design draft as a design file through the terminal.

[0054] Optionally, the terminal may also install and run a client with a code generation program that supports the front-end code generation method shown in the embodiments of this application. This code generation program is used to generate the front-end code corresponding to the input design file.

[0055] Optionally, the code generation program and the UI design tool described above may be different programs within the same client. Alternatively, the code generation program and the UI design tool may be programs within different clients.

[0056] It should be noted that the implementation form of the UI design tool and code generation program described in the embodiments of this application is not limited. For example, it can be an application that needs to be downloaded and installed, a mini-program that does not need to be installed, a web application, a browser plugin, etc.

[0057] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but it is not limited to these.

[0058] Optionally, the server can be a server providing background services for the code generation program in the terminal. That is, the terminal uploads the design file selected by the user to the server, and the server generates the front-end code corresponding to the uploaded design file. Optionally, the server can provide background services for code generation programs in multiple terminals.

[0059] Alternatively, the terminal can communicate with the server via a network, such as a wireless or wired network.

[0060] For example, the method shown in the embodiments of this application includes two stages: a model training stage and a code generation stage. The specific content of the model training stage and the code generation stage are shown below.

[0061] • Code generation phase.

[0062] (1) Obtain the design file.

[0063] Obtain design file 10, which can be a design file from any UI design tool. Design file 10 includes at least one UI element and is used to indicate at least one of the following: type information, attribute information, and hierarchical relationship of the at least one UI element. Type information includes whether the UI element is a text element, button element, container element, image element, etc. Attribute information includes basic attributes, style attributes, layout attributes, special attributes, etc. Basic attributes include ID, name, type, etc. Style attributes include color, border, shadow, etc. Layout attributes include position, size, alignment, etc. Special attributes include text style, interaction state, etc. Hierarchical relationship: If the first UI element includes a second UI element, then the second UI element can be considered a child element of the first UI element.

[0064] The design file 10 can be in different data formats, such as JSON (JavaScript Object Notation), XML (eXtensible Markup Language), etc.

[0065] Optionally, the method for obtaining the design file 10 varies depending on the UI design tool. For example, the design file 10 can be obtained using the API supported by the UI design tool; or, the design file can be downloaded as a compressed package from the UI design tool and then decompressed to obtain the design file 10.

[0066] (2) Convert the design files into a unified intermediate representation.

[0067] Optionally, the unified intermediate representation 11 is a tree structure containing all UI elements and their attributes, where each node represents a UI element and its attributes. The unified intermediate representation 11 is used to standardize the representation of different types of design files.

[0068] Optionally, different types of design files refer to design files from different UI design tools; or, different types of design files refer to design files with different data formats from the same UI design tool; or, different types of design files refer to design files with different data formats from different UI design tools. Specifically, the types of design files can be distinguished from the following two dimensions: UI design tool; data format; and data organization form. Data organization form refers to the structure and format of data when stored, processed, and represented.

[0069] Optionally, converting the design file 10 into a unified intermediate representation 11 includes the following stages: standardization processing; hierarchical structure conversion; attribute extraction and mapping.

[0070] For example, standardization refers to transforming data from different standards into data that conforms to a unified standard. Standardization includes at least one of the following: unifying the coordinate system, i.e., converting the coordinate systems of different UI design tools into a unified standard; standardizing color representation, such as using RGBA or HEX format; unifying size units, such as converting all sizes to pixels (px); and standardizing attribute names, such as using a unified attribute naming convention.

[0071] For example, the hierarchy transformation is used to convert the hierarchy in the design file 10 into a tree structure. The hierarchy transformation includes at least one of the following: establishing parent-child node relationships; handling component and instance relationships; merging similar layers; and handling special container types.

[0072] For example, establishing a parent-child node relationship, such as for two UI elements that have an inclusion relationship, setting the included UI element as a child node of the UI element that contains it.

[0073] For example, handling the relationship between components and instances involves some UI design tools. These tools allow designers to design a component and add it to the design draft. After the component is added to the design draft as a UI element (this process can be called instantiation), these UI elements can be called instances of the component. One component can correspond to multiple instances. These instances usually have the same element attributes as the component, but designers can adjust each instance to have element attributes that are different from the component. Handling the relationship between components and instances means distinguishing between the element attributes inherited from the component and the element attributes that are different from the component for each UI element that belongs to the instance. This ensures that when generating front-end code later, appropriate inheritance relationships can be generated, guaranteeing the standardization and readability of the front-end code.

[0074] For example, merging similar layers refers to a process similar to the relationship between processing components and instances. Layers are the names of UI elements in another part of the UI design tool. Merging similar layers can be understood as identifying similar layers through type information or other information, so that some code such as CSS code corresponding to these similar layers can be merged when generating front-end code, thereby optimizing the generated front-end code.

[0075] For example, attribute extraction and mapping is used to extract attribute information of each UI element and map this attribute information to the corresponding node of the UI element.

[0076] (3) Input the unified intermediate representation into the generative language model to obtain the front-end code.

[0077] The unified intermediate representation 11 is input into the generative language model 12, which uses the unified intermediate representation 11 as prompt words to obtain the front-end code 13 corresponding to the design file 10.

[0078] Optionally, the generated front-end code 13 includes at least one of the following: HTML, CSS, and JavaScript.

[0079] Optionally, for the generated front-end code 13, each element, tag, attribute name, or style name in the front-end code is generated based on the semantic information of its corresponding node in the unified intermediate representation 11. The naming of these elements, tags, attribute names, or style names conforms to the naming conventions in front-end development.

[0080] Optionally, the generated front-end code 13 can be formatted and have its dependencies handled to produce the final front-end code. Code formatting refers to using a code formatting tool to uniformly format the generated front-end code, making the final code more compliant with coding standards. Dependency handling involves adding the necessary dependencies to the front-end code to ensure its successful execution.

[0081] Optionally, the generated front-end code 13 can be integrated into a complete front-end project. For example, the generated code can be written into files such as index.html, styles.css, and script.js.

[0082] • Model training phase.

[0083] (1) Construct the dataset.

[0084] Design files and front-end code are collected from open-source projects, and text pairs are formed to obtain the training dataset.

[0085] Optionally, in order to enable the generative language model to understand the structural relationship between the tree structure of the intermediate representation and the DOM tree in HTML, as well as the mapping relationship between the attributes in each node and the attributes in CSS, in addition to the training data consisting of the complete intermediate representation and the complete front-end code, an additional part of the training data can be constructed based on the two contents that need to be learned mentioned above. This will enable the trained generative language model to understand or learn the above mapping relationship, thereby generating front-end code better.

[0086] For example, a generative language model needs to understand common component naming conventions, identify the functions and uses of components, analyze semantic keywords in the names, and understand the relationships between nodes, such as the logical relationship between parent and child components, the functional hierarchy of components, and the interaction relationships between components, based on the names of each node in the intermediate representation.

[0087] Therefore, a mapping list can be constructed for the generative language model. This mapping list can exist as an external database or be input into the generative language model as prompt words, allowing the generative language model to learn directly.

[0088] (2) Based on the training data 20 consisting of intermediate representation and front-end code, train the pre-trained generative language model 21 to obtain the trained generative language model 12.

[0089] For example, based on the intermediate representation, a prompt word is generated, such as "Please convert the intermediate representation of the following UI element into HTML and CSS code: ${JSON.stringify(ast, null, 2)}".

[0090] Require:

[0091] 1. Generate semantic HTML structure

[0092] 2. Use BEM naming conventions

[0093] 3. Ensure a responsive layout.

[0094] Here, ${JSON.stringify(ast, null, 2)} represents the intermediate representation of the input.

[0095] Optionally, the prompt word is input into the pre-trained generative language model 21 to obtain the predicted front-end code output by the pre-trained generative language model. The model loss is calculated based on the similarity between the predicted front-end code and the actual front-end code. Based on the model loss, the pre-trained generative language model 21 is trained to obtain the trained generative language model 12.

[0096] Figure 2 A flowchart illustrating a front-end code generation method provided in an exemplary embodiment of this application is shown. This method is executed by a computer device, which may be one described above. Figure 1 The computer device shown in the figure. The method includes:

[0097] Step 210: Obtain the design file, which indicates at least one UI element and the attribute information of each UI element.

[0098] A design file is a file exported from any UI design tool. A design file corresponds to at least one front-end page; or, in other words, a design file corresponds to at least one user interface.

[0099] Optionally, the design file stores at least one UI element and attribute information for each of the at least one UI element. The UI elements and attribute information in the design file can be stored in different data formats, such as JSON (JavaScript Object Notation), XML (eXtensible Markup Language), etc.

[0100] Step 220: Convert the design file into an intermediate representation stored in a tree structure. Each node in the intermediate representation corresponds to at least one UI element in the UI elements. The intermediate representation is used to standardize the representation of different types of design files.

[0101] Each UI element in at least one UI element saved in the design file is used as a node in a tree structure to obtain an intermediate representation stored in a tree structure.

[0102] Optionally, the intermediate representation is used to standardize the representation of different types of design draft files; or, the intermediate representation is used to uniformly represent different types of design draft files; or, the intermediate representation is used to format different types of design draft files.

[0103] In this context, different types of design draft files refer to design draft files from different UI design tools; or, different types of design draft files refer to design draft files with different data formats within the same UI design tool; or, different types of design draft files refer to design draft files with different data formats within different UI design tools. Specifically, the types of design draft files can be distinguished from the following two dimensions: UI design tool; data format; and data organization form. The data organization form refers to the structure and format of data during storage, processing, and representation.

[0104] Generally speaking, the data format of the same UI design tool is the same. That is, for a UI design tool, the data format of the exported design files is fixed, such as JSON format or XML format.

[0105] However, in some embodiments, the data format of a design file exported by a UI design tool is selected by the user; that is, a design file exported by a UI design tool can be in either JSON or XML format. For different UI design tools, the corresponding data formats can be the same or different. For example, UI design tool 1 uses JSON format, while UI design tool 2 uses either JSON or XML format, and so on.

[0106] Optionally, for design files with the same data format, their corresponding data organization can be different or the same. In particular, design files exported from different UI design tools typically have different data organization formats. For example, when a user designs the same front-end page using different UI design tools, the attribute information of the UI elements in the exported design files may differ. For instance, for UI design tool 1, the size of UI elements is in millimeters (mm); for UI design tool 2, the size is in pixels (px); and for UI design tool 3, the size is in points (lb). As another example, for UI design tool 1, the color of UI elements is represented in hexadecimal notation, such as #FF0000; for UI design tool 2, the color is represented in RGB notation, such as rgb(255, 0, 0); and for UI design tool 3, the color is represented in RGBA notation, such as rgba(255, 0, 0, 0.5). For example, different UI design tools may use different coordinate systems when representing location information. This means that for the same UI element in the design draft of the same front-end page, different UI design tools may use different coordinates to represent that UI element. Furthermore, the same attribute may have different attribute names or different size granularities depending on the UI design tool.

[0107] Optionally, the intermediate representation is used to standardize different types of design draft files, which means using a unified tree structure to represent design draft files of different data types, and / or converting design draft files with different data organization forms into a standardized data organization form, such as the same unit, color representation, coordinate system, etc.

[0108] Step 230: Input the intermediate representation into the generative language model to obtain the front-end code corresponding to the design file.

[0109] The intermediate representation is input into the generative language model to obtain the front-end code corresponding to the design draft file output by the generative language model.

[0110] Optionally, the front-end code includes at least one of the following: HTML, CSS, and JavaScript.

[0111] In summary, the method provided in this application illustrates a method for converting a design draft file into an intermediate representation and then inputting the intermediate representation into a generative language model to ultimately obtain the front-end code corresponding to the design draft file. On one hand, this method enables the rapid conversion of design draft files into front-end code, eliminating the need for front-end developers to write front-end code step-by-step based on the design drafts created by product designers. This improves front-end code development efficiency, shortens the software development cycle, and reduces software development costs. Furthermore, for front-end developers, the step from design draft to front-end code is a simple but tedious task. This process often leads to minor program errors such as syntax or naming errors during the front-end code writing process, which are often difficult to debug. Therefore, having a computer generate front-end code from the design draft file improves the reliability and quality of the front-end code. In addition, quickly converting the design draft into runnable front-end code facilitates rapid software iteration and design verification.

[0112] On the other hand, in the process of generating front-end code, the front-end code is not directly generated based on the design draft file. Instead, the design draft file is first converted into an intermediate representation with a tree structure, and then the front-end code is generated based on this intermediate representation. By converting the design draft file into an intermediate representation, the design draft file is standardized, ensuring the consistency of the input to the generative language model. This allows the generative language model to better understand the input information, reducing ambiguity introduced by different types of design draft files and improving the reliability and accuracy of the front-end code generated by the generative language model. Furthermore, converting the design draft file into an intermediate representation before inputting it into the generative language model also makes the method provided in this application applicable to design draft files from different UI design tools. This ensures that the corresponding front-end code can be obtained for design draft files from different UI design tools, improving the versatility of the front-end code generation process.

[0113] Next, we will introduce the specific process of converting the design file into an intermediate representation.

[0114] Based on Figure 2 In an alternative embodiment, such as Figure 3 As shown, step 220 can be implemented as steps 221 to 223.

[0115] Step 221: Obtain the first UI element from at least one UI element in the design file; and parse the node attributes of the first UI element, wherein the node attributes of the first UI element satisfy the attribute specification of the intermediate representation.

[0116] Optionally, if the design file stores at least one UI element as a list, an element is retrieved from the list as the first UI element, such as retrieving the i-th element from the list. Alternatively, if the design file stores at least one UI element as a tree, the UI element corresponding to the root node of the tree is retrieved as the first UI element.

[0117] For example, at least one UI element stored as a list is shown below [1 [2, 3, 4 [5, 6]], 7]. The design file includes 7 UI elements. When reading the design file, the UI elements that can be read as the first UI element are UI element 1 and UI element 7. The remaining UI elements 2 to 6 cannot be read directly, but need to be obtained by traversing the child elements of UI element 1.

[0118] Optionally, the attribute specifications represented in the intermediate representation include at least one of the following: coordinate system; unit of measurement; color representation; whether nullables are allowed; default value; and attribute name naming conventions.

[0119] Optionally, the node attributes of the first UI element can be more or fewer than the attribute information of the first UI element. That is, during the parsing process, new attributes can be added to the node attributes of the first UI element, and these attributes do not exist in the attribute information of the first UI element; alternatively, some attribute information from the first UI element can be omitted from the node attributes of the first UI element; the attribute name and attribute value of the attribute information from the first UI element in the node attributes can also be changed, and so on.

[0120] Step 222: If the first UI element has at least one child element, traverse the at least one child element to obtain the node attributes of the at least one child element; based on the first UI element, the node attributes of the first UI element, the at least one child element, and the node attributes of the at least one child element, add a first subtree in the intermediate representation. The first subtree is used to indicate the first UI element and the at least one child element of the first UI element.

[0121] Optionally, if the first UI element has at least one child element, the at least one child element is recursively traversed to obtain its node attributes. The at least one child element includes at least one direct child element; or, the first UI element has at least two child elements, which include at least one direct child element and at least one indirect child element. A direct child element is a child element directly associated with the first UI element; an indirect child element is a child element associated with a child element of the first UI element, such as a child element associated with a direct child element or an indirect child element of the first UI element. Each UI element includes child element attributes. A child element directly associated with a UI element is a child element within that UI element's child element attributes, or a child element at the first level of the child element attributes, or a child element that can be directly read based on the child element attributes. A child element indirectly associated with a UI element is a child element at the nth level of the child element attributes, where n is a positive integer greater than 1; or a child element that cannot be directly read based on the UI element's child element attributes.

[0122] For example, at least one UI element stored as a list is shown below [1 [2, 3, 4 [5, 6]], 7]. The design file includes 7 UI elements. If the first UI element is UI element 1, then UI element 1 has child elements. Specifically, there are 3 direct child elements, namely UI element 2, UI element 3 and UI element 4, and 2 indirect child elements, namely UI element 5 and UI element 6. When iterating through at least one child element of UI element 1, the child elements that can be directly read from the child element properties of UI element 1 are considered direct children, namely UI element 2, UI element 3, and UI element 4. For example, first, the property information of UI element 2 is read, and it is determined whether the child element property of UI element 2 is empty, i.e., whether UI element 2 has child elements. Since UI element 2 has no child elements, only the property information of UI element 2 is read and returned. After returning the property information of UI element 2, since UI element 1 still has child elements whose property information has not been read, the property information of the next child element, i.e., the property information of UI element 3, is read. Since UI element 3 also has no child elements, the property information of UI element 3 is read and returned. The process continues to read the next child element... The attribute information of the elements, specifically the attribute information of UI element 4, is not directly returned after reading the attribute information of UI element 4, since UI element 4 has child elements. Instead, based on the child element attributes in the attribute information of UI element 4, the child elements of UI element 4, namely UI elements 5 and 6, are read. For example, first, the attribute information of UI element 5 is read. Since UI element 5 has no child elements, only the attribute information of UI element 5 is read and returned. Next, the attribute information of UI element 6 is read. Since UI element 6 also has no child elements, the attribute information of UI element 6 is read and returned. At this point, UI element 4 no longer has any child elements whose attribute information has not been read. Therefore, the attribute information of UI element 4 and its child elements (UI elements 5 and 6) is returned. After returning the attribute information of UI element 4 and its child elements, UI element 1 no longer has any child elements whose attribute information has not been read. Therefore, the attribute information of UI element 1 and its corresponding at least one child element is returned, completing the traversal of the child elements of UI element 1. It should be noted that the above attribute information needs to be parsed to obtain node attributes that satisfy the attribute specifications of the intermediate representation. This step is not shown in the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto.

[0123] It should be noted that the at least one UI element stored in the list manner described above can also be regarded as stored in a forest manner, that is, UI element 1 corresponds to a tree, UI element 7 corresponds to a tree. The embodiments of this application do not limit the name of the storage method, but no matter how the at least one UI element is stored, it can be converted into an intermediate representation of a tree structure using the method shown in the embodiments of this application.

[0124] In some embodiments, step 222 includes at least one of the following: if the first UI element has at least one direct child element, parse the node attributes of at least one direct child element, wherein the node attributes of at least one direct child element satisfy the attribute specification of the intermediate representation; if the first UI element has at least one indirect child element, parse the node attributes of at least one indirect child element, wherein the node attributes of at least one indirect child element satisfy the attribute specification of the intermediate representation; and add a first subtree in the intermediate representation based on at least two of the first UI element, the node attributes of the first UI element, and at least two of the node attributes of at least one direct child element, at least one direct child element, at least one indirect child element, and at least one indirect child element.

[0125] Optionally, if at least one child element is a direct child element, then step 222 above can be implemented as follows: parsing to obtain the node attributes of at least one direct child element; based on the first UI element, the node attributes of the first UI element, at least one direct child element, and the node attributes of at least one direct child element, adding a first subtree in the intermediate representation. The first subtree includes two levels: the first level of the first subtree is the node corresponding to the first UI element, and the second level of the first subtree is the node corresponding to at least one direct child element.

[0126] Optionally, the first UI element has at least two child elements, which include at least one direct child element and at least one indirect child element. Step 222 above can be implemented as follows: parsing to obtain the node attributes of at least one direct child element and at least one indirect child element; based on the first UI element, the node attributes of the first UI element, at least one direct child element, at least one child element's node attributes, at least one indirect child element, and at least one indirect child element's node attributes, a first subtree is added to the intermediate representation. The first subtree includes n+2 levels, where n is a positive integer. The first level of the first subtree is the node corresponding to the first UI element, the second level of the first subtree is the node corresponding to at least one direct child element, and the third to n+2 levels of the first subtree are the nodes corresponding to at least one indirect child element.

[0127] It should be noted that the first UI element, at least one child element, at least one direct child element, and at least one indirect child element mentioned above when adding the first subtree can be understood as an element identifier. This element identifier is used to uniquely identify the UI element in the intermediate representation. This element identifier is generated by the computer device when traversing the UI element.

[0128] Step 223: If the first UI element does not have any child elements, add the first node corresponding to the first UI element in the intermediate representation based on the first UI element and its node attributes.

[0129] Optionally, if the first UI element does not have any child elements, that is, if the first UI element is represented as a leaf node in the intermediate representation, then the first node corresponding to the first UI element can be added directly in the intermediate representation based on the first UI element and the node attributes of the first UI element.

[0130] In summary, the method provided in this application illustrates a method for converting design draft files into intermediate representations. This requires reading at least one UI element from the design draft file one by one and parsing the attribute information of at least one UI element into node attributes that satisfy the attribute specifications of the intermediate representation. That is, on the one hand, it deconstructs the storage structure of at least one UI element in the design draft file and converts it into a tree structure in the intermediate representation; on the other hand, it also needs to convert attribute information in the design draft file that does not conform to the attribute specifications of the intermediate representation into node attributes. This achieves a standardized expression of the design draft file. The standardized design draft file helps ensure the consistency of the input to the generative language model, enabling the generative language model to better understand the input information, reduce ambiguity introduced by different types of design draft files, and improve the reliability and accuracy of the front-end code generated by the generative language model.

[0131] Furthermore, a scenario is illustrated where at least one child element includes both direct and indirect child elements. Since front-end pages or user interfaces often have many functional sections, such as menu bars or carousels at different levels, it may be necessary to associate a single UI element with multiple levels of child elements. The intermediate representation shown in this application preserves the parent-child relationships of various UI elements in the design file, facilitating the generation of front-end code that conforms to the layout of the design file based on these parent-child relationships, thus improving the standardization and reliability of front-end code generation.

[0132] The following methods can be used to parse the node attributes of the first UI element and obtain the node attributes of at least one child element, as mentioned above.

[0133] • Parsing node attributes.

[0134] In some embodiments, the method further includes: obtaining attribute information of UI elements from a design file; performing element parsing based on the attribute information of the UI elements and the attribute specifications of the intermediate representation to determine the node attributes of the UI elements; wherein, element parsing includes at least one of the following: unified coordinate system; standardized color representation; unified size unit; standardized attribute name; attribute extraction and mapping; adding target attributes.

[0135] In some embodiments, the parsing of node attributes can be performed before the design file is converted into an intermediate representation, that is, before step 220. This can be understood as performing data cleaning on the design file before converting it to an intermediate representation. Alternatively, it can be performed during the process of converting the design file into an intermediate representation, that is, during the process of steps 221 to 223 above. This application embodiment does not limit this.

[0136] For example, for attribute information involving coordinates, such as position and angle, it is necessary to perform coordinate system transformation based on the standard coordinate system shown in the attribute specification of the intermediate representation and the position and angle information in the attribute information, so as to achieve coordinate system unification.

[0137] For example, for attribute information involving color, such as line color, fill color, and font color, it is necessary to convert the color representation shown in the intermediate attribute specification and the color information in the attribute information to achieve standardization of color representation. For example, converting the hexadecimal representation into a unified RGB representation.

[0138] For example, for attribute information involving dimensions, such as line length, shape size, and line thickness, it is necessary to perform size conversion based on the size unit shown in the attribute specification of the intermediate representation and the length, size, and thickness information in the attribute information, thereby achieving uniformity of size units. For example, for line thickness, 1 pound ≈ 0.35 millimeters.

[0139] For example, for attribute names in some attribute information, an attribute mapping table can be set in the attribute specification of the intermediate representation. Based on this attribute mapping table, the attribute names in the attribute information are converted into attribute names that satisfy the attribute specification of the intermediate representation. That is, based on the attribute names of the attribute information of the first UI element, the attribute names of the corresponding node attributes of the attribute information of the first UI element are retrieved from the attribute mapping table; based on the attribute names of the node attributes of the first UI element and the attribute values ​​of the attribute information of the first UI element, the node attributes of the first UI element are determined.

[0140] For example, the attribute specification of the intermediate representation includes the attribute names that need to be extracted and mapped. That is, the node attributes of the nodes in the intermediate representation can be some of the attribute information of the UI element, rather than all of the attribute information. Optionally, the attribute names that need to be extracted and mapped can be related to the type information of the UI element. For example, for a UI element with the type information of "text", the attribute names that need to be extracted and mapped can include at least one of the following: text content, font size, font, font weight, line height, character spacing, alignment, and text style. For a UI element with the type information of "image", the attribute names that need to be extracted and mapped can include at least one of the following: image source and fill method. For a UI element with the type information of "container", the attribute names that need to be extracted and mapped can include at least one of the following: layout method, arrangement direction, main axis alignment, cross axis alignment, and spacing.

[0141] For example, the attribute specification in the intermediate representation also includes target attributes corresponding to the target type. These target attributes are not explicitly displayed in the attribute information of the UI element, but need to be added accordingly based on the UI element's type information, element name, etc. That is, when the type information in the UI element's attribute information is the target type and / or the element name of the UI element is the target name, a target attribute is added to the UI element. Target attributes, such as attributes reflecting the UI element's interaction method, are used to indicate the interaction operation or trigger operation corresponding to the node. For example, for a container type UI element named a scroll container, a "scroll attribute" is added to the UI element; for a container type UI element named a grid container, a "grid attribute" is added to the UI element; for a container type UI element named a carousel container, a "swipe toggle attribute" is added to the UI element, and so on. That is, the attribute information of the first UI element includes at least one of type information and element name; when the type information of the first UI element is a target type, a target attribute is added to the node corresponding to the first UI element to determine the node attribute of the first UI element; when the element name of the first UI element is a target name, a target attribute is added to the node corresponding to the first UI element to determine the node attribute of the first UI element; when the type information of the first UI element is a target type and the element name of the first UI element is a target name, a target attribute is added to the node of the first UI element to determine the node attribute of the first UI element; wherein, the target type and / or target name are used to implicitly indicate the interaction method of the first UI element, and the target attribute is used to indicate the interaction method of the first UI element. Alternatively, through the above method, the implicit information in the design file is transformed into display information in the intermediate representation, thereby improving the generation efficiency and accuracy of the front-end code. Optionally, the interaction methods of UI elements, in addition to the interaction methods related to container-type UI elements mentioned above, also include click interactions and long-press interactions for button-type UI elements, hover interactions for image-type UI elements, etc., which are not limited in this embodiment.

[0142] In some embodiments, parsing the node attributes of the first UI element includes: obtaining attribute information of the first UI element from the design file; performing element parsing based on the attribute information of the first UI element and the attribute specification of the intermediate representation to determine the node attributes of the first UI element; wherein, element parsing includes at least one of the following: unified coordinate system; standardized color representation; unified size unit; standardized attribute name; attribute extraction and mapping; adding target attribute.

[0143] In some embodiments, parsing to obtain the node attributes of at least one direct child element includes: obtaining attribute information of the i-th direct child element from the design file, where i is a positive integer; performing element parsing based on the attribute information of the i-th direct child element and the attribute specification of the intermediate representation to determine the node attributes of the i-th direct child element; setting i = i + 1, until the node attributes of at least one direct child element are parsed; wherein, element parsing includes at least one of the following: unifying the coordinate system; standardizing color representation; unifying the size unit; standardizing attribute names; attribute extraction and mapping; adding target attributes.

[0144] In some embodiments, parsing to obtain the node attributes of at least one indirect child element includes: obtaining attribute information of the i-th indirect child element from the design file, where i is a positive integer; performing element parsing based on the attribute information of the i-th indirect child element and the attribute specification of the intermediate representation to determine the node attributes of the i-th indirect child element; setting i = i + 1, until the node attributes of at least one indirect child element are parsed; wherein, element parsing includes at least one of the following: unifying the coordinate system; standardizing color representation; unifying the size unit; standardizing attribute names; attribute extraction and mapping; adding target attributes.

[0145] In summary, the method provided in this application illustrates a node attribute parsing approach. It parses the attribute information of each UI element in the design draft file to obtain node attributes that meet the attribute specifications of the intermediate representation. This not only ensures the format specifications of the intermediate representation but also guarantees the attribute specifications, improving the standardization of the intermediate representation. The standardized design draft file helps ensure the consistency of the input to the generative language model, enabling the model to better understand the input information and reducing ambiguity introduced by different types of design draft files and attribute information with different specifications. This improves the reliability and accuracy of the front-end code generated by the generative language model. Especially regarding attribute information, the front-end code contains attributes that need to be generated based on the attribute information in the design draft file, such as layout and style. If generated directly based on the attribute information in the design draft file, the generative language model will have difficulty correctly mapping attribute information with different specifications to the corresponding front-end code. Alternatively, the training cost for correctly mapping attribute information with different specifications to the corresponding front-end code is high (because this requires targeted training on attribute information with different specifications to ensure that the generative language model can recognize different specifications). This significantly increases the application cost of the front-end code generation method. By using the attribute specification of the intermediate representation to parse the attribute information in the design file to obtain the node attributes of each UI element, the generative language model can then generate the front-end code based on these node attributes. In this case, the generative language model only needs to learn the attribute specification of the intermediate representation, which improves the accuracy of front-end code generation and reduces the training cost of the generative language model.

[0146] The following shows how to obtain the front-end code corresponding to the design file based on the intermediate representation.

[0147] • Generation of front-end code.

[0148] Based on Figure 2 In an alternative embodiment, such as Figure 4 As shown, step 230 can be implemented as steps 231 and 232.

[0149] Step 231: Based on the intermediate representation, obtain the generated prompt words.

[0150] Optionally, prompt words are constructed based on the intermediate representation. For example, the generated prompt words are shown below:

[0151] Please convert the intermediate representation of the following UI element into HTML and CSS code: "Intermediate Representation".

[0152] Require:

[0153] 1. Generate semantic HTML structure

[0154] 2. Use BEM naming conventions

[0155] 3. Ensure responsive layout >

[0156] The text within the angle brackets <> is the generated prompt word. "Intermediate representation" actually refers to a tree-structured code, which is represented by text here. However, the present application does not limit the display format of the intermediate representation.

[0157] Optionally, the generation prompt is used to indicate the generation specifications for the front-end code; in other words, the generation prompt includes an intermediate representation and the generation specifications. The generation specifications are as described above regarding semantic structure, meaning that HTML tag naming must meet semantic requirements. For example, for a navigation UI element, when generating its corresponding HTML code, a name related to navigation (navigation) is used. Similarly, child elements of the navigation UI element also use navigation-related naming, such as a list of elements in the navigation called `navigation__list`, list items in the list called `navigation__item`, links corresponding to list items called `navigation__link`, and so on.

[0158] Step 232: Input the generated prompt words into the generative language model to obtain the front-end code corresponding to the design file.

[0159] For example, the generated prompt words are input into a generative language model to obtain the front-end code output by the generative model. Taking the generated prompt words as an example, the obtained front-end code includes the HTML and CSS code corresponding to the design file.

[0160] In some embodiments, the method further includes: formatting the front-end code according to a code template to obtain front-end code that meets the code generation specifications.

[0161] Optionally, the code template includes at least one of the following: an HTML template, a CSS template, and a JS template. The code template is used to indicate the format of front-end code that conforms to code generation specifications. For example, the code template indicates the indentation method, maximum line length, use of blank lines, variable and function declarations, etc., to conform to code generation specifications. Indentation methods include using spaces or tabs (usually 2 or 4 spaces). Maximum line length is limited to a maximum length per line (e.g., no more than 80 or 120 characters) to improve code readability. Blank line usage includes using blank lines between logically related code blocks, between function and class definitions, etc. Variable and function declarations include avoiding declaring multiple variables on the same line, placing function and variable declarations at the top of the code, and arranging them in logical order, etc. Formatting the front-end code improves the readability and maintainability of the generated front-end code, making it easier for front-end developers to further check and adjust the generated front-end code, resulting in more maintainable front-end code.

[0162] In some embodiments, the method further includes: saving the front-end code as a front-end project, wherein the front-end project includes the file corresponding to the front-end code.

[0163] Optionally, the design file includes at least one user interface, each user interface corresponding to at least one piece of front-end code. Running this front-end code will generate the user interface. For each user interface, its corresponding front-end code is saved as at least one file, and these files are integrated to obtain the corresponding front-end project. Directly integrating the generated front-end code into a front-end project allows front-end developers to run and verify the generated code directly, eliminating the need for them to manually categorize, create, copy, and paste the front-end code for each user interface, and save the files. This further shortens development time and improves efficiency.

[0164] In summary, the method provided in this application illustrates the process of inputting intermediate representations into a generative language model to obtain front-end code. Specifically, prompt words are generated based on the intermediate representations, thereby limiting the front-end code output by the generative language model to meet the generation specifications. This allows the generated front-end code to be used directly, improving the standardization, readability, and maintainability of the front-end code.

[0165] Furthermore, the training method for generative language models is shown.

[0166] • Training of generative language models.

[0167] Based on Figure 2 In an alternative embodiment, such as Figure 5 As shown, the method further includes steps 310 to 330.

[0168] Step 310: Obtain the pre-trained generative language model.

[0169] Optionally, the generative language model is pre-trained, and the pre-training of the generative language model is carried out on a device with hardware environment that can support pre-training, using a large amount of corpus.

[0170] Optionally, the pre-trained generative language model can support the generation of front-end code based on intermediate representations.

[0171] Step 320: Construct the training dataset.

[0172] Optionally, a training dataset is constructed to meet the generation requirements of generating front-end code based on intermediate representations.

[0173] In some embodiments, the training dataset corresponds to at least one training task, which includes at least one of the following: a code generation task, a structure mapping task, and a style mapping task; the structure mapping task refers to the training task of training a generative language model to learn the mapping relationship between nodes in an intermediate representation and nodes in a document object model; the style mapping task refers to the training task of training a generative language model to learn the mapping relationship between node attributes of nodes in an intermediate representation and front-end attributes in front-end code; step 320 includes at least one of the following: when at least one training task includes a code generation task, obtaining a first intermediate representation and a first front-end code, determining the first intermediate representation and the first front-end code as training data, the first intermediate representation and the first front-end code corresponding to a design draft file; when at least one training task includes a structure mapping task, obtaining a second intermediate representation and a second front-end code, determining the second intermediate representation and the second front-end code as training data, the nodes in the second intermediate representation do not include node attributes, and the second front-end code does not include front-end attributes; when at least one training task includes a style mapping task, obtaining a third intermediate representation and a third front-end code, determining the third intermediate representation and the third front-end code as training data, the third intermediate representation includes node attributes of a first type of node, and the second front-end code includes front-end attributes corresponding to the first type of node.

[0174] Among them, front-end properties are attribute information in the front-end code, such as the tag attributes of tags in HTML code, style attributes in CSS code, and so on.

[0175] Optionally, in the case where the training dataset corresponds to a code generation task, step 320 can be implemented as follows: obtaining a first intermediate representation and a first front-end code, determining the first intermediate representation and the first front-end code as training data, and the first intermediate representation and the first front-end code corresponding to a design file.

[0176] For example, the code generation task is a training task to generate the corresponding front-end code for a design draft file. Therefore, the training data consists of a first intermediate representation corresponding to the design draft file and the first front-end code. Typically, the design draft file corresponds to a front-end page or user interface. The design draft file and the first front-end code can originate from an open-source project; or a project written by developers themselves; or the design draft file originates from an open-source project, while the first front-end code is written by developers based on the design draft file, and so on. The first intermediate representation is obtained by converting the design draft file based on the method shown in step 220 and its corresponding optional embodiments.

[0177] Optionally, the first front-end code is the standardized front-end code. That is, the first front-end code is adjusted according to the generation specifications.

[0178] Optionally, in the case of a structure mapping task corresponding to the training dataset, step 320 can be implemented as follows: obtaining a second intermediate representation and a second front-end code, determining the second intermediate representation and the second front-end code as training data, wherein the nodes in the second intermediate representation do not include node attributes, and the second front-end code does not include front-end attributes.

[0179] For example, the structure mapping task refers to the training task that trains a generative language model to learn the mapping relationship between nodes in the intermediate representation and nodes in the document object model. In other words, the structure mapping task is a training task focused on the mapping relationship between nodes in the intermediate representation and nodes in the document object model. The purpose of the structure mapping task is to enable the generative language model to learn how to map each node in the intermediate representation to different types of HTML tags. Therefore, the training data used for training does not include attribute-related information; that is, nodes in the second intermediate representation do not include node attributes, or in other words, the nodes do not have corresponding node attributes; the second front-end code does not include front-end attributes, or in other words, the second front-end code does not have the front-end attributes corresponding to the nodes in the document object model. It should be noted that the attributes corresponding to the nodes in the second intermediate representation and the nodes in the document object model refer to style attributes. That is, the aforementioned second intermediate representation and second front-end code can be understood as follows: the nodes in the second intermediate representation do not include the first node attributes, and the second front-end code does not include the first front-end attributes; or, the nodes in the second intermediate representation include the second node attributes, and the second front-end code includes the second front-end attributes. The first node attribute and the first front-end attribute are style-related attributes, such as line thickness, length, color, position, etc.; the second node attribute and the second front-end attribute are attributes related to the node or front-end code itself, such as node type, tag type, node identifier, etc.

[0180] For example, the structure mapping task enables generative language models to determine the corresponding tag type in the front-end code based on the type of the node in the intermediate representation. For instance, a node of type CONTAINER is mapped to an HTML tag of div or section; a node of type TEXT is mapped to an HTML tag of p or span; a node of type IMAGE is mapped to an HTML tag of img; a node with interactive features is mapped to an HTML tag of button or a, and so on.

[0181] Optionally, in the case of a style mapping task corresponding to the training dataset, step 320 can be implemented as follows: obtaining a third intermediate representation and a third front-end code, determining the third intermediate representation and the third front-end code as training data, wherein the third intermediate representation includes the node attributes of the first type of node, and the second front-end code includes the front-end attributes corresponding to the first type of node.

[0182] For example, the style mapping task refers to the training task that trains a generative language model to learn the mapping relationship between node attributes in the intermediate representation and front-end attributes in the front-end code. In other words, the style mapping task is a training task focused on the mapping relationship between node attributes in the intermediate representation and front-end attributes in the front-end code. The purpose of the style mapping task is to enable the generative language model to learn how to map node attributes to style attributes that satisfy CSS rules. CSS rules include, but are not limited to, naming conventions, size units, coordinate systems, etc.

[0183] Optionally, the style mapping task also includes training the generative language model to learn the mapping relationship between target type nodes in the intermediate representation and target front-end attributes in the front-end code. That is, for a target type node, the generative language model needs to learn to add target attributes to that node, or in other words, the generative language model needs to learn the front-end attributes implicitly indicated by the node's type information. For example, the generative language model needs to add target attributes according to the node's type information, node name, etc. In other words, when the node's type information is the target type and / or the node name is the target name, target attributes are generated for the tag or style corresponding to that node. For example, for a container type node named "Scroll Container," the front-end attribute corresponding to "Scroll Attribute" is added to that node; for a container type node named "Grid Container," the front-end attribute corresponding to "Grid Attribute" is added to that node; for a container type node named "Carousel Container," the front-end attribute corresponding to "Slide Toggle Attribute" is added to that node, and so on. Alternatively, without needing a node name, the generative language model adds target attributes to the node based on the node's type information and contextual information (such as the information corresponding to the node's parent node, sibling nodes, and child nodes).

[0184] Optionally, in the aforementioned code generation, structure mapping, and style mapping tasks, the generative language model can also learn how to generate tags and / or front-end code names corresponding to nodes according to naming conventions. That is, it can be trained using first, second, or third front-end code that conforms to the naming conventions, thereby enabling the generative language model to learn how to generate front-end code that meets the naming conventions.

[0185] Step 330: Train a generative language model based on the training dataset.

[0186] Optionally, the training dataset includes at least one training data point, and a generative language model is trained based on the at least one training data point.

[0187] In some embodiments, the training dataset includes at least one training data, and the at least one training data includes an intermediate representation to be trained and a front-end code to be trained; step 330 above includes: obtaining the intermediate representation to be trained and the front-end code to be trained; traversing at least one node in the intermediate representation to be trained; predicting the predicted label corresponding to each node in the at least one node based on the at least one node; calculating the model loss based on the predicted label corresponding to each node in the at least one node and the real label in the front-end code to be trained; and training a generative language model based on the model loss.

[0188] Optionally, the front-end code to be trained includes the HTML to be trained, and the HTML to be trained includes at least one real tag. Optionally, each of the at least one real tag includes tag attributes (also known as front-end attributes). Each of the at least one real tag may also not include tag attributes.

[0189] Optionally, after obtaining the intermediate representation to be trained and the front-end code to be trained, the intermediate representation to be trained and the front-end code to be trained are segmented and encoded to obtain a first feature vector and a second feature vector. The first feature vector is the feature vector corresponding to the intermediate representation to be trained, and the second feature vector is the feature vector corresponding to the front-end code to be trained. Based on the feature vector corresponding to the first node in at least one node in the intermediate representation to be trained, the predicted label corresponding to the first node is predicted. This process is repeated to obtain the predicted label corresponding to each node in at least one node. Based on the predicted label corresponding to at least one node, the corresponding predicted front-end code is obtained. Based on the predicted front-end code and the real front-end code in the front-end code to be trained, the model loss is calculated, and the generative language model is trained based on the model loss. The calculation of the model loss is based on the feature vector and the second feature vector of the predicted front-end code.

[0190] Optionally, the model loss includes at least one of the following: the loss between the predicted label and the true label; the loss between the predicted front-end code and the true front-end code. It should be noted that the design of the model loss can be based on the front-end code generation specifications. For example, in the case of a structure mapping task, when calculating the loss between the predicted front-end code and the true front-end code, a weighted cross-entropy loss can be used. For structures that the generative language model is expected to learn, the weights of the corresponding labels can be increased, etc. This application does not limit this aspect.

[0191] Optionally, a generative language model can be trained based on the model loss. For example, the model parameters of the generative language model can be updated using the backpropagation algorithm. That is, based on the model loss, the gradient vector of each parameter in the model parameters is calculated using the backpropagation algorithm, and the model parameters are updated in the opposite direction of the gradient vector to train the generative language model.

[0192] In some embodiments, at least one node includes node attributes; step 330 further includes: predicting the predicted front-end attribute corresponding to each node in the at least one node based on the node attributes of the at least one node; calculating the model loss based on the predicted front-end attribute corresponding to each node in the at least one node and the real front-end attribute in the front-end code to be trained; and training a generative language model based on the model loss.

[0193] Optionally, the predicted front-end attributes include at least one of the following: the tag attributes of the tag corresponding to each node; and the style attributes corresponding to each node. That is, in some embodiments, the generative language model also needs to learn whether node attributes should be generated as tag attributes or style attributes. In other words, the generative language model also needs to learn which node attributes should be generated as HTML code and which node attributes should be generated as CSS code.

[0194] Optionally, after obtaining the intermediate representation to be trained and the front-end code to be trained, the intermediate representation to be trained and the front-end code to be trained are segmented and encoded to obtain a first feature vector and a second feature vector. The first feature vector is the feature vector corresponding to the intermediate representation to be trained, and the second feature vector is the feature vector corresponding to the front-end code to be trained. At least one node in the intermediate representation to be trained has a corresponding node attribute. Based on the feature vector corresponding to the first node in at least one node in the intermediate representation to be trained, the predicted front-end attribute corresponding to the first node is predicted. This process is repeated to obtain the predicted front-end attribute corresponding to each node in at least one node. Based on the predicted front-end attribute corresponding to at least one node, the corresponding predicted front-end code is obtained. Based on the predicted front-end code and the real front-end code in the front-end code to be trained, the model loss is calculated, and a generative language model is trained based on the model loss. The calculation of the model loss is based on the feature vector and the second feature vector of the predicted front-end code.

[0195] Optionally, the model loss includes at least one of the following: the loss between predicted front-end attributes and real front-end attributes; the loss between predicted front-end code and real front-end code; and the loss between predicted labels and real labels.

[0196] In summary, the method provided in this application illustrates the training process of a generative language model, from the construction of the training dataset to the training of the generative language model based on the training dataset. This enables the generative language model to better generate compliant front-end code based on intermediate representations. By training the generative language model to generate front-end code, the trained generative language model can quickly convert design drafts into runnable front-end code, facilitating rapid software iteration and design verification. Furthermore, it can reduce errors that may be introduced during manual coding, ensuring consistency between the design draft and the final implementation.

[0197] Furthermore, the construction method of the training dataset is shown, with training datasets specifically built according to the training tasks. To enable the generative language model to learn how to generate corresponding front-end code based on the intermediate representation corresponding to the design draft, training data for the code generation task can be constructed. To enable the generative language model to learn how to map nodes in the intermediate representation to nodes specified in the HTML document object model (DOM), i.e., learning the generation specifications for various types of tags, training data for the structure mapping task can be constructed. To enable the generative language model to learn how to map node attributes of each node in the intermediate representation to front-end attributes, i.e., learning the mapping specifications between node attributes and front-end attributes, training data for the style mapping task can be constructed. This approach allows the generative language model to benefit from features shared across different tasks, learning more general representations and thus improving the model's generalization ability across tasks. Furthermore, some features are easy to learn in certain tasks but difficult to learn in others. This approach allows the generative language model to efficiently learn these features through different tasks, thereby improving the performance of front-end code generation.

[0198] Furthermore, the document demonstrates the specific process of training a generative language model based on a training dataset. This process includes node prediction and attribute prediction. Node prediction maps nodes in the intermediate representation to tags in the front-end code (represented by UI elements in the design file). Correct node mapping ensures the reliability and interactivity of the subsequently generated front-end code, thus guaranteeing the writing of subsequent business-related code. Attribute prediction maps node attributes in the intermediate representation to front-end attributes. It also generates corresponding front-end attributes based on information implicit in some node attributes, ensuring that the generated front-end code matches the design draft after execution.

[0199] The following section will further introduce the entire process of generating front-end code and training generative language models.

[0200] • The process of generating front-end code.

[0201] For example, such as Figure 6 As shown, the front-end code generation process mainly includes two stages: design draft parsing and code generation.

[0202] (1) Analysis of the design draft.

[0203] Design draft parsing is the process of converting design drafts from UI design tools into structured data that can be processed later.

[0204] Step 1: Input the design draft.

[0205] To support the input formats of various design tools, a specific parser is written for each tool. These parsers convert the file formats of different UI design tools into a unified intermediate representation (IR). In other words, different parsing methods are used for different UI design tool types 30, as shown below.

[0206] For UI design tool 1: UI design tool 1 provides an API (Application Programming Interface), so we use parsing method one to obtain the JSON data of the design draft (i.e., the design draft file) through the API.

[0207] For UI Design Tool 2: The design file exported by UI Design Tool 2 is a compressed package containing multiple JSON files. Therefore, parsing method two is adopted to obtain the design file by decompressing and parsing these JSON files.

[0208] For UI-related tool 3: The design draft exported by UI design tool 3 is also a compressed package containing multiple XML files. Therefore, parsing method 3 is adopted to obtain the design draft file by decompressing and parsing these XML files.

[0209] Although the above method can parse design files 31 from different UI design tools, the formats and data representations of UI elements in different types of design files 31 are different. Therefore, it is necessary to perform a data processing / cleaning operation first to convert the raw data from different sources into a unified intermediate representation.

[0210] Step 2: Unify the AST structure.

[0211] The design file 31 is input into the AST converter 32 to obtain a unified AST structure (i.e., a unified intermediate representation).

[0212] To simplify subsequent processing, the parsing results from different design tools are converted into a unified intermediate representation. This intermediate representation can be a tree structure containing all UI elements and their attributes.

[0213] class UIElement: # Defines the UI element class

[0214] def __init__(self, element_type, properties, children = None): # Initialization method for UI element classes

[0215] self.element_type = element_type # Element type

[0216] self.properties = properties # element properties

[0217] self.children = children; if children else[] #list of child elements

[0218] class DeSignDocument: # Defines the design document class

[0219] def__init__(self):

[0220] self.elements = [] # List of elements in the design document

[0221] def add_element(self, element):

[0222] self.elements.append(element) # Add elements to the design document

[0223] For example, the process of using the AST converter 32 to convert the design file 31 into an intermediate representation includes at least one of the following: standardization processing; hierarchical structure conversion; attribute extraction and mapping.

[0224] • Standardization Processing Stage: After obtaining the design draft file 31, standardization processing is required. This includes at least one of the following:

[0225] - Unified coordinate system: Convert the coordinate systems of different tools into a unified standard;

[0226] -Standardized color representation: Use RGBA or HEX format uniformly;

[0227] - Standardize size units: Convert all dimensions to pixels (px);

[0228] - Standardize attribute names: Use a consistent attribute naming convention.

[0229] • Hierarchical structure transformation: Converting the original hierarchical structure into a standard tree structure, including at least one of the following:

[0230] -Establish parent-child node relationships;

[0231] - Handling component and instance relationships;

[0232] -Merge similar layers;

[0233] - Handling special container types.

[0234] • Attribute extraction and mapping: Extract and map attributes for each UI element. The attributes to be extracted must include at least one of the following:

[0235] -Basic attributes: ID, name, type, etc.;

[0236] - Style attributes: color, border, shadow, etc.;

[0237] - Layout properties: position, size, alignment, etc.;

[0238] -Special attributes: text style, interactive state, etc.

[0239] The above-described component and instance relationships are designed for component reuse mechanisms in UI design tools. In some UI design tools, a component is a reusable template, while an instance is a specific use of that component. A template can create different instances in different places on the page. In the intermediate representation, the following is required:

[0240] -Preserve component definition: Record all properties and structure of the original component;

[0241] - Mark instance association: Add a reference to the original component in the instance node;

[0242] - Handle property overriding: Record the properties that are overridden in the instance.

[0243] For example:

[0244]

[0245]

[0246] Optionally, the component's properties can be directly referenced in the instance node, or they can be stored separately in other files.

[0247] For example, taking an intermediate representation as an AST structure, the final template of the intermediate representation is as follows: Figures 7 to 9 As shown, where, Figure 7 This represents the attribute information corresponding to the AST tree, with each branch representing different attribute information of a node in the AST tree. Figure 8 This refers to the attribute information corresponding to the layout attributes of a node in the AST tree. Figure 9 This refers to the attribute information corresponding to the unique attributes of a node in an AST tree.

[0248] The core attributes include:

[0249] -id: A unique identifier for the node, used to uniquely locate the node throughout the entire AST.

[0250] -type: Node type, such as container, text, image, button, etc.

[0251] -name: Node name, usually derived from the layer name in the design tool.

[0252] Layout properties include:

[0253] -position: Defines the position and size information of the node.

[0254] -constraints: Defines the constraints on a node within its parent container.

[0255] -padding / margin: Defines the inner and outer margins of the node.

[0256] Style attributes include:

[0257] -fills: Defines the fill style of the node, which can be solid color, gradient, etc.

[0258] -borders: Defines the border style of the node.

[0259] -shadows: Defines the shadow effect of the node.

[0260] Specific / proprietary attributes include:

[0261] -text: A proprietary attribute of text nodes, containing text content and formatting information.

[0262] -image: A proprietary attribute of image nodes, containing information about the image source and display method.

[0263] -container: A proprietary attribute of the container node that contains layout information.

[0264] This unified AST structure design takes into account: completeness, ensuring that the AST structure contains all the necessary information in the design file; scalability, supporting the easy addition of new attributes and types; understandability, with intuitive attribute naming and a clear structural hierarchy; and universality, suitable for data conversion between different design tools.

[0265] The following is a brief introduction to the AST converter 32 for a UI design tool 1, which can also be called the UI design tool 1 parser.

[0266]

[0267]

[0268] The following is a brief introduction to the AST converter 32 for a UI design tool 2, which can also be called the UI design tool 2 parser.

[0269]

[0270] The format of the design file 31 corresponding to UI design tool 2 is shown below. In UI design tool 2, each UI element in design file 31 is displayed in the form of layers. The design file includes a list of pages, and each page includes various layers displayed hierarchically.

[0271]

[0272] It should be noted that the attributes of nodes and pages can refer to the node attribute types included in the above AST, but there will be certain differences in definition and representation. This application does not limit these differences.

[0273] (2) Code generation.

[0274] After obtaining a unified AST structure, it can be input into a generative language model 33, which will generate at least one of HTML 34, CSS 35, and JS 36, depending on the prompt words input by the user. For example, if the prompt words input by the user are to generate HTML and CSS, then the generative speech model will only generate HTML 34 and CSS 35.

[0275] For example, the AST corresponding to one of the product cards is input into the generative language model 33. First, the AST structure is converted into a prompt word format that the generative language model 33 can understand. This process includes serializing the AST structure and adding necessary contextual information.

[0276] For example, the front-end page (or webpage) corresponding to the product card is as follows: Figure 10 As shown in design draft 50, which includes an image and text, this design draft 50 is converted into a unified intermediate representation, namely an AST tree 51. This AST tree includes three nodes: the root node is a container node, and the root node has two connected child nodes, one of which is an image and the other is text. This AST tree 51 is then input into the generative language model 33 for HTML generation 34 and CSS generation 35, resulting in front-end code 52. The front-end code 52 includes the HTML and CSS code corresponding to design draft 50.

[0277] Generating front-end code using a generative language model offers the following advantages: correct HTML semantic structure; appropriate accessibility attributes; compliance with SEO best practices; clear class naming; and reasonable style organization.

[0278] During code generation, the generative language model analyzes the AST structure and generates the corresponding code. This process is recursive, starting from the root node and processing each node layer by layer:

[0279] First, the generative language model identifies the node type and selects the appropriate HTML tag. For example:

[0280] -The CONTAINER type is usually converted to div or section;

[0281] -TEXT type is converted to p or span;

[0282] - Convert IMAGE type to img;

[0283] - Special interactive components may be converted to buttons or a tags.

[0284] For each node, the generative language model analyzes its style attributes and generates the corresponding CSS rules:

[0285] - Handle layout properties (position, flex, etc.);

[0286] - Handle visual styles (colors, shadows, etc.);

[0287] - Handle interactive states (hover, active, etc.).

[0288] The code generation process described above is automated, but generative language models consider several factors to ensure the quality of the generated code: the selection of semantic tags; the organization and reuse of styles; the implementation of responsive design; browser compatibility; and code maintainability.

[0289] Optionally, after generating the front-end code, it can be further standardized through code integration 37, code formatting 38, and dependency handling 39 to output the final code 40. Code formatting 38 is used to format the front-end code to meet the code generation specifications. Dependency handling 39 is used to add corresponding dependency declarations based on the packages, classes, objects, etc. used by the front-end code, so that the code can automatically import its corresponding dependencies at runtime, thereby supporting the direct execution of the generated front-end code.

[0290] For example, based on predefined HTML, CSS, and JavaScript templates, the generated front-end code is normalized to ensure that the generated code conforms to best practices and standards.

[0291] For example, the generated HTML, CSS, and JavaScript code are integrated into a complete front-end project. First, the output directory is checked and created if it doesn't exist. The generated code is then written into its corresponding files: index.html, styles.css, and script.js.

[0292] • The training process of generative language models.

[0293] Specifically, such as Figure 11 As shown.

[0294] (1) Analysis of the design draft.

[0295] The process of parsing the design draft is similar to that shown in "·Generating Front-end Code" above, and will not be repeated here.

[0296] (2) Model training.

[0297] Model training includes building a dataset, training a generative language model (using GPT as an example here), and optimizing the model so that it can understand the design drafts and generate the corresponding front-end code.

[0298] Step 3: Construct the training dataset.

[0299] Collect a large number of design drafts and their corresponding front-end code as training data. Data can be collected from the following aspects:

[0300] - Open source projects: Collect design drafts and front-end code from open source projects.

[0301] - Exporting from design tools: Export the design draft using design tools (such as Figma, Sketch, Adobe XD) and manually write the corresponding front-end code.

[0302] - Synthetic Data: Generates synthetic design drafts and front-end code.

[0303] The collected design drafts and front-end code 41 are converted into a format suitable for model training. The intermediate representation (IR) of the design drafts and the front-end code 41 are converted into training data pairs 42, which are used as training data. The specific code implementation is shown below.

[0304]

[0305] Step 4: Train the generative language model.

[0306] For example, using a Generative Pre-trained Transformer (GPT) model, we train the GPT model to understand the design and generate the corresponding front-end code. Below is an example of training a model using Hugging Face's Transformers library:

[0307]

[0308]

[0309] The above training process is to enable the GPT model to learn and understand the AST structure 43 and learn the code mapping relationship 44.

[0310] Model fine-tuning can improve the accuracy of model generation and the quality of code. Model fine-tuning refers to further training a pre-trained model using task-specific data.

[0311] Fine-tuning the model is similar to the initial training process, but a smaller learning rate and longer training time can be used so that the model can better adapt to the data for the specific task.

[0312] During fine-tuning, model evaluation needs to be performed45 to continuously assess the model's performance, and model optimization should be performed based on the evaluation results46. A validation set can be used to evaluate the quality of the generated model.

[0313] In summary, the method provided in this application has the following beneficial effects:

[0314] 1. Improve development efficiency

[0315] - Automated code generation.

[0316] - Reduce manual coding: By automatically generating HTML, CSS, and JavaScript code, it reduces the time and effort developers need to manually write code.

[0317] - Rapid prototyping: It can quickly convert design drafts into runnable front-end code, facilitating rapid iteration and design verification.

[0318] 2. Maintain design consistency

[0319] - A consistent coding style.

[0320] - Predefined templates: Use predefined code templates to ensure that the generated code conforms to best practices and standards, maintaining consistency in code style.

[0321] - Reduce human error: Automatic code generation can reduce errors that may be introduced during manual coding, ensuring consistency between the design and the final implementation.

[0322] 3. Improve code quality

[0323] - In accordance with best practices.

[0324] - Standardized code: By using predefined templates and rules, the generated code conforms to best practices in front-end development, improving code quality.

[0325] - Maintainability: The generated code has a clear structure and is easy to maintain and extend.

[0326] 4. Improve collaboration efficiency

[0327] - Seamless integration of design and development.

[0328] -Simultaneous design and development: Designers can directly convert design drafts into code, reducing communication costs and time between design and development.

[0329] - Rapid feedback: Developers can quickly generate code and test it, promptly identify and resolve issues, and improve team collaboration efficiency.

[0330] 5. Flexibility and scalability

[0331] - Customization and extensions.

[0332] -Custom templates: You can customize code templates according to project requirements to generate front-end code that meets specific needs.

[0333] - Extended functionality: The system can be extended to support more design elements and interaction logic to meet the needs of different projects.

[0334] 6. Improve user experience

[0335] -Respond quickly to needs.

[0336] - Rapid iteration: Through automation tools, we can quickly respond to changes in user needs and carry out rapid iteration and updates.

[0337] - Consistent User Experience: Ensure consistency between design and implementation to provide a consistent user experience.

[0338] 7. Reduce costs

[0339] - Saves manpower and time.

[0340] - Reduce repetitive work: Automation tools can reduce repetitive manual coding work and save labor costs.

[0341] - Shorten the development cycle: Improve development efficiency, shorten the project development cycle, and reduce project costs.

[0342] Please refer to Figure 12 This diagram illustrates a structural block diagram of a front-end code generation apparatus provided in an exemplary embodiment of this application. The apparatus has the functionality to implement the aforementioned example of a soft front-end code generation method; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the computer device described above, or it can be installed within a computer device. Figure 12 As shown, the device may include: an acquisition module 410, a conversion module 420, and a generation module 430.

[0343] The acquisition module 410 is used to acquire a design draft file, wherein the design draft file is used to indicate at least one UI element and attribute information of each UI element in the at least one UI element;

[0344] The conversion module 420 is used to convert the design file into an intermediate representation stored in a tree structure, wherein each node in the intermediate representation corresponds to a UI element in the at least one UI element, and the intermediate representation is used to standardize the representation of different types of design files;

[0345] The generation module 430 is used to input the intermediate representation into the generative language model to obtain the front-end code corresponding to the design file.

[0346] In some embodiments, the conversion module 420 includes an acquisition submodule and an addition submodule.

[0347] A submodule is configured to obtain a first UI element from at least one UI element in the design file; and parse the node attributes of the first UI element, wherein the node attributes of the first UI element satisfy the attribute specification of the intermediate representation.

[0348] A submodule is added to traverse the at least one child element when the first UI element has at least one child element, and obtain the node attributes of the at least one child element; based on the first UI element, the node attributes of the first UI element, the at least one child element and the node attributes of the at least one child element, a first subtree is added to the intermediate representation, and the first subtree is used to indicate the first UI element and the at least one child element of the first UI element.

[0349] Adding a submodule is also used to add a first node corresponding to the first UI element in the intermediate representation based on the first UI element and the node attributes of the first UI element when the first UI element does not have child elements.

[0350] In some embodiments, a submodule is added, including a parsing unit and an adding unit.

[0351] The parsing unit is used to parse the node attributes of the at least one direct child element when the first UI element has at least one direct child element, wherein the at least one direct child element satisfies the attribute specification of the intermediate representation.

[0352] The parsing unit is used to parse the node attributes of the at least one indirect child element when the first UI element has at least one indirect child element, wherein the node attributes of the at least one indirect child element satisfy the attribute specification of the intermediate representation.

[0353] An adding unit is used to add the first subtree to the intermediate representation based on at least two of the first UI element, the node attributes of the first UI element, and the node attributes of the at least one direct child element, the at least one direct child element, the at least one indirect child element, and the at least one indirect child element.

[0354] In some embodiments, the acquisition submodule is further configured to acquire attribute information of the first UI element from the design file; perform element parsing based on the attribute information of the first UI element and the attribute specification of the intermediate representation to determine the node attributes of the first UI element; wherein the element parsing includes at least one of the following: unified coordinate system; standardized color representation; unified size unit; standardized attribute name; attribute extraction and mapping; adding target attribute.

[0355] In some embodiments, the generation module 430 is further configured to obtain a generation prompt word based on the intermediate representation; and input the generation prompt word into the generative language model to obtain the front-end code corresponding to the design file.

[0356] In some embodiments, the device further includes a training module.

[0357] The training module is used to obtain a pre-trained generative language model; construct a training dataset; and train the generative language model based on the training dataset.

[0358] In some embodiments, the training dataset includes at least one training data, the at least one training data including an intermediate representation to be trained and a front-end code to be trained; the training module is further configured to acquire the intermediate representation to be trained and the front-end code to be trained; traverse at least one node in the intermediate representation to be trained; predict the predicted label corresponding to each of the at least one node based on the at least one node; calculate the model loss based on the predicted label corresponding to each of the at least one node and the real label in the front-end code to be trained; and train the generative language model based on the model loss.

[0359] In some embodiments, the at least one node includes node attributes; the training module is further configured to predict the predicted front-end attribute corresponding to each node in the at least one node based on the node attributes of the at least one node; calculate the model loss based on the predicted front-end attribute corresponding to each node in the at least one node and the real front-end attribute in the front-end code to be trained; and train the generative language model based on the model loss.

[0360] In some embodiments, the training dataset corresponds to at least one training task, which includes at least one of the following: a code generation task, a structure mapping task, and a style mapping task; the structure mapping task refers to a training task that trains the generative language model to learn the mapping relationship between nodes in the intermediate representation and nodes in the document object model; the style mapping task refers to a training task that trains the generative language model to learn the mapping relationship between node attributes in the intermediate representation and front-end attributes in the front-end code; the training module is further configured to, when the at least one training task includes a code generation task, obtain a first intermediate representation and a first front-end code, and determine the first intermediate representation and the first front-end code as... The training data includes a design file corresponding to the first intermediate representation and the first front-end code. When at least one training task includes a structure mapping task, a second intermediate representation and a second front-end code are obtained, and the second intermediate representation and the second front-end code are determined as the training data. The nodes in the second intermediate representation do not include node attributes, and the second front-end code does not include front-end attributes. When at least one training task includes a style mapping task, a third intermediate representation and a third front-end code are obtained, and the third intermediate representation and the third front-end code are determined as the training data. The third intermediate representation includes node attributes of a first type of node, and the second front-end code includes front-end attributes corresponding to the first type of node.

[0361] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0362] Figure 13 A structural block diagram of a server provided in an exemplary embodiment of this application is shown.

[0363] The server 800 includes a central processing unit (CPU) 801, a system memory 804 including random access memory (RAM) 802 and read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. The server 800 also includes a basic input / output system (I / O system) 806 to facilitate information transfer between various devices within the server, and a mass storage device 807 for storing the operating system 813, application programs 814, and other program modules 815.

[0364] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 for user input, such as a mouse or keyboard. Both the display 808 and the input device 809 are connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include the input / output controller 810 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.

[0365] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable storage media provide non-volatile storage for the server 800. That is, the mass storage device 807 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0366] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 804 and mass storage device 807 described above can be collectively referred to as memory.

[0367] The memory stores one or more programs, which are configured to be executed by one or more central processing units 801. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 801 executes the one or more programs to implement the methods provided by the above method embodiments.

[0368] According to various embodiments of this application, the server 800 can also be connected to a remote computer device on a network, such as the Internet. That is, the server 800 can be connected to a network 812 via a network interface unit 811 connected to the system bus 805, or the network interface unit 811 can be used to connect to other types of networks or remote computer device systems (not shown).

[0369] The memory further includes one or more programs stored in the memory, and the one or more programs include steps executed by the server in the method provided in the embodiments of this application.

[0370] Figure 14 A structural block diagram of a terminal provided in an exemplary embodiment of this application is shown.

[0371] The terminal 900 can be a portable mobile terminal, also referred to as a mobile terminal in this embodiment. Examples include smartphones, tablets, MP3 players (Moving Picture Experts Group Audio Layer III), and MP4 players (Moving Picture Experts Group Audio Layer IV). The terminal 900 may also be referred to as user equipment, portable terminal, or other names.

[0372] Typically, terminal 900 includes a processor 901 and a memory 902.

[0373] Processor 901 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 901 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0374] The memory 902 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement the front-end code generation method provided in the embodiments of this application.

[0375] In some embodiments, the terminal 900 may also optionally include: a peripheral device interface 903 and at least one peripheral device. Specifically, the peripheral device includes at least one of: a radio frequency circuit 904, a touch display screen 905, a camera 906, an audio circuit 907, and a power supply 908.

[0376] Peripheral device interface 903 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 901 and memory 902. In some embodiments, processor 901, memory 902 and peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 901, memory 902 and peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0377] The radio frequency (RF) circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 904 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0378] The touch display screen 905 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. The touch display screen 905 also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to the processor 901 for processing. The touch display screen 905 is used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one touch display screen 905, located on the front panel of the terminal 900; in other embodiments, there may be at least two touch display screens, respectively located on different surfaces of the terminal 900 or in a folded design; in some embodiments, the touch display screen 905 may be a flexible display screen, located on a curved or folded surface of the terminal 900. Furthermore, the touch display screen 905 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The touch display screen 905 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0379] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is used for video calls or selfies, and the rear-facing camera is used for taking photos or videos. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, and a wide-angle camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, and panoramic shooting and VR (Virtual Reality) shooting by fusion of the main camera and the wide-angle camera. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0380] Audio circuit 907 provides an audio interface between the user and terminal 900. Audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to processor 901 for processing, or input to radio frequency circuit 904 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of terminal 900. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from processor 901 or radio frequency circuit 904 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, audio circuit 907 may also include a headphone jack.

[0381] Power supply 908 is used to power the various components in terminal 900. Power supply 908 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 908 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0382] In some embodiments, the terminal 900 further includes one or more sensors 909. The one or more sensors 909 include, but are not limited to, an accelerometer 910, a gyroscope 911, a pressure sensor 912, an optical sensor 913, and a proximity sensor 914.

[0383] Accelerometer 910 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by terminal 900. For example, accelerometer 910 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 901 can control touch screen 905 to display the user interface in landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 910. Accelerometer 910 can also be used for games or for acquiring user motion data.

[0384] The gyroscope sensor 911 can detect the orientation and rotation angle of the terminal 900. The gyroscope sensor 911, in conjunction with the accelerometer sensor 910, can collect the user's 3D movements on the terminal 900. Based on the data collected by the gyroscope sensor 911, the processor 901 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0385] The pressure sensor 912 can be disposed on the side bezel of the terminal 900 and / or on the lower layer of the touch display screen 905. When the pressure sensor 912 is disposed on the side bezel of the terminal 900, it can detect the user's grip signal on the terminal 900 and perform left / right hand recognition or quick operation based on the grip signal. When the pressure sensor 912 is disposed on the lower layer of the touch display screen 905, it can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 905. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0386] An optical sensor 913 is used to collect ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the touch screen 905 based on the ambient light intensity collected by the optical sensor 913. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 913.

[0387] The proximity sensor 914, also known as a distance sensor, is typically located on the front of the terminal 900. The proximity sensor 914 is used to detect the distance between the user and the front of the terminal 900. In one embodiment, when the proximity sensor 914 detects that the distance between the user and the front of the terminal 900 is gradually decreasing, the processor 901 controls the touchscreen display 905 to switch from a screen-on state to a screen-off state; when the proximity sensor 914 detects that the distance between the user and the front of the terminal 900 is gradually increasing, the processor 901 controls the touchscreen display 905 to switch from a screen-off state to a screen-on state.

[0388] Those skilled in the art will understand that Figure 14 The structure shown does not constitute a limitation on terminal 900 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0389] In an exemplary embodiment, this application provides a chip that includes programmable logic circuits and / or program instructions. When the chip is run on a computer device, it is used to implement the front-end code generation method provided in the above method embodiments.

[0390] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the aforementioned method for generating front-end code.

[0391] In an exemplary embodiment, a computer program product is also provided, which, when executed by a processor, is used to implement the aforementioned method for generating front-end code.

[0392] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0393] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating front-end code, characterized in that, The method includes: Obtain a design file, which indicates at least one UI element and attribute information of each UI element. The design file is converted into an intermediate representation stored in a tree structure. Each node in the intermediate representation corresponds to one UI element in the at least one UI element. The intermediate representation is used to standardize the representation of different types of design files. The intermediate representation is input into the generative language model to obtain the front-end code corresponding to the design file.

2. The method according to claim 1, characterized in that, The process of converting the design file into an intermediate representation stored in a tree structure includes: Obtain a first UI element from at least one UI element in the design file; and parse the node attributes of the first UI element, wherein the node attributes of the first UI element satisfy the attribute specification of the intermediate representation; If the first UI element has at least one child element, traverse the at least one child element to obtain the node attributes of the at least one child element; based on the first UI element, the node attributes of the first UI element, the at least one child element and the node attributes of the at least one child element, add a first subtree to the intermediate representation, the first subtree being used to indicate the first UI element and at least one child element of the first UI element; If the first UI element does not have any child elements, a first node corresponding to the first UI element is added to the intermediate representation based on the first UI element and its node attributes.

3. The method according to claim 2, characterized in that, When the first UI element has at least one child element, the at least one child element is traversed to obtain the node attributes of the at least one child element; based on the first UI element, the node attributes of the first UI element, the at least one child element, and the node attributes of the at least one child element, a first subtree is added to the intermediate representation, including at least one of the following: When the first UI element has at least one direct child element, the node attributes of the at least one direct child element are parsed and the node attributes of the at least one direct child element satisfy the attribute specification of the intermediate representation. When the first UI element has at least one indirect child element, the node attributes of the at least one indirect child element are parsed and the node attributes of the at least one indirect child element satisfy the attribute specification of the intermediate representation. The first subtree is added to the intermediate representation based on the first UI element, the node attributes of the first UI element, and at least two of the at least one direct child element, the node attributes of the at least one direct child element, the at least one indirect child element, and the node attributes of the at least one indirect child element.

4. The method according to claim 2 or 3, characterized in that, The parsing process yields the node attributes of the first UI element, including: Obtain the attribute information of the first UI element from the design file; Based on the attribute information of the first UI element and the attribute specification of the intermediate representation, element parsing is performed to determine the node attributes of the first UI element; The element parsing includes at least one of the following: unified coordinate system; standardized color representation; unified size unit; standardized attribute name; attribute extraction and mapping; and adding target attribute.

5. The method according to claim 4, characterized in that, The element parsing includes normalized attribute names; the attribute specification of the intermediate representation includes an attribute mapping table, which is used to indicate the mapping relationship between attribute information and node attributes; The step of performing element parsing based on the attribute information of the first UI element and the attribute specification of the intermediate representation to determine the node attributes of the first UI element includes: Based on the attribute name of the attribute information of the first UI element, the attribute name of the node attribute of the first UI element is retrieved from the attribute mapping table; The node attributes of the first UI element are determined based on the attribute name of the node attribute of the first UI element and the attribute value of the attribute information of the first UI element.

6. The method according to claim 4, characterized in that, The element parsing includes adding target attributes; the attribute information of the first UI element includes at least one of type information and element name; The step of performing element parsing based on the attribute information of the first UI element and the attribute specification of the intermediate representation to determine the node attributes of the first UI element includes at least one of the following: If the type information of the first UI element is the target type, add the target attribute to the node corresponding to the first UI element to determine the node attribute of the first UI element; If the element name of the first UI element is the target name, add the target attribute to the node corresponding to the first UI element to determine the node attribute of the first UI element. When the type information of the first UI element is the target type and the element name of the first UI element is the target name, the target attribute is added to the node of the first UI element to determine the node attribute of the first UI element. Wherein, the target type and / or the target name are used to implicitly indicate the interaction mode of the first UI element, and the target attribute is used to indicate the interaction mode of the first UI element.

7. The method according to any one of claims 1 to 6, characterized in that, The intermediate representation is input into a generative language model to obtain the front-end code corresponding to the design file, including: Based on the intermediate representation, the generated prompt words are obtained; The generated prompt words are input into the generative language model to obtain the front-end code corresponding to the design file.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Obtain a pre-trained generative language model; Build the training dataset; The generative language model is trained based on the training dataset.

9. The method according to claim 8, characterized in that, The training dataset includes at least one training data, and the at least one training data includes an intermediate representation to be trained and a front-end code to be trained. The process of training the generative language model based on the training dataset includes: Obtain the intermediate representation to be trained and the front-end code to be trained; Traverse at least one node in the intermediate representation to be trained; Based on the at least one node, predict the predicted label corresponding to each of the at least one node; The model loss is calculated based on the predicted label corresponding to each node in the at least one node and the real label in the front-end code to be trained. The generative language model is trained based on the model loss.

10. The method according to claim 9, characterized in that, The at least one node includes node attributes; The method further includes: Based on the node attributes of the at least one node, predict the predicted front-end attribute corresponding to each of the at least one node; The model loss is calculated based on the predicted front-end attributes corresponding to each of the at least one node and the real front-end attributes in the front-end code to be trained. The generative language model is trained based on the model loss.

11. The method according to any one of claims 8 to 10, characterized in that, The training dataset corresponds to at least one training task, and the at least one training task includes at least one of the following: code generation task, structure mapping task, and style mapping task; the structure mapping task refers to the training task of training the generative language model to learn the mapping relationship between nodes in the intermediate representation and nodes in the document object model; the style mapping task refers to the training task of training the generative language model to learn the mapping relationship between node attributes in the intermediate representation and front-end attributes in the front-end code. The construction of the training dataset includes at least one of the following: When the at least one training task includes a code generation task, a first intermediate representation and a first front-end code are obtained, and the first intermediate representation and the first front-end code are determined as training data, wherein the first intermediate representation and the first front-end code correspond to a design file. In the case where the at least one training task includes a structure mapping task, a second intermediate representation and a second front-end code are obtained, and the second intermediate representation and the second front-end code are determined as the training data, wherein the nodes in the second intermediate representation do not include node attributes, and the second front-end code does not include front-end attributes. In the case where the at least one training task includes a style mapping task, a third intermediate representation and a third front-end code are obtained, and the third intermediate representation and the third front-end code are determined as the training data. The third intermediate representation includes node attributes of a first type of node, and the second front-end code includes front-end attributes corresponding to the first type of node.

12. A front-end code generation device, characterized in that, The device includes: An acquisition module is used to acquire a design draft file, wherein the design draft file is used to indicate at least one UI element and attribute information of each UI element in the at least one UI element; The conversion module is used to convert the design file into an intermediate representation stored in a tree structure. Each node in the intermediate representation corresponds to a UI element in the at least one UI element. The intermediate representation is used to standardize the representation of different types of design files. The generation module is used to input the intermediate representation into the generative language model to obtain the front-end code corresponding to the design draft file.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the front-end code generation method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method for generating front-end code as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, and a processor reads from and executes the computer program to implement the front-end code generation method as described in any one of claims 1 to 11.

Images