UI design file generation method and device, storage medium and program product

Abstract

The embodiment of the invention provides a UI design file generation method and device, a storage medium and a program product. In the embodiment of the invention, a mode of generating the UI design file based on the image is provided, the original image is understood through the front-end code generation model, the HTML page matched with the original image is generated, and the front-end code corresponding to the HTML page is output, so that the front-end code output by the model is converted into the UI design file. In the whole process, the visual design is quickly generated through the original image, the generation time of the UI design file is remarkably shortened, the working efficiency is improved, the generation process is simplified, even non-professional personnel can easily operate, the dependence on professional design software and designers is reduced, the manpower and time cost is reduced, and the resource utilization efficiency is improved. In addition, the structure and the style of the HTML page are automatically adjusted and generated according to the original image through the front-end code generation model, and the consistency and the attractiveness of the design are ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, storage medium and program product for generating a UI design file. Background Art

[0002] Currently, UI (User Interface) designers can perform corresponding UI design work based on UI design tools (such as Sketch). For example, Sketch is a vector drawing application suitable for UI designers, and UI designers can design UI pages on Sketch. Among them, when designing UI pages on UI design tools, UI designers hand-draw sketches according to user needs, convert the sketches into wireframes, select corresponding components from the component library and drag them into the line drawing to generate UI design files corresponding to the UI page.

[0003] However, this UI page provision method requires a high level of professionalism from UI designers. For non-professionals, there are problems such as high learning costs and complex operations, which affect the user experience. Summary of the invention

[0004] Multiple aspects of the present application provide a UI design file generation method, device, storage medium and program product to reduce dependence on professional design software and designers, reduce manpower and time costs in the design process, and improve resource utilization efficiency.

[0005] An embodiment of the present application provides a method for generating a UI design file, comprising: acquiring an original image, the original image including original elements; generating a first prompt word according to the original image and a first prompt word template, the first prompt word template including first task description information for describing a task that a front-end code generation model needs to perform; calling the front-end code generation model according to the first prompt word to generate a target hypertext markup language HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information, wherein the target visual elements included in the target HTML page are adapted to the original elements; and mapping the target front-end code into a target UI design file corresponding to the target HTML page according to a mapping relationship between visual elements and layer elements, wherein the target layer elements included in the target UI design file are adapted to the target visual elements.

[0006] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement each step in the UI design file generation method provided in the embodiment of the present application.

[0007] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements each step in the UI design file generation method provided in the embodiment of the present application.

[0008] An embodiment of the present application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the processor implements the steps in the UI design file generation method provided in the embodiment of the present application.

[0009] In an embodiment of the present application, a method for generating a UI design file based on an image is provided, wherein the original image is understood by a front-end code generation model, an HTML page adapted to the original image is generated, and the front-end code corresponding to the HTML page is output, thereby converting the front-end code output by the model into a UI design file. In the whole process, the visual design is quickly generated through the original image, which not only significantly reduces the generation time of the UI design file and improves work efficiency, but also simplifies the generation process of the UI design file, so that even non-professional designers can easily get started, reduce dependence on professional design software and designers, reduce manpower and time costs in the design process, and improve resource utilization efficiency. In addition, the structure and style of the generated HTML page are automatically adjusted according to the original image through the front-end code generation model to ensure the consistency and aesthetics of the design and reduce human errors in the design process. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1a A flowchart of a method for generating a UI design file provided for an exemplary embodiment of the present application;

[0012] Figure 1b A schematic diagram of an original image, an HTML page with visual element size information, and an HTML page obtained by parsing a UI design file provided by an exemplary embodiment of the present application;

[0013] Figure 2a A schematic diagram of a UI design file generation page provided for an exemplary embodiment of the present application;

[0014] Figure 2b A schematic diagram of another UI design file generation page provided for an exemplary embodiment of the present application;

[0015] Figure 2c A schematic diagram of another UI design file generation page provided for an exemplary embodiment of the present application;

[0016] Figure 2d A schematic diagram of another UI design file generation page provided for an exemplary embodiment of the present application;

[0017] Figure 3 A schematic diagram of a front-end code generation model and a front-end code update model that cooperate with each other to generate a UI design file provided by an exemplary embodiment of the present application;

[0018] Figure 4 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0021] The various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0022] In view of the above technical problems, in an embodiment of the present application, a method for generating UI design files based on images is provided, in which the original image is understood by the front-end code generation model, an HTML page adapted to the original image is generated, and the front-end code corresponding to the HTML page is output, thereby converting the front-end code output by the model into a UI design file. In the whole process, the visual design is quickly generated through the original image, which not only significantly reduces the generation time of the UI design file and improves work efficiency, but also simplifies the generation process of the UI design file. Even non-professional designers can easily get started, reduce dependence on professional design software and designers, reduce manpower and time costs in the design process, and improve resource utilization efficiency. In addition, the structure and style of the generated HTML page are automatically adjusted according to the original image through the front-end code generation model to ensure the consistency and aesthetics of the design and reduce human errors in the design process.

[0023] A solution provided by an embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0024] Figure 1a A flowchart of a method for generating a UI design file is provided for an exemplary embodiment of the present application. Figure 1a As shown, the method includes:

[0025] 101. Obtain an original image, where the original image includes original elements;

[0026] 102. Generate a first prompt word according to the original image and a first prompt word template, where the first prompt word template includes first task description information, which is used to describe a task that needs to be performed by the front-end code generation model;

[0027] 103. Calling a front-end code generation model according to the first prompt word to generate a target hypertext markup language HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information, wherein the target visual elements included in the target HTML page are adapted to the original elements;

[0028] 104. According to the mapping relationship between the visual elements and the layer elements, the target front-end code is mapped into a target UI design file corresponding to the target HTML page, and the target layer elements included in the target UI design file are adapted to the target visual elements.

[0029] In this embodiment, Figure 1a The execution subject of the method shown can be a terminal device, for example, a desktop computer, a laptop computer, a smart phone or an IOT (Internet of Things) device and a smart wearable device (such as a smart watch, a smart bracelet), etc.

[0030] In this embodiment, the original image is an image that has the need to generate a UI page. The original image is not limited. For example, the original image may include but is not limited to: hand-drawn sketches, UI page screenshots, and images from other sources. Images from other sources may include but are not limited to: images downloaded from the Internet, images taken by a camera or terminal device, etc. The original image can be the above-mentioned various image sources, which greatly expands the user's usage scenarios and facilitates different types of design needs.

[0031] The original image includes original elements, which are basic elements constituting the original image and can define the image structure and content of the original image. The original elements can be visually independent objects. For example, the original elements can include but are not limited to: text, symbols, lines, and graphics. The original elements correspond to attribute information, such as attribute information that can include but is not limited to: color, shape, texture, shadow, and line thickness.

[0032] The present embodiment provides a front-end code generation model for generating front-end code for an original image. The front-end code generation model can be implemented as a large language model (LLM). For example, the front-end code generation model includes but is not limited to: a generative model (GPT) based on a transformer and a pre-trained model (BERT) based on a transformer.

[0033] Among them, the front-end code generation model corresponds to a first prompt word template, and the first prompt word template includes at least first task description information, and the first task description information is used to describe the task that the front-end code generation model needs to perform. In addition, the first prompt word template can also include: role information and requirement information. Role information refers to the role played by the front-end code generation model. For example, the role information can be a front-end code developer, and the requirement information is a requirement for the front-end code generation model to generate a Hypertext Markup Language (HTML) page and its corresponding front-end code.

[0034] In this embodiment, the first prompt word can be generated according to the original image and the first prompt word template. The implementation method of generating the first prompt word according to the original image and the first prompt word template is not limited. For example, the original image is filled into the first prompt word template in a local storage location to obtain the first prompt word. For another example, the original image feature is encoded to obtain a coded feature, and the feature code is filled into the first prompt word template to obtain the first prompt word. The method of encoding the original image is not limited. For example, the encoding method may include but is not limited to: base64 encoding, lossy compression encoding, lossless compression encoding, and image feature encoding. Base64 encoding is a method of converting binary data into a string format. Lossy compression is an encoding method that reduces the image size by reducing part of the information in the original image, such as JPEG format. Lossless compression is an encoding method that reduces the image size without losing image quality, such as PNG and GIF formats. Image feature encoding is a method of extracting feature information from an original image, such as HOG (Histogram of Oriented Gradients), SIFT (Scale Invariant Feature Transform), etc.

[0035] The following is an exemplary display of the first prompt word.

[0036] Example A1:

[0037] ##Role information: Front-end code developer.

[0038] ##Description of the first task:

[0039] Please generate an HTML page based on the original image provided. Please restore the layout and style of the original image in the HTML page as accurately as possible, and output the front-end code corresponding to the HTML page.

[0040] The following is the Base64 encoding of the original image: [Base64 encoded string];

[0041] Please make sure the HTML page structure is clear. If there is text content in the image, please try to identify it and embed it correctly into the HTML page.

[0042] Example A2:

[0043] ##Role information: Front-end code developer.

[0044] ##Description of the first task:

[0045] You use Cascading Style Sheets (CSS), HTML, and scripting languages ​​(JavaScript, JS) to build an HTML page based on the original image provided by the user. You need to constantly update the built HTML page so that the style and layout of the HTML page are close to the original image;

[0046] Returns the complete opening and closing tags (such as,) tags, which indicates that the output includes all content from the beginning to the end, that is, the front-end code of the entire HTML page.

[0047] ## Request Information

[0048] - Make sure that the original image matches the generated HTML page in terms of background color, text color, font size, font family, padding, margins, and borders.

[0049] - The HTML page includes the text information in the original image.

[0050] -All functionality and content can be fully implemented, rather than having comments to indicate "more elements should be here". For example, if an HTML page needs a navigation bar, all the navigation links can be written directly, rather than adding comments to indicate that more links can be added as needed.

[0051] - If the original image shows multiple original elements of the same type (e.g., news items or list items, etc.), then write the corresponding HTML code according to the actual number of original elements. For example, if there are 15 news items, then there are 15 corresponding independent content tags in the front-end code of the HTML page (e.g., <article>) or container label (e.g. ) to represent each entry.

[0052] - For images in the original image, use placeholder information for the image and in the image tag (e.g., ) to provide a detailed description for the subsequent generation of images in HTML pages. The alt attribute is a type of attribute information used to provide alternative text for an image.

[0053] In this embodiment, the front-end code generation model is called according to the first prompt word. In the front-end code generation model, the target HTML page corresponding to the original image and its corresponding target front-end code are generated according to the first task description information. This process includes the extraction, classification and reconstruction of the original elements to ensure that the generated HTML page can accurately reflect the user's design intent.

[0054] Wherein, the target HTML page includes target visual elements. The target visual elements refer to the elements presented on the target HTML page, and the target visual elements may include but are not limited to: text, pictures, buttons, and form controls, etc. These visual elements are defined by the HTML structure and styled by CSS to ensure that they are displayed in the expected manner. The target visual elements can accurately reflect the design intent in the original image. Wherein, the target visual elements included in the target HTML page are adapted to the original elements in the original image. Adaptation means that in the process of converting the original elements into visual elements, the design intent and visual effects of the original image are accurately reflected in the target HTML page. For example, adaptation can be implemented to maintain the visual consistency of the original elements and the visual elements, ensuring that the visual elements in the HTML page (such as attribute information such as color, font, spacing, etc.) are consistent with the original elements in the original image. For another example, the interactive elements (such as buttons, links, form controls, etc.) in the static original image are converted into HTML elements, CSS elements or JavaScript elements with actual functions, etc.

[0055] In this embodiment, the UI design file is a file used to store various elements on the UI page. The UI design file may include layers, components, styles, layouts, and other design elements. The UI design file may use a variety of file formats. For example, the UI design file may use but is not limited to: ".sketch" and ".psd" and other file formats. Among them, ".sketch" is a vector drawing file format. ".psd" is an image editing file format that supports complex image editing functions such as multiple layers, masks, adjustment layers, special effects, etc.

[0056] Among them, the UI design file includes layer elements, which are the basic units in a UI design file and are used to build and organize visual designs. Each layer represents an independent object or component, and the properties of each layer element can be adjusted individually without affecting other layer elements. Layer elements may include but are not limited to: shape layers, text layers, image layers, vector layers, effect layers, and combined layers. Among them, shape layers may include but are not limited to: geometric shape layers such as rectangles, ellipses, and polygons. Text layers are used to add text content. Image layers are used for imported pictures or drawn bitmaps. Vector layers are used for graphics created using vector paths, which are easy to scale without distortion. Effect layers are used to apply visual effects such as shadows, glows, and blurs. Combined layers are used to group multiple layers together so that they can be moved or adjusted together.

[0057] There is a mapping relationship between visual elements and layer elements. This mapping relationship refers to the process of mapping the visual expression in web design (defined by HTML and CSS) to the layer structure in graphic design tools (such as Sketch). This mapping relationship specifically involves parsing the layout, style and content of HTML, creating corresponding layer elements, and building UI design files.

[0058] In this embodiment, the target front-end code can be mapped to the target UI design file corresponding to the target HTML page according to the mapping relationship between the visual elements and the layer elements. Among them, the target layer elements included in the target UI design file are adapted to the target visual elements. Among them, adaptation means that in the process of converting the target visual elements into the target layer elements, the design intent and visual effects of the target HTML page are accurately reflected in the target UI design file. For example, adaptation can be implemented to maintain the visual consistency between the target layer elements and the target visual elements, ensuring that the visual elements in the HTML page (such as color, font, spacing and other attribute information) are consistent with the target layer elements in the target UI design file.

[0059] In the case of generating a target UI design file, the target UI design file can be stored, for example, locally in the terminal. When needed, the target UI design file can be parsed by a UI design application deployed on the terminal device to obtain an HTML page. Furthermore, in the case of generating an HTML page and a front-end code by the front-end code generation model, the size information of each visual element can be output on the HTML page for the debugger to fine-tune the size information.

[0060] The following is an exemplary display of the original image, the HTML page with the size information of the visual elements, and the HTML page obtained by parsing the target UI design file. Figure 1b As shown. Figure 1b In the example, the size of the visual element is 600×300, but the illustration is not limited thereto. The text information included in the original image and the HTML page are respectively: "Fresh Fruit Selection", "Nutritional Combination" and "Direct Supply from Orchard".

[0061] In an embodiment of the present application, a method for generating a UI design file based on an image is provided, wherein the original image is understood by a front-end code generation model, an HTML page adapted to the original image is generated, and the front-end code corresponding to the HTML page is output, thereby converting the front-end code output by the model into a UI design file. In the whole process, the visual design is quickly generated through the original image, which not only significantly reduces the generation time of the UI design file and improves work efficiency, but also simplifies the generation process of the UI design file, so that even non-professional designers can easily get started, reduce dependence on professional design software and designers, reduce manpower and time costs in the design process, and improve resource utilization efficiency. In addition, the structure and style of the generated HTML page are automatically adjusted according to the original image through the front-end code generation model to ensure the consistency and aesthetics of the design and reduce human errors in the design process.

[0062] In an optional embodiment, the implementation method for obtaining the original image is not limited. For example, the storage path of the original image can be pre-set, and the original image for which the UI design file needs to be generated can be stored in the storage path. In the case where a UI design file needs to be generated, the original image can be automatically obtained based on the storage path. For another example, a UI design file generation page is provided in an embodiment of the present application, and the UI design file generation page can be displayed, and the UI design file generation page includes an image upload control; the user can upload the original image through the image upload control, and the terminal device can obtain the original image in response to the upload operation on the image upload control.

[0063] Optionally, the UI design file generation page further includes: a page generation control and a model selection control. The user can initiate a model selection operation on the model selection control to select a front-end code generation model for generating an HTML page and trigger the page generation control. The terminal device can respond to the trigger operation on the page generation control, generate a first prompt word according to the original image and the first prompt word template, and call the front-end code generation model according to the first prompt word to generate a target HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information.

[0064] like Figure 2a As shown, a UI design file generation page is exemplarily shown, which includes: a model selection control 201, an image upload control 202, and a page generation control 203. Further, it also includes: a file export control 204. Among them, when the file export control is triggered, the terminal device can respond to the triggering operation of the file export control 204 and store the target UI design file in a local specified path for subsequent use by the user.

[0065] In the subsequent use process, the target UI design file can be parsed by the UI design application on the terminal device to convert the target UI design file into an HTML page and publish it for viewing by users.

[0066] Figure 3 The exemplary embodiment of the present application provides an example of a front-end code generation model and a front-end code update model that cooperate with each other to generate a target UI design file. The front-end code generation model and the front-end code update model can be implemented as the same model or different models. Figure 3 In the example, the front-end code generation model and the front-end code update model are implemented as different models.

[0067] In an optional embodiment, when an HTML page is generated, a target HTML page can also be displayed. When a page adjustment is required for the target HTML page, the page adjustment is performed on the target HTML page through the front-end code update model. The front-end code update model is used to adjust the front-end code of the target HTML page to achieve the purpose of adjusting the target HTML page. Among them, the front-end code update model corresponds to a second prompt word template, and the second prompt word template is used to describe the task that the front-end code update model needs to perform. The second prompt word template includes second task description information, and the second task description information indicates how to adjust the target HTML page.

[0068] Among them, a target HTML page can be displayed, and a target HTML page has multiple target visual elements; a user can initiate an adjustment operation for any target visual element, and accordingly, in response to the adjustment operation for any target visual element, attribute adjustment description information for any target visual element can be obtained; a second prompt word is generated according to the attribute adjustment description information of any target visual element and a second prompt word template; a front-end code update model is called according to the second prompt word to adjust at least part of the front-end code associated with any target visual element in the front-end code according to the second task description information and the attribute adjustment description information to obtain the adjusted front-end code, and an adjusted target HTML page corresponding to the adjusted front-end code is displayed; according to the mapping relationship between the visual element and the layer element, the adjusted front-end code is mapped into an adjusted target UI design file.

[0069] Among them, the implementation method of mapping the adjusted front-end code into the adjusted target UI design file according to the mapping relationship between visual elements and layer elements is not limited. Please refer to the introduction of "Mapping the target front-end code into the target UI design file corresponding to the target HTML page according to the mapping relationship between visual elements and layer elements", which will not be repeated here.

[0070] The attribute adjustment description information is a description information that defines how to adjust the attribute information of any target visual element, and the attribute adjustment description information may include but is not limited to: style adjustment description information, layout adjustment description information, and hierarchical relationship adjustment description information. For example, the style adjustment description information may be "adjust the background to blue" or "change text A1 to text A2", etc. For another example, the layout adjustment description information may be "extend the length of the entire page", etc. The hierarchical relationship adjustment description information may be "adjust information B from category C to category B", etc.

[0071] Among them, the implementation method of generating the second prompt word according to the adjustment description information and the second prompt word template of any target visual element is not limited. For example, the second prompt word template includes a blank prompt word, the adjustment description information can be filled into the blank prompt word, and the second prompt word template filled with the adjustment description information is used as the second prompt word. For another example, the second task description information includes placeholder information corresponding to the adjustment description information, the adjustment description information can be filled into the placeholder information, and the second prompt word template filled with the adjustment description information is used as the second prompt word.

[0072] One implementation of calling the front-end code update model according to the second prompt word may be inputting the second prompt word into the front-end code update model.

[0073] Among them, according to the second task description information and the attribute adjustment description information, at least part of the front-end code associated with any target visual element is adjusted in the front-end code to obtain the adjusted front-end code. The implementation method is not limited. For example, according to the second task description information and the attribute adjustment description information, the original attribute information of any target visual element is adjusted in the front-end code to obtain the target attribute information of any target visual element to obtain the adjusted front-end code. For example, in the front-end code update model, according to the attribute adjustment description information, the identification information of the visual element to be adjusted in the front-end code is determined, and the original attribute information of the visual element to be adjusted is obtained based on the identification information; according to the attribute adjustment description information, the original attribute information is adjusted to update the front-end code. Among them, the identification information of the visual element to be adjusted can be the user identifier (User Identifier, UID) of the visual element.

[0074] Optionally, in response to an editing operation on any target visual element, the implementation method for obtaining the attribute description information for any target visual element is not limited. The following provides a specific implementation method, in response to a trigger operation on any target visual element, an input box corresponding to any target visual element is displayed; the trigger operation can be a mouse click operation or a screen touch operation, which is not limited. The user can initiate an input operation on the input box, and accordingly, the terminal device can respond to the input operation on the input box and use the text information in the input box as the attribute adjustment description information for any target visual element. Among them, the input operation is not limited. For example, when an input box is displayed on a page, the user can enter the attribute adjustment description information into the input box, and the input box corresponds to a confirmation control. When the confirmation control is triggered, the terminal device can respond to the trigger operation on the confirmation control and determine to initiate the input operation on the input box. Figure 2b As shown, an example of displaying a target HTML page 205 on a UI design file generation page is provided. The target HTML page 205 is implemented as a basic information page, which includes: four configuration controls: photo, nickname, skill certificate, and work information, a save control, and information completion progress information, such as 25%. Among them, when the photo 2051 is triggered, it can be displayed Figure 2c As shown. Figure 2c In the example, the input box corresponding to the photo 2051 is displayed, which includes a text input area, a confirmation control, and a cancel control. The user can enter "change the background to gray" in the input box and click the confirmation control. In response to the click operation on the confirmation control, the terminal device calls the front-end code to update the model, changes the background information of the visual element to which the photo belongs to to gray, and displays Figure 2d In this embodiment, an intuitive and easy-to-use user interface is designed, which allows users to easily input original images, generate UI design files, and adjust properties such as styles and layouts, thereby lowering the usage threshold and improving the user experience. Furthermore, the adjustment effect of the HTML page can be fed back in real time, further optimizing the design process and ensuring that the final output meets expectations.

[0075] In an optional embodiment, the implementation method of converting the target front-end code into a target UI design file corresponding to the target HTML page according to the mapping relationship between visual elements and layer elements is not limited. An implementation method of converting the target front-end code into a target UI design file corresponding to the target HTML page according to the mapping relationship between visual elements and layer elements includes: parsing the target front-end code to obtain a document structure tree and node attribute information corresponding to the target HTML page, wherein the document structure tree includes multiple nodes corresponding to multiple target visual elements; generating multiple layer elements according to the multiple nodes included in the document structure tree and the mapping relationship between visual elements and layer elements; generating layer attribute information of the multiple layer elements according to the node attribute information; determining the hierarchical relationship between the multiple layer elements according to the hierarchical relationship between the multiple nodes; generating the target UI design file corresponding to the target HTML page according to the hierarchical relationship between the multiple layer elements, the multiple layer elements and their layer attribute information.

[0076] The document structure tree is a hierarchical node tree that is parsed and constructed for the target front-end code. The document structure tree represents the relationship between the target visual elements in the target HTML page, wherein the multiple nodes included in the document structure tree correspond to the multiple target visual elements included in the target HTML page, for example, one node may correspond to one target visual element. For example, the document structure tree may be implemented as a Document Object Model Tree (DOM) tree.

[0077] For example, the nodes included in the document structure tree can be implemented as tags in the front-end code, and the nodes may include but are not limited to:

[0078] 1) Tags used to define different parts of a page, such as the head tag <header>, Navigation Tags <nav>, Main content tags <main>, Chapter Tags <section>And article tags <article>etc. tags.

[0079] 2) Used as a container or inline element to wrap other content, such as a block-level element tag and inline elements .

[0080] 3) Title tags, such as <h1> to< / h1> <h6>, paragraph tags, such as, .

[0081] 4) Form elements are labels used for user interaction, such as <button>, <input> , <textarea> wait.< / textarea>

[0082] Among them, multiple layer elements can be generated according to multiple nodes included in the document structure tree and the mapping relationship between visual elements and layer elements. For example, one node can be mapped to one layer element, or one node can be mapped to multiple layer elements, or multiple nodes can be mapped to one layer element, and this is not limited.

[0083] For example, a node is implemented as a title tag ( <header>), the node can be mapped to a text layer.

[0084] For another example, a visual element corresponding to a node includes multiple sub-elements, each of which corresponds to a layer element, such as a Visual elements can have multiple sub-elements, such as images, paragraph text, or other nested , that is, a node can be mapped into multiple layer elements.

[0085] For another example, multiple visual elements may be mapped into a layer element, wherein the multiple visual elements include a title, content text, a button, etc. The multiple visual elements are added to a toolbar, and the toolbar is mapped into a layer element.

[0086] The layer attribute information of multiple layer elements can be generated according to the node attribute information. The node attribute information can include but is not limited to: style information and layout information, etc. For example, the node attribute information can be implemented as CCS information.

[0087] There is a hierarchical relationship between multiple nodes, which refers to the subordinate relationship between nodes, and the hierarchical relationship corresponds to the hierarchical relationship between target visual elements. For example, a target visual element can contain multiple child elements, and the target visual element itself is a child element of its parent element. Multiple child elements under the same parent element are sibling elements. Based on this, the hierarchical relationship between multiple layer elements can be determined according to the hierarchical relationship between multiple nodes. For example, the hierarchical relationship between multiple layer elements can be determined based on the corresponding relationship between multiple nodes and multiple layer elements.

[0088] Optionally, the implementation method of generating a target UI design file corresponding to a target HTML page according to the hierarchical relationship between multiple layer elements, multiple layer elements and their layer attribute information is not limited. The following provides an implementation method of generating a target UI design file corresponding to a target HTML page according to multiple layer elements and their style information and hierarchical relationship. The attribute information of multiple target layer elements is converted into UI design description information using a structured syntax format, and the UI design description information is provided to the server, so that the server can serialize the UI design description information to obtain and return the target UI design file corresponding to the target HTML page.

[0089] The structured syntax format may include, but is not limited to, Domain Specific Language-JavaScript Object Notation (JSON), Ordered Graph Data Language (OGDL), or YAML, etc.

[0090] Serialization refers to the process of converting UI design description information into a format that can be stored or transmitted. For example, if the target UI design file is in binary format, the UI design description information can be converted into the target UI design file by some binary serialization method. Alternatively, a serialization tool that converts the UI design description information into a UI design file can be deployed on the server side, and the UI design description information can be input into the serialization tool for serialization to obtain a UI design file.

[0091] Regarding the embodiments of the present application Figure 1a The detailed implementation and beneficial effects of each step in the method have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0092] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 103 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0093] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0094] Figure 4 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown in FIG. Figure 4 As shown, the device includes: a memory 44 and a processor 45.

[0095] The memory 44 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device.

[0096] The processor 45 is coupled to the memory 44 and is used to execute the computer program in the memory 44, so as to: obtain an original image, the original image includes original elements; generate a first prompt word according to the original image and a first prompt word template, the first prompt word template includes first task description information, which is used to describe the task that the front-end code generation model needs to perform; call the front-end code generation model according to the first prompt word to generate a target hypertext markup language HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information, the target visual elements included in the target HTML page are adapted to the original elements; according to the mapping relationship between the visual elements and the layer elements, map the target front-end code into a target UI design file corresponding to the target HTML page, the target layer elements included in the target UI design file are adapted to the target visual elements.

[0097] In an optional embodiment, the processor 45 is also used to: display a target HTML page, the target HTML page has multiple target visual elements; in response to an adjustment operation on any target visual element, obtain attribute adjustment description information for any target visual element; generate a second prompt word based on the attribute adjustment description information of any target visual element and a second prompt word template, the second prompt word template including second task description information for describing the task that needs to be performed by the front-end code update model; call the front-end code update model based on the second prompt word to adjust at least part of the front-end code associated with any target visual element in the front-end code based on the second task description information and the attribute adjustment description information to obtain the adjusted front-end code, and display the adjusted target HTML page corresponding to the adjusted front-end code; and map the adjusted front-end code into an adjusted target UI design file based on the mapping relationship between visual elements and layer elements.

[0098] In an optional embodiment, when the processor 45 obtains the attribute description information for any target visual element in response to an editing operation on any target visual element, it is specifically used to: display an input box corresponding to any target visual element in response to a trigger operation on any target visual element; and use the text information in the input box as the attribute adjustment description information for any target visual element in response to an input operation on the input box.

[0099] In an optional embodiment, when the processor 45 converts the target front-end code into a target UI design file corresponding to the target HTML page according to the mapping relationship between the visual elements and the layer elements, it is specifically used to: parse the target front-end code to obtain a document structure tree and node attribute information corresponding to the target HTML page, wherein the multiple nodes included in the document structure tree correspond to the multiple target visual elements; generate multiple layer elements according to the multiple nodes included in the document structure tree and the mapping relationship between the visual elements and the layer elements; generate layer attribute information of the multiple layer elements according to the node attribute information; determine the hierarchical relationship between the multiple layer elements according to the hierarchical relationship between the multiple nodes; and generate the target UI design file corresponding to the target HTML page according to the hierarchical relationship between the multiple layer elements, the multiple layer elements and their layer attribute information.

[0100] In an optional embodiment, when the processor 45 generates a target UI design file corresponding to a target HTML page based on multiple layer elements and their style information and hierarchical relationships, it is specifically used to: use a structured syntax format to convert the attribute information of multiple target layer elements into UI design description information, and provide the UI design description information to the server so that the server can serialize the UI design description information to obtain and return the target UI design file corresponding to the target HTML page.

[0101] In an optional embodiment, when acquiring the original image, the processor 45 is specifically used to: display a UI design file generation page, which includes an image upload control and a page generation control; in response to an upload operation on the image upload control, acquire the original image; before generating the first prompt word based on the original image and the first prompt word template, the method also includes: in response to a trigger operation on the page generation control, generate the first prompt word based on the original image and the first prompt word template.

[0102] In an optional embodiment, when the processor 45 generates the first prompt word according to the original image and the first prompt word template, it is specifically used to: encode the original image to obtain the encoding feature; and fill the encoding feature into the first prompt word template to obtain the first prompt word.

[0103] Regarding the embodiments of the present application Figure 4 The detailed implementation and beneficial effects of each step in the illustrated device have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0104] Further, if Figure 4 As shown, the electronic device also includes: a communication component 46, a display 47, a power component 48, an audio component 49 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 In addition, Figure 4 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT (Internet of Things) device and a smart wearable device (such as a smart watch, a smart bracelet), or a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 4 If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 4 Components within the dashed box.

[0105] Accordingly, the present application also provides a computer-readable storage medium storing a computer program, which can achieve the above-mentioned Figure 1a The steps in the method embodiment shown may be executed by an electronic device.

[0106] Accordingly, the embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, causes the processor to implement the above Figure 1a The steps in the method embodiment shown may be executed by an electronic device.

[0107] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0108] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0109] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0110] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0111] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0112] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0113] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0114] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0116] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0117] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0118] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0119] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0120] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application. < / header> < / button> < / h6> < / article> < / section> < / main> < / nav> < / header> < / article>

Claims

1. A method for generating a UI design file, characterized in that: include: Acquire an original image, wherein the original image includes an original element; generating a first prompt word according to the original image and a first prompt word template, wherein the first prompt word template includes first task description information, which is used to describe a task that needs to be performed by a front-end code generation model; The front-end code generation model is called according to the first prompt word to generate a target hypertext markup language HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information, wherein the target visual elements included in the target HTML page are adapted to the original elements; According to the mapping relationship between visual elements and layer elements, the target front-end code is mapped into a target UI design file corresponding to the target HTML page, and the target layer elements included in the target UI design file are adapted to the target visual elements.

2. The method according to claim 1, characterized in that Also includes: Displaying the target HTML page, the target HTML page having multiple target visual elements; In response to an adjustment operation on any target visual element, acquiring attribute adjustment description information on the any target visual element; Generate a second prompt word according to the attribute adjustment description information of any target visual element and the second prompt word template, wherein the second prompt word template includes second task description information, which is used to describe the task that needs to be performed by the front-end code update model; Calling the front-end code update model according to the second prompt word, adjusting at least part of the front-end code associated with any target visual element in the front-end code according to the second task description information and the attribute adjustment description information, so as to obtain an adjusted front-end code, and displaying an adjusted target HTML page corresponding to the adjusted front-end code; According to the mapping relationship between the visual elements and the layer elements, the adjusted front-end code is mapped into the adjusted target UI design file.

3. The method according to claim 2, characterized in that In response to an editing operation on any target visual element, acquiring attribute description information on any target visual element includes: In response to a triggering operation on any target visual element, displaying an input box corresponding to the any target visual element; In response to an input operation on the input box, the text information in the input box is used as attribute adjustment description information for any target visual element.

4. The method according to claim 1, characterized in that: According to the mapping relationship between the visual elements and the layer elements, the target front-end code is converted into a target UI design file corresponding to the target HTML page, including: Parsing the target front-end code to obtain a document structure tree and node attribute information corresponding to the target HTML page, wherein the document structure tree includes a plurality of nodes corresponding to the plurality of target visual elements; Generate multiple layer elements according to the multiple nodes included in the document structure tree and in combination with the mapping relationship between the visual elements and the layer elements; Generate layer attribute information of the plurality of layer elements according to the node attribute information; Determining the hierarchical relationship between the plurality of layer elements according to the hierarchical relationship between the plurality of nodes; A target UI design file corresponding to the target HTML page is generated according to the hierarchical relationship between the multiple layer elements, the multiple layer elements and their layer attribute information.

5. The method according to claim 4, characterized in that Generating a target UI design file corresponding to the target HTML page according to the multiple layer elements and their style information and hierarchical relationships, including: Using a structured syntax format, the attribute information of the multiple target layer elements is converted into UI design description information, and the UI design description information is provided to the server so that the server can serialize the UI design description information to obtain and return the target UI design file corresponding to the target HTML page.

6. The method according to any one of claims 1 to 5, characterized in that: Get the original image, including: Displaying a UI design file generation page, wherein the UI design file generation page includes an image upload control and a page generation control; In response to an upload operation on an image upload control, obtaining an original image; Before generating the first prompt word according to the original image and the first prompt word template, the method further includes: generating the first prompt word according to the original image and the first prompt word template in response to a trigger operation on a page generation control.

7. The method according to any one of claims 1 to 5, characterized in that: Generating a first prompt word according to the original image and a first prompt word template includes: Encoding the original image to obtain encoding features; Fill the encoding feature into the first prompt word template to obtain a first prompt word.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 6.

Images