UI design file generation method, device, storage medium and program product

Abstract

Embodiments of the present application provide a UI design file generation method and device, a storage medium and a program product. In the embodiments of the present application, a method for generating a UI design file based on an image is provided. The original image is understood by a front-end code generation model, an HTML page adapted to the original image is generated, and the front-end code corresponding to the HTML page is output, so as to convert the front-end code output by the model into a UI design file. In the whole process, the visual design is quickly generated through the original image, the generation time of the UI design file is significantly reduced, the work efficiency is improved, and the generation process is simplified. Even non-professionals can easily get started, the dependence on professional design software and designers is reduced, the human and time costs are reduced, and the resource utilization efficiency is improved. In addition, the structure and style of the HTML page are automatically adjusted by the front-end code generation model according to the original image, so as to ensure the consistency and beauty of the design.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, device, storage medium, and program product for generating UI design files. Background Technology

[0002] Currently, UI (User Interface) designers can perform UI design work using UI design tools (such as Sketch). Sketch, for example, is a vector drawing application suitable for UI designers, allowing them to design UI pages. The process involves UI designers sketching based on user needs, converting the sketch into wireframes, and then dragging and dropping appropriate components from the component library onto the wireframe to generate the corresponding UI design file.

[0003] However, this method of providing UI pages requires a high level of expertise from UI designers. For non-professionals, it presents problems such as high learning costs and complex operation, which affects the user experience. Summary of the Invention

[0004] This application provides a method, device, storage medium, and program product for generating UI design files, which reduces reliance on professional design software and designers, lowers manpower and time costs in the design process, and improves resource utilization efficiency.

[0005] This application provides a method for generating a UI design file, comprising: acquiring an original image, the original image including original elements; generating a first prompt word based on the original image and a first prompt word template, the first prompt word template including first task description information for describing the task to be performed by a front-end code generation model; calling the front-end code generation model based on the first prompt word to generate a target Hypertext Markup Language (HTML) page corresponding to the original image and its corresponding target front-end code based on the first task description information, wherein the target visual elements included in the target HTML page are adapted to the original elements; and mapping the target front-end code to a target UI design file corresponding to the target HTML page based on the mapping relationship between visual elements and layer elements, wherein the target layer elements included in the target UI design file are adapted to the target visual elements.

[0006] This application also provides an electronic device, including: a memory and a processor; the memory for storing a computer program; and the processor, coupled to the memory, for executing the computer program to implement the steps in the UI design file generation method provided in this application.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the UI design file generation method provided in this application.

[0008] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps in the UI design file generation method provided in this application.

[0009] This application provides a method for generating UI design files based on images. A front-end code generation model understands the original image, generates an HTML page adapted to the original image, and outputs the corresponding front-end code for the HTML page, thereby converting the front-end code output by the model into a UI design file. Throughout this process, the rapid generation of visual designs from the original image significantly reduces the UI design file generation time, improves work efficiency, and simplifies the UI design file generation process. Even non-professional designers can easily get started, reducing reliance on professional design software and designers, lowering manpower and time costs in the design process, and improving resource utilization efficiency. Furthermore, the front-end code generation model automatically adjusts the structure and style of the generated HTML page based on the original image, ensuring design consistency and aesthetics, and reducing human error in the design process. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0011] Figure 1a A flowchart illustrating a UI design file generation method provided for an exemplary embodiment of this application;

[0012] Figure 1b A schematic diagram of an original image, an HTML page with visual element size information, and an HTML page obtained by parsing a UI design file, provided for an exemplary embodiment of this application;

[0013] Figure 2a A schematic diagram of a UI design file generation page provided for an exemplary embodiment of this application;

[0014] Figure 2b A schematic diagram of another UI design file generation page provided for an exemplary embodiment of this application;

[0015] Figure 2c A schematic diagram of another UI design file generation page provided for an exemplary embodiment of this application;

[0016] Figure 2d A schematic diagram of yet another UI design file generation page provided for an exemplary embodiment of this application;

[0017] Figure 3 A schematic diagram illustrating the cooperation between a front-end code generation model and a front-end code update model to generate UI design files, provided as an exemplary embodiment of this application;

[0018] Figure 4 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0021] The various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0022] To address the aforementioned technical issues, this application provides a method for generating UI design files based on images. A front-end code generation model understands the original image, generates an HTML page adapted to the original image, and outputs the corresponding front-end code for the HTML page, thereby converting the front-end code output by the model into a UI design file. Throughout this process, the rapid generation of visual designs from the original image significantly reduces the UI design file generation time, improves work efficiency, and simplifies the UI design file generation process. Even non-professional designers can easily get started, reducing reliance on professional design software and designers, lowering manpower and time costs in the design process, and improving resource utilization efficiency. Furthermore, the front-end code generation model automatically adjusts the structure and style of the generated HTML page based on the original image, ensuring design consistency and aesthetics, and reducing human error in the design process.

[0023] The following describes in detail a solution provided by an embodiment of this application, with reference to the accompanying drawings.

[0024] Figure 1a This is a flowchart illustrating a method for generating UI design documents, provided as an exemplary embodiment of this application. Figure 1a As shown, the method includes:

[0025] 101. Obtain the original image, which includes the original elements;

[0026] 102. Generate a first prompt word based on the original image and the first prompt word template. The first prompt word template includes first task description information, which describes the task that the front-end code generation model needs to perform.

[0027] 103. Based on the first prompt word, call the front-end code generation model to generate the target hypertext markup language HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information. The target visual elements included in the target HTML page are adapted to the original elements.

[0028] 104. Based on the mapping relationship between visual elements and layer elements, map the target front-end code to the target UI design file corresponding to the target HTML page. The target UI design file includes target layer elements and target visual elements that are adapted.

[0029] In this embodiment, Figure 1a The subject executing the method shown can be a terminal device, such as a desktop computer, laptop computer, smartphone, IoT (Internet of Things) device, or smart wearable device (such as a smartwatch or smart bracelet).

[0030] In this embodiment, the original image is an image required to generate a UI page. The original image is not limited; for example, it can include, but is not limited to, hand-drawn sketches, UI page screenshots, and images from other sources. Images from other sources can include, but are not limited to, images downloaded from the internet, images taken with a camera or terminal device, etc. The fact that the original image can be from all of these image sources greatly expands the user's usage scenarios and facilitates different types of design needs.

[0031] The original image includes original elements, which are the basic elements that constitute the original image. These elements define the image structure and content and can be visually independent objects. For example, original elements can include, but are not limited to, text, symbols, lines, and graphics. Original elements also have corresponding attribute information, such as, but not limited to, color, shape, texture, shadow, and line thickness.

[0032] This embodiment provides a front-end code generation model for generating front-end code for original images. The front-end code generation model can be implemented as a Large Language Model (LLM), and for example, it includes, but is not limited to, generative models based on Transformers (GPT) and pre-trained models based on Transformers (BERT).

[0033] The front-end code generation model has a corresponding first prompt word template, which includes at least a first task description. This first task description describes the task the front-end code generation model needs to perform. In addition, the first prompt word template may also include role information and requirement information. Role information refers to the role played by the front-end code generation model; for example, the role could be a front-end code developer. Requirement information refers to the requirements for the front-end code generation model to generate Hyper Text Markup Language (HTML) pages and their corresponding front-end code.

[0034] In this embodiment, a first prompt word can be generated based on the original image and the first prompt word template. The implementation method for generating the first prompt word based on the original image and the first prompt word template is not limited. For example, the original image can be stored locally and then filled into the first prompt word template to obtain the first prompt word. Another example is encoding the features of the original image to obtain encoded features, and then filling these encoded features into the first prompt word template to obtain the first prompt word. The method of encoding the original image is not limited. For example, encoding methods may include, but are not limited to: base64 encoding, lossy compression encoding, lossless compression encoding, and image feature encoding. Base64 encoding is a method of converting binary data into a string format. Lossy compression is a method of reducing image size by reducing some information in the original image, such as JPEG format. Lossless compression is a method of reducing image size without sacrificing image quality, such as PNG and GIF formats. Image feature encoding is a method of extracting feature information from the original image, such as HOG (Histogram of Oriented Gradients) and SIFT (Scale Invariant Feature Transform).

[0035] The following is an example demonstration of the first prompt word.

[0036] Example A1:

[0037] ## Role Information: Front-end code developer.

[0038] ##First Task Description Information:

[0039] Please generate an HTML page based on the provided original image. Please reproduce the layout and style of the original image in the HTML page as accurately as possible, and output the corresponding front-end code for the HTML page.

[0040] The following is the Base64 encoding of the original image: [Base64 encoded string];

[0041] Please ensure that the HTML page structure is clear. If there is text content in the image, please try to identify it and embed it correctly into the HTML page.

[0042] Example A2:

[0043] ## Role Information: Front-end code developer.

[0044] ##First Task Description Information:

[0045] You build an HTML page using Cascading Style Sheets (CSS), HTML, and a scripting language (JavaScript, JS) based on the original image provided by the user. You need to continuously update the built HTML page so that the style and layout of the HTML page are close to the original image.

[0046] Returns the complete front-end code within the start and end tags (e.g., , ), indicating that the output includes everything from start to end, i.e., the front-end code of the entire HTML page.

[0047] ##Required Information

[0048] - Ensure that the original image matches the generated HTML page in terms of background color, text color, font size, font family, padding, margins, and borders.

[0049] - The HTML page includes text information from the original image.

[0050] - All functionality and content can be fully implemented without relying on comments indicating "more elements should be here." For example, if an HTML page needs a navigation bar, all navigation links can be written directly instead of adding comments saying more links can be added as needed.

[0051] - If the original image displays multiple original elements of the same type (e.g., news items or list items), then write the corresponding HTML code according to the actual number of original elements. For example, if there are 15 news items, then the front-end code of the HTML page will have 15 corresponding individual content tags (e.g., ...). <article>) or container label (e.g., () is used to represent each entry.

[0052] - For images in the original image, placeholder information is used for that image, and in the image label (e.g., The `alt` attribute of an image provides a detailed description, guiding the subsequent generation of the image in the HTML page. The `alt` attribute is an attribute that provides alternative text information for the image.

[0053] In this embodiment, the front-end code generation model is invoked according to the first prompt word. In the front-end code generation model, the target HTML page corresponding to the original image and its corresponding target front-end code are generated according to the first task description information. This process includes the extraction, classification and reconstruction of the original elements to ensure that the generated HTML page can accurately reflect the user's design intent.

[0054] The target HTML page includes target visual elements. Target visual elements refer to the elements presented on the target HTML page, which can include, but are not limited to, text, images, buttons, and form controls. These visual elements are defined by the HTML structure and styled using CSS to ensure they are displayed as intended. Target visual elements accurately reflect the design intent of the original image. Furthermore, the target visual elements in the target HTML page are adapted to the original elements in the original image. Adaptation means that during the process of converting the original elements into visual elements, the design intent and visual effects of the original image are accurately reflected in the target HTML page. For example, adaptation can be achieved by maintaining visual consistency between the original elements and the visual elements, ensuring that the visual elements in the HTML page (such as color, font, spacing, and other attribute information) are consistent with the original elements in the original image. Another example is converting interactive elements (such as buttons, links, and form controls) in a static original image into functional HTML, CSS, or JavaScript elements.

[0055] In this embodiment, a UI design file is a file used to store various elements on a UI page. A UI design file can include layers, components, styles, layouts, and other design elements. UI design files can use various file formats, such as, but not limited to, ".sketch" and ".psd". ".sketch" is a vector drawing file format. ".psd" is an image editing file format that supports complex image editing functions such as multiple layers, masks, adjustment layers, and special effects.

[0056] UI design files include layer elements, which are the basic units used to build and organize visual designs. Each layer represents an independent object or component, and the properties of each layer element can be adjusted independently without affecting other layer elements. Layer elements can include, but are not limited to: shape layers, text layers, image layers, vector layers, effect layers, and grouped layers. Shape layers can include, but are not limited to: geometric shape layers such as rectangles, ellipses, and polygons. Text layers are used to add text content. Image layers are used for imported images or drawn bitmaps. Vector layers are used for graphics created using vector paths, which are easy to scale without distortion. Effect layers are used to apply visual effects such as shadows, glows, and blurs. Grouped layers are used to group multiple layers together for simultaneous movement or adjustment.

[0057] There is a mapping relationship between visual elements and layer elements. This mapping relationship refers to the process of mapping the visual presentation in web design (defined by HTML and CSS) to the layer structure in graphic design tools (such as Sketch). Specifically, this mapping relationship involves parsing the layout, styles, and content of HTML to create corresponding layer elements in order to build the UI design file.

[0058] In this embodiment, the target front-end code can be mapped to the target UI design file corresponding to the target HTML page based on the mapping relationship between visual elements and layer elements. The target UI design file includes the adaptation of target layer elements and target visual elements. Adaptation means that during the process of converting target visual elements into target layer elements, the design intent and visual effects of the target HTML page are accurately reflected in the target UI design file. For example, adaptation can be implemented to maintain visual consistency between target layer elements and target visual elements, ensuring that the visual elements in the HTML page (such as attributes like color, font, and spacing) are consistent with the target layer elements in the target UI design file.

[0059] In this process, when generating a target UI design file, the file can be stored, for example, locally on the terminal. When needed, a UI design application deployed on the terminal device can parse the target UI design file to obtain an HTML page. Furthermore, when the front-end code generation model generates both the HTML page and the front-end code, the size information of each visual element can be output on the HTML page for debugging personnel to fine-tune the dimensions.

[0060] The following examples demonstrate the original image, an HTML page with size information of visual elements, and an HTML page obtained by parsing the target UI design file. Figure 1b As shown. In Figure 1b The illustration uses a visual element with a size of 600×300 as an example, but it is not limited to this. The text information included in the original image and the HTML page are "Fresh Fruit Selection", "Nutritional Combination", and "Direct from Orchard", respectively.

[0061] This application provides a method for generating UI design files based on images. A front-end code generation model understands the original image, generates an HTML page adapted to the original image, and outputs the corresponding front-end code for the HTML page, thereby converting the front-end code output by the model into a UI design file. Throughout this process, the rapid generation of visual designs from the original image significantly reduces the UI design file generation time, improves work efficiency, and simplifies the UI design file generation process. Even non-professional designers can easily get started, reducing reliance on professional design software and designers, lowering manpower and time costs in the design process, and improving resource utilization efficiency. Furthermore, the front-end code generation model automatically adjusts the structure and style of the generated HTML page based on the original image, ensuring design consistency and aesthetics, and reducing human error in the design process.

[0062] In one optional embodiment, the method of obtaining the original image is not limited. For example, a storage path for the original image can be preset, and the original image to be used to generate the UI design file can be stored in this storage path. When the UI design file needs to be generated, the original image can be automatically obtained based on this storage path. As another example, this application embodiment provides a UI design file generation page, which can be displayed and includes an image upload control; the user can upload the original image through the image upload control, and the terminal device can obtain the original image in response to the upload operation of the image upload control.

[0063] Optionally, the UI design file generation page also includes a page generation control and a model selection control. Users can initiate a model selection operation using the model selection control to choose the front-end code generation model for generating the HTML page, and trigger the page generation control. The terminal device can respond to the trigger operation on the page generation control by generating a first prompt based on the original image and the first prompt template, and then calling the front-end code generation model based on the first prompt to generate the target HTML page corresponding to the original image and its corresponding target front-end code according to the first task description information.

[0064] like Figure 2a As shown, an exemplary UI design file generation page is displayed, which includes: a model selection control 201, an image upload control 202, and a page generation control 203. Furthermore, it also includes: a file export control 204. When the file export control is triggered, the terminal device can respond to the trigger operation of the file export control 204 by storing the target UI design file to a specified local path for later use by the user.

[0065] In subsequent use, the target UI design file can be parsed using a UI design application on the terminal device to convert it into an HTML page and publish it for users to view.

[0066] Figure 3 This application provides an exemplary embodiment of a front-end code generation model and a front-end code update model that work together to generate a target UI design file. The front-end code generation model and the front-end code update model can be implemented as the same model or as different models. Figure 3 The example uses the front-end code generation model and the front-end code update model as different implementations.

[0067] In an optional embodiment, when generating an HTML page, the target HTML page can also be displayed. If page adjustments to the target HTML page are required, these adjustments are made using a front-end code update model. The front-end code update model adjusts the front-end code of the target HTML page to achieve the desired page adjustment. The front-end code update model corresponds to a second prompt word template, which describes the tasks that the front-end code update model needs to perform. The second prompt word template includes second task description information, indicating how to adjust the target HTML page.

[0068] This system can display a target HTML page containing multiple target visual elements. Users can initiate adjustment operations on any target visual element. Correspondingly, in response to these adjustments, the system can obtain attribute adjustment descriptions for each target visual element. A second prompt is generated based on the attribute adjustment descriptions and a second prompt template. The system then calls the front-end code to update the model based on the second prompt. Based on the second prompt, at least a portion of the front-end code associated with each target visual element is adjusted according to the second task description and attribute adjustment descriptions to obtain the adjusted front-end code. The system then displays the adjusted target HTML page corresponding to the adjusted front-end code. Finally, based on the mapping relationship between visual elements and layer elements, the adjusted front-end code is mapped to an adjusted target UI design file.

[0069] The implementation method of mapping the adjusted front-end code to the adjusted target UI design file based on the mapping relationship between visual elements and layer elements is not limited. For details, please refer to the introduction on "mapping the target front-end code to the target UI design file corresponding to the target HTML page based on the mapping relationship between visual elements and layer elements". It will not be repeated here.

[0070] Attribute adjustment descriptions define how to adjust the attributes of any target visual element. These descriptions can include, but are not limited to, style adjustment descriptions, layout adjustment descriptions, and hierarchy adjustment descriptions. For example, a style adjustment description could be "change the background to blue" or "change text A1 to text A2." Similarly, a layout adjustment description could be "increase the overall page length." Hierarchical adjustment descriptions could be "move information B from category C to category B."

[0071] The implementation method for generating the second prompt word based on the adjustment description information of any target visual element and the second prompt word template is not limited. For example, if the second prompt word template includes a blank prompt word, the adjustment description information can be filled into the blank prompt word, and the second prompt word template filled with the adjustment description information can be used as the second prompt word. As another example, if the second task description information includes placeholder information corresponding to the adjustment description information, the adjustment description information can be filled into the placeholder information, and the second prompt word template filled with the adjustment description information can be used as the second prompt word.

[0072] One implementation of calling the front-end code update model based on the second prompt word is to input the second prompt word into the front-end code update model.

[0073] The implementation method of adjusting at least a portion of the front-end code associated with any target visual element based on the second task description information and attribute adjustment description information to obtain the adjusted front-end code is not limited. For example, based on the second task description information and attribute adjustment description information, the original attribute information of any target visual element in the front-end code is adjusted to obtain the target attribute information of any target visual element, thereby obtaining the adjusted front-end code. For example, in the front-end code update model, based on the attribute adjustment description information, the identification information of the visual element to be adjusted in the front-end code is determined, and the original attribute information of the visual element to be adjusted is obtained based on the identification information; based on the attribute adjustment description information, the original attribute information is adjusted to update the front-end code. The identification information of the visual element to be adjusted may be the user identifier (UID) of the visual element.

[0074] Optionally, the implementation method for obtaining attribute description information for any target visual element in response to an editing operation on that target visual element is not limited. A specific implementation method is provided below, in response to a trigger operation on any target visual element, displaying an input box corresponding to that target visual element; the trigger operation can be a mouse click operation or a screen touch operation, and is not limited thereto. The user can initiate an input operation on the input box, and correspondingly, the terminal device can respond to the input operation on the input box by using the text information in the input box as attribute adjustment description information for any target visual element. The input operation is not limited; for example, when an input box is displayed on the page, the user can enter attribute adjustment description information into the input box. The input box corresponds to a confirmation control; when the confirmation control is triggered, the terminal device can respond to the trigger operation on the confirmation control to determine whether to initiate an input operation on the input box. Figure 2b As shown, this provides an example of displaying a target HTML page 205 on a UI design file generation page. The target HTML page 205 is implemented as a basic information page, which includes four configuration controls: photo, nickname, skill certificates, and job information; a save control; and information completion progress information, such as 25%. Specifically, when photo 2051 is triggered, it can display... Figure 2c As shown. In Figure 2c The image displays an input box corresponding to photo 2051. This input box includes a text input area, a confirmation control, and a cancellation control. The user can enter "Change the background to gray" in the input box and click the confirmation control. In response to this click, the terminal device calls the front-end code to update the model, changing the background information of the visual element containing the photo to gray, and then displays it. Figure 2d The page shown is illustrated. In this embodiment, an intuitive and easy-to-use user interface was designed, enabling users to easily input original images, generate UI design files, and adjust attributes such as style and layout, lowering the barrier to entry and improving the user experience. Furthermore, the adjustment effect of the HTML page can be fed back in real time, further optimizing the design process and ensuring that the final output meets expectations.

[0075] In one optional embodiment, the implementation method of converting target front-end code into a target UI design file corresponding to a target HTML page based on the mapping relationship between visual elements and layer elements is not limited. One implementation method for converting target front-end code into a target UI design file corresponding to a target HTML page based on the mapping relationship between visual elements and layer elements includes: parsing the target front-end code to obtain a document structure tree and node attribute information corresponding to the target HTML page, wherein multiple nodes in the document structure tree correspond to multiple target visual elements; generating multiple layer elements based on the multiple nodes in the document structure tree and the mapping relationship between visual elements and layer elements; generating layer attribute information for the multiple layer elements based on the node attribute information; determining the hierarchical relationship between the multiple layer elements based on the hierarchical relationship between the multiple nodes; and generating a target UI design file corresponding to the target HTML page based on the hierarchical relationship between the multiple layer elements, the multiple layer elements, and their layer attribute information.

[0076] The document structure tree (VSH) is a hierarchical tree of nodes constructed by parsing the target front-end code. It represents the relationships between various target visual elements within the target HTML page. Multiple nodes in the VSH correspond to multiple target visual elements within the target HTML page; for example, one node can correspond to one target visual element. For instance, the VSH can be implemented as a Document Object Model Tree (DOM).

[0077] For example, the nodes included in the document structure tree can be implemented as tags in the front-end code, and the nodes can include, but are not limited to:

[0078] 1) Tags used to define different parts of a page, such as header tags. <header>Navigation tags <nav>Main content tags <main>Chapter Tags <section>and article tags <article>Tags such as "etc."

[0079] 2) Used as a container or inline element to wrap other content, such as block-level element tags. and inline elements .

[0080] 3) Title tags, such as, <h1> to< / h1> <h6>Paragraph tags, such as .

[0081] 4) Form elements are tags used for user interaction, such as... <button>, <input> , <textarea> wait.< / textarea>

[0082] This involves generating multiple layer elements based on the document structure tree, combined with the mapping relationship between visual elements and layer elements. For example, one node can be mapped to one layer element, or one node can be mapped to multiple layer elements, or multiple nodes can be mapped to one layer element; there are no restrictions on this.

[0083] For example, the node is implemented as a title label ( <header>This node can be mapped to a text layer.

[0084] For example, a node's visual element may include multiple child elements, and each child element may correspond to a layer element, such as a... A visual element can contain multiple child elements, such as images, paragraph text, or other nested elements. This means that a node can be mapped to multiple layer elements.

[0085] For example, multiple visual elements can be mapped to a single layer element. These visual elements include titles, content text, buttons, etc. Adding these multiple visual elements to a toolbar allows you to map the toolbar to a single layer element.

[0086] This allows for the generation of layer attribute information for multiple layer elements based on node attribute information. Node attribute information may include, but is not limited to, style information and layout information. For example, node attribute information can be implemented as CCS information.

[0087] Within this framework, a hierarchical relationship exists between multiple nodes. This hierarchy refers to the subordination between nodes, corresponding to the hierarchical relationship between target visual elements. For example, a target visual element can contain multiple child elements, and this target visual element itself is a child element of its parent element. Multiple child elements under the same parent element are sibling elements. Based on this, the hierarchical relationship between multiple layer elements can be determined according to the hierarchical relationship between multiple nodes. For instance, the hierarchical relationship between multiple layer elements can be determined based on the correspondence between multiple nodes and multiple layer elements.

[0088] Optionally, the implementation method for generating the target UI design file corresponding to the target HTML page based on the hierarchical relationship between multiple layer elements, the multiple layer elements and their layer attribute information is not limited. The following provides an implementation method for generating the target UI design file corresponding to the target HTML page based on multiple layer elements, their style information and hierarchical relationship. Using a structured syntax format, the attribute information of multiple target layer elements is converted into UI design description information, which is then provided to the server for serialization to obtain and return the target UI design file corresponding to the target HTML page.

[0089] The structured syntax format may include, but is not limited to: Domain Specific Language - JavaScript Object Notation (JSON), Ordered Graph Data Language (OGDL), or Data Serialization Language (YAML), etc.

[0090] Serialization refers to the process of converting UI design description information into a format that can be stored or transmitted. For example, if the target UI design file is in binary format, the UI design description information can be converted into the target UI design file using some binary serialization methods. Alternatively, a serialization tool can be deployed on the server to convert the UI design description information into a UI design file. The UI design description information can be input into this serialization tool to perform serialization and obtain the UI design file.

[0091] Regarding the embodiments provided in this application Figure 1a The detailed implementation methods and beneficial effects of each step in the method shown have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0092] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 103 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0093] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the terms "first," "second," etc., used herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0094] Figure 4 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application, such as... Figure 4 As shown, the device includes a memory 44 and a processor 45.

[0095] Memory 44 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device.

[0096] The processor 45, coupled to the memory 44, is used to execute a computer program in the memory 44 for: acquiring an original image, the original image including original elements; generating a first prompt word based on the original image and a first prompt word template, the first prompt word template including first task description information for describing the task that the front-end code generation model needs to perform; invoking the front-end code generation model based on the first prompt word to generate a target Hypertext Markup Language (HTML) page corresponding to the original image and its corresponding target front-end code based on the first task description information, the target HTML page including target visual elements adapted to the original elements; and mapping the target front-end code to a target UI design file corresponding to the target HTML page based on the mapping relationship between visual elements and layer elements, the target UI design file including target layer elements adapted to the target visual elements.

[0097] In an optional embodiment, the processor 45 is further configured to: display a target HTML page, the target HTML page having multiple target visual elements; in response to an adjustment operation for any target visual element, obtain attribute adjustment description information for any target visual element; generate a second prompt word based on the attribute adjustment description information of any target visual element and a second prompt word template, the second prompt word template including second task description information for describing the task that the front-end code needs to perform to update the model; call the front-end code to update the model according to the second prompt word, so as to adjust at least a portion of the front-end code associated with any target visual element in the front-end code according to the second task description information and the attribute adjustment description information, so as to obtain the adjusted front-end code, and display the adjusted target HTML page corresponding to the adjusted front-end code; and map the adjusted front-end code to the adjusted target UI design file according to the mapping relationship between visual elements and layer elements.

[0098] In an optional embodiment, when the processor 45 obtains attribute description information for any target visual element in response to an editing operation for any target visual element, it is specifically used to: display an input box corresponding to any target visual element in response to a trigger operation for any target visual element; and use the text information in the input box as attribute adjustment description information for any target visual element in response to an input operation for the input box.

[0099] In an optional embodiment, when the processor 45 converts the target front-end code into a target UI design file corresponding to the target HTML page based on the mapping relationship between visual elements and layer elements, it specifically performs the following steps: parses the target front-end code to obtain the document structure tree and node attribute information corresponding to the target HTML page, wherein the document structure tree includes multiple nodes corresponding to multiple target visual elements; generates multiple layer elements based on the multiple nodes included in the document structure tree and the mapping relationship between visual elements and layer elements; generates layer attribute information for the multiple layer elements based on the node attribute information; determines the hierarchical relationship between the multiple layer elements based on the hierarchical relationship between the multiple nodes; and generates a target UI design file corresponding to the target HTML page based on the hierarchical relationship between the multiple layer elements, the multiple layer elements, and their layer attribute information.

[0100] In an optional embodiment, when the processor 45 generates the target UI design file corresponding to the target HTML page based on multiple layer elements, their style information, and hierarchical relationships, it is specifically used to: convert the attribute information of multiple target layer elements into UI design description information using a structured syntax format, provide the UI design description information to the server so that the server can serialize the UI design description information to obtain the target UI design file corresponding to the target HTML page and return it.

[0101] In an optional embodiment, when acquiring the original image, the processor 45 is specifically configured to: display a UI design file generation page, the UI design file generation page including an image upload control and a page generation control; acquire the original image in response to an upload operation on the image upload control; and before generating a first prompt word based on the original image and the first prompt word template, the method further includes: generating a first prompt word based on the original image and the first prompt word template in response to a trigger operation on the page generation control.

[0102] In an optional embodiment, when the processor 45 generates the first prompt word based on the original image and the first prompt word template, it is specifically used to: encode the original image to obtain encoded features; and fill the encoded features into the first prompt word template to obtain the first prompt word.

[0103] Regarding the embodiments provided in this application Figure 4 The detailed implementation methods and beneficial effects of each step in the device shown have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0104] Furthermore, such as Figure 4 As shown, the electronic device also includes other components such as a communication component 46, a display 47, a power supply component 48, and an audio component 49. Figure 4 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 4 The components shown. Additionally... Figure 4 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a desktop computer, laptop computer, smartphone, IoT (Internet of Things) device, or smart wearable device (such as a smartwatch or smart bracelet), or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a desktop computer, laptop computer, smartphone, or other terminal device, it may include... Figure 4 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., it may be omitted. Figure 4 The component within the dashed box.

[0105] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the above-described functions. Figure 1a The steps in the method embodiment shown can be performed by an electronic device.

[0106] Accordingly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to perform the above-described... Figure 1a The steps in the method embodiment shown can be performed by an electronic device.

[0107] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0109] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0110] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0111] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0116] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0117] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0118] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0120] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application. < / header> < / button> < / h6> < / article> < / section> < / main> < / nav> < / header> < / article>

Claims

1. A UI design file generation method, characterized by, The method comprises: obtaining an original image, the original image comprising original elements; generating a first prompt word according to the original image and a first prompt word template, the first prompt word template comprising first task description information for describing a task to be performed by a front-end code generation model; calling the front-end code generation model according to the first prompt word to generate a target HTML page corresponding to the original image and target front-end code corresponding to the target HTML page according to the first task description information, the target HTML page comprising target visual elements adapted to the original elements; mapping the target front-end code into a target UI design file corresponding to the target HTML page according to a mapping relationship between visual elements and layer elements, the target UI design file comprising target layer elements adapted to the target visual elements.

2. The method of claim 1, wherein, The method further comprises: displaying the target HTML page, the target HTML page comprising a plurality of target visual elements; in response to an adjustment operation on any target visual element, obtaining attribute adjustment description information for the any target visual element; generating a second prompt word according to the attribute adjustment description information for the any target visual element and a second prompt word template, the second prompt word template comprising second task description information for describing a task to be performed by a front-end code update model; calling the front-end code update model according to the second prompt word to adjust at least part of the front-end code associated with the any target visual element in the front-end code according to the second task description information and the attribute adjustment description information to obtain adjusted front-end code, and displaying an adjusted target HTML page corresponding to the adjusted front-end code; mapping the adjusted front-end code into the adjusted target UI design file according to the mapping relationship between visual elements and layer elements.

3. The method of claim 2, wherein, In response to an editing operation on any target visual element, obtaining attribute description information for the any target visual element, comprising: in response to a trigger operation on any target visual element, displaying an input box corresponding to the any target visual element; in response to an input operation on the input box, taking text information in the input box as attribute adjustment description information for the any target visual element.

4. The method of claim 1, wherein, Converting the target front-end code into a target UI design file corresponding to the target HTML page according to the mapping relationship between visual elements and layer elements, comprising: parsing the target front-end code to obtain a document structure tree and node attribute information corresponding to the target HTML page, the document structure tree comprising a plurality of nodes corresponding to a plurality of target visual elements; generating a plurality of layer elements according to the plurality of nodes included in the document structure tree in combination with the mapping relationship between visual elements and layer elements; generating layer attribute information of the plurality of layer elements according to the node attribute information; determining a hierarchical relationship between the plurality of layer elements according to a hierarchical relationship between the plurality of nodes. According to the hierarchical relationship among the plurality of layer elements, the plurality of layer elements and the layer attribute information thereof, a target UI design file corresponding to the target HTML page is generated.

5. The method of claim 4, wherein, According to the plurality of layer elements and the style information and hierarchical relationship thereof, a target UI design file corresponding to the target HTML page is generated, including: Using a structured syntax format, the attribute information of the plurality of target layer elements is converted into UI design description information, and the UI design description information is provided to the server, so that the server serializes the UI design description information to obtain a target UI design file corresponding to the target HTML page and returns.

6. The method according to any one of claims 1 to 5, characterized in that, Obtaining an original image, including: Displaying a UI design file generation page, the UI design file generation page including an image upload control and a page generation control; In response to an upload operation on the image upload control, an original image is obtained; Before generating a first prompt word according to the original image and a first prompt word template, the method further includes: in response to a trigger operation on the page generation control, generating a first prompt word according to the original image and a first prompt word template.

7. The method according to any one of claims 1 to 5, characterized in that, According to the original image and the first prompt word template, a first prompt word is generated, including: Encoding the original image to obtain an encoded feature; The encoded feature is filled into the first prompt word template to obtain the first prompt word.

8. An electronic device, comprising: Including: Memory and processor; The memory is used to store a computer program; the processor is coupled with the memory and is used to execute the computer program to realize the steps in the method of any one of claims 1-6.

9. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor realizes the steps in the method of any one of claims 1-6.

10. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instruction is executed by the processor, the processor realizes the steps in the method of any one of claims 1-6.

Images