Model training method and apparatus, device, medium, and product

Abstract

This application discloses a model training method, apparatus, device, medium, and product. The method includes: acquiring sample prompt information and acquiring reference interface code corresponding to the sample prompt information, wherein the sample prompt information is used to express the requirements for generating the interface; inputting the sample prompt information into a first model to obtain predicted interface code; based on the reference interface code and the reference interface corresponding to the reference interface code, obtaining an evaluation result for the predicted interface code, wherein the evaluation result is used to indicate the similarity between the predicted interface code and the reference interface code; and training the first model based on the evaluation result. By evaluating the predicted interface code and using the evaluation result to guide model updates, the model training efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device, medium, and product. Background Technology

[0002] A graphical user interface (GUI) code generation agent is an artificial intelligence system that automatically generates GUI code. The GUI code generation agent is used to dynamically generate or modify the graphical interface in real time to meet the user's interface generation requirements.

[0003] In related technologies, a GUI code is generated in the strategy model of the GUI code generation agent to generate GUI code corresponding to the interface generation requirements, and the interface image corresponding to the GUI code is obtained. Based on the interface image and the target interface image corresponding to the interface generation requirements, the interface similarity between the interface image and the target interface image is obtained as a reward score through a Vision-Language Model (VLM). The strategy model is updated based on the reward score.

[0004] However, the above training method processes interface images through VLM, resulting in a coarse interface recognition process, low image processing efficiency, and low interface similarity accuracy, leading to low model training efficiency. Summary of the Invention

[0005] This application provides a model training method, apparatus, device, medium, and product. The technical solutions provided by this application include the following aspects.

[0006] According to one aspect of the embodiments of this application, a model training method is provided, the method comprising: Obtain sample prompt information and the reference interface code corresponding to the sample prompt information, wherein the sample prompt information is used to express the requirements for generating the interface; Input the sample prompt information into the first model to obtain the prediction interface code; Based on the reference interface code and the reference interface corresponding to the reference interface code, an evaluation result is obtained for the predicted interface code, and the evaluation result is used to indicate the degree of similarity between the predicted interface code and the reference interface code; The first model is trained based on the evaluation results.

[0007] According to one aspect of the embodiments of this application, a model training apparatus is provided, the apparatus comprising: The first acquisition module is configured to acquire sample prompt information and acquire reference interface code corresponding to the sample prompt information, wherein the sample prompt information is used to express the requirements for generating the interface; The generation module is configured to input the sample prompt information into the first model to obtain the prediction interface code; The second acquisition module is configured to acquire an evaluation result for the predicted interface code based on the reference interface code and the reference interface corresponding to the reference interface code, wherein the evaluation result is used to indicate the degree of similarity between the predicted interface code and the reference interface code. The training module is configured to train the first model based on the evaluation results.

[0008] According to one aspect of the embodiments of this application, a terminal device is provided, the terminal device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described model training method.

[0009] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described model training method.

[0010] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium, and a processor reading from the computer-readable storage medium and executing the computer program to implement the above-described model training method.

[0011] The technical solution provided in this application can bring the following beneficial effects: Based on the predicted interface code corresponding to the sample prompt information obtained through the first model, the predicted interface corresponding to the predicted interface code is evaluated based on the reference interface code. The evaluation results are used to assess the similarity between the predicted interface code generated by the first model and the reference interface code. The evaluation results guide the model update, expand the evaluation of the interface from the dimension of interface code, improve the accuracy of evaluating training data during training, and thus improve the model training accuracy and efficiency. Furthermore, generating interface code through the trained model can improve the consistency between the interface code and the generation requirements, thereby improving the model generation accuracy. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a computer system provided in an exemplary embodiment of this application; Figure 2 This is a flowchart of a model training method provided in an exemplary embodiment of this application; Figure 3 This is a flowchart of a model training method provided in another exemplary embodiment of this application; Figure 4 This is a schematic diagram illustrating the acquisition of layout information provided in an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the interface code generation model training process provided in an exemplary embodiment of this application; Figure 6 This is a flowchart of a model training method provided in yet another exemplary embodiment of this application; Figure 7 This is a block diagram of a model training apparatus provided in an exemplary embodiment of this application; Figure 8 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0015] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0016] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user, processor, and computer device data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0018] Please refer to Figure 1 This illustration shows a schematic diagram of a model training system provided in one embodiment of this application. The computer system 100 includes: a terminal 120, or a server 140, or both a terminal 120 and a server 140. In the model training method provided in this embodiment, the executing entity for each step can be a computer device within the computer system 100, wherein the computer device is equipped with a model training apparatus for applying the model training method. In some embodiments, the computer device includes hardware devices that require interface generation, such as touch devices, mixed input devices, wearable devices, and IoT devices. Optionally, the aforementioned computer device can be a terminal 120 or a server 140, etc., and this embodiment does not limit this.

[0019] The model training method provided in this application embodiment can be executed independently by terminal 120, independently by server 140, or jointly by terminal 120 and server 140; no limitation is made here. In some embodiments, computer system 100 can be implemented as the system architecture of the model training method.

[0020] The device types of terminal 120 include at least one of the following: smartphones, desktop computers, laptop computers, personal computers (PCs), tablet computers, e-book readers, smart robots, augmented reality (AR) devices, virtual reality (VR) devices, Moving Picture Experts Group Audio Layer III (MP3) players, Moving Picture Experts Group Audio Layer IV (MP4) players, smart TVs, in-vehicle terminals, wearable devices, and other electronic devices. This application embodiment does not limit the types of devices included in the embodiments.

[0021] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0022] Those skilled in the art will understand that the number of the aforementioned devices can be more or less. For example, there may be only one device, or there may be dozens or hundreds of devices, or even more. This application does not limit the number or type of devices.

[0023] Server 140 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Server 140 provides functional services for implementing model training. Optionally, server 140 undertakes the main computing work, and terminal 120 undertakes the secondary computing work; or, server 140 undertakes the secondary computing work, and terminal 120 undertakes the main computing work; or, server 140 and terminal 120 collaborate on computing using a distributed computing architecture.

[0024] In one example, the model training method is implemented independently by server 140. Illustratively, server 140 obtains sample prompt information through upload or download. For example, it receives an upload operation to upload sample prompt information to the prompt information management platform on server 140, or a download operation to download sample prompt information from a database or open-source data platform to server 140; and server 140 obtains the reference interface code corresponding to the sample prompt information. The sample prompt information is input into the first model deployed on server 140 to obtain prediction interface code. Optionally, a prediction interface corresponding to the sample prompt information is generated based on the prediction interface code, wherein the first model is used to generate interface code that meets the interface generation requirements corresponding to the prompt information.

[0025] In the example above, server 140 obtains an evaluation result for the predicted interface code based on the reference interface code and the reference interface corresponding to the reference interface code. The evaluation result is used to indicate the similarity between the predicted interface code and the reference interface code. A first model is trained based on the evaluation result. Optionally, the first model takes action under the sample prompt information through a reinforcement learning method, that is, generates the predicted interface code. The evaluation result is used as feedback for the first model, and the model parameters of the first model are updated based on the feedback, thereby updating the strategy of the first model to generate interface code, so that the interface code generated by the first model better matches the requirements of the sample prompt information.

[0026] Optionally, server 140 obtains a first evaluation value based on the similarity requirement between the interface structure of the reference interface and the interface structure of the predicted interface; if the first evaluation value reaches a preset threshold, a second evaluation value is obtained based on the interface style of the reference interface and the interface style of the predicted interface. Based on the first and second evaluation values, server 140 trains a first model. Optionally, the evaluation result includes the first and second evaluation values.

[0027] In another example, the model training method is implemented collaboratively by terminal 120 and server 140. Illustratively, terminal 120 acquires sample prompt information and the corresponding reference interface code. The sample prompt information expresses the requirement for generating the interface. Server 140, which houses the first model, receives the sample prompt information from terminal 120 and inputs it into the first model to obtain the predicted interface code. Based on the reference interface code and its corresponding reference interface, server 140 obtains an evaluation result for the predicted interface code. This evaluation result indicates the similarity between the predicted interface code and the reference interface code. Server 140 then trains the first model based on the evaluation result.

[0028] Schematic, server 140 obtains a first evaluation value based on the similarity requirement between the interface structure of the reference interface and the interface structure of the predicted interface; if the first evaluation value reaches a preset threshold, server 140 obtains a second evaluation value based on the interface style of the reference interface and the interface style of the predicted interface; a first model is trained based on the first and second evaluation values ​​to obtain an interface code generation model. Optionally, server 140 also deploys a second model, which is a pre-trained large language model for evaluating the style consistency of interface images. The second model is used to obtain the second evaluation value corresponding to the sample prompt information based on the image style requirements of the interface; that is, the second model is a reward model corresponding to the image style requirements.

[0029] Optionally, server 140 sends the trained first model, i.e., the interface code generation model, to terminal 120. Terminal 120 executes the interface code generation task through the interface code generation model to obtain the generation result corresponding to the interface code generation task, i.e., the interface code that meets the interface generation requirements. Alternatively, server 140 executes the interface code generation task through the interface code generation model to obtain the interface code corresponding to the interface code generation task, and sends the interface code to terminal 120, which receives the interface code.

[0030] It is worth noting that the aforementioned terminal 120 refers to an electronic device with output display capabilities and input control capabilities. The aforementioned server 140 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud security, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application embodiment does not limit this.

[0031] Cloud technology refers to a managed technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0032] In some embodiments, the server 140 described above can also be implemented as a node in a blockchain system.

[0033] A GUI code generation agent is an artificial intelligence system that automatically generates GUI code. This agent dynamically generates or modifies graphical interfaces in real-time to meet user interface generation requirements. In related technologies, the agent generates GUI code corresponding to the interface generation requirements through a policy model and obtains the corresponding interface image. Based on this interface image and the target interface image corresponding to the interface generation requirements, a Visual Similarity Model (VLM) is used to obtain the interface similarity between the interface image and the target interface image as an evaluation value. The policy model is then updated based on the evaluation value. The VLM, acting as a reward model, evaluates the interface image generated from the interface code output by the policy model to obtain an evaluation value.

[0034] To illustrate, when the user instruction corresponding to the interface generation requirement is obtained, the user instruction is input into the policy model to output the interface code; the interface code is rendered to obtain the interface image; the interface image and the target interface image are input into the reward model, and the interface similarity between the interface image and the target interface image in the visual dimension is evaluated by the reward model, and a scalar result is output as the evaluation value; based on the evaluation value, the policy model is updated by a reinforcement learning algorithm.

[0035] However, the above training method has at least one of the following problems.

[0036] 1. Using Visual Modeling (VLM) to process interface images results in a coarse interface recognition process. VLM only provides an evaluation result of interface similarity based on the overall similarity of image data, making it difficult to identify structural detail errors in the interface. Optionally, structural detail errors include, but are not limited to, situations where the button color and shape in the interface image generated by the policy model are only slightly different from the target interface image, but the text on the buttons contains spelling errors or other anomalies; or, the overall interface style of the interface image is consistent with the target interface image, but key interactive elements are missing, such as hidden input boxes not being displayed. VLM's accuracy in identifying these structural detail errors is low. If VLM fails to identify structural detail errors and outputs a high evaluation value, it misleads the policy model. After the policy model updates based on this high evaluation value, it will learn incorrect code logic, resulting in low model training accuracy.

[0037] 2. During the process of training a policy model through reinforcement learning, there is a process in which the policy model interacts with user commands to generate complete interface code, which is the rollout phase. In the rollout phase, a large number of evaluation values ​​corresponding to user commands need to be obtained to improve statistical stability. However, each time an evaluation value is obtained, a large number of parameters of VLM need to be called for image processing, resulting in high inference latency and low evaluation value acquisition efficiency. This leads to low model training throughput and high computational resource consumption of computing units such as graphics processing units (GPUs), resulting in low model training efficiency.

[0038] 3. Before pre-training, VLM performs overall image consistency annotation on the sample interface image and the corresponding target interface image from the user's perspective, and trains VLM based on this annotation and the sample interface image. That is, VLM outputs a global subjective evaluation result for the interface image. For example, if the full score is 10, it outputs 8 as the evaluation result. The policy model cannot obtain information about the interface components with deviations from the evaluation value corresponding to this evaluation result, resulting in a coarse interface similarity and low accuracy of interface evaluation. It is unable to allocate the evaluation value to specific policy actions at a finer granularity, resulting in a slow convergence speed and low model training efficiency during training.

[0039] To address the aforementioned issues, this application provides a model training method. This method involves acquiring sample prompt information, inputting the prompt information into a first model and outputting interface code, generating a predicted interface corresponding to the sample prompt information based on the interface code, obtaining a first evaluation value based on the interface structure of a reference interface and the predicted interface, and obtaining a second evaluation value based on the interface style of the reference interface and the predicted interface when the first evaluation value reaches a preset threshold. The method then trains the first model based on the first and second evaluation values ​​to obtain an interface code generation model. The first model is a pre-acquired large language model used for generating interface code.

[0040] Figure 2 This is a flowchart of a model training method provided in one embodiment of this application. The method is performed by a computer device (which can be implemented as follows). Figure 1 The method can be executed by the terminal 120 or the server 140 shown, or it can be executed by the terminal, the server, or both. This embodiment takes the method being executed by the server as an example, and the method includes the following steps 210 to 240.

[0041] Step 210: Obtain the sample prompt information and the corresponding reference interface code.

[0042] Sample prompts are used to express the requirements for generating the interface. In some embodiments, within the application environment corresponding to the GUI code agent or the GUI world model, the prompts describe the requirements for generating the interface. These requirements are implemented as an interface generation task, which instructs the generation of a first interface upon execution of the interface generation task. The GUI world model includes a neural network model capable of simulating dynamic changes in the GUI environment. This neural network model predicts the first interface, its interface state, and the interaction results after performing interaction tasks on the first interface, thereby dynamically generating, displaying, and modifying the GUI without formal user interaction.

[0043] The sample prompt information describes the interface generation requirements, including at least one aspect of the application scenario of the first interface, the interface components in the first interface, and the corresponding interactive operations of the first interface. For example, the sample prompt information can be implemented as the text prompt "Generate an interface with username and password input boxes," or as the text prompt "Generate a display interface including the interface components shown in the image," along with an image prompt.

[0044] The first interface serves at least one purpose, such as displaying or conveying information, providing a perceptual environment, or performing interactive tasks. The first interface is the interface to be generated, including a predicted interface and a reference interface. The predicted interface is the interface generated by the first model based on sample prompts, aiming to meet the interface generation requirements. The reference interface is independent of the first model, separate from the prediction and generation process, and consistent with the interface generation requirements. Compared to the predicted interface, the reference interface is an objectively existing standard interface, and its consistency with the interface generation requirements is determined by a preset verification standard. The sample prompts are used to instruct the generation of the predicted interface; the consistency between the predicted interface and the interface generation requirements is uncertain and needs to be evaluated. The differences between the reference interface and the predicted interface can improve the accuracy of quantitatively evaluating the consistency between the predicted interface and the interface generation requirements.

[0045] In some embodiments, the reference interface is generated through reference interface code. The reference interface or the method of obtaining the reference interface is predefined or manually created based on interface generation requirements; optionally, after obtaining the reference interface, it undergoes multiple verification processes such as manual review, unit testing, and syntax checking to ensure that the format of the reference interface conforms to preset specifications. The reference interface is unique or relatively accurate; for example, the reference interface is at least one interface selected through multiple evaluations. Optionally, the methods for obtaining the reference interface include, but are not limited to, manual writing or creation, preset rule generation, preset model generation, real-time acquisition or reading, etc.; this application does not limit this.

[0046] Specifically, unit testing of the reference interface involves using pre-written, minimal-granularity automated test cases to verify whether at least one interface component or at least one function in the corresponding reference interface code can be displayed as expected, ensuring the accuracy of the interface component display and the logical correctness of the function code. Syntax checking of the reference interface involves static analysis of the reference interface code according to the official specifications of a specific programming language or code format. This detects and corrects potential syntax errors such as spelling mistakes and incorrect attribute values, ensuring the structural legality of the reference interface code and ensuring the correct acquisition of the reference interface.

[0047] Optionally, the first interface corresponding to the sample prompt information can be a GUI, or it can be a static display interface, a guide interface, or any other form of user interface (UI). Other forms of UI include, but are not limited to, text user interface (TUI), voice user interface (VUI), natural user interface (NUI), command line interface (CLI), and fictional user interface (FUI).

[0048] Sample prompts are content entered by the user in real time or in advance, including but not limited to query information, instruction information, and example information. Query information is a conditional request expression entered by the user to obtain specific information or trigger a system response. This request expression is used to specify the target object or scope boundary for retrieval or computation processing, including keyword forms, question text forms, etc. Sample prompts include at least one modal information, where at least one modal information refers to prompts corresponding to at least one information type, including but not limited to text, images, audio, video, motion information, and biometric information. In some embodiments, the methods for obtaining sample prompts include but are not limited to at least one of the following.

[0049] 1. The user input interface generates a task description, and sample prompt information is obtained based on the text or image content corresponding to the task description generated by the interface. Illustratively, upon receiving an input description prompt, the user inputs the content corresponding to the task description into the input device interface based on software or hardware controls. Optionally, when the task description is generated as text content, if the input device is a physical keyboard, the input device driver obtains the corresponding character code and generates the corresponding text sequence based on the pressed keyboard controls; if the input device is a touchscreen, the input device driver obtains the corresponding coordinate mapping information based on the touched coordinates on the touchscreen to obtain the corresponding text; if the input device is a voice input device, the corresponding text is recognized based on the acquired sound wave signal.

[0050] 2. Based on the user requirements corresponding to the interface interaction task, automatically generate sample task prompt information, including but not limited to: obtaining the corresponding interface generation requirements through prediction processing based on the user requirements corresponding to the interface interaction task, thereby obtaining sample prompt information; wherein, the interface generation task to be executed corresponding to the sample prompt information belongs to at least one step in the complete interface interaction task process.

[0051] 3. Retrieve pre-configured sample prompts from a database or prompt information platform. For example, query and read pre-stored sample prompts from a structured database; or read pre-stored sample files containing sample prompts from a local or distributed file system; or call third-party data service interfaces, such as Application Programming Interfaces (APIs), to pull, subscribe to, or collect sample prompts in real time.

[0052] It is worth noting that the above-described methods for obtaining sample prompt information are merely illustrative examples, and the specific methods for obtaining task prompt words are not limited in the embodiments of this application.

[0053] Step 220: Input the sample prompt information into the first model to obtain the prediction interface code.

[0054] The first model is a pre-acquired large language model used to generate interface code. As the strategy model corresponding to the interface code generation model, the first model can be a low-level model with general language capabilities, pre-trained on large-scale unlabeled data. For example, the first model can be a Multimodal Large Language Model (MLLM). Optionally, a predicted interface corresponding to sample prompt information can be generated based on the predicted interface code.

[0055] Upon receiving sample prompts and contextual information, the first model automatically infers and generates predicted interface code that meets the interface generation requirements. This predicted interface code serves as a coded representation of the predicted interface, essentially a structured representation of the predicted interface. The contextual information, during the generation of the predicted interface code as output, provides the first model with external information or constraints, guiding it to focus on the specific domain, document, or knowledge scope corresponding to the current sample prompts. Predicting the interface code based on contextual and sample prompt information improves the accuracy, traceability, and interpretability of the first model's output, and enhances the relevance between the generated result (the predicted interface code) and the application scenario and code context corresponding to the sample prompts, thereby improving the consistency between the predicted interface code and the interface generation requirements.

[0056] In some embodiments, the first model outputs a corresponding generated result, i.e., a predicted interface code, based on given sample prompts and an autoregressive generation mechanism. The process of generating the predicted interface code includes, but is not limited to, at least one of the following steps.

[0057] 1. The sample prompt information is converted into a vector representation that the first model can process through feature processing, that is, the sample prompt information is encoded to obtain a prompt vector sequence. Specifically, when the sample prompt information is implemented as text prompt information, the sample prompt information is mapped to a text word sequence and the corresponding embedding vector of the text word sequence through a word segmenter. When the sample prompt information is implemented as image prompt information, the sample prompt information is extracted into an image feature sequence through a visual encoder, such as a Vision Transformer (ViT), and the image feature sequence is mapped to the text embedding space to combine the image prompt information and the text prompt information to obtain the prompt vector sequence that serves as the basis for the first model to generate the prediction interface code.

[0058] In some embodiments, the embedding vector corresponding to the sample prompt information is obtained through a text encoder. For example, the text encoder includes, but is not limited to, the Word to Vector (Word2Vec) model, the Global Vectors (GloVe) model, and the FastText model.

[0059] 2. The encoded prompt vector sequence is input into the Transformer architecture of the first model. Through a multi-layer self-attention mechanism feedforward network, the correlation strength between each element (such as a text word or image patch) in the sample prompt information and all other elements except itself is calculated, thereby establishing the global dependency structure corresponding to the sample prompt information. The context representation vector is obtained by encoding based on this relationship structure.

[0060] 3. The first model predicts the probability distribution of the next text word on a per-text word basis based on the context representation vector. At each prediction step, the text word with the highest probability is selected, or, according to the sampling strategy, the target text word is selected as the newly generated text word. The newly generated text word is appended to the context representation vector as input for the next step. This process iterates until a terminator is generated, such as an end-of-sequence (EOS) symbol, or the maximum length limit is reached, resulting in a generated sequence composed of newly generated text words arranged in order.

[0061] 4. The generated sequence is mapped back to readable text through inverse word segmentation, or the representation is restored to images or other forms through a decoder to form the final output result, which is the prediction interface code.

[0062] It is worth noting that the above process of obtaining the prediction interface code is only an example, and the specific process of outputting the prediction interface code from the input sample prompt information of the first model in this embodiment is not limited.

[0063] Interface code is a computer program instruction used to generate a predicted interface that conforms to the interface generation requirements corresponding to the sample prompt information. Optionally, interface code includes at least one of predicted interface code and reference interface code. Interface code describes the structure, layout, style, and executable interactive behaviors of the predicted interface through declarative or imperative code syntax, thereby defining the interface structural logic and visual style. It is a machine language carrier that transforms interface generation requirements into a visual predicted interface. For example, interface code can be implemented as at least one of machine languages ​​such as Hyper Text Markup Language (HTML), eXtensible Markup Language (XML), and Cascading Style Sheets (CSS), and this application does not limit this to any particular language.

[0064] Based on the predicted interface code generated by the first model, the predicted interface code is rendered and parsed in a lightweight sandbox environment to obtain the predicted interface. The lightweight sandbox environment refers to an isolated execution environment with low resource consumption and fast startup, supporting the parallel execution of multiple instances of the predicted interface code as a process-level or container-level isolated environment. Illustratively, the lightweight sandbox environment includes, but is not limited to: 1. A headless browser, which refers to a browser or browser simulation engine that can run in the background and does not have a recognizable graphical interface; 2. A layout engine, which is the mathematical solver corresponding to the predicted interface code, used to recursively calculate the declarative layout description, i.e., the predicted interface code, into the final position and size information of each interface component presented on the predicted interface. Optionally, the predicted interface includes, but is not limited to, the interface image and interface structure of the predicted interface.

[0065] In some embodiments, generating a prediction interface corresponding to sample prompt information based on the prediction interface code includes, but is not limited to, at least one of the following steps.

[0066] 1. The structural portion of the prediction interface code generated by the first model is loaded into the rendering engine. This structural portion can be implemented as a string or file presented in HTML or XML format. The rendering engine is the aforementioned lightweight sandbox environment. The rendering engine's parser converts the original interface code into Document Object Model (DOM) tree data. This DOM tree data represents the structured node hierarchy of the prediction interface; in other words, the DOM tree data is the interface structure of the prediction interface. For example, the rendering engine's parser can be implemented using the Robolectric framework.

[0067] 2. Parse the inline styles, internal style sheets, and CSS files corresponding to the prediction interface code, and calculate the style attributes of each DOM node in the DOM tree data, such as size, color, font, and margins. Based on the style attributes, associate the preset style rules with the DOM nodes to generate the layout tree information corresponding to the prediction interface code.

[0068] The CSS file includes rendering styles that describe the structural parts of the predictive interface code. These rendering styles are separate from the structural parts of the predictive interface code. The rendering styles can be interface style information such as layout, color, and font. Rendering styles can come from multiple style definition sources, and these styles can be applied to the same interface elements according to a preset priority rule. Multiple interface elements, their corresponding style properties, and script behaviors constitute the interface components. Definition sources include, but are not limited to: author styles (CSS files written by developers); user styles (styles customized by visitors in the browser used to render the predictive interface, such as larger fonts for improved accessibility); and user agent styles (default styles provided by the browser).

[0069] Style attributes include, but are not limited to: control characteristics, such as button or input box type, interaction state, etc.; text characteristics, such as font, font size, alignment, etc.; and interaction characteristics, such as clickable area, focus state, etc.

[0070] 3. Traverse the layout tree information corresponding to the predicted interface code. Based on the layout rules in the layout tree, obtain the precise coordinates and dimensions of each DOM node in the predicted interface, i.e., the predicted position information, to achieve layout processing. Layout rules include, but are not limited to: box model information, used to specify the content, padding, border, and margin of each interface element, thereby obtaining the actual width and height proportion of the interface element in the predicted interface; float information, used to remove interface elements from the default arrangement of interface elements and move them left or right until the outer edge of the interface element touches the edge of the containing block or another interface element containing float information, thereby achieving the effect of text wrapping around an image; positioning information, used to indicate the positioning method of the interface element in the interface structure, such as setting offsets through properties like top, bottom, left, and right when the basic positioning is determined by the position property. Optionally, positioning information includes, but is not limited to, relative positioning, absolute positioning, fixed positioning, and sticky positioning.

[0071] 4. Rasterize the visualization information of each interface element in the layout tree information after typesetting into pixel data; perform drawing and compositing processing on the pixel data to obtain the bitmap corresponding to the predicted interface. Among them, the drawing processing is used to fill the pixel data with visual attributes such as color, border, and shadow; the compositing processing is used to overlay multiple drawn interface elements in sequence.

[0072] 5. Output the prediction interface; including but not limited to outputting the bitmap corresponding to the prediction interface as an image file or displaying it on the target screen, or outputting the prediction DOM tree data, screenshot data, page metric information, etc. corresponding to the prediction interface through the developer tool interface.

[0073] It is worth noting that the above-described process of obtaining the prediction interface is merely an illustrative example, and the specific process of generating the prediction interface based on the prediction interface code in this application embodiment is not limited. The above-described process of generating the prediction interface is a rendering process, which does not depend on a physical display device. At least one step of the processing is completed in memory, which can realize automated and batch interface generation and interface image acquisition.

[0074] Step 230: Based on the reference interface code and the reference interface corresponding to the reference interface code, obtain the evaluation results for the prediction interface code.

[0075] The evaluation results are used to indicate the degree of similarity between the predicted interface code and the reference interface code.

[0076] In some embodiments, the evaluation result includes a first evaluation value, or the evaluation result includes a first evaluation value and a second evaluation value. The first evaluation value indicates the degree of structural similarity between the predicted interface and the reference interface, and the second evaluation value indicates the degree of style similarity between the predicted interface and the reference interface.

[0077] The reference interface is generated based on reference interface code. The process of generating the reference interface from the reference interface code is consistent with the process of generating the prediction interface from the prediction interface code obtained from the first model. The interface structure of the prediction interface is obtained during the generation of the prediction interface; similarly, the interface structure of the reference interface is obtained during the generation of the reference interface. Alternatively, the interface structure of the reference interface can be directly obtained as pre-configured data included in the reference interface.

[0078] The interface code is used to generate a first interface that meets the interface generation requirements corresponding to the sample prompt information. This first interface includes interface information and interface components for implementing interactive or display tasks. Optionally, the interface image of the first interface is a screenshot. In some embodiments, the interface information of the first interface is organized and displayed in a graphical and spatial manner. Taking an interactive component as an example, when the interface component receives an interactive operation from the user via a pointer device or touch device, the interface state of the first interface is updated in real time, providing the user with feedback in at least one aspect—visual or auditory.

[0079] In some embodiments, from the perspective of interface code, the interface component includes at least one interface element. The interface element includes, but is not limited to, document structure elements for defining the overall framework of the page, text elements for displaying text information, form elements for user input and data submission, multimedia elements for embedding resources such as images, audio and video, list elements for organizing grouped data items, semantic tag elements for describing the meaning of content and enhancing readability, and embedded elements for embedding external content or defining interactive areas.

[0080] In some embodiments, from a visualization perspective, the first interface includes interface components for implementing interactive tasks or demonstrating human tasks. Illustratively, taking an interactive interface component as an example, the interface component is a control that implements a specific function in the form of a graphical component, including but not limited to: 1. Container components, such as windows, panels, tabs, etc. 2. Information components, such as icons, progress bars, dialog boxes, etc. 3. Input components, such as text boxes, drop-down lists, sliders, etc. 4. Output components, such as static labels, status bars, tooltips, etc. 5. Control components, such as buttons, radio buttons and checkboxes, switches, etc. 6. Navigation components, such as menu bars, hierarchical navigation, toolbars, etc.

[0081] It is worth noting that the above classification of interface components is only an illustrative example. Interface components may include one or more specific functions, and the functions corresponding to different interface components may be the same or overlap. The specific implementation of interface components is not limited in the embodiments of this application.

[0082] The similarity requirement between the interface structure of the reference interface and the interface structure of the predicted interface is a quantitative constraint set during the evaluation of the predicted interface, defining the degree of commonality and difference between the interface structures of the reference interface and the predicted interface. It is used to specify the evaluation criteria for evaluating the predicted interface. The similarity requirement can be characterized by a mathematical function that quantifies the difference or degree of similarity, such as by distance metrics (e.g., Euclidean distance, cosine similarity) or overlap metrics (e.g., Jaccard coefficient, F1 score).

[0083] In some embodiments, the interface structure of the reference interface is obtained based on the reference interface code, and the interface structure of the predicted interface is obtained based on the predicted interface code; a first evaluation value is determined by performing similarity matching processing on the interface structure of the reference interface and the interface structure of the predicted interface, the first evaluation value being used to indicate the degree of structural similarity between the predicted interface and the reference interface; and an evaluation result is obtained based on the first evaluation value.

[0084] In the above embodiments, a first evaluation value is obtained based on the similarity requirement between the interface structure of the reference interface and the interface structure of the predicted interface. The interface structure is the layout tree information corresponding to the first interface, such as DOM tree data or view tree data, where the view is used to indicate the interface components in the first interface, and the view tree data is used to describe the tree-like organizational structure of at least one interface component in the first interface. Optionally, the first interface includes a predicted interface and a reference interface. In some embodiments, the interface structure includes the position information and text information of the interface components.

[0085] In the above embodiments, the engineering implementation forms corresponding to the similarity requirements include, but are not limited to, at least one of the following forms.

[0086] 1. Judgment Criteria: As a binary filtering mechanism for qualified / unqualified, schematically, if the similarity value exceeds the judgment threshold, the first evaluation value corresponding to the similarity requirement is obtained; otherwise, the first evaluation value corresponding to the similarity requirement is obtained.

[0087] 2. Scoring criteria: As a quantitative evaluation indicator, the similarity value is directly used as the evaluation value. Indicatively, the evaluation value can range from 0 to 100%. The higher the evaluation value, the more similar the interface structure of the reference interface is to the interface structure of the predicted interface, that is, the more the interface structure of the predicted interface meets the interface generation requirements corresponding to the sample prompt information.

[0088] 3. Sorting Criteria: As a benchmark for comparison, illustratively, when the first model generates multiple prediction interfaces, the interface structures of the prediction interfaces are sorted from high to low according to their similarity to the interface structures of the reference interfaces, thereby obtaining the first evaluation value based on the order of the interface structures of the prediction interfaces.

[0089] 4. Optimization objective: As a loss function, for example, during the training of the first model, minimize the difference between the interface structure of the prediction interface and the interface structure of the reference interface. This difference can be realized as mean squared error, contrastive loss, etc.

[0090] 5. Clustering or grouping criteria: For example, group reference interfaces with high similarity to the interface structure of the predicted interface into one category, and obtain the first evaluation value based on the category of the reference interface.

[0091] 6. Warning Triggering Condition: As an anomaly detection threshold, schematically, when the similarity value between the interface structure of the test interface and the interface structure of the prediction interface is lower than the preset anomaly detection threshold, the first evaluation value corresponding to triggering the warning is obtained; otherwise, the first evaluation value corresponding to not triggering the warning is obtained.

[0092] It is worth noting that the above-mentioned similarity requirements are merely illustrative examples, and the specific implementation of obtaining the first evaluation value based on the interface structure of the reference interface and the interface structure of the predicted interface is not limited in the embodiments of this application.

[0093] In some embodiments, if the first evaluation value meets the preset requirements, a second evaluation value is obtained based on the interface style of the reference interface and the interface style of the prediction interface; and an evaluation result is obtained based on the first evaluation value and the second evaluation value.

[0094] Schematic illustration: The preset requirement can be implemented as follows: the first evaluation value reaches a preset threshold. The preset threshold indicates the lower limit of the first evaluation value. If the first evaluation value does not reach the preset threshold, the prediction interface is determined to not meet the interface generation requirements corresponding to the sample prompt information. If the first evaluation value reaches the preset threshold, the interface structure of the prediction interface is determined to meet the interface generation requirements corresponding to the sample prompt information. The interface generation requirements corresponding to the sample prompt information include, but are not limited to, the required content of the generated interface described in the sample prompt information and the first interface requirement indicated by the reference interface.

[0095] In some embodiments, interface style refers to the set of consistent features presented by the first interface in terms of visual design and interaction or display, including visual elements such as color, font, control shape, spacing, animation effects, and layout patterns, as well as interaction or display rules. Interface style is used to define the overall visual tone and behavioral norms of the first interface, ensuring that different pages or interface components maintain consistency in visual and interactive or display operations, thereby improving user recognition efficiency and user experience.

[0096] Optionally, the interface generation requirements include at least one of the interface structure requirements for the prediction interface and the interface style requirements for the prediction interface; the interface structure requirements correspond to the evaluation criteria for the interface structure and detail dimensions, such as whether the interface components are complete and whether the layout meets the requirements of the first interface; the interface style requirements correspond to the evaluation criteria for the overall image style dimension of the interface, such as whether the colors of the prediction interface are harmonious, whether the crowding affects the visual experience, and whether the style consistency meets the requirements of the first interface.

[0097] In some embodiments, the methods for obtaining a second evaluation value based on the interface style of the reference interface and the interface style of the predicted interface include, but are not limited to: obtaining a second evaluation value based on the consistency requirement between the interface style of the reference interface and the interface style of the predicted interface, wherein the principle of obtaining a second evaluation value based on the consistency requirement is the same as the principle of obtaining a first evaluation value based on the similarity requirement between the interface structure of the reference interface and the interface structure of the predicted interface, and will not be elaborated here; or, directly obtaining a second evaluation value corresponding to the predicted interface based on a preset reward association relationship, wherein the reward association relationship can be a reward function or a pre-trained large language model. Optionally, the second evaluation value can be implemented as a second reward score.

[0098] Step 240: Train the first model based on the evaluation results.

[0099] Based on the evaluation results, the first model is trained through reinforcement learning. The evaluation results are the evaluation results of the interface structure and interface style from the perspective of the first model through trial and error interaction with the environment. Based on the evaluation results, the first model is trained to learn a better generation strategy than the current generation strategy indicated by the environment, and maximizes the accumulation of reward signals indicated by the evaluation results of each training round during multiple rounds of iterative training.

[0100] In a schematic representation, the agent in the reinforcement learning training process is a decision-making entity that executes actions and learns policies, such as the first model; the environment is the external system through which the agent interacts, such as the preset rules corresponding to obtaining evaluation results; the state is the representation of the environment at a certain moment in the training process, such as the currently generated interface code or the predicted interface; the action is the operation that the agent can perform in a certain state during the training process, such as the interface code corresponding to generating sample prompt information; the policy is a mapping function from state to action, used to determine the mapping relationship between state A and action B in the agent's "do action B in state A"; the reward signal is the immediate scalar feedback from the environment to the agent's actions, used to measure the degree to which the action conforms to the agent's application requirements, such as the evaluation result; and the cumulative reward is the total reward obtained by the agent in the complete interaction sequence.

[0101] The evaluation result, as a reward signal, is the core supervisory signal in the reinforcement learning training process. It transforms the abstract "whether the interface code generation task of the first model is successful" into a specific numerical indicator, clarifying the corresponding optimization direction of the first model during training. For example, for high evaluation values, the strategy is adjusted to increase the probability of action occurrence, and for low evaluation values, the strategy is adjusted to decrease the probability of action occurrence. Furthermore, the first evaluation value and the second evaluation value are combined and integrated through weighted combination to integrate different evaluation dimensions, enabling the first model to optimize multiple objectives simultaneously during training, avoiding the reward hacking phenomenon caused by a single indicator.

[0102] In some embodiments, the application scenarios for interface code generation models to generate interfaces include, but are not limited to, at least one of the following: 1. Automatically generating editable interface prototypes through natural language descriptions; 2. Converting design drafts or text descriptions into front-end code; 3. Dynamically generating custom layouts based on user preferences to adapt to personalized layouts; 4. Generating automated test cases and building virtual test interfaces; 5. Automatically generating interfaces that conform to the Web Content Accessibility Guidelines (WCAG) standards based on semantics to assist in accessibility design. The above application scenarios are merely illustrative examples and are not intended to limit the scope of the application.

[0103] In summary, in this embodiment, based on the predicted interface code corresponding to the sample prompt information obtained through the first model, the predicted interface corresponding to the predicted interface code is evaluated based on the reference interface code. The evaluation result is used to assess the similarity between the predicted interface code generated by the first model and the reference interface code. The evaluation result guides the model update, expands the evaluation of the interface from the dimension of interface code, improves the accuracy of evaluating training data during training, thereby improving the model training accuracy and efficiency. Furthermore, generating interface code through the trained model can improve the consistency between the interface code and the generation requirements, thus improving the model generation accuracy.

[0104] Figure 3 This is a flowchart of a model training method provided in another embodiment of this application. The method is performed by a computer device (which can be implemented as follows). Figure 1 The method can be executed by the terminal 120 or the server 140 shown, or it can be executed by the terminal, the server, or both. This embodiment takes the method being executed by the server as an example, and the method further includes steps 310 to 320.

[0105] Step 310: Obtain the prediction layout information of the prediction interface and the reference layout information of the reference interface.

[0106] In some embodiments, a reference interface corresponding to the sample prompt information is collected from at least one target platform via a preset network interface. Optionally, reference interface code corresponding to the reference interface is obtained from the target platform based on keywords corresponding to the sample prompt information, and the reference interface corresponding to the sample prompt information is generated based on the reference interface code; or, the reference interface is directly collected from the target platform based on keywords corresponding to the sample prompt information, such as obtaining reference layout information directly from the web page information of the target platform. For example, based on the sample prompt information "generate a login interface with username and password input boxes", the keyword "login_activity" is obtained, and the reference interface code corresponding to the reference interface is collected from the target platform based on the keyword, thereby generating the reference interface. Optionally, the reference interface obtained from the publicly disclosed target platform can be implemented as obtaining at least one of the following: reference interface code, reference interface, reference interface image, and reference layout information.

[0107] The target platform can be implemented as an open-source engine, that is, the core, reusable driver and computing module of the software system. The source code of this module is publicly available, and it can obtain publicly available reference interface code or reference layout information with keyword tags by running preset scripts. Obtaining keywords corresponding to sample prompts includes, but is not limited to, receiving keyword text given by the user based on sample prompts, obtaining keywords by calculating statistical features such as word frequency in sample prompts, extracting keywords through pre-trained deep learning models, extracting keywords based on syntactic structure according to preset grammatical rules (such as proper nouns or core verbs), or obtaining text content with specific styles (such as large font size, centered display, etc.) as keywords from multimodal sample prompts.

[0108] To illustrate, high-quality pre-configured interface code snippets can be extracted or collected directly from open-source code engines, code hosting platforms, or from real-world application (App) packages via preset automated scripts or APIs. Here, "high-quality" means that the interface code snippets meet the interface generation requirements corresponding to the sample prompt information, and "real-world App package" refers to an application installation package released to end users that runs in a real Internet environment.

[0109] By directly collecting pre-configured reference interfaces, the bottleneck of computational power that relies entirely on the slow generation of training data by the model to be trained is broken during the reinforcement learning training process. This reduces the cost of quickly acquiring massive amounts of real code samples, increases the speed of sample construction corresponding to training data, enriches the distribution characteristics corresponding to interface code samples, ensures the diversity of training data, and expands the reference breadth of training data by using the reference interface code obtained from the target platform as the real business distribution.

[0110] In some embodiments, the layout information of the first interface includes the predicted layout information of the predicted interface and the reference layout information of the reference interface; the layout information of the first interface includes the text information and position information of at least one interface component in the first interface. For illustrative purposes, please refer to... Figure 4 , Figure 4 This is a schematic diagram illustrating the acquisition of layout information provided in an exemplary embodiment of this application, such as... Figure 4 As shown, taking the acquisition of predicted layout information of the predicted interface as an example, the interface code is rendered and parsed in a lightweight sandbox environment to obtain the predicted interface, including the predicted layout information of the predicted interface. Taking the browser 410 executing the rendering process as an example, the process of converting the original interface code into DOM tree data 420 and then into layout tree information 430 is as follows: Specifically, the browser main thread 440 converts the DOM nodes that have already had their style attributes calculated into layout tree information for interface layout. Among them, the DOM nodes that have already had their style attributes calculated represent the calculated styles, which are the inline styles, internal style sheets, and CSS files corresponding to the interface code parsed by the browser. The actual set of style attributes calculated for each DOM node is the output result of the style calculation, including but not limited to the color, font size, width, and height of the interface elements.

[0111] like Figure 4 As shown, there is a correspondence between DOM nodes and UI elements. For example, multiple DOM nodes correspond to one UI element, or one DOM node corresponds to multiple UI elements. The DOM tree data 420 is processed for layout to obtain layout tree information 430. Layout tree information 430 indicates how UI elements occupy space in the predicted UI and the positional relationships between UI elements. Layout tree information 430 includes block-level fluid layout, which refers to a layout mode that uses a block-level formatting context as the layout rule to arrange block-level UI elements vertically. Block-level UI elements are arranged vertically according to block-level rules, with each corresponding UI element occupying a single line. The layout text is the leaf node in the layout tree information 430 that indicates plain text content. The layout text carries the actual rendered character data and is contained within an inline box. Optionally, the inline box can be an explicitly defined inline container by the developer or an anonymous inline box automatically generated within a block-level container.

[0112] like Figure 4As shown, the main thread 440 is the core execution thread in the browser rendering process, responsible for at least one task, such as constructing the DOM tree data 420, calculating style patterns, handling layout and typesetting, and executing scripts, to drive the complete rendering process. It is worth noting that obtaining the reference layout information of the reference interface is based on the same principle as obtaining the predicted layout information of the predicted interface, and will not be elaborated here; alternatively, the reference layout information can be directly collected during the process of obtaining the reference interface.

[0113] Step 320: Obtain the first evaluation value based on the reference layout information and the predicted layout information.

[0114] The first evaluation value is obtained by comparing and calculating the key element sets in the reference layout information and the predicted layout information. The comparison and calculation are performed based on preset calculation rules, which are deterministic logical rules defined manually without relying on machine learning models, statistical inference, or probability sampling. Exemplarily, the calculation rules include, but are not limited to, conditional judgments, arithmetic operations, string matching, and threshold comparisons. In some embodiments, predicted layout information is extracted from the predicted interface code, and reference layout information is extracted from the reference interface. Based on the reference layout information and the predicted layout information, objective verification is performed to evaluate the predicted interface and obtain the first evaluation value. Optionally, the first evaluation value can be implemented as a first reward score.

[0115] In some embodiments, the predicted layout information includes predicted text information of at least one interface component in the predicted interface; a text matching degree is obtained based on the predicted text information and reference text information in the reference layout information, the text matching degree being used to indicate the text consistency between the predicted interface and the reference interface; and a first evaluation value is obtained based on the text matching degree.

[0116] To illustrate, text information refers to the text content displayed on interface components, not descriptive text for the first interface, such as the "Login" text on a button component or the "Username" text on a label component. In other words, text information refers to the visible text content of the interface. Based on the obtained text information, the text matching degree is obtained based on the predicted text information and the reference text information in the reference layout information. The text matching degree can be implemented as text recall, which is the proportion of predicted text information that actually exists in the predicted interface that overlaps with the reference text information. Where predicted text information overlaps with reference text information, it means that the predicted text information has been correctly identified. The correctness of the predicted text information is determined according to the interface generation requirements.

[0117] The first evaluation value is obtained based on predicted text information and reference text information, realizing a deep deconstructive evaluation of the quality of the generated prediction interface. This avoids relying solely on superficial visual similarity and forces a precise evaluation of the text content generated at the code level. This fundamentally eliminates the possibility that the evaluation model corresponding to the reward signal may be deceived by inferior sample interfaces that are "visually similar but structurally incorrect." The evaluation benchmark is transformed from subjective judgment of the evaluation model to objective calculation, ensuring the logical and functional correctness of the prediction interface code corresponding to the prediction interface used to train the first model. This improves the objectivity and absolute fairness of the evaluation process and increases the efficiency of model training.

[0118] In some embodiments, the predicted layout information includes the predicted position information of at least one interface component in the predicted interface; a position matching degree is obtained based on the predicted position information and the reference position information, the position matching degree being used to indicate the consistency of component positions between the predicted interface and the reference interface; and a first evaluation value is obtained based on the position matching degree.

[0119] Schematic, the location information can be implemented as the bounding box coordinates of at least one UI element in the UI component. The bounding box coordinates correspond to the layout rules in the process of obtaining the first UI based on the UI code, and are used to accurately locate the position and occupied area of ​​the UI element on the target screen. The bounding box coordinates include, but are not limited to, the x-coordinate (x) of the top-left corner of the bounding box, the y-coordinate (y) of the top-left corner of the bounding box, the width (width, w) of the bounding box, and the height (height, h) of the bounding box.

[0120] In the above embodiments, when the predicted position information and the reference position information are obtained, the position matching degree is obtained based on the predicted position information and the reference position information. The position matching degree can be realized as the intersection over union (IoU) ratio of the bounding box coordinates, which is the ratio of the intersection area to the union area between the bounding box of the interface element in the predicted position information and the bounding box of the interface element in the reference position information. The value range of the bounding box coordinate IoU is [0,1]. The bounding box coordinate IoU of 1 is used to indicate that the bounding box of the interface element in the predicted position information and the bounding box of the interface element in the reference position information are completely coincident, that is, the predicted position information and the reference position information are completely consistent. The bounding box coordinate IoU of 0 is used to indicate that the bounding box of the interface element in the predicted position information and the bounding box of the interface element in the reference position information do not overlap, that is, the predicted position information and the reference position information are completely inconsistent.

[0121] Optionally, the location matching degree obtained based on the reference location information in the predicted location information and the reference location information can be targeted at key interface components in at least one interface component in the predicted interface. The location matching degree can be implemented as a precise numerical matching result, such as the intersection-union ratio of bounding box coordinates; the location matching degree can also be implemented as a relative matching result of location information, such as the orientation relationship between at least two interface components, or the interface component being located in a specific area of ​​the predicted interface. For example, based on the reference location information of the reference interface, when the search box of the predicted interface is located to the lower right of the refresh button, the first evaluation value is obtained as a preset higher evaluation value of 0.8; or, based on the reference location information of the reference interface, when the enter button of the predicted interface is located in the (1,1) area of ​​the predicted interface, the first evaluation value is obtained as a preset higher evaluation value of 0.9, wherein the predicted interface is pre-divided into multiple positioning areas, and the (1,1) area is the positioning area located in the first row and first column.

[0122] The first evaluation value is obtained based on predicted and reference position information, realizing a deep deconstructive evaluation of the quality of the generated prediction interface. This avoids relying solely on superficial visual similarity and forces a precise evaluation of the position information corresponding to the DOM node structure generated at the code level. This fundamentally eliminates the possibility that the evaluation model corresponding to the reward signal may be deceived by inferior sample interfaces that are "visually similar but structurally incorrect." The evaluation benchmark is transformed from subjective judgment of the evaluation model to objective calculation, ensuring the logical and functional correctness of the prediction interface code corresponding to the prediction interface used to train the first model. This improves the objectivity and absolute fairness of the evaluation process and increases the efficiency of model training.

[0123] In some embodiments, the first evaluation value is obtained for at least one of location information and text information, that is, the first evaluation value is obtained based on at least one of text matching degree and location matching degree; when both text matching degree and location matching degree are obtained, a weighted fusion process is performed on the text matching degree and location matching degree to obtain the first evaluation value.

[0124] For example, when the predicted layout information is implemented as predicted DOM tree data {"type": "button", "text":"Login", "bbox": [100,500,200,50]}, the predicted text information "Login" and the predicted position information corresponding to the predicted interface are obtained: the bounding box coordinates are [100,500,200,50]. When the reference layout information is implemented as reference DOM tree data {"type": "button", "text": "Login", "bbox": [105,524,195,74]}, the reference text information "Login" and the reference position information corresponding to the reference interface are obtained: the bounding box coordinates are [105,524,195,74]. Through calculation rules, the text matching degree is obtained as text recall = 1, and the position matching degree is obtained as bounding box coordinate intersection-union ratio = 0.8. A weighted fusion process is performed on the text matching degree and the position matching degree to obtain the first evaluation value. The first evaluation value = text matching degree × text weight + position matching degree × position weight. Accordingly, the first evaluation value = 1 × 0.5 + 0.8 × 0.5 = 0.9.

[0125] In some embodiments, if the first evaluation value does not reach a preset threshold, a preset penalty value is obtained, which is used to indicate the degree of structural difference between the predicted interface and the reference interface; the first model is trained based on the penalty value. The preset threshold is used to truncate predicted interfaces that do not meet the interface generation requirements; for example, if the first evaluation value is 0.9 and the preset threshold is 0.6, the first evaluation value reaches the preset threshold, and a second evaluation value is obtained; if the first evaluation value is 0.4 and the preset threshold is 0.6, the first evaluation value does not reach the preset threshold, and a penalty value is obtained as a reward signal, which is used to indicate the degree of structural difference between the predicted interface and the reference interface in reinforcement learning terminology; wherein, the reward signal is a generalized scalar feedback value used to indicate the training direction of the first model, and the reward signal may include at least one of positive numbers (such as the first evaluation value or the second evaluation value), negative numbers (such as the penalty value), and zero.

[0126] The first evaluation value is the evaluation result calculated based on the DOM tree data obtained from the rendering process. For evaluation results that are lower than the preset threshold, the prediction result corresponding to the evaluation result is directly rejected. That is, the preset penalty value is obtained as the reward information to train the first model, skipping the more costly process of obtaining the second evaluation value corresponding to the interface style, that is, the visual evaluation process. This enables rapid filtering of inferior samples, improves the funnel mechanism based on the first evaluation value, and improves the training efficiency of the model.

[0127] In some embodiments, a predicted interface image of the predicted interface is acquired, and a reference interface image of the reference interface is acquired; the predicted interface image, the reference interface image, and preset evaluation prompts are input into a second model to output a second evaluation value, the second model being used to evaluate the interface style corresponding to the interface image; optionally, the second model is a pre-trained large language model. Indicatively, the interface image and the reference interface image are implemented in Portable Network Graphics (PNG) format; the second model is a VLM serving as a subjective visual scoring engine.

[0128] In some embodiments, sample interface image pairs are obtained, each sample interface image pair is labeled with a reference evaluation value, and the reference evaluation value is used to indicate the consistency of image style between sample interface image pairs; a second model is trained based on the sample interface image pairs.

[0129] The second model is trained by using sample interface image pairs and their corresponding reference evaluation values. The correspondence between the sample interface image pairs and the reference evaluation values ​​provides the basis for training the second model. Based on the reference evaluation values, the difference between the second evaluation values ​​predicted by the second model and the reference evaluation values ​​can be obtained, objectively quantifying the performance of the second evaluation value prediction model and improving the training efficiency and accuracy of the second model during iterative training.

[0130] In the above embodiments, the reference evaluation value is labeled from a subjective perspective and is used to express the consistency of image style between two interface images in a sample interface image pair. The subjective perspective uses the user's perceived style as the target of the final interface style evaluation; the second model uses the reference evaluation value labeled from the subjective perspective to simulate the user's subjective visual experience, evaluate the consistency of image style between the predicted interface and the reference interface, and obtain a second evaluation value. Illustratively, subjective visual experience includes, but is not limited to, typography style and subjective aesthetics, and can be evaluated from aspects such as color, overall layout, component style, animation and interactive feedback, text font and spacing.

[0131] The reference evaluation values, from a subjective perspective, include, but are not limited to, at least one of the following steps: 1. Manually annotating the reference evaluation values ​​for sample interface image pairs. 2. Performing preprocessing on the reference evaluation values, such as outlier removal, data cleaning and standardization, and noise reduction, to obtain interface style evaluation results with group interpretability, thereby improving the objectivity of the reference evaluation values ​​at the analytical level. 3. Annotating the reference evaluation values ​​to the sample interface image pairs.

[0132] The pre-defined evaluation prompts are pre-defined instructions used to define the roles and behavioral guidelines of the second model. These prompts typically do not contain dynamically injected specific knowledge content. Understandably, the second model is used to simulate the user's ability to evaluate the consistency of image style between the predicted interface image and the reference interface image. The reference evaluation values, labeled from a subjective perspective, express the human eye's visual experience of image style. For example, the evaluation prompts are implemented as "evaluate the visual style consistency between the predicted interface image and the reference interface image."

[0133] The second model is a machine learning model trained on reference evaluation values ​​labeled from a subjective perspective on sample interface images. In other words, the second model simulates the human eye's visual perception of image quality and evaluates the consistency of image style between the interface image of the predicted interface and the interface image of the reference interface with subjective evaluation ability to obtain a second evaluation value. Thus, the predicted interface is evaluated by combining the interface structure and interface style to avoid reward deception and evaluation illusion. This reduces computational costs and improves inference speed and throughput, guiding the first model to perform iterative training.

[0134] For illustrative purposes, please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of the interface code generation model training process provided in an exemplary embodiment of this application, such as... Figure 5 As shown, in the first stage, user command 511 and reference context 512 are input to the first model 510, and the interface code sample generated by the first model 510 is output, which is the predicted interface code corresponding to the predicted interface. Here, user command 511 is sample prompt information, and the first model 510 is a pre-acquired base model used to generate the interface code. The predicted interface code generated by the first model 510 is stored in the interface code sample pool 520. Interface code samples, which are reference interface codes corresponding to the reference interface, are collected from at least one target platform 530 through a preset network interface 531. The network interface 531 collects reference interfaces based on keywords in the sample prompt information. The reference interface codes collected from the target platform 530 are stored in the interface code sample pool 520. It can be understood that the interface code samples in the interface code sample pool 520 include the predicted interface code generated by the first model 510 obtained through the sample exploration path, and the reference interface code collected from the target platform 530 obtained through the sample filling path.

[0135] like Figure 5As shown, in the second stage, the interface code samples in the interface code sample pool 520 are input into the headless layout engine sandbox 540 to parse and render the predicted interface code and the reference interface code, generating corresponding screenshots of the predicted interface and the reference interface, as well as the predicted layout information of the predicted interface and the reference layout information of the reference interface. The predicted layout information is implemented as predicted DOM tree data, and the reference layout information is implemented as reference DOM tree data. A first evaluation value is obtained based on the interface structure of the reference interface and the interface structure of the predicted interface; that is, a structural detail evaluation 550 is performed on the predicted DOM tree data and the reference DOM tree data to obtain the first evaluation value. The predicted layout information includes predicted text information and predicted position information of at least one interface component in the predicted interface, and the reference layout information includes reference text information and reference position information of at least one interface component in the reference interface. Accordingly, the structural detail evaluation 550 includes, but is not limited to, obtaining text matching degree based on the predicted text information and the reference text information, and obtaining position matching degree based on the predicted position information and the reference position information. For example, the text matching degree can be implemented as text recall, and the position matching degree can be implemented as the intersection-union ratio (IUU).

[0136] like Figure 5 As shown, the first evaluation value is input into the threshold determiner 560. If the first evaluation value reaches the threshold, a second evaluation value is obtained based on the interface style of the predicted interface. The interface image of the predicted interface is obtained through the headless layout engine sandbox 540, and a reference interface image of the reference interface is obtained. The interface image is implemented as an interface screenshot, and the reference interface image is implemented as a reference interface screenshot. An overall image evaluation 570 is performed on the interface screenshot and the reference interface screenshot. The overall image evaluation 570 includes, but is not limited to, inputting the interface screenshot, the reference interface screenshot, and preset evaluation prompts into the second model 571 to output the second evaluation value. The second model 571 is a pre-trained large language model used to evaluate the interface image from at least two style perspectives, such as consistency of typography style and visual aesthetics with the reference interface image.

[0137] like Figure 5As shown, when the first evaluation value is input into the threshold determiner 560, if the first evaluation value does not reach the threshold, the sample filtering mechanism 561 is triggered, returning a preset low reward signal, i.e., the penalty score, as the reward signal for the current training round. It can be understood that in the third stage, the comprehensive reward fusion model 580 comprehensively processes at least one of the first and second evaluation values ​​to obtain the reward signal. This comprehensive processing includes, but is not limited to: when the first evaluation value reaches the preset threshold and a second evaluation value is obtained, weighted fusion of the first and second evaluation values ​​to obtain the reward signal; when the first evaluation value does not reach the preset threshold, the penalty score is used as the reward signal. Based on this reward signal, reinforcement learning is performed to update the first model 590, adjusting the model parameters of the first model 510, and obtaining the prediction interface code for the next training round through the updated first model, iterating training until the training requirements are met.

[0138] In summary, in this embodiment, based on the predicted interface code corresponding to the sample prompt information obtained through the first model, the predicted interface corresponding to the predicted interface code is evaluated based on the reference interface code. The evaluation result is used to assess the similarity between the predicted interface code generated by the first model and the reference interface code. The evaluation result guides the model update, expands the evaluation of the interface from the dimension of interface code, improves the accuracy of evaluating training data during training, thereby improving the model training accuracy and efficiency. Furthermore, generating interface code through the trained model can improve the consistency between the interface code and the generation requirements, thus improving the model generation accuracy.

[0139] The method provided in this embodiment evaluates the prediction layout information of the prediction interface by introducing a millisecond-level pure Central Processing Unit (CPU) objective scoring engine. It visualizes the interface structure as layout information, thereby objectively evaluating the consistency between the prediction interface and the interface generation requirements. This implements a funnel mechanism to filter out inferior samples whose first evaluation value does not reach a preset threshold, avoids invalid inference calls to the second model, reduces evaluation costs, improves overall training speed and throughput, and improves model training efficiency.

[0140] Figure 6 This is a flowchart of a model training method provided in another embodiment of this application. The method is implemented using a computer device (which can be configured as follows). Figure 1 The method can be executed by the terminal 120 or the server 140 shown, or it can be executed by the terminal, the server, or both. This embodiment takes the method being executed by the server as an example, and the method further includes steps 242 to 244.

[0141] Step 242: Obtain a reward signal based on the first evaluation value and / or the second evaluation value.

[0142] In some embodiments, a reward signal is obtained by fusing the first evaluation value and the second evaluation value. The fusing of the first evaluation value and the second evaluation value includes, but is not limited to, at least one of the following methods.

[0143] 1. Obtain the first weight corresponding to the first evaluation value, and obtain the second weight corresponding to the second evaluation value; perform a weighted summation of the first evaluation value and the second evaluation value based on the first weight and the second weight to obtain the reward signal. For example, when the first weight is a and the second weight is b, the reward signal = first evaluation value × first weight + second evaluation value × second weight. In this case, the reward signal is a linear combination of the evaluation values ​​of each dimension, and each dimension's evaluation value contributes independently to the reward signal.

[0144] 2. Multiply and fuse the first and second evaluation values. For example, the reward signal = first evaluation value × second evaluation value. At this point, the reward signal is zero when the evaluation value of any dimension is 0. That is, the interface generation requirements for each evaluation dimension must be met simultaneously to obtain positive feedback for the training of the first model. For example, when the first evaluation value is 0.9 and the second evaluation value is 0.9, the reward signal = 0.9 × 0.9 = 0.81.

[0145] It is worth noting that the above-described method of merging the first and second evaluation values ​​is merely an illustrative example, and this application does not limit the specific method for obtaining the reward signal. For example, the merging process may also include harmonic average processing, geometric average processing, extreme value merging processing, etc.

[0146] Step 244: Iteratively train the first model based on the reward signal until it meets the training requirements.

[0147] In some embodiments, based on the reward signal, the policy loss value for the current round is obtained, and the policy loss value is used to indicate the difference between the prediction interface and the reference interface; based on the policy loss value for the current round, the gradient value of the first model for the current round is obtained through the backpropagation algorithm; based on the gradient value for the current round, the model parameters of the first model are adjusted during the training of the current round until they meet the training requirements. Optionally, if the adjusted first model meets the training requirements, the updating of the first model is stopped.

[0148] In some embodiments, a first model is iteratively trained using the Group Relative Policy Optimization (GRPO) algorithm based on reward signals, wherein the first model may be an actor model. The GRPO algorithm training process for the first model includes, but is not limited to, at least one of the following steps: First, based on the same sample prompt information, at least one answer path is generated in parallel by the first model to obtain a sampling group; Given the reward signals corresponding to each answer path, the average reward signal corresponding to at least one answer path in the sampling group is obtained, and this average reward signal is used as the evaluation baseline; The average reward signal of each answer path is subtracted from the corresponding average reward signal to obtain the relative reward signal of each answer path; Policy loss values ​​with different weights are obtained based on the relative reward signals of each answer path, and the policy loss values ​​are used to indicate the difference between the answer path and the target answer path corresponding to the interface generation requirements; Based on the policy loss values, the gradient value of the first model in the current round is obtained through the backpropagation algorithm; Based on the gradient value of the current round, the model parameters of the first model are adjusted in the training of the current round until the training requirements are met to obtain the interface code generation model.

[0149] Optionally, based on the policy loss value, the direction and magnitude of the adjustment processing corresponding to the model parameters of the first model are obtained through backpropagation, i.e., the second gradient value. Illustratively, the partial derivative of the policy loss function with respect to the model parameters of the first model is obtained as the gradient value of the model parameters of the first model in the current round. Specifically, after updating the model parameters of the first model based on the gradient value of the current round, the policy loss value decreases accordingly, thereby reducing the difference between the answer path and the target answer path corresponding to the interface generation requirements.

[0150] In some embodiments, the model parameters of the first model are updated using an optimization algorithm based on the gradient value of the model parameters of the first model in the current iteration. Through backpropagation, the gradient value of the preset policy loss function relative to the model parameters of the first model in the current iteration is automatically calculated. This allows for the determination of the adjustment process that should be performed on the model parameters of the first model in the current iteration, as well as the urgency of the adjustment process. This adjusts the model parameters of the first model, improving the training efficiency of the model parameters. It also ensures that the policy loss value can be reduced in each iteration. The policy loss value is obtained through the policy loss function, providing a continuous quantitative indicator to clarify the direction of reinforcement learning training, providing a basis for further training of the first model, and improving the accuracy and training efficiency of the interface code generated by the first model.

[0151] The model parameters of the first model are adjusted based on gradient values ​​through multiple iterations. These iterations are controlled by training parameters, which include, but are not limited to: 1. Learning rate: Used to control the step size of parameter updates in each round. The size of the learning rate affects the efficiency of training. For example, if the learning rate is too large, the parameter iteration trajectory will be unstable, making it difficult to reach the convergence condition and ending the iteration. If the learning rate is too small, it will lead to slow convergence or cause the iteration process to stop prematurely in the suboptimal solution region.

[0152] 2. Batch size: This indicates the size of the subset of sample data input to the first model at one time during each round of iterative training; that is, the number of samples contained in the batch. Batch size affects the stability of the iterative training process. For example, a larger batch results in higher accuracy in obtaining the gradient values ​​for the current round. Furthermore, batch size affects the processor's memory requirements; a larger batch requires more storage space during training. Additionally, batch size also affects the convergence speed during iterative training. A larger batch allows for more data to be processed in parallel during each round, resulting in a higher iteration speed per round, but more iterations are needed to reach convergence, thus leading to slower convergence. Optionally, the batch size can be adjusted based on the available storage space.

[0153] 3. Training rounds refer to the number of iterations during the training process, indicating an optional implementation of the convergence condition. The number of training rounds is determined based on the complexity of the interface code generation task corresponding to the sample prompts. For example, complex interface code generation tasks, such as those with complex interface requirements, numerous inference steps, and long interface code, have a larger decision space, and therefore require more iterations in the iterative training process.

[0154] The number of training epochs affects the computational complexity and model training quality during training. To illustrate, too many training epochs can lead to overfitting of the first model and lower accuracy in performing UI code generation tasks. Too many training epochs can also lead to lower generalization performance of the first model and lower accuracy in performing UI code generation tasks.

[0155] It is worth noting that the above-mentioned method of setting training parameters is only an illustrative example, and the specific process of adjusting the parameters of the first model based on gradient values ​​through multiple rounds of iteration is not limited in the embodiments of this application.

[0156] In summary, in this embodiment, based on the predicted interface code corresponding to the sample prompt information obtained through the first model, the predicted interface corresponding to the predicted interface code is evaluated based on the reference interface code. The evaluation result is used to assess the similarity between the predicted interface code generated by the first model and the reference interface code. The evaluation result guides the model update, expands the evaluation of the interface from the dimension of interface code, improves the accuracy of evaluating training data during training, thereby improving the model training accuracy and efficiency. Furthermore, generating interface code through the trained model can improve the consistency between the interface code and the generation requirements, thus improving the model generation accuracy.

[0157] The method provided in this embodiment obtains a first evaluation value and a second evaluation value, enabling the first model to continuously interact with the environment corresponding to the sample prompt information, autonomously explore and learn the target interface code that meets the interface generation requirements; through the guidance of reward signals, the interface code generation model's understanding and satisfaction of the interface generation intent corresponding to the sample prompt information is enhanced, thereby improving the efficiency and accuracy of training the first model.

[0158] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0159] Please refer to Figure 7 This illustrates a block diagram of a model training apparatus provided in one embodiment of this application. Figure 7 As shown, the device includes: The first acquisition module 710 is configured to acquire sample prompt information and the reference interface code corresponding to the sample prompt information. The sample prompt information is used to express the requirements for generating the interface. The generation module 720 is configured to input sample prompt information into the first model to obtain the prediction interface code, and generate the prediction interface corresponding to the sample prompt information based on the interface code. The first model is a pre-acquired large language model used to generate the interface code. The second acquisition module 730 is configured to acquire an evaluation result for the predicted interface code based on the reference interface code and the reference interface corresponding to the reference interface code. The evaluation result is used to indicate the degree of similarity between the predicted interface code and the reference interface code. Training module 740 is configured to train the first model based on the evaluation results.

[0160] In an optional embodiment, the second acquisition module 730 is further configured to acquire the interface structure of the reference interface based on the reference interface code, and to acquire the interface structure of the prediction interface based on the prediction interface code. The second acquisition module 730 is further configured to perform similarity matching processing on the interface structure of the reference interface and the interface structure of the prediction interface to determine a first evaluation value, the first evaluation value being used to indicate the degree of structural similarity between the prediction interface and the reference interface. The second acquisition module 730 is also configured to acquire the evaluation result based on the first evaluation value.

[0161] In an optional embodiment, the second acquisition module 730 is further configured to acquire prediction layout information of the prediction interface and reference layout information of the reference interface. The second acquisition module 730 is also configured to acquire a first evaluation value based on reference layout information and predicted layout information.

[0162] In an optional embodiment, the predicted layout information includes predicted text information of at least one interface component in the predicted interface. The second acquisition module 730 is also configured to acquire a text matching degree based on the reference text information in the predicted text information and the reference layout information, wherein the text matching degree is used to indicate the text consistency between the predicted interface and the reference interface. The second acquisition module 730 is also configured to acquire a first evaluation value based on the text matching degree.

[0163] In an optional embodiment, the predicted layout information includes the predicted position information of at least one interface component in the predicted interface. The second acquisition module 730 is also configured to acquire a position matching degree based on the reference position information in the predicted position information and the reference layout information, wherein the position matching degree is used to indicate the consistency of component positions between the predicted interface and the reference interface. The second acquisition module 730 is also configured to acquire a first evaluation value based on the location matching degree.

[0164] In an optional embodiment, the second acquisition module 730 is further configured to acquire a second evaluation value based on the interface style of the reference interface and the interface style of the prediction interface if the first evaluation value meets the preset requirements. The second acquisition module 730 is also configured to acquire the evaluation result based on the first evaluation value and the second evaluation value.

[0165] In an optional embodiment, the second acquisition module 730 is further configured to acquire a prediction interface image of the prediction interface and a reference interface image of the reference interface. The second acquisition module 730 is further configured to input the predicted interface image, the reference interface image, and the preset evaluation prompt words into the second model to output a second evaluation value. The second model is used to evaluate the interface style corresponding to the interface image.

[0166] In an optional embodiment, the training module 740 is further configured to obtain a reward signal based on a first evaluation value and / or a second evaluation value; Training module 740 is also configured to iteratively train the first model based on the reward signal until it meets the training requirements.

[0167] In an optional embodiment, the training module 740 is further configured to obtain a policy loss value for the current round based on a reward signal, the policy loss value being used to indicate the difference between the prediction interface and the reference interface. The training module 740 is also configured to obtain the gradient value of the first model for the current round based on the policy loss value of the current round through the backpropagation algorithm; The training module 740 is also configured to adjust the model parameters of the first model in the current training round based on the gradient value of the current round until the training requirements are met.

[0168] In an optional embodiment, the second acquisition module 730 is further configured to acquire a preset penalty value if the first evaluation value does not reach a preset threshold. The penalty value is used to indicate the degree of structural difference between the prediction interface and the reference interface. The second acquisition module 730 is also configured to train the first model based on the penalty value.

[0169] In summary, the apparatus provided in this embodiment, based on the predicted interface code corresponding to the sample prompt information obtained through the first model, evaluates the predicted interface corresponding to the predicted interface code based on the reference interface code, and obtains the evaluation result to assess the similarity between the predicted interface code generated by the first model and the reference interface code; the evaluation result guides the model update, expands the evaluation of the interface from the dimension of interface code, improves the accuracy of evaluating training data during training, thereby improving the model training accuracy and efficiency; and by generating interface code through the trained model, the consistency between the interface code and the generation requirements can be improved, thus improving the model generation accuracy.

[0170] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0171] Figure 8This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. The computer device is a server 140. Typically, the server 140 includes a first processor 801 and a first memory 802.

[0172] The first processor 801 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The first processor 801 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The first processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the first processor 801 may integrate a GPU, which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the first processor 801 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0173] The first memory 802 may include one or more computer-readable storage media, which may be non-transitory. The first memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the first memory 802 is used to store at least one instruction, which is executed by the first processor 801 to implement the model training method provided in the method embodiments of this application.

[0174] In some embodiments, the server 140 may optionally include an input interface 803 and an output interface 804. The first processor 801, the first memory 802, and the input interface 803 and output interface 804 can be connected via a bus or signal line. Various peripheral devices can be connected to the input interface 803 and output interface 804 via a bus, signal line, or circuit board. The input interface 803 and output interface 804 can be used to connect at least one input / output (I / O) related peripheral device to the first processor 801 and the first memory 802. In some embodiments, the first processor 801, the first memory 802, and the input interface 803 and output interface 804 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the first processor 801, the first memory 802, and the input interface 803 and output interface 804 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0175] Figure 9 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Optionally, the computer device 900 is a terminal.

[0176] The computer device 900 can be a portable mobile terminal, also referred to as a mobile terminal in this embodiment. Examples include: smartphones, tablets, laptops, e-book readers, MP3 players, MP4 players, etc. The computer device 900 may also be referred to as user equipment, portable terminal, or other names.

[0177] Typically, computer device 900 includes: a second processor 901 and a second memory 902.

[0178] The second processor 901 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The second processor 901 may be implemented using at least one hardware form selected from DSP, FPGA, and PLA. The second processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the second processor 901 may integrate a GPU, which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, the second processor 901 may also include an AI processor, which is used to handle computational operations related to machine learning.

[0179] The second memory 902 may include one or more computer-readable storage media, which may be tangible and non-transitory. The second memory 902 may also include high-speed random access memory devices and non-volatile storage devices, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the second memory 902 are used to store at least one instruction, which is executed by the second processor 901 to implement the model training methods provided in the various method embodiments of this application.

[0180] In some embodiments, the computer device 900 may also optionally include a peripheral device interface 903 and at least one peripheral device. Specifically, the peripheral device includes at least one of a radio frequency circuit 904, a touch display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 908. The computer device 900 also includes one or more sensors 909. The one or more sensors 909 include, but are not limited to, an accelerometer 910, a gyroscope 911, a pressure sensor 912, an optical sensor 913, and a proximity sensor 914.

[0181] Those skilled in the art will understand that Figure 8 , Figure 9 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0182] On the other hand, embodiments of this application provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the model training method provided in the embodiments of this application as described above.

[0183] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the model training method provided in the embodiments of this application as described above.

[0184] On the other hand, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the model training method provided in the embodiments of this application as described above.

[0185] On the other hand, embodiments of this application provide a computer device including the processor described above. Optionally, the processor is a GPU. The computer device can be at least one of a portable computer, a desktop computer, a server, a server cluster, an AI computing cluster, and a cloud computing cluster. The AI ​​computing cluster can also be simply referred to as an intelligent computing cluster or a smart computing cluster.

[0186] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0187] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0188] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0189] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A model training method, characterized in that, The method includes: Obtain sample prompt information and the reference interface code corresponding to the sample prompt information, wherein the sample prompt information is used to express the requirements for generating the interface; Input the sample prompt information into the first model to obtain the prediction interface code; Based on the reference interface code and the reference interface corresponding to the reference interface code, an evaluation result is obtained for the predicted interface code, and the evaluation result is used to indicate the degree of similarity between the predicted interface code and the reference interface code; The first model is trained based on the evaluation results.

2. The method of claim 1, wherein, The step of obtaining the evaluation result for the predicted interface code based on the reference interface code and the reference interface corresponding to the reference interface code includes: The interface structure of the reference interface is obtained based on the reference interface code, and the interface structure of the prediction interface is obtained based on the prediction interface code. A first evaluation value is determined by performing similarity matching processing on the interface structure of the reference interface and the interface structure of the predicted interface. The first evaluation value is used to indicate the degree of structural similarity between the predicted interface and the reference interface. The evaluation result is obtained based on the first evaluation value.

3. The method of claim 2, wherein, The step of performing similarity matching between the interface structure of the reference interface and the interface structure of the predicted interface to determine the first evaluation value includes: Obtain the prediction layout information of the prediction interface and the reference layout information of the reference interface; The first evaluation value is obtained based on the reference layout information and the predicted layout information.

4. The method of claim 3, wherein, The prediction layout information includes the prediction text information of at least one interface component in the prediction interface. The step of obtaining the first evaluation value based on the reference layout information and the predicted layout information includes: The text matching degree is obtained based on the predicted text information and the reference text information in the reference layout information. The text matching degree is used to indicate the text consistency between the predicted interface and the reference interface. The first evaluation value is obtained based on the text matching degree.

5. The method of claim 3, wherein, The predicted layout information includes the predicted position information of at least one interface component in the predicted interface. The step of obtaining the first evaluation value based on the reference layout information and the predicted layout information includes: The position matching degree is obtained based on the predicted position information and the reference position information in the reference layout information. The position matching degree is used to indicate the consistency of component positions between the predicted interface and the reference interface. The first evaluation value is obtained based on the location matching degree.

6. The method of claim 2, wherein, Obtaining the evaluation result based on the first evaluation value includes: If the first evaluation value meets the preset requirements, a second evaluation value is obtained based on the interface style of the reference interface and the interface style of the prediction interface. The evaluation result is obtained based on the first evaluation value and the second evaluation value.

7. The method of claim 6, wherein, The step of obtaining a second evaluation value based on the interface style of the reference interface and the interface style of the predicted interface when the first evaluation value meets the preset requirements includes: Obtain the prediction interface image of the prediction interface, and obtain the reference interface image of the reference interface. The predicted interface image, the reference interface image, and the preset evaluation prompts are input into the second model to output the second evaluation value. The second model is used to evaluate the interface style corresponding to the interface image.

8. The method according to claim 6, characterized in that, Training the first model based on the evaluation results includes: A reward signal is obtained based on the first evaluation value and / or the second evaluation value; The first model is iteratively trained based on the reward signal until it meets the training requirements.

9. The method according to claim 8, characterized in that, The iterative training of the first model based on the reward signal until the training requirements are met includes: Based on the reward signal, the strategy loss value for the current round is obtained, and the strategy loss value is used to indicate the difference between the prediction interface and the reference interface; Based on the policy loss value of the current round, the gradient value of the first model for the current round is obtained through the backpropagation algorithm; Based on the gradient value of the current round, the model parameters of the first model are adjusted in the current round of training until they meet the training requirements.

10. The method according to claim 2, characterized in that, The method further includes: If the first evaluation value does not meet the preset requirements, a preset penalty value is obtained, which is used to indicate the degree of structural difference between the prediction interface and the reference interface; The first model is trained based on the penalty value.

11. A model training device, characterized in that, The device includes: The first acquisition module is configured to acquire sample prompt information and acquire reference interface code corresponding to the sample prompt information, wherein the sample prompt information is used to express the requirements for generating the interface; The generation module is configured to input the sample prompt information into the first model to obtain the prediction interface code; The second acquisition module acquires an evaluation result for the predicted interface code based on the reference interface code and the reference interface corresponding to the reference interface code. The evaluation result is used to indicate the degree of similarity between the predicted interface code and the reference interface code. The training module is configured to train the first model based on the evaluation results.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the model training method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the model training method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the model training method as described in any one of claims 1 to 10.

Images