Code generation method and device based on large model, storage medium and computer equipment

Abstract

The invention discloses a code generation method and device based on a large model, a storage medium and computer equipment. The method comprises the steps that a UI prototype draft of a target service is acquired; performing element identification on the UI prototype draft through a picture identification large model, and determining element information in the UI prototype draft; generating a code generation cue word based on the element information and a preset cue word, inputting the code generation cue word into a code generation large model to obtain an intermediate code corresponding to the UI prototype draft, and arranging a target service code of the target service based on the intermediate code. The automatic process from UI design to code generation is achieved, the tedious process of manually writing UI components and debugging interaction logic in traditional development is avoided, repeated work and human errors are reduced, developers can put more energy into core service function development, then development efficiency is improved, code quality is guaranteed, and the development cost is reduced. And rapid and high-quality delivery of the project is promoted.

Description

Technical Field

[0001] This application relates to the field of code generation technology, and in particular to a code generation method, apparatus, storage medium and computer equipment based on a large model. Background Technology

[0002] In traditional front-end development workflows, developers work from scratch, creating UI components based on prototypes provided by UX designers or product managers and integrating them into the business system. This process has many drawbacks, especially in UI component interaction design. Developers must manually develop and debug each component individually, resulting in a massive workload and being extremely time-consuming. When faced with complex interfaces, the repetitive design elements and layout configurations force developers to spend a significant amount of time and energy on these repetitive tasks, preventing them from fully focusing on implementing business functions. This not only significantly reduces development efficiency but can also negatively impact development quality due to distracted attention and time constraints, adversely affecting the smooth progress of the project.

[0003] Therefore, there is an urgent need for a technical solution that can effectively address the above problems to improve the efficiency and quality of front-end development. Summary of the Invention

[0004] In view of this, embodiments of this application provide a code generation method, apparatus, storage medium, and computer device based on a large model.

[0005] According to one aspect of this application, a code generation method based on a large model is provided, the method comprising: Obtain the UI prototype of the target business; The UI prototype is used to identify elements by recognizing a large model of images, thereby determining the element information in the UI prototype. Based on the element information and preset prompt words, code generation prompt words are generated. The code generation prompt words are then input into the code generation model to obtain the intermediate code corresponding to the UI prototype. The target business code of the target business is then arranged based on the intermediate code.

[0006] Optionally, the preset prompts include the code generation specifications for the target business; the step of inputting the code generation prompts into the large code generation model to obtain the intermediate code corresponding to the UI prototype includes: The code generation prompts are input into the code generation model, and the model generates intermediate code corresponding to the UI prototype based on the element information according to the code generation specifications.

[0007] Optionally, before performing element recognition on the UI prototype using image recognition large model to determine the element information in the UI prototype, the method further includes: Obtain UI design draft training samples, wherein the UI design draft training samples include UI design draft samples marked with basic element information, the marked basic element information including the element type and position; The general image recognition model is trained using the UI design sample to obtain the image recognition model.

[0008] Optionally, the UI design draft training samples include a first UI design draft training sample and a second UI design draft training sample. The first UI design draft training sample includes a first UI design draft sample marked with basic element information. The second UI design draft training sample includes a second UI design draft sample marked with basic element information and element description information of the second UI design draft sample. The element description information sample includes the relative positional relationship and hierarchical structure information between elements. The step of training a general image recognition model using the UI design sample to obtain the image recognition model includes: The convolutional neural network model is trained using the first UI design sample to obtain an element information recognition model; The element information recognition model is integrated into a general image recognition model to obtain an initial image recognition model. The second UI design draft sample is input into the initial image recognition model. The element information recognition model is used to predict the basic element information of the second UI design draft sample. Based on the element basic information prediction results, the general image recognition model is used to analyze the relative positional relationships and hierarchical structure of the elements in the second UI design draft sample to obtain element description information prediction results. Based on the element description information prediction results and the element description information, the parameters of the initial image recognition model are optimized to obtain the image recognition model. The step of identifying elements in the UI prototype using a large image recognition model to determine element information in the UI prototype includes: The UI prototype is input into the image recognition model to predict the basic element information of the UI prototype. Based on the predicted basic element information, the relative position and hierarchical structure information between elements in the UI prototype are identified, and the element information is output. The element information includes the basic element information, the relative position and hierarchical structure information between elements.

[0009] Optionally, the step of training the convolutional neural network model using the first UI design sample to obtain the element information recognition model includes: The first UI design draft sample is divided into a first sub-sample and a second sub-sample; The convolutional neural network model is trained in one stage based on the first subsample to optimize the parameters of the convolutional and fully connected layers of the convolutional neural network model, and then trained in two stages based on the second subsample to optimize the parameters of the fully connected layers of the convolutional neural network model, thereby obtaining the element information recognition model. The step of integrating the element information recognition model into a general image recognition model to obtain an initial image recognition model includes: A low-rank matrix is ​​added to the general image recognition model, and the element information recognition model is then integrated into the general image recognition model with the added low-rank matrix to obtain an initial image recognition model. The parameters of the initial image recognition model are optimized by tuning the parameters of the low-rank matrix.

[0010] Optionally, after obtaining the intermediate code corresponding to the UI prototype, the method further includes: A UI interface is generated based on the intermediate code, and the accuracy of the intermediate code is verified based on the UI interface. If the intermediate code passes the accuracy verification, then the target service code for the target service is compiled based on the intermediate code; If the intermediate code fails the accuracy check, it is determined that there is an error in element information identification or intermediate code generation.

[0011] Optionally, the step of verifying the accuracy of the intermediate code based on the UI interface includes: The UI interface and the UI prototype are compared to determine whether the UI interface matches the UI prototype. If there is no match, it is determined that the intermediate code has failed the accuracy check and that there is an error in the identification of element information; If a match is found, the UI interface is subjected to interactive function verification to verify whether the interactive functions of the interactive elements in the element information are normal, and if the interactive functions are normal, the intermediate code is verified by a code inspection tool to verify whether it conforms to the encoding specifications. If the interactive function is abnormal or does not conform to the coding specifications, it is determined that the intermediate code has failed the accuracy verification and that there is an error in the intermediate code generation.

[0012] Optionally, after determining that there is an element information recognition error, the method further includes: Obtain the image comparison results of the UI interface and the UI prototype, generate element recognition prompts based on the image comparison results, input the element recognition prompts and the UI prototype into the image recognition model, redetermine the element information in the UI prototype, and regenerate and verify the accuracy of the intermediate code. If the intermediate code fails the accuracy verification, manually annotate the element information of the UI prototype, and iteratively train the image recognition model using the annotated UI prototype. After determining that an intermediate code generation error exists, the method further includes: If the interaction function is abnormal, an element interaction abnormality prompt word is constructed based on the element information of the element with abnormal interaction function. Based on the element interaction abnormality prompt word, the element information and the preset prompt word, the code generation prompt word is reconstructed. The code generation model is then used to regenerate intermediate code based on the regenerated code generation prompt word, and the accuracy of the intermediate code is re-verified. If the intermediate code fails the accuracy verification, the intermediate code is manually corrected. The code generation model is then iteratively trained using the corrected intermediate code and the element information. If the code does not conform to the encoding standard, an encoding standard prompt word is constructed based on the non-compliant part of the intermediate code. The code generation prompt word is reconstructed based on the encoding standard prompt word, the element information, and the preset prompt word. The intermediate code is regenerated using the code generation model based on the regenerated code generation prompt word, and the accuracy of the intermediate code is re-verified. If the intermediate code fails the accuracy verification, it is manually corrected. The code generation model is then iteratively trained using the corrected intermediate code and the element information.

[0013] According to another aspect of this application, a code generation apparatus based on a large model is provided, the apparatus comprising: The prototype acquisition module is used to acquire the UI prototype of the target business. The element recognition module is used to identify elements in the UI prototype by recognizing the large model of the image, and to determine the element information in the UI prototype. The code generation module is used to generate code generation prompts based on the element information and preset prompts, input the code generation prompts into the code generation model, obtain the intermediate code corresponding to the UI prototype, and then compile the target business code of the target business based on the intermediate code.

[0014] Optionally, the preset prompt words include the code generation specifications for the target business; the code generation module is specifically used for: The code generation prompts are input into the code generation model, and the model generates intermediate code corresponding to the UI prototype based on the element information according to the code generation specifications.

[0015] Optionally, the element identification module is specifically used for: Before performing element recognition on the UI prototype using the image recognition large model to determine the element information in the UI prototype, a UI design draft training sample is obtained. The UI design draft training sample includes UI design draft samples marked with basic element information, which includes the element type and position. The general image recognition model is trained using the UI design sample to obtain the image recognition model.

[0016] Optionally, the UI design draft training samples include a first UI design draft training sample and a second UI design draft training sample. The first UI design draft training sample includes a first UI design draft sample marked with basic element information. The second UI design draft training sample includes a second UI design draft sample marked with basic element information and element description information of the second UI design draft sample. The element description information sample includes the relative positional relationship and hierarchical structure information between elements. The element recognition module is specifically used for: A convolutional neural network model is trained using the first UI design draft sample to obtain an element information recognition model. This element information recognition model is then integrated into a general image recognition model to obtain an initial image recognition model. The second UI design draft sample is input into this initial image recognition model. The element information recognition model predicts basic element information from the second UI design draft sample, and the general image recognition model analyzes the relative positions and hierarchical structure of elements in the second UI design draft sample based on the predicted basic element information to obtain element description information prediction results. Based on the element description information prediction results and the element description information, the parameters of the initial image recognition model are fine-tuned to obtain the final image recognition model. The UI prototype is input into the image recognition model to predict the basic element information of the UI prototype. Based on the predicted basic element information, the relative position and hierarchical structure information between elements in the UI prototype are identified, and the element information is output. The element information includes the basic element information, the relative position and hierarchical structure information between elements.

[0017] Optionally, the element identification module is specifically used for: The first UI design draft sample is divided into a first sub-sample and a second sub-sample; The convolutional neural network model is trained in one stage based on the first subsample to optimize the parameters of the convolutional and fully connected layers. Then, the trained convolutional neural network model is trained in two stages based on the second subsample to optimize the parameters of the fully connected layers, thus obtaining the element information recognition model. A low-rank matrix is ​​added to the general image recognition model, and the element information recognition model is then integrated into the general image recognition model with the added low-rank matrix to obtain an initial image recognition model. The parameters of the initial image recognition model are optimized by tuning the parameters of the low-rank matrix.

[0018] Optionally, the device further includes: a verification module, used for: After obtaining the intermediate code corresponding to the UI prototype, a UI interface is generated based on the intermediate code, and the accuracy of the intermediate code is verified based on the UI interface. If the intermediate code passes the accuracy verification, then the target service code for the target service is compiled based on the intermediate code; If the intermediate code fails the accuracy check, it is determined that there is an error in element information identification or intermediate code generation.

[0019] Optionally, the verification module is specifically used for: The UI interface and the UI prototype are compared to determine whether the UI interface matches the UI prototype. If there is no match, it is determined that the intermediate code has failed the accuracy check and that there is an error in the identification of element information; If a match is found, the UI interface is subjected to interactive function verification to verify whether the interactive functions of the interactive elements in the element information are normal, and if the interactive functions are normal, the intermediate code is verified by a code inspection tool to verify whether it conforms to the encoding specifications. If the interactive function is abnormal or does not conform to the coding specifications, it is determined that the intermediate code has failed the accuracy verification and that there is an error in the intermediate code generation.

[0020] Optionally, the element recognition module is further configured to: after determining that there is an element information recognition error, obtain the image comparison result between the UI interface and the UI prototype, generate element recognition prompt words based on the image comparison result, input the element recognition prompt words and the UI prototype into the image recognition big model, and re-determine the element information in the UI prototype; The code generation module is also used to: regenerate intermediate code; The verification module is also used to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, manually annotate the element information of the UI prototype. The element recognition module is also used to: iteratively train the image recognition model using the annotated UI prototype; The code generation module is further configured to: after determining that there is an intermediate code generation error, if the interactive function is abnormal, construct an element interaction abnormality prompt word based on the element information of the element with abnormal interactive function, reconstruct a code generation prompt word based on the element interaction abnormality prompt word, the element information and the preset prompt word, and regenerate the intermediate code based on the regenerated code generation prompt word through the code generation big model; The verification module is further configured to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, then manually correct the intermediate code. The code generation module is further configured to: iteratively train the large code generation model using the corrected intermediate code and the element information; The code generation module is further configured to: after determining that there is an intermediate code generation error, if it does not conform to the encoding standard, construct an encoding standard prompt word based on the part of the intermediate code that does not conform to the encoding standard, reconstruct a code generation prompt word based on the encoding standard prompt word, the element information and the preset prompt word, and regenerate the intermediate code based on the regenerated code generation prompt word through the code generation big model; The verification module is further configured to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, then manually correct the intermediate code. The code generation module is also used to iteratively train the large code generation model using the corrected intermediate code and the element information.

[0021] According to another aspect of this application, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the above-described code generation method based on a large model.

[0022] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described code generation method based on a large model.

[0023] By employing the above technical solutions, this application provides a code generation method, apparatus, storage medium, and computer device based on a large model. This method acquires a UI prototype of the target business, accurately identifies element information within the large model using image recognition, generates code generation prompts by combining preset prompts, and then inputs the code to generate an intermediate code from the large model, thereby arranging the target business code. This application automates the process from UI design to code generation, avoiding the tedious process of manually writing UI components and debugging interaction logic in traditional development. It reduces repetitive work and human error, allowing developers to focus more on core business function development, thereby improving development efficiency, ensuring code quality, and promoting rapid and high-quality project delivery.

[0024] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a code generation method based on a large model provided in an embodiment of this application is shown. Figure 2 A flowchart illustrating another code generation method based on a large model provided in an embodiment of this application is shown. Figure 3 A schematic diagram of the structure of a code generation device based on a large model provided in an embodiment of this application is shown. Detailed Implementation

[0026] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0027] This embodiment provides a code generation method based on a large model, such as Figure 1 As shown, the method includes: Step 101: Obtain the UI prototype of the target business.

[0028] Step 102: Use image recognition to identify elements in the UI prototype and determine the element information in the UI prototype.

[0029] Step 103: Generate code generation prompts based on the element information and preset prompts, input the code generation prompts into the code generation model to obtain the intermediate code corresponding to the UI prototype, and then compile the target business code of the target business based on the intermediate code.

[0030] This application provides a code generation method based on a large model. This method optimizes the traditional front-end development process by automatically identifying UI prototype elements and generating code. Figure 2 As shown, specifically, the process begins by obtaining UI prototypes for the target business, such as design drafts provided by UX / product personnel, including login pages and product list pages. These prototypes are then converted to a unified image format. Next, an image recognition model is used to parse the prototypes, automatically identifying element information, such as the login page's "username input box" (including borders, rounded corners, and placeholder text attributes), "password input box" (including hide / show functionality), "submit button" (including hover color change and click feedback effects), and "error message area" (including dynamic display logic), along with their interaction details. Finally, preset prompts (such as "Help generate Vue 3") are used to further refine the prototypes. The code is defined using the Composition API, script tags are defined using `lang=ts`, style tags are defined using Sass and support scoped functionality, and the UI component library is preferably Element-Plus. Based on the identified element information, structured code generation prompts are generated. These prompts are then input into the code to generate a large model, outputting intermediate code (static interaction design code, primarily defining the element types and layout configurations for the final interaction) that can be directly used for orchestrating business code. For example, this generates complete React component code including state management, event binding, and style configuration. In a real-world scenario, traditional development requires developers to manually write code for each component on the login page and debug the interaction logic, a time-consuming and error-prone process. With this method, the image recognition large model can quickly and accurately extract element information, and the code generation large model can automatically generate intermediate code conforming to framework specifications based on the prompts. Developers only need to adjust a small portion of the business logic to complete the integration. This method not only avoids the efficiency loss caused by repeatedly writing UI components, but also reduces human errors, such as inconsistent styles and missing interaction logic, through automated identification and generation. This allows developers to focus more on the implementation of core business functions, thereby improving development efficiency while ensuring code quality, and ultimately achieving the dual goals of rapid project delivery and high-quality implementation.

[0031] By applying the technical solution of this embodiment, a UI prototype of the target business is obtained. A large-scale image recognition model is used to accurately identify element information within the prototype. Preset prompts are combined with code generation to generate prompts, which are then input into the large-scale code generation model to obtain intermediate code. This is used to arrange the target business code. This embodiment of the application realizes an automated process from UI design to code generation, avoiding the tedious process of manually writing UI components and debugging interaction logic in traditional development. It reduces repetitive work and human error, allowing developers to focus more on core business function development, thereby improving development efficiency, ensuring code quality, and promoting rapid and high-quality project delivery.

[0032] Optionally, in this embodiment of the application, the preset prompt word includes the code generation specification of the target business; the step of inputting the code generation prompt word into the code generation large model to obtain the intermediate code corresponding to the UI prototype includes: inputting the code generation prompt word into the code generation large model, so as to generate the intermediate code corresponding to the UI prototype based on the element information using the code generation specification as the standard through the code generation large model.

[0033] In this embodiment, the preset prompt words include the code generation specifications for the target business. The code generation model uses these specifications as a standard, combined with element information extracted by the image recognition model, to generate intermediate code. Taking a bank software login page as an example, the image recognition model can identify the following element information: For instance, the "username input box" is a text input type, located in the top left area of ​​the page (coordinates X:50px, Y:100px, width 200px, height 30px); the "password input box" is also a text input type, located directly below the username input box (relative vertical spacing 20px), both at the same form container level; the "submit button" is a button type, located to the right of the password input box (horizontal spacing 30px), at the second level within the form container. Combined with the preset code generation specifications (such as "using TypeScript language, following React framework component naming conventions, the form container must wrap all input components and set uniform margins"), the code generation model can automatically generate intermediate code that conforms to bank security standards and development specifications. For example, it can generate React component code with type constraints, ensuring that input box type validation, button click event binding, and other logic all comply with the specifications. By standardizing constraints and matching element information, the inconsistencies caused by manual coding in traditional development are avoided, such as chaotic component naming and incorrect hierarchical structure. At the same time, it reduces the coding and debugging time of developers for repetitive UI elements, allowing them to focus more on the development of business logic specific to banking software (such as account verification and transaction record query). This improves development efficiency while ensuring code quality, achieving high efficiency and standardized uniformity in the UI development stage of banking software.

[0034] It should be noted that the above-mentioned large code generation model is obtained by optimizing and training a general large model with code generation capabilities based on the element information of the UI design draft and the intermediate code of the UI design draft.

[0035] Optionally, in this embodiment, the method further includes: obtaining UI design draft training samples, wherein the UI design draft training samples include UI design draft samples labeled with basic element information, the labeled basic element information including the element type and position; and training a general image recognition model using the UI design draft samples to obtain the image recognition model. Specifically, the UI design draft training samples include a first UI design draft training sample and a second UI design draft training sample. The first UI design draft training sample includes a first UI design draft sample labeled with basic element information, and the second UI design draft training sample includes a second UI design draft sample labeled with basic element information and element description information of the second UI design draft sample. The element description information sample includes the relative positional relationship and hierarchical structure information between elements. Training the general image recognition model using the UI design draft samples to obtain the image recognition model includes: training a convolutional neural network model using the first UI design draft samples to obtain the element type and position. An information recognition model is used; the element information recognition model is integrated into a general image recognition model to obtain an initial image recognition model. The second UI design draft sample is input into the initial image recognition model to predict the basic element information of the second UI design draft sample through the element information recognition model. Based on the prediction results of the basic element information, the general image recognition model is used to analyze the relative positional relationships and hierarchical structure of the elements in the second UI design draft sample to obtain the element description information prediction results. Based on the element description information prediction results and the element description information, the parameters of the initial image recognition model are optimized to obtain the image recognition model.

[0036] In this embodiment, a phased and refined training method is proposed for building a large-scale image recognition model. This aims to improve the model's ability to analyze elements in UI design drafts, enabling it to accurately identify basic element information (type, position) as well as complex relative positional relationships and hierarchical structure information between elements. This lays the foundation for generating high-quality code based on this information. The specific process is as follows: First, to train a large-scale image recognition model with comprehensive analytical capabilities, two types of UI design draft training samples are constructed. Each type of sample has a specific training objective: First UI design draft training samples: These samples focus on labeling basic element information. Taking the initial design draft of a bank software as an example, the "login button" in the design draft is labeled as a button type, and its position coordinates on the page (e.g., X:200px, Y:150px) and size (e.g., 80×40px) are clearly defined. Through training with a large number of these samples, the model can initially master the ability to identify UI element types and locate element positions, which is the foundation for subsequent analysis of complex relationships. Second UI design draft training samples: These samples are further enriched based on the first type. In addition to containing basic element information, they also label the relative positional relationships and hierarchical structure information between elements. Taking the upgraded bank software design draft as an example, while marking the "transfer button" as its button type and position size, it also records that the horizontal spacing between it and the "recipient input box" is 10px, and that the "transfer button" is located in the second layer within the "operation panel" container. This is crucial for accurately understanding the layout logic of the UI design draft and helps the model grasp the spatial relationships and hierarchical relationships between elements. Furthermore, this application embodiment adopts a phased model training method. First, basic information recognition model training is performed, using the first UI design draft sample to train the convolutional neural network (CNN) model. The CNN model has powerful feature extraction capabilities in the field of image recognition. Through learning from a large number of samples, it can learn to recognize different types of UI elements (such as buttons, input boxes, text labels, etc.) and locate their positions in the image. After this stage of training, a model that can initially recognize the basic information of elements is obtained, providing a foundation for subsequent training. Second, the trained element information recognition model is connected to a general image recognition model to form an initial image recognition model. The general image recognition model has a wider range of image recognition capabilities, but it lacks sufficient understanding of the specific element relationships in the UI design draft. By integrating with the element information recognition model, the initial model, while inheriting the advantages of the general model, initially possesses the ability to recognize basic information about UI elements. Next, a second UI design sample is input into the initial image recognition model. The element information recognition model predicts the basic information of the elements in the sample, determining the type and position of each element. Then, based on these prediction results, the general image recognition model further analyzes the relative positional relationships (such as horizontal and vertical spacing) and hierarchical structure information (such as the container level to which the element belongs) between elements.By comparing the model's predictions with the actual information labeled in the samples, the error is calculated and the model parameters are adjusted. This process is continuously iterated, gradually optimizing the model's ability to resolve complex relationships, ultimately resulting in a large image recognition model capable of accurately identifying all key information in UI design drafts.

[0037] Taking a real-world application of banking software as an example, the image recognition model trained as described above can accurately identify various element information in UI design drafts. For instance, in the "account balance display module," the model can accurately identify the "balance number" as text and determine that it is located at the top of the module; it also identifies the "currency unit label" as text, located to the right of the number, with a horizontal spacing of 5px. Furthermore, the model can determine that both elements are in the first level within the "balance card" container. This element information provides reliable input for subsequent code generation, ensuring that the generated intermediate code accurately reflects the layout and interaction logic of the UI design draft. When developers use this intermediate code to arrange target business code, they no longer need to manually adjust layout issues caused by errors in element information parsing, thus greatly improving development efficiency while ensuring code quality. This embodiment trains a large image recognition model capable of parsing basic element information and complex relationships in UI design drafts through a hierarchical classification and training sample construction method and a phased reinforcement model training method. This provides strong support for code generation methods based on large models, improving the efficiency and quality of front-end development.

[0038] Optionally, in this embodiment of the application, the step of identifying elements in the UI prototype using an image recognition model to determine element information in the UI prototype includes: inputting the UI prototype into the image recognition model to predict the basic element information of the UI prototype using the image recognition model, and identifying the relative positions and hierarchical structure information between elements in the UI prototype based on the predicted basic element information, and outputting element information, wherein the element information includes the basic element information, the relative positions between elements, and the hierarchical structure information.

[0039] In this embodiment, the image recognition model first performs a pixel-level scan of the input UI prototype to identify the basic information of all individual elements. For example, on the transfer page, the model will first locate the "recipient account input box" (element type: text input box; position: 20% below the top of the page, coordinates X:80px, Y:120px, size 300×40px), the "amount input box" (same type, located 30px below the recipient account vertically), and the "confirm transfer button" (element type: button; position: 20px to the right of the amount input box horizontally, coordinates X:400px, Y:190px). Then, based on the predicted basic information, the model further analyzes the relative positional relationships and hierarchical structure between the elements. For example, on the transfer page, the model recognizes that the "Recipient Account Input Box" and the "Amount Input Box" are vertically aligned (with a vertical spacing of 30px), and both are within the same form container (hierarchical structure: form container → input box group). Simultaneously, the "Confirm Transfer Button" and the "Amount Input Box" are horizontally adjacent (with a horizontal spacing of 20px), but belong to an independent button hierarchy outside the form container. This information provides complete layout logic input for the subsequent code generation of the large model, ensuring that the generated intermediate code accurately reproduces the spatial structure and hierarchical relationships in the design draft, avoiding layout errors or missing interaction logic due to incorrect element relationship parsing. This step, through phased recognition of basic information and complex relationships, ensures both the accuracy of element positioning and captures the spatial relationships and hierarchical logic between elements. This enables the image recognition large model to comprehensively and accurately parse the key information in the UI prototype, laying the foundation for efficient and high-quality code generation.

[0040] Optionally, in this embodiment, training the convolutional neural network model using the first UI design draft sample to obtain the element information recognition model includes: dividing the first UI design draft sample into a first sub-sample and a second sub-sample; performing a first-stage training on the convolutional neural network model based on the first sub-sample to optimize the parameters of the convolutional and fully connected layers of the convolutional neural network model; and performing a second-stage training on the convolutional neural network model after the first-stage training based on the second sub-sample to optimize the parameters of the fully connected layers of the convolutional neural network model, thereby obtaining the element information recognition model. The step of integrating the element information recognition model into a general image recognition model to obtain an initial image recognition model includes: adding a low-rank matrix to the general image recognition model, and integrating the element information recognition model into the general image recognition model with the added low-rank matrix to obtain an initial image recognition model. Parameter optimization of the initial image recognition model is achieved by optimizing the parameters of the low-rank matrix.

[0041] In this embodiment, the first UI design draft sample is divided into a first sub-sample (such as a basic UI element diagram of a banking software) and a second sub-sample (such as an upgraded UI diagram with a complex layout). During the first stage of training, the first sub-sample is used to optimize the convolutional layers (responsible for extracting low-level features such as edges and textures) and fully connected layers (responsible for initial classification and localization) of the convolutional neural network. The second stage of training uses the second sub-sample, optimizing only the fully connected layer parameters, enabling the model to more precisely calibrate position coordinates and identify element types. This layered training avoids overfitting or underfitting problems that may result from one-step training. When connecting the trained element information recognition model to a general image recognition model, a low-rank matrix is ​​used as a bridge. A low-rank matrix is ​​a structure with fewer parameters but strong expressive power; adjusting its parameters can indirectly optimize the performance of the entire large model. The resulting large-scale image recognition model can output complete element information in one go, including basic information (type, position), relative positional relationships (e.g., the "transfer button" is 20 pixels to the right of the "amount input box"), and hierarchical structure (e.g., the "button" belongs to the second level of the "operation panel" container). This ensures that the generated intermediate code can directly reproduce the layout logic of the design draft, eliminating the need for developers to manually adjust element relationships and significantly reducing debugging time. This solution enhances the model's ability to parse UI elements through phased training and achieves efficient fine-tuning by combining low-rank matrices. Ultimately, it constructs a large-scale image recognition model that can accurately identify complex UI relationships and quickly adapt to design changes, providing a more reliable and efficient input foundation for front-end code generation.

[0042] In this embodiment of the application, optionally, after obtaining the intermediate code corresponding to the UI prototype, the method further includes: generating a UI interface based on the intermediate code, and performing an accuracy verification on the intermediate code based on the UI interface; if the intermediate code passes the accuracy verification, then arranging the target business code of the target business based on the intermediate code; if the intermediate code fails the accuracy verification, then determining that there is an element information recognition error or an intermediate code generation error.

[0043] In this embodiment, the generated intermediate code is used to visualize the UI elements, layout, and style described by the intermediate code through a specific rendering mechanism or code parsing tool, constructing the corresponding UI interface and transforming the abstract intermediate code into a tangible interface presentation. For example, for the intermediate code of a bank software login page, rendering can generate a complete interface containing elements such as a username input box, a password input box, and a login button. The generated UI interface is then compared with the original UI prototype. The basic information such as element type, position, size, and color is checked to ensure consistency with the prototype. Simultaneously, the relative positional relationships (such as horizontal spacing and vertical alignment) and hierarchical structure (such as the container hierarchy to which elements belong) between elements are accurately reproduced. For example, it is checked whether the login button is centered below the username and password input boxes in the generated interface, maintains appropriate vertical spacing from them, and is within the correct container hierarchy. When the generated UI interface is highly consistent with the UI prototype in all key information and meets the preset accuracy standards, the intermediate code is deemed to have passed the accuracy check. At this point, the target business code for the target business can be further programmed based on the intermediate code. If significant differences are found between the generated UI and the UI prototype during the comparison process, such as element position offsets, type errors, or disordered hierarchical relationships, the intermediate code is deemed to have failed the accuracy check. In this case, the root cause can be determined to be either an error in element information recognition (e.g., inaccurate parsing of elements in the UI prototype by the large image recognition model) or an error in intermediate code generation (e.g., logical or encoding errors during the conversion of element information into intermediate code). Further investigation and correction of the specific problem can then be carried out.

[0044] Optionally, in this embodiment of the application, the step of verifying the accuracy of the intermediate code based on the UI interface includes: comparing the UI interface and the UI prototype to determine whether the UI interface matches the UI prototype; If there is no match, it is determined that the intermediate code has failed the accuracy check and that there is an error in the identification of element information; If a match is found, the UI interface is subjected to interactive function verification to verify whether the interactive functions of the interactive elements in the element information are normal, and if the interactive functions are normal, the intermediate code is verified by a code inspection tool to verify whether it conforms to the encoding specifications. If the interactive function is abnormal or does not conform to the coding specifications, it is determined that the intermediate code has failed the accuracy verification and that there is an error in the intermediate code generation.

[0045] In this embodiment, firstly, image comparison technology is used to compare the generated UI interface with the original UI prototype. This process analyzes the differences between the two images pixel by pixel to determine whether they are highly consistent in terms of overall layout, element appearance (such as shape, color, and size). For example, for the product details page UI of an e-commerce app, the comparison checks whether the display position of the product image, the font size and color of the price tag, and the shape and position of the purchase button in the generated interface and the prototype are completely matched. If significant differences are found, it indicates that the generated UI interface fails to accurately reproduce the design of the UI prototype. It is initially determined that the intermediate code has failed the accuracy verification, and the root cause of the problem lies in the incorrect identification of element information. That is, when the image recognition model parses the UI prototype, it does not accurately identify the basic information of the elements (such as type, position, style, etc.), resulting in deviations in the generated intermediate code. If the image comparison results show that the UI interface matches the UI prototype, the interactive functions of the UI interface are further verified. For interactive elements in the UI interface (such as buttons, input boxes, drop-down menus, etc.), the actual operation of the user is simulated to check whether the interactive functions of these elements are normal. For example, in the product details page of the aforementioned e-commerce app, should clicking the "Buy" button redirect to the order confirmation page correctly? Should the search results be displayed correctly after entering keywords in the search input box? Should the drop-down menu expand and collapse correctly? If the interactive functions malfunction, such as buttons not responding or input boxes being unreadable, it indicates a problem with the intermediate code in describing the element's interactive logic, failing to accurately convey the element's interactive behavior. In this case, the intermediate code is deemed to have failed the accuracy check, and an error in its generation is confirmed—that is, an error occurred in the intermediate code when converting element information into executable code logic. If the interactive function verification passes, it indicates that the intermediate code is basically correct in describing the element's interactive logic. At this point, a code inspection tool is used to check the intermediate code for coding standards. The code inspection tool will scan and analyze the intermediate code according to preset coding standards (such as code indentation, variable naming rules, commenting standards, etc.) to check whether the intermediate code conforms to these standards. If the intermediate code does not conform to the coding standards, it may affect the readability, maintainability, and scalability of the code, and may even lead to compilation errors or runtime errors. Therefore, if the intermediate code is found to be inconsistent with the encoding standard, it is also determined that the intermediate code has failed the accuracy verification, and it is confirmed that there is an error in the intermediate code generation, that is, the intermediate code did not follow the unified encoding standard during the generation process. This embodiment provides comprehensive assurance for the quality of the intermediate code through a multi-layered accuracy verification mechanism of image comparison, interactive function verification, and encoding standard checking.

[0046] Optionally, after determining that there is an element information recognition error, the method further includes: obtaining the image comparison result between the UI interface and the UI prototype; generating element recognition prompts based on the image comparison result; inputting the element recognition prompts and the UI prototype into the image recognition large model; re-determining the element information in the UI prototype; and re-generating and verifying the accuracy of the intermediate code. If the intermediate code fails the accuracy verification, manually annotating the element information in the UI prototype; and iteratively training the image recognition large model using the annotated UI prototype.

[0047] In this embodiment, when intermediate code verification fails and is determined to be due to element information recognition error, the specific erroneous element is first located through image comparison results (e.g., the offset position of the "transfer button" in a bank app). Based on the comparison differences, element recognition prompts are generated (e.g., "Focus on recognizing the vertical centering features of button-like elements within a form container"). These prompts, along with the original UI prototype, are then input into the image recognition model again. The model re-parses the element information based on the prompts, generates a new intermediate code, and verifies it again. If it still fails, the prototype is manually annotated (e.g., annotating the precise coordinate range and hierarchy of the "transfer button"). The annotated data is used to iteratively train the model, improving its ability to recognize similar elements. This mechanism, through a closed loop of "problem-correction-reinforcement," achieves progressive optimization from single error correction to model capability improvement. On one hand, prompts generated based on image comparison results guide the model to refocus on key element features, reducing repeated errors; on the other hand, feedback training with manually annotated data allows the model to continuously learn the recognition logic of complex UI scenarios (e.g., dynamic layouts, nested containers), gradually reducing reliance on manual verification. Ultimately, this closed-loop mechanism not only improves the accuracy of single code generation, but also enhances the system's adaptability to diverse UI prototypes through model iteration, continuously reducing the error rate in long-term development and improving the stability and efficiency of the front-end code generation process.

[0048] In this embodiment of the application, optionally, after determining that there is an intermediate code generation error, the method further includes: if the interaction function is abnormal, constructing an element interaction abnormality prompt word based on the element information of the element with abnormal interaction function, reconstructing a code generation prompt word based on the element interaction abnormality prompt word, the element information and the preset prompt word, regenerating the intermediate code based on the regenerated code generation prompt word through the code generation big model, and re-verifying the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, manually correcting the intermediate code, and iteratively training the code generation big model using the corrected intermediate code and the element information; If the code does not conform to the encoding standard, an encoding standard prompt word is constructed based on the non-compliant part of the intermediate code. The code generation prompt word is reconstructed based on the encoding standard prompt word, the element information, and the preset prompt word. The intermediate code is regenerated using the code generation model based on the regenerated code generation prompt word, and the accuracy of the intermediate code is re-verified. If the intermediate code fails the accuracy verification, it is manually corrected. The code generation model is then iteratively trained using the corrected intermediate code and the element information.

[0049] In this embodiment, when intermediate code fails due to interactive function anomalies or coding style non-compliance, the system initiates a targeted correction process: Interactive function anomaly scenario: Taking the unresponsive "transfer button" in a bank app as an example, the system extracts the button's element information (e.g., type is button, located in the second level of the form container), generates an interaction anomaly prompt "the button needs to be bound to the onSubmit event," and reconstructs the code generation instruction based on preset coding style prompts, driving the large code model to regenerate intermediate code. If it still fails, the intermediate code is manually corrected (e.g., supplementing event binding logic), and the corrected intermediate code is used with the original element information to train the model, strengthening its ability to learn interactive logic. Coding style non-compliance scenario: If the intermediate code variable naming violates camelCase rules, a coding style prompt "variable names must use lowerCamelCase" is generated, merged with element information, and the prompt is reconstructed to regenerate compliant intermediate code. If this fails, the naming is manually standardized, and the model is iteratively trained using positive example data to improve its adherence to coding style guidelines. This mechanism, through a closed-loop design of "problem type identification - intelligent prompt guidance - model iteration reinforcement," achieves progressive optimization from single error correction to system capability enhancement. On the one hand, for two common types of errors—interaction anomalies and coding standard errors—specific prompts are generated to guide the model to refocus on key logic, reducing recurring errors. On the other hand, the fusion training of manually corrected data and original information enables the model to continuously learn complex interaction logic and coding standards, gradually reducing reliance on manual verification. Ultimately, this closed-loop optimization not only improves the accuracy of single code generation but also enhances the system's adaptability to diverse UI scenarios and coding requirements through model iteration. In long-term development, it continuously reduces the error rate, improves the stability and efficiency of the front-end code generation process, and achieves a qualitative leap from "passive error correction" to "proactive optimization."

[0050] Furthermore, as Figure 1 In terms of specific implementation of the method, this application provides a code generation device based on a large model, such as... Figure 3 As shown, the device includes: The prototype acquisition module is used to acquire the UI prototype of the target business. The element recognition module is used to identify elements in the UI prototype by recognizing the large model of the image, and to determine the element information in the UI prototype. The code generation module is used to generate code generation prompts based on the element information and preset prompts, input the code generation prompts into the code generation model, obtain the intermediate code corresponding to the UI prototype, and then compile the target business code of the target business based on the intermediate code.

[0051] Optionally, the preset prompt words include the code generation specifications for the target business; the code generation module is specifically used for: The code generation prompts are input into the code generation model, and the model generates intermediate code corresponding to the UI prototype based on the element information according to the code generation specifications.

[0052] Optionally, the element identification module is specifically used for: Before performing element recognition on the UI prototype using the image recognition large model to determine the element information in the UI prototype, a UI design draft training sample is obtained. The UI design draft training sample includes UI design draft samples marked with basic element information, which includes the element type and position. The general image recognition model is trained using the UI design sample to obtain the image recognition model.

[0053] Optionally, the UI design draft training samples include a first UI design draft training sample and a second UI design draft training sample. The first UI design draft training sample includes a first UI design draft sample marked with basic element information. The second UI design draft training sample includes a second UI design draft sample marked with basic element information and element description information of the second UI design draft sample. The element description information sample includes the relative positional relationship and hierarchical structure information between elements. The element recognition module is specifically used for: A convolutional neural network model is trained using the first UI design draft sample to obtain an element information recognition model. This element information recognition model is then integrated into a general image recognition model to obtain an initial image recognition model. The second UI design draft sample is input into this initial image recognition model. The element information recognition model predicts basic element information from the second UI design draft sample, and the general image recognition model analyzes the relative positions and hierarchical structure of elements in the second UI design draft sample based on the predicted basic element information to obtain element description information prediction results. Based on the element description information prediction results and the element description information, the parameters of the initial image recognition model are fine-tuned to obtain the final image recognition model. The UI prototype is input into the image recognition model to predict the basic element information of the UI prototype. Based on the predicted basic element information, the relative position and hierarchical structure information between elements in the UI prototype are identified, and the element information is output. The element information includes the basic element information, the relative position and hierarchical structure information between elements.

[0054] Optionally, the element identification module is specifically used for: The first UI design draft sample is divided into a first sub-sample and a second sub-sample; The convolutional neural network model is trained in one stage based on the first subsample to optimize the parameters of the convolutional and fully connected layers. Then, the trained convolutional neural network model is trained in two stages based on the second subsample to optimize the parameters of the fully connected layers, thus obtaining the element information recognition model. A low-rank matrix is ​​added to the general image recognition model, and the element information recognition model is then integrated into the general image recognition model with the added low-rank matrix to obtain an initial image recognition model. The parameters of the initial image recognition model are optimized by tuning the parameters of the low-rank matrix.

[0055] Optionally, the device further includes: a verification module, used for: After obtaining the intermediate code corresponding to the UI prototype, a UI interface is generated based on the intermediate code, and the accuracy of the intermediate code is verified based on the UI interface. If the intermediate code passes the accuracy verification, then the target service code for the target service is compiled based on the intermediate code; If the intermediate code fails the accuracy check, it is determined that there is an error in element information identification or intermediate code generation.

[0056] Optionally, the verification module is specifically used for: The UI interface and the UI prototype are compared to determine whether the UI interface matches the UI prototype. If there is no match, it is determined that the intermediate code has failed the accuracy check and that there is an error in the identification of element information; If a match is found, the UI interface is subjected to interactive function verification to verify whether the interactive functions of the interactive elements in the element information are normal, and if the interactive functions are normal, the intermediate code is verified by a code inspection tool to verify whether it conforms to the encoding specifications. If the interactive function is abnormal or does not conform to the coding specifications, it is determined that the intermediate code has failed the accuracy verification and that there is an error in the intermediate code generation.

[0057] Optionally, the element recognition module is further configured to: after determining that there is an element information recognition error, obtain the image comparison result between the UI interface and the UI prototype, generate element recognition prompt words based on the image comparison result, input the element recognition prompt words and the UI prototype into the image recognition big model, and re-determine the element information in the UI prototype; The code generation module is also used to: regenerate intermediate code; The verification module is also used to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, manually annotate the element information of the UI prototype. The element recognition module is also used to: iteratively train the image recognition model using the annotated UI prototype; The code generation module is further configured to: after determining that there is an intermediate code generation error, if the interactive function is abnormal, construct an element interaction abnormality prompt word based on the element information of the element with abnormal interactive function, reconstruct a code generation prompt word based on the element interaction abnormality prompt word, the element information and the preset prompt word, and regenerate the intermediate code based on the regenerated code generation prompt word through the code generation big model; The verification module is further configured to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, then manually correct the intermediate code. The code generation module is further configured to: iteratively train the large code generation model using the corrected intermediate code and the element information; The code generation module is further configured to: after determining that there is an intermediate code generation error, if it does not conform to the encoding standard, construct an encoding standard prompt word based on the part of the intermediate code that does not conform to the encoding standard, reconstruct a code generation prompt word based on the encoding standard prompt word, the element information and the preset prompt word, and regenerate the intermediate code based on the regenerated code generation prompt word through the code generation big model; The verification module is further configured to: re-verify the accuracy of the intermediate code; if the intermediate code fails the accuracy verification, then manually correct the intermediate code. The code generation module is also used to iteratively train the large code generation model using the corrected intermediate code and the element information.

[0058] It should be noted that other corresponding descriptions of the functional units involved in the code generation device based on a large model provided in this application embodiment can be found in [reference]. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0059] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0060] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0061] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0062] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0063] It should be noted that the user personal information involved in the embodiments of this application is all authorized (with the knowledge and consent) by the relevant parties or fully authorized by all parties, and the executing entity can obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals. It should be noted that if any software tools or components other than those of this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use.

[0064] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A code generation method based on a large model, characterized in that, The method includes: Obtain the UI prototype of the target business; The UI prototype is used to identify elements by recognizing a large model of images, thereby determining the element information in the UI prototype. Based on the element information and preset prompt words, code generation prompt words are generated. The code generation prompt words are then input into the code generation model to obtain the intermediate code corresponding to the UI prototype. The target business code of the target business is then arranged based on the intermediate code.

2. The method according to claim 1, characterized in that, The preset prompt words include the code generation specifications for the target business; The step of inputting the code generation prompts into the large code generation model to obtain the intermediate code corresponding to the UI prototype includes: The code generation prompts are input into the code generation model, and the model generates intermediate code corresponding to the UI prototype based on the element information according to the code generation specifications.

3. The method according to claim 1, characterized in that, Before identifying elements in the UI prototype using image recognition large model to determine element information in the UI prototype, the method further includes: Obtain UI design draft training samples, wherein the UI design draft training samples include UI design draft samples marked with basic element information, the marked basic element information including the element type and position; The general image recognition model is trained using the UI design sample to obtain the image recognition model.

4. The method according to claim 3, characterized in that, The UI design draft training samples include a first UI design draft training sample and a second UI design draft training sample. The first UI design draft training sample includes a first UI design draft sample marked with basic element information. The second UI design draft training sample includes a second UI design draft sample marked with basic element information and element description information of the second UI design draft sample. The element description information sample includes the relative positional relationship and hierarchical structure information between elements. The step of training a general image recognition model using the UI design sample to obtain the image recognition model includes: The convolutional neural network model is trained using the first UI design sample to obtain an element information recognition model; The element information recognition model is integrated into a general image recognition model to obtain an initial image recognition model. The second UI design draft sample is input into the initial image recognition model. The element information recognition model is used to predict the basic element information of the second UI design draft sample. Based on the element basic information prediction results, the general image recognition model is used to analyze the relative positional relationships and hierarchical structure of the elements in the second UI design draft sample to obtain element description information prediction results. Based on the element description information prediction results and the element description information, the parameters of the initial image recognition model are optimized to obtain the image recognition model.

5. The method according to claim 4, characterized in that, The step of identifying elements in the UI prototype using a large image recognition model to determine element information in the UI prototype includes: The UI prototype is input into the image recognition model to predict the basic element information of the UI prototype. Based on the predicted basic element information, the relative position and hierarchical structure information between elements in the UI prototype are identified, and the element information is output. The element information includes the basic element information, the relative position and hierarchical structure information between elements.

6. The method according to claim 4, characterized in that, The step of training a convolutional neural network model using the first UI design sample to obtain an element information recognition model includes: The first UI design draft sample is divided into a first sub-sample and a second sub-sample; The convolutional neural network model is trained in one stage based on the first subsample to optimize the parameters of the convolutional and fully connected layers of the convolutional neural network model, and then trained in two stages based on the second subsample to optimize the parameters of the fully connected layers of the convolutional neural network model, thereby obtaining the element information recognition model. The step of integrating the element information recognition model into a general image recognition model to obtain an initial image recognition model includes: A low-rank matrix is ​​added to the general image recognition model, and the element information recognition model is then integrated into the general image recognition model with the added low-rank matrix to obtain an initial image recognition model. The parameters of the initial image recognition model are optimized by tuning the parameters of the low-rank matrix.

7. The method according to any one of claims 1 to 6, characterized in that, After obtaining the intermediate code corresponding to the UI prototype, the method further includes: A UI interface is generated based on the intermediate code, and the accuracy of the intermediate code is verified based on the UI interface. If the intermediate code passes the accuracy verification, then the target service code for the target service is compiled based on the intermediate code; If the intermediate code fails the accuracy check, it is determined that there is an error in element information identification or intermediate code generation.

8. The method according to claim 7, characterized in that, The accuracy verification of the intermediate code based on the UI interface includes: The UI interface and the UI prototype are compared to determine whether the UI interface matches the UI prototype. If there is no match, it is determined that the intermediate code has failed the accuracy check and that there is an error in the identification of element information; If a match is found, the UI interface is subjected to interactive function verification to verify whether the interactive functions of the interactive elements in the element information are normal, and if the interactive functions are normal, the intermediate code is verified by a code inspection tool to verify whether it conforms to the encoding specifications. If the interactive function is abnormal or does not conform to the coding specifications, it is determined that the intermediate code has failed the accuracy verification and that there is an error in the intermediate code generation.

9. The method according to claim 8, characterized in that, After determining that there is an element information recognition error, the method further includes: Obtain the image comparison results of the UI interface and the UI prototype, generate element recognition prompts based on the image comparison results, input the element recognition prompts and the UI prototype into the image recognition model, redetermine the element information in the UI prototype, and regenerate and verify the accuracy of the intermediate code. If the intermediate code fails the accuracy verification, manually annotate the element information of the UI prototype, and iteratively train the image recognition model using the annotated UI prototype. After determining that an intermediate code generation error exists, the method further includes: If the interaction function is abnormal, an element interaction abnormality prompt word is constructed based on the element information of the element with abnormal interaction function. Based on the element interaction abnormality prompt word, the element information and the preset prompt word, the code generation prompt word is reconstructed. The code generation model is then used to regenerate intermediate code based on the regenerated code generation prompt word, and the accuracy of the intermediate code is re-verified. If the intermediate code fails the accuracy verification, the intermediate code is manually corrected. The code generation model is then iteratively trained using the corrected intermediate code and the element information. If the code does not conform to the encoding standard, an encoding standard prompt word is constructed based on the non-compliant part of the intermediate code. The code generation prompt word is reconstructed based on the encoding standard prompt word, the element information, and the preset prompt word. The intermediate code is regenerated using the code generation model based on the regenerated code generation prompt word, and the accuracy of the intermediate code is re-verified. If the intermediate code fails the accuracy verification, it is manually corrected. The code generation model is then iteratively trained using the corrected intermediate code and the element information.

10. A code generation device based on a large model, characterized in that, The device includes: The prototype acquisition module is used to acquire the UI prototype of the target business. The element recognition module is used to identify elements in the UI prototype by recognizing the large model of the image, and to determine the element information in the UI prototype. The code generation module is used to generate code generation prompts based on the element information and preset prompts, input the code generation prompts into the code generation model, obtain the intermediate code corresponding to the UI prototype, and then compile the target business code of the target business based on the intermediate code.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

12. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 9.

Images