User interface code generation method, system and equipment based on multi-modal large model

Abstract

The invention relates to the field of man-machine interaction and software engineering, and provides a user interface code generation method, system and equipment based on a multi-modal large model, and the method comprises the following steps: constructing a page conversion diagram according to a UI design drawing; extracting a UI component in the UI design drawing and analyzing the function of the UI component to obtain a UI component analysis result; integrating the page conversion diagram, the UI component analysis result and the UI design diagram to generate cue words; and inputting the cue word into the multi-modal large model, and generating a user interface code by the multi-modal large model according to the cue word. The problems that in current UI development, efficiency is low, complex design is difficult to process, and interaction logic understanding is limited are solved. According to the method, powerful technical support is provided for wide application and further development of the declarative UI framework, and progress of the whole UI development ecosystem is promoted.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction and software engineering, and in particular to a method, system and device for generating user interface code based on a multimodal large model. Background Art

[0002] With the rapid development of the mobile application ecosystem, the UI development paradigm has gradually shifted from the traditional imperative to the declarative. Although declarative frameworks such as ArkUI, React Native, and Flutter have simplified the UI development process to a certain extent, the conversion process from UI design to functional declarative UI code is still complex and time-consuming, requiring a lot of manual coding. Existing technologies have obvious deficiencies in handling complex designs and interaction logic, especially when adapting to emerging declarative UI frameworks, they face many challenges.

[0003] Current UI code generation methods mainly target traditional imperative UI frameworks and are difficult to effectively extend to the declarative UI field. Directly applying multimodal large models (MLLMs) for UI code generation, while showing potential, still faces problems such as low component recognition accuracy, limited understanding of interaction logic, inconsistent cross-page behavior, and code reliability. In addition, existing large models lack a deep understanding of emerging declarative UI frameworks such as ArkUI, making it difficult to fully leverage the advantages of these frameworks. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a user interface code generation method, system and device based on a multimodal large model, aiming to achieve efficient and accurate conversion from UI design to functional declarative UI code, while solving key challenges such as complex design understanding, interaction logic recognition, cross-page consistency and code reliability, and providing comprehensive automated support for the development of declarative UI frameworks.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for generating user interface code based on a multimodal large model, comprising the following steps:

[0007] Build a page transition diagram based on the UI design diagram;

[0008] Extract UI components from UI design diagrams and analyze their functions to obtain UI component analysis results;

[0009] Integrate the page transition diagram, UI component analysis results, and UI design diagram to generate prompt words;

[0010] The prompt words are input into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

[0011] In some embodiments, constructing a page transition diagram according to the UI design diagram specifically includes:

[0012] Provide the UI design diagram and the definition of the page transition diagram to the multimodal big model. The multimodal big model analyzes the UI design diagram and generates the page transition diagram according to the provided prompts.

[0013] In some embodiments, extracting UI components from the UI design diagram and analyzing their functions specifically includes:

[0014] Use computer vision technology to perform zero-shot object detection on UI design images to identify UI components and generate bounding boxes of UI components;

[0015] Perform instance segmentation of the UI component using an image segmentation algorithm using the bounding box as a guide for segmentation to obtain a segmented UI component image;

[0016] The segmented UI component images are provided to the multimodal large model, and the multimodal large model extracts the UI component types and infers the functions of the UI components based on the UI component images.

[0017] In some embodiments, the step of integrating the page transition diagram, the UI component analysis results, and the UI design diagram to generate prompt words specifically includes:

[0018] Integrate the page transition diagram, UI component analysis results, and UI design diagram to ensure the consistency and integrity of the three information;

[0019] Based on the integrated data, context descriptions and task instructions are constructed to form prompt words for generating user interface codes.

[0020] In some embodiments, the context description includes overall UI layout, page hierarchy, component functionality, and interaction logic, and the task instructions include code structure requirements, framework-specific syntax, and component implementation details.

[0021] In some embodiments, the prompt words are generated based on a prompt template, and the method further includes dynamically adjusting the prompt template to automatically adjust the detail and emphasis of the prompt according to the complexity of the UI design diagram and the target UI framework.

[0022] In some embodiments, it also includes:

[0023] Create a corresponding knowledge graph for the target UI framework;

[0024] When generating user interface code, the multimodal large model queries the knowledge graph to obtain relevant UI framework specific information and integrates the information into the prompt word.

[0025] In some embodiments, after the multimodal macromodel generates the user interface code according to the prompt word, it further includes:

[0026] The generated user interface code is iteratively optimized, including navigation consistency verification and compile-time error correction.

[0027] In a second aspect, the present invention further provides a user interface code generation system based on a multimodal large model, comprising:

[0028] A page transition diagram construction module is used to construct a page transition diagram according to a UI design diagram;

[0029] The UI component extraction and analysis module is used to extract UI components in the UI design diagram and analyze their functions to obtain UI component analysis results;

[0030] The prompt word synthesis module is used to integrate the page transition diagram, UI component analysis results and UI design diagram to generate prompt words;

[0031] The user interface code generation module is used to input prompt words into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

[0032] In a third aspect, the present invention also provides a user interface code generation device based on a multimodal large model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the user interface code generation method based on the multimodal large model as described above when executing the computer program.

[0033] In summary, the present invention has the following beneficial effects compared with the prior art:

[0034] The user interface code generation method, system and device based on the multimodal large model provided by the present invention first construct a page transition diagram according to the UI design diagram to capture the interactive logic of the UI; then extract the UI components in the UI design diagram and analyze their functions to obtain the UI component analysis results, providing structured information for subsequent code generation; then integrate the page transition diagram, UI component analysis results and UI design diagram to generate structured and informative prompt words, and input them into the multimodal large model to generate user interface code. This phased, multi-step method ensures the comprehensiveness and accuracy from UI design to user interface code implementation, and effectively realizes the automation of UI intelligent construction. It solves the problems of low efficiency, difficult to handle complex design, and limited understanding of interactive logic in current UI development. The present invention provides strong technical support for the widespread application and further development of the declarative UI framework, and promotes the progress of the entire UI development ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 An overall flow chart of a method for generating user interface code based on a multimodal large model provided in this embodiment;

[0036] Figure 2 A flowchart of a method for generating user interface code based on a multimodal large model provided in this embodiment. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0038] In order to better understand the embodiments of the present invention, some professional terms appearing in the text are first explained.

[0039] UI: User Interface. The user interface is the core part of human-computer interaction and is the medium for users to interact with computer systems. It includes graphical user interfaces (GUIs), command line interfaces (CLIs), and other forms of interactive interfaces. In mobile application and web development, UI usually refers to graphical elements that users can see and interact with, such as buttons, text boxes, images, etc.

[0040] MLLM: Multimodal Large Language Model. Multimodal large models are an important development in the field of artificial intelligence, which expands the capabilities of traditional large models that only process text. MLLM is able to process and understand multiple types of input data at the same time, such as text, images, audio, and video. This multimodal capability allows the model to perform more complex tasks such as image description, visual question answering, cross-modal retrieval, etc. In UI development, MLLM can be used to understand design images and generate corresponding code, or generate UI designs based on text descriptions.

[0041] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for generating user interface code based on a multimodal large model, aiming to convert a UI design drawing into a functional declarative UI code, and the method comprises the following steps:

[0042] S1. Build a page transition diagram based on the UI design diagram;

[0043] The UI design drawing is a UI design draft produced by UI designers using design software such as Sketch, Adobe Photoshop, Figma and other design tools. UI design drawings usually include the product's interface, layout, icons, buttons and other design elements, showing the product's appearance and user interaction experience. UI design drawings usually consist of multiple interfaces, each of which has corresponding design patterns and controls, as well as related page elements, interaction rules, design styles and other details.

[0044] The Page Transition Graph (PTG) is a graphical structure used to represent the navigation and transition relationships between pages in a multi-page application. In a page transition graph, nodes usually represent different pages or views of an application, while edges represent transitions or navigation paths between pages. The page transition graph can help developers and designers better understand and plan the overall structure and user flow of an application, and is particularly useful for the development and maintenance of complex applications. In the automatic generation of user interface code, the page transition graph can be used to guide the generation of consistent user interface code and navigation logic across pages.

[0045] This step is used to capture the interaction logic of the UI design diagram. This embodiment uses a multimodal large language model (MLLM) to build a page transition diagram. First, the UI design diagram and the definition of the page transition diagram are provided to the multimodal large model. The multimodal large model analyzes the UI design diagram and automatically generates a comprehensive page transition diagram based on the provided prompts, and outputs its analysis results in a structured JSON format.

[0046] To ensure the accuracy and completeness of the captured interaction logic, the generated page transition diagram can also be reviewed by a designer or developer to confirm the accuracy and completeness of the interaction logic and make necessary corrections or additions.

[0047] The introduction of page transition graph aims to provide a clear and structured representation of interaction logic for multimodal large models, effectively bridging the gap in expressing dynamic interactions in static UI design. This approach can not only accurately capture the navigation relationship between pages, but also represent the conditions and operations that trigger these navigations, providing a solid foundation for subsequent user interface code generation.

[0048] S2, extracting UI components in the UI design diagram and analyzing their functions to obtain UI component analysis results;

[0049] This step converts the complex UI design drawing into a structured component representation so that the multimodal large model can be more easily understood and processed. In this embodiment, this step includes three key stages: first, automatic component detection is performed, specifically, computer vision technology can be used to perform zero-sample object detection on the UI design drawing to identify UI components, and generate a bounding box of the UI component. In this way, potential UI components can be identified without a large amount of training for UI-specific data sets; then automatic instance segmentation is performed, specifically, the bounding box can be used as a guide for segmentation, and the image segmentation algorithm can be used to perform instance segmentation on the UI component to obtain the segmented UI component image and complete the extraction of the UI component; finally, UI component classification and functional analysis are performed, which can be completed by applying a multimodal large model, specifically providing the segmented UI component image to the multimodal large model, and the multimodal large model extracts the UI component type and infers the potential function of the UI component based on the UI component image. This method combines the advantages of object detection and instance segmentation, which can not only efficiently identify UI components, but also generate accurate segmentation results.

[0050] S3, integrating the page transition diagram, UI component analysis results, and UI design diagram to generate prompt words;

[0051] This step is the key link to integrate the previous pre-processed data into comprehensive prompt words, which aims to enhance the ability of the multimodal large model to generate user interface code that accurately reflects the original design visual elements and underlying interaction logic. In this embodiment, this step specifically includes: integrating the page transition diagram, UI component analysis results, and UI design diagram to ensure the consistency and integrity of the three information, which can be completed through a specific algorithm; based on the integrated data, construct a detailed context description to form prompt words for generating user interface code based on the integrated data. Among them, the context description includes the overall UI layout, page hierarchy, component functions, and interaction logic. In order to guide the multimodal large model to generate high-quality user interface code, the prompt words also include task instructions, and the task instructions include key points such as code structure requirements, framework-specific syntax, and component implementation details.

[0052] In a preferred embodiment, the prompt words are generated based on a prompt template, and the structure and editing of the prompt template can be optimized through repeated experiments and iterations. Preferably, the prompt template can also be dynamically adjusted to automatically adjust the detail and emphasis of the prompt according to the complexity of the UI design drawing and the target UI framework.

[0053] This comprehensive and sophisticated cue synthesis process provides a structured and informative context for the multimodal large model, enabling it to generate more accurate and functionally complete user interface code.

[0054] S4. Input the prompt words into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

[0055] As mentioned above, in UI development, the multimodal big model can be used to understand the design image and generate the corresponding code. This step is to input the comprehensive prompt words built earlier into the multimodal big model, which can analyze the information in the prompt words and automatically generate the user interface code corresponding to the UI design drawing.

[0056] In a preferred embodiment, after the multimodal large model generates the initial user interface code, iterative code optimization can also be performed to systematically identify and correct potential problems in the generated code to ensure the high quality of the final output. Iterative code optimization specifically includes navigation consistency verification and compile-time error correction of the generated user interface code. For navigation consistency verification, a special static analysis tool can be used to parse the generated code and extract all implemented navigation logic, which is compared in detail with the page transition diagram to verify navigation consistency. For compile-time errors, the code can be systematically processed through an automated compilation check script, key information in the error message can be extracted, and combined with relevant code snippets to form a structured prompt feedback to the multimodal large model for correction. The entire iterative optimization process includes steps such as initial code generation, navigation consistency check, navigation correction, project compilation, compile-time error analysis and error correction. By limiting the number of iterations, while ensuring the quality of the code, the efficiency of the process is also ensured, and high-quality, fully functional user interface code is finally output.

[0057] In order to solve the problem of limited knowledge of large multimodal models when dealing with emerging UI frameworks (such as ArkUI), the Retrieval-Augmented Generation (RAG) method can also be introduced. The core of this method is to build and utilize knowledge graphs (KG) to supplement the model's understanding of a specific framework. Retrieval-Augmented Generation is an AI technology that combines information retrieval and text generation. In this method, the system first retrieves relevant information from a large knowledge base, and then uses this retrieved information to enhance the generation capabilities of the language model. In the context of UI development, RAG can be used to retrieve documentation or code examples for a specific UI framework, and then use this information to generate more accurate UI code that conforms to the specifications of a specific framework.

[0058] Specifically, first, a corresponding knowledge graph is created for the target UI framework, where the target UI framework refers to the UI framework applied by the finally generated user interface code, and the knowledge graph includes at least one of the grammatical rules, design patterns, component libraries, and best practice information of the UI framework. Before generating the code corresponding to the UI design drawing, the multimodal large model queries the knowledge graph to obtain relevant UI framework-specific information and integrates the information into the prompt word. To ensure that the system always keeps abreast of the latest framework features and best practices, a mechanism can be set up to regularly update the knowledge graph and explore various technologies to optimize the information retrieval and integration process. Through this retrieval-enhanced generation method, the performance of the system in processing emerging mainstream frameworks is significantly improved.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) The present invention introduces a multi-page application interaction logic understanding technology based on a page transition graph. By constructing and analyzing a page transition graph, it is possible to fully capture the state transition and routing logic in complex applications, which is essential for generating routing configuration and state management code in a declarative UI framework. This method makes up for the shortcomings of static UI design in expressing dynamic interactions and provides key support for generating fully functional declarative multi-page applications.

[0061] (2) This invention proposes a framework-specific knowledge enhancement method based on retrieval-enhanced generation. This innovation is specifically aimed at the rapid development and diversity of declarative UI frameworks. By building and utilizing knowledge graphs for specific declarative UI frameworks, the accuracy of code generation and framework compatibility are significantly improved. This approach enables the system to better adapt to emerging declarative UI frameworks (such as ArkUI) and generate code that conforms to the latest framework features and best practices of the declarative programming paradigm.

[0062] (3) The present invention adopts a high-precision UI component recognition method that integrates computer vision and multimodal large models. This method can not only accurately identify and segment each component in the UI design, but also deeply understand the function and purpose of each component, providing rich context information for subsequent declarative user interface code generation. Compared with traditional methods, the method of the present invention is more suitable for the componentization and state-driven characteristics of declarative UI, and can better capture the hierarchical relationship and data flow between components, which is the key to generating high-quality declarative user interface code.

[0063] (4) In order to further improve the quality of generated code, the present invention designs an iterative optimization mechanism for code generation for declarative UI frameworks. This mechanism systematically identifies and corrects potential problems in the generated code through multiple rounds of iterations, with a particular focus on challenges unique to declarative UI, such as the correctness of component properties, consistency of state management, and accurate implementation of declarative routing. This optimization mechanism not only checks the syntactic correctness, but also verifies whether the generated code accurately reflects the design intent.

[0064] In summary, the present invention greatly improves the efficiency of UI development, transforms the complex manual coding process into automatic generation, and significantly shortens the product development cycle; the present invention reduces human errors and improves code quality through precise AI-driven analysis; the present invention promotes seamless collaboration between designers and developers, and narrows the gap between design concepts and actual implementations; the present invention provides strong technical support for the widespread application and further development of declarative UI frameworks, and promotes the progress of the entire UI development ecosystem.

[0065] Based on the same inventive concept, an embodiment of the present invention further provides a user interface code generation system based on a multimodal large model, comprising:

[0066] A page transition diagram construction module is used to construct a page transition diagram according to a UI design diagram;

[0067] The UI component extraction and analysis module is used to extract UI components in the UI design diagram and analyze their functions to obtain UI component analysis results;

[0068] The prompt word synthesis module is used to integrate the page transition diagram, UI component analysis results and UI design diagram to generate prompt words;

[0069] The user interface code generation module is used to input prompt words into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

[0070] It should be noted that the technical details that are not described in detail in the embodiment of the user interface code generation system based on the multimodal big model can be referred to the user interface code generation method based on the multimodal big model as described above provided in any embodiment of the present invention, and will not be repeated here.

[0071] Based on the same inventive concept, an embodiment of the present invention also provides a user interface code generation device based on a multimodal large model, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the user interface code generation method based on the multimodal large model as described above when executing the computer program.

[0072] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of the electronic device. In this embodiment, the processor is used to run the program code or process data stored in the memory, such as the program code of the vehicle collision warning method.

[0073] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory can also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory can also include both the internal storage unit of the electronic device and its external storage device. In this embodiment, the memory is generally used to store the operating method and various application software installed in the electronic device, such as the program code of the vehicle collision warning method, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.

[0074] Based on the same inventive concept, the present invention also provides a readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of a user interface code generation method based on a multimodal large model as described above are implemented.

[0075] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0076] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating user interface code based on a multimodal large model, characterized in that: The steps include: Build a page transition diagram based on the UI design diagram; Extract UI components from UI design diagrams and analyze their functions to obtain UI component analysis results; Integrate the page transition diagram, UI component analysis results, and UI design diagram to generate prompt words; The prompt words are input into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

2. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: The construction of a page transition diagram according to the UI design diagram specifically includes: Provide the UI design diagram and the definition of the page transition diagram to the multimodal big model. The multimodal big model analyzes the UI design diagram and generates the page transition diagram according to the provided prompts.

3. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: The extracting of UI components from the UI design diagram and analyzing their functions specifically include: Use computer vision technology to perform zero-shot object detection on UI design images to identify UI components and generate bounding boxes of UI components; Perform instance segmentation of the UI component using an image segmentation algorithm using the bounding box as a guide for segmentation to obtain a segmented UI component image; The segmented UI component images are provided to the multimodal large model, and the multimodal large model extracts the UI component types and infers the functions of the UI components based on the UI component images.

4. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: The step of integrating the page transition diagram, the UI component analysis results and the UI design diagram to generate prompt words specifically includes: Integrate the page transition diagram, UI component analysis results, and UI design diagram to ensure the consistency and integrity of the three information; Based on the integrated data, context descriptions and task instructions are constructed to form prompt words for generating user interface codes.

5. The method for generating user interface code based on a multimodal large model as claimed in claim 4, characterized in that: The context description includes the overall UI layout, page hierarchy, component functions and interaction logic, and the task instructions include code structure requirements, framework-specific syntax and component implementation details.

6. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: The prompt words are generated based on a prompt template. The method also includes dynamically adjusting the prompt template to automatically adjust the detail and emphasis of the prompt according to the complexity of the UI design diagram and the target UI framework.

7. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: Also includes: Create a corresponding knowledge graph for the target UI framework; When generating user interface code, the multimodal large model queries the knowledge graph to obtain relevant UI framework specific information and integrates the information into the prompt word.

8. The method for generating user interface code based on a multimodal large model according to claim 1, characterized in that: After the multimodal large model generates the user interface code according to the prompt word, it also includes: The generated user interface code is iteratively optimized, including navigation consistency verification and compile-time error correction.

9. A user interface code generation system based on a multimodal large model, characterized in that: include: A page transition diagram construction module is used to construct a page transition diagram according to a UI design diagram; The UI component extraction and analysis module is used to extract UI components in the UI design diagram and analyze their functions to obtain UI component analysis results; The prompt word synthesis module is used to integrate the page transition diagram, UI component analysis results and UI design diagram to generate prompt words; The user interface code generation module is used to input prompt words into the multimodal large model, and the multimodal large model generates user interface codes according to the prompt words.

10. A user interface code generation device based on a multimodal large model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the user interface code generation method based on the multimodal large model as described in claims 1-8 when executing the computer program.

Images