An AI large model-based front-end visual layout generation method and system

Abstract

The embodiment of the application provides a front-end visual layout generation method and system based on an AI large model, and relates to the technical field of financial front-end application development. After obtaining design input data including a target technology stack and a front-end design image, a design prompt word is matched based on the design input data. Then, a code data stream is generated according to the design prompt word using a target model, and the code data stream is rendered in real time in a pre-constructed isolated rendering environment. In response to an update instruction for the rendering result of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, and version archiving is performed on the complete code after the iterative optimization. The method can rely on an AI large model to reconstruct a front-end code generation process, build an efficient and stable front-end and back-end collaborative mechanism through a standardized whole-process design, improve the development efficiency, restoration accuracy and code standardization of a financial front-end page, and solve the problem of low front-end application development efficiency.

Description

Technical Field

[0001] This application relates to the field of financial front-end application development technology, and in particular to a front-end visualization layout generation method and system based on AI large model. Background Technology

[0002] Financial front-end applications refer to user-interactive, visual applications that directly reach users in the financial industry. By integrating financial data with an intuitive user interface, financial front-end applications transform complex financial information into actionable interface elements for financial business consultation, product demonstrations, and trading. With the accelerated digital transformation of the financial industry, users have increasingly personalized and diversified needs for financial products and services. Financial institutions need to rapidly iterate their products to respond to market changes, making shortening the development cycle and improving development efficiency of financial front-end applications a core industry demand.

[0003] In the development of financial front-end applications, developers need to manually write application interface programs to recreate the design based on the user interface (UI) design drafts, thereby generating the front-end pages. This manual programming development model consumes a significant amount of developers' time in the front-end page style layout development process. During development, details such as colors, fonts, component layouts, and interactive effects from the design drafts must be meticulously matched and manually coded. Furthermore, because front-end pages in financial scenarios involve complex data display, form validation, and responsive layouts adapted to multiple terminals, the workload for style development is further increased.

[0004] Meanwhile, due to differences in developers' technical skills and coding habits, it's difficult to guarantee consistency between the page styles reproduced by different developers and the design drafts, resulting in inconsistent visual fidelity. Furthermore, in manual coding mode, code quality control relies on developers' subjective awareness, making it difficult to effectively guarantee uniformity in coding standards. This easily leads to problems such as redundant code, inconsistent naming, and logical redundancy, increasing the difficulty of later code maintenance and version iteration. Moreover, potential code vulnerabilities may affect the stability of the front-end application, failing to fully meet the financial industry's requirements for system reliability, security, and scalability. Summary of the Invention

[0005] In view of this, embodiments of this application provide a front-end visual layout generation method and system based on AI large model to solve the problem of low efficiency in front-end application development under manual coding mode.

[0006] According to a first aspect of this application, a front-end visual layout generation method based on an AI large model is provided, the method comprising: Acquire design input data, which includes the target technology stack and front-end design images; Design prompts are matched based on the design input data, and the design prompts include visual element prompts matched according to the target technology stack; the visual element prompts are used to characterize at least one visual element identified from the front-end design image; The target model is used to generate a code data stream based on the design prompts; the code data stream includes multiple code snippets, which are associated with interface elements; the target model is a large language model selected from multiple parallel-running localized deployment models. The code data stream is rendered in real time within a pre-built isolated rendering environment; In response to an update instruction for the rendering result of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, and version archiving is performed on the iteratively optimized complete code.

[0007] In some embodiments, after obtaining the design input data, the method further includes: A connection request is generated based on the input data; The connection request is sent to the backend service to establish a real-time bidirectional communication connection between the frontend and the backend; the real-time bidirectional communication connection is used to synchronize the system's running progress and execute streaming data transmission. The system sends a setting request to the backend service through the real-time bidirectional communication connection, thereby triggering the backend service to process the setting request in pipeline mode and to perform connection settings and connection exception cleanup in response to the setting request.

[0008] In some embodiments, after obtaining the design input data, the method further includes: Extract image parameters from the front-end design image, wherein the image parameters include at least one of image resolution and image file size; Obtain an image judgment threshold, wherein the image judgment threshold includes at least one of a resolution threshold and a file size threshold corresponding to the image parameters; If the image parameters exceed the image judgment threshold, parameter adjustment is performed on the front-end design image, and the parameter adjustment includes at least one of proportional scaling and lossless compression. The front-end design image with adjusted parameters is converted into a unified format, and redundant metadata in the front-end design image is removed.

[0009] In some embodiments, matching design prompts based on the design input data includes: Extract the target technology stack and the front-end design image from the design input data; The target technology stack is invoked according to the prompt word database, which includes multiple different versions of prompt word sets; the multiple versions of prompt word sets are used to represent multiple versions of code variants; Identify the visual elements in the front-end design image, wherein the visual elements include at least one of element color, element shape, element classification, and layout parameters between elements; Match the visual element with the prompt words in the prompt word database, and combine the matched prompt words into the design prompt words.

[0010] In some embodiments, a code data stream is generated using a target model based on the design cue words, including: Obtain model matching information, which includes at least one of the design input data and the current system state; the current system state is used to characterize the model response speed of the locally deployed model. The target model is selected based on the model matching information; Input the design prompts into the target model to obtain the code snippets output by the target model based on the design prompts; The code snippets are combined into the code data stream.

[0011] In some embodiments, combining the code fragments into the code data stream includes: Iterate through the element prompts in the design prompts to determine the total number of interface elements to be generated; Record the number of code snippets output by the target model; The real-time generation progress of the complete code is calculated by comparing the number of code snippets with the total number of interface elements. The real-time generation progress is bound to the code snippet to encapsulate the real-time generation progress into the code data stream.

[0012] In some embodiments, rendering the code data stream in real time within a pre-built isolated rendering environment includes: Extract code snippets from the code data stream, and assemble the code snippets into complete code according to the design prompts; The isolated rendering environment is created using an embedded browser window sandbox framework; Add interface elements to the isolated rendering environment according to the complete code, wherein the interface elements are associated with the code snippet; A preview interface is rendered based on the interface elements, and the preview interface is used to present the rendering result of the code data stream.

[0013] In some embodiments, in response to an update instruction for the rendering result of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, including: Obtain the update instruction, which is an instruction input by the user regarding the rendering result of the code data stream; the update instruction includes supplementary images and supplementary text; Context information is generated based on the update instruction, and the context information includes the supplementary image and supplementary prompt words matched according to the supplementary text; The target model is used to generate supplementary code based on the context information.

[0014] In some embodiments, version archiving of the iteratively optimized complete code includes: Record the historical modification history of the supplementary code; Obtain a stop iteration command, wherein the stop iteration command is a command input by the user for the rendering result corresponding to the supplementary code, or the stop iteration command is a command generated based on the update command and combined with preset iteration judgment conditions; In response to the stop iteration command, the supplementary code and the historical modification records are archived and stored based on the source code-based version management mechanism.

[0015] According to a second aspect of this application, a front-end visual layout generation system based on an AI large model is provided, the system comprising: The front-end module is used to acquire design input data, which includes the target technology stack and the front-end design image; The prompt word matching module is used to match design prompt words based on the design input data. The design prompt words include visual element prompt words matched according to the target technology stack. The visual element prompt words are used to characterize at least one visual element identified from the front-end design image. A code generation module is used to generate a code data stream based on the design prompts using a target model; the code data stream includes multiple code snippets, which are associated with interface elements; the target model is a large language model selected from multiple parallel-running localized deployment models. A real-time rendering module is used to render the code data stream in real time within a pre-built isolated rendering environment; The iterative optimization module is used to perform iterative optimization on the complete code corresponding to the code data stream in response to the update instruction for the rendering result of the code data stream, and to archive the version of the iteratively optimized complete code.

[0016] By employing the above technical solutions, embodiments of this application provide a front-end visual layout generation method and system based on an AI large-scale model. After acquiring design input data including the target technology stack and front-end design images, design prompts are matched based on the design input data. Then, the target model generates a code data stream based on the design prompts. This code data stream is then rendered in real-time in a pre-built isolated rendering environment. In response to update instructions for the rendering results of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, and the optimized complete code is archived. This method can reconstruct the front-end code generation process based on an AI large-scale model. Through standardized end-to-end design, it constructs an efficient and stable front-end and back-end collaboration mechanism, improving the development efficiency, accuracy, and code standardization of financial front-end pages, and solving the problem of low front-end application development efficiency.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of the front-end visual layout generation method based on an AI large model provided in the embodiments of this application; Figure 2 This is a schematic diagram of the front-end visual layout generation system architecture provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the overall process of generating a front-end visual layout, as provided in the embodiments of this application. Figure 4 A schematic diagram of the version archiving process provided in this application embodiment; Figure 5 This is a schematic diagram of the front-end visualization layout generation system based on an AI large model, provided in an embodiment of this application. Detailed Implementation

[0019] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0020] In this embodiment, the AI-based large-scale model-based front-end visualization layout generation method and system can be used for financial front-end application development. Here, the financial front-end application refers to a user-interactive, visual application directly facing users in the financial industry. By integrating financial data with an intuitive user interface, financial front-end applications can transform complex financial information into operable interface elements for financial business consultation, product display, and trading.

[0021] In the development of financial front-end applications, developers need to recreate the user interface (UI) design drafts by writing application interface programs to generate front-end pages. In some embodiments, to generate front-end pages, after obtaining the UI design drafts, a technology stack can be selected based on project requirements. A technology stack refers to the complete set of technologies required to build, run, and maintain an application. For example, a technology stack may include stylesheets such as React, Vue, Bootstrap, HTML, Angular, and TypeScript, component libraries, or complete customization.

[0022] After selecting a technology stack, use tools like Vite or Create React App to quickly initialize the project, configuring development standards such as ESLint, Prettier, and path aliases to set up the environment. Then, create the basic project structure and global styles, establishing a code skeleton and visual tone that match the UI design drafts to ensure overall style consistency.

[0023] Next, by deconstructing the design drafts, a component tree is constructed to transform the design drawings into a reusable component structure. Then, following priorities such as from the outside in, from the largest to the smallest, and from top to bottom, the visual reproduction of static components is achieved. During the visual reproduction process, static components can be added sequentially through layout structure implementation, general component filling, page assembly, and style implementation.

[0024] After adding static components, the static page is transformed into a usable front-end interface by adding interactions and dynamic data. The added interactions and dynamic data can include state management, event handling, simulated Application Programming Interface (API) data, and controlled form components. Then, responsive cross-platform adaptation ensures the interface's usability across different resolutions and devices.

[0025] The development of financial front-end applications consumes a significant amount of developers' time in the style and layout development process. This requires developers to meticulously coordinate details such as colors, fonts, component arrangement, and interactive effects from the design drafts, and then manually implement them through coding. Furthermore, front-end pages in financial scenarios involve complex data display, form validation, and responsive layouts adapted to multiple terminals, which further increases the workload of style development and reduces the efficiency of front-end application development.

[0026] Meanwhile, due to differences in developers' technical skills and coding habits, it's difficult to guarantee consistency between the page styles reproduced by different developers and the design drafts, resulting in inconsistent visual fidelity. Furthermore, in manual coding mode, code quality control relies on developers' subjective awareness, making it difficult to effectively guarantee uniformity in coding standards. This easily leads to problems such as redundant code, inconsistent naming, and logical redundancy, increasing the difficulty of later code maintenance and version iteration. Moreover, potential code vulnerabilities may affect the stability of the front-end application, failing to fully meet the financial industry's requirements for system reliability, security, and scalability.

[0027] To address the low efficiency of front-end application development under manual coding, this application provides a front-end visual layout generation method based on an AI-powered large-scale model in some embodiments. This method, designed with intelligent, standardized, and iterative approaches, reduces the time invested in style layout during the financial front-end development process, allowing developers to focus on core business logic construction, performance iteration optimization, and security system building. The method achieves a one-to-one accurate replication of the page's visual effect from the design draft, while ensuring the quality, compliance, and standardization of the generated code. This helps financial institutions quickly respond to market dynamics and business needs, improve product iteration efficiency and core competitiveness, and drive the transformation of financial front-end development towards intelligence and efficiency.

[0028] The method can be applied to electronic devices with data processing capabilities. As a front-end interactive device in the application development process, the electronic device includes, but is not limited to, computers, servers, mobile terminals, smart wearable devices, and industrial control machines. For ease of description, in this application embodiment, a front-end electronic device or a front-end visual layout generation system is used as the execution subject of the method. It should be understood that the method can also be applied to other types of execution subjects, which are not shown one by one in this application embodiment. Figure 1 As shown, the method includes: S101. Obtain design input data.

[0029] When generating a front-end visual layout, design input data can be obtained first. This design input data is the set of data entered by the user on the front-end electronic device through data input-related interactive operations. The design input data includes the target technology stack and the front-end design image.

[0030] For example, such as Figure 2 As shown, the front-end visual layout generation system includes a front-end interaction layer, a back-end service layer, and a data storage layer. As the starting point of the front-end visual layout generation process, the front-end interaction layer is used for data input; its core task is to complete user input, providing the foundation for subsequent processes. Users can perform two core operations based on the front-end interaction layer. The first core operation is selecting the target technology stack. The front-end visual layout generation process supports multiple front-end technology stacks such as React, Bootstrap, HTML, and Vue. Users can choose one from these stacks as the target technology stack based on their specific development needs.

[0031] The second core operation is uploading the front-end design image. Users perform file upload interactions using the file upload control provided by the front-end interaction layer, uploading the user interface design draft as the front-end visual layout generation system for layout generation. To meet the design needs of different users, the front-end visual layout generation system supports multiple image formats such as PNG, JPG, and JPEG to ensure that the front-end design image can be accurately recognized by the subsequent AI model.

[0032] Once the input data is acquired, the front-end visual layout generation system can automatically proceed to the next step, establishing a communication connection between the front-end interaction layer and the back-end service to ensure subsequent data transmission and progress synchronization. By constructing an efficient data flow channel, bidirectional communication between the front-end and back-end is enabled. For example, the front-end establishes a bidirectional communication connection via WebSocket to achieve real-time interaction between requests and results. The back-end uses a pipeline model to process requests in layers, sequentially completing the establishment and cleanup of WebSocket connections, extraction and validation of request parameters, system status broadcasting, creation of custom prompts, AI code generation, and post-processing of results, ensuring a standardized workflow.

[0033] Therefore, in some embodiments, after acquiring design input data, the electronic device can also generate a connection request based on the input data and send the connection request to the backend service to establish a real-time bidirectional communication connection between the frontend and the backend. This real-time bidirectional communication connection is used to synchronize system operation progress and execute streaming data transmission. Then, a setting request is sent to the backend service through the real-time bidirectional communication connection to trigger the backend service to process the setting request in a pipeline mode and, in response to the setting request, perform connection setup and connection exception cleanup.

[0034] For example, after obtaining the design input data, the front-end visual layout generation system can enter the input processing and AI generation stage. Before input processing, real-time bidirectional communication between the front-end and back-end can be established through the WebSocket protocol to ensure that the code streaming output and the visualization of the generation progress are synchronized.

[0035] Establishing a WebSocket bidirectional communication connection is the core support for front-end and back-end data interaction, connecting input processing with subsequent image optimization, AI generation, and other processes. When establishing a WebSocket bidirectional communication connection, the front-end electronic device can first send a connection request to establish a WebSocket connection with the back-end, enabling real-time bidirectional communication between the front-end and back-end and breaking the unidirectional limitation of HTTP requests.

[0036] The WebSocket bidirectional communication connection allows for the synchronization of system progress during communication, such as image optimization and AI code generation, ensuring users can view these updates in real-time on the front end. The WebSocket bidirectional communication connection can also be used for streaming data transmission, particularly for pushing AI-generated code to the front end in real-time, thus enhancing the user experience.

[0037] After establishing a WebSocket bidirectional communication connection, the backend service can use a pipelined approach to handle setup requests from the frontend electronic device. These setup requests include WebSocket connection settings, so upon receiving a setup request, the backend service can configure the WebSocket connection accordingly. Furthermore, the backend service can perform connection exception cleanup to ensure communication stability and prevent process anomalies caused by connection interruptions.

[0038] After establishing a two-way communication connection, the front-end visual layout generation system can perform multi-framework adaptation and image optimization for initial data processing. To this end, the front-end visual layout generation system can deploy a multi-framework adaptation and image optimization module. This module supports adaptation to mainstream front-end frameworks such as React, Vue, Bootstrap, and HTML, and is compatible with PNG, JPG, and JPEG image uploads. Furthermore, this module has a built-in image processing engine to detect image size and quality. When it detects images exceeding preset limits, it automatically adjusts the size or compression quality of the front-end design image and converts it to a specific format to reduce file size and ensure efficient subsequent parsing.

[0039] Therefore, in some embodiments, after obtaining the design input data, the front-end visual layout generation system can extract image parameters of the front-end design image, wherein the image parameters include at least one of image resolution and image file size.

[0040] Next, an image judgment threshold is obtained, which includes at least one of a resolution threshold and a file size threshold corresponding to image parameters. The extracted image parameters are then compared with the preset image judgment threshold. If the image parameters do not exceed the threshold, the front-end design image is directly used as input to the AI ​​model for code generation. If the image parameters exceed the threshold, parameter adjustments are performed on the front-end design image, including at least one of proportional scaling and lossless compression. The parameter-adjusted front-end design image is then converted to a unified format, and redundant metadata is removed.

[0041] For example, the core objective of the image optimization stage is to standardize the front-end design images uploaded by users to adapt to the recognition requirements of the AI ​​model and ensure the visual fidelity of subsequent code generation. Therefore, the front-end visual layout generation system can first perform image parameter detection, that is, extract the core parameters of the uploaded image, including image resolution and file size, through an image parameter detection unit, providing a basis for subsequent judgments. Then, through threshold judgment, the extracted image parameters are compared with the image judgment thresholds preset by the AI ​​model to determine whether the image exceeds the limits, such as being too large in size or file size.

[0042] If the image parameters exceed the preset image judgment threshold, the parameters of the input front-end design image need to be adjusted. During the parameter adjustment process, the system can use a proportional scaling algorithm to adjust the image size to avoid image distortion. Furthermore, a lossless compression strategy is adopted for quality compression, performing lossless compression on the input front-end design image. If the lossless compression method cannot meet the size requirements, lossy compression is performed with the principle of minimizing visual sharpness loss.

[0043] Image optimization processing for front-end design images also includes format unification adjustment. Regardless of whether the front-end design images have undergone parameter adjustments, they are all ultimately converted to JPEG format, and redundant metadata in the images are removed to reduce file size, so as to ensure the recognition accuracy and efficiency of the AI ​​model.

[0044] As can be seen, in the image optimization stage, core parameters such as image resolution and file size can be extracted first through the image parameter detection unit and compared with the preset thresholds of the AI ​​model. Parameter adjustments can then be made based on the comparison results. Size adjustment in parameter adjustment uses a proportional scaling algorithm to avoid image distortion; quality compression in parameter adjustment adopts a lossless compression priority strategy. When lossless compression cannot meet the size requirements, lossy compression is performed with the principle of minimizing visual sharpness loss. Finally, the image is uniformly converted to JPEG format, and redundant metadata is removed to ensure compatibility with the AI ​​model's recognition accuracy and efficiency.

[0045] S102. Match design prompts based on design input data.

[0046] After acquiring and optimizing the design input data, design prompts can be matched based on the input data. These prompts include visual element prompts matched according to the target technology stack. The visual element prompts characterize at least one visual element identified from the front-end design image. For example, matching prompts such as "red," "bold," and "100px" can represent visual elements such as color, font, and spacing, respectively.

[0047] In order to match design prompts, in some embodiments, when matching design prompts based on design input data, the target technology stack and front-end design image can be extracted from the design input data first, and the prompt database can be called according to the target technology stack. The prompt database includes multiple sets of prompts of different versions; the multiple sets of prompts of different versions are used to represent multiple versions of code variants.

[0048] Then, visual elements are identified in the front-end design image. These visual elements include at least one of element color, element shape, element category, and layout parameters between elements. Based on the identification results, prompts associated with the visual elements are matched in a prompt database, and the matched prompts are combined into design prompts.

[0049] For example, such as Figure 3 As shown, after image optimization, the front-end visual layout generation system can enter the AI ​​generation stage. The AI ​​generation stage is the core functional component of the system, responsible for generating compliant front-end code based on the optimized front-end design image and the selected target technology stack. Simultaneously, the AI ​​generation stage can support secondary optimization, forming an iterative closed loop.

[0050] Before AI generation, adaptive design prompts need to be generated. The system automatically matches the corresponding prompt database based on the user's selected target technology stack, ensuring that the design prompts input into the AI ​​model clearly instruct the AI ​​model to accurately replicate visual details such as color, font, and spacing in the front-end design images, thus ensuring the visual fidelity of the generated code.

[0051] Therefore, when entering the AI ​​generation stage, an AI model with image recognition capabilities can be used to identify visual elements in the front-end design image. This allows for the identification of various visual controls in the front-end interface, such as buttons, information boxes, graphics, and display effects. Furthermore, features are extracted from the visual details of each visual control, such as color, font, and spacing, to determine the visual elements contained in the front-end design image.

[0052] Then, based on the visual elements, match the corresponding prompts in the prompt database associated with the target technology stack, such as prompts for red, bold, and 100px. The matched prompts are then combined into design prompts, such as "Write the HTML code according to the layout of image A, where button B is red, the text on button B is bold, and the distance between button B and button C is 100px."

[0053] S103. Use the target model to generate code data flow based on design prompts.

[0054] After matching the design hints, the design hints and the front-end design image can be input into the target model. The target model then generates code based on the design hints, forming a code data stream. The target model is a large language model selected from multiple parallel-running localized deployment models.

[0055] For example, the front-end visual layout generation system can integrate locally deployed AI models such as Qwen3-VL-Plus, GLM-4.6V, and Minimax-VL-01. These AI models support custom parameter configurations to adapt to local application needs. The AI ​​model can generate four versions of code by default to meet diverse development requirements. During the AI ​​generation phase, a target model can be selected from multiple locally deployed AI models based on the current system status and actual business needs.

[0056] The code data stream includes multiple code snippets. Each code snippet is program code automatically generated by the target model based on design prompts and front-end design images. Each code snippet can independently represent a UI element, meaning that there is a relationship between the code snippet and the UI element.

[0057] For example, by performing visual element recognition on the front-end design image, it is determined that the front-end design image contains the interface element "Button C". The prompts corresponding to "Button C" include "the color is green, the text displayed on Button B is in bold, and the distance between Button C and Button B is 100px". Based on the above prompts, the corresponding code snippet can be generated, including: "background-color: green / #2ecc71 / rgb(46,204,113); font-family: Microsoft YaHei; margin: 0100px", which is used to present the corresponding visual effect of "Button C" in the front-end interface.

[0058] To generate code data streams, in some embodiments, when generating code data streams using a target model based on design prompts, model matching information can be obtained first, and a target model can be selected based on the model matching information. The model matching information includes at least one of design input data and the current system state; the current system state is used to characterize the model response speed of the locally deployed model.

[0059] For example, when determining the target model, one can first identify AI models that can generate program code under the target technology stack, such as multiple AI models like Qwen3-VL-Plus, GLM-4.6V, and Minimax-VL-01 for local deployment. Then, by extracting the current system state, the AI ​​model with the fastest response speed among the multiple AI models can be selected as the target model.

[0060] The design prompts are then input into the target model to obtain code snippets output by the target model based on the design prompts. These code snippets are then combined into a code data stream. Furthermore, while combining the code snippets into the code data stream, the total number of UI elements to be generated can be determined by iterating through the element prompts in the design prompts, and the number of code snippets output by the target model is recorded. By comparing the number of code snippets with the total number of UI elements, the real-time generation progress of the complete code is calculated. This real-time generation progress is then bound to the code snippets to encapsulate the real-time generation progress into the code data stream.

[0061] The AI ​​model can generate multiple versions of code, meaning it generates four different code variants by default. The generated code can be streamed to the front end via WebSocket, allowing users to monitor the generation progress in real time. For example, if the Qwen3-VL-Plus model is determined to have the fastest response time, it can be used as the target model. Design prompts are then input into this target model to obtain code snippets based on the design prompts. These snippets can then be combined into a code data stream.

[0062] When combining code snippets into a code data stream, the total number of UI elements to be generated can be determined by iterating through the element prompts in the design prompts. For example, if the design prompts include visual element prompts for button B, button C, graphic D, and animation E, the total number of UI elements can be determined to be 4. Simultaneously, based on the output of the Qwen3-VL-Plus model, the number of code snippets output by the target model is recorded, such as 2 code snippets. Then, by comparing the number of code snippets with the total number of UI elements, the real-time generation progress of the complete code is calculated; that is, when two code snippets are generated, the real-time generation progress is 2 / 4 = 50%. Finally, the real-time generation progress is bound to the code snippets to encapsulate the real-time generation progress into the code data stream.

[0063] As can be seen, the AI ​​generation stage employs multiple AI models deployed locally to operate in parallel, adaptively matching the target technology stack, and introducing corresponding CSS frameworks and JavaScript libraries to accurately replicate the visual details of the design images, such as color, font, and spacing. It generates four code versions by default for selection, improving the adaptability of code generation.

[0064] S104. Render code data streams in real time within a pre-built isolated rendering environment.

[0065] After generating the code data stream, the backend service can push the code data stream to the frontend electronic device, so that the frontend electronic device can render the code data stream in real time in a pre-built isolated rendering environment, so that the user can perform preview verification.

[0066] In some embodiments, when rendering code data streams in real time within a pre-built isolated rendering environment, code snippets can be extracted from the code data stream first, and then assembled into complete code according to design prompts. An isolated rendering environment is then created using an embedded browser window sandbox framework. Interface elements are then added to the isolated rendering environment according to the complete code, and a preview interface is rendered based on these interface elements. The interface elements are associated with the code snippets; the preview interface is used to present the rendering result of the code data stream.

[0067] For example, after obtaining the code data stream, the front-end visual layout generation system can enter the preview and verification stage. This stage verifies the effectiveness of the AI-generated code and supports real-time user editing and visual fidelity verification, providing a basis for subsequent iterations and optimizations. During this stage, iframe sandbox technology can be used to build an isolated rendering environment to intercept malicious scripts and cross-domain requests, ensuring system security.

[0068] Based on the isolated rendering environment, multiple view switching previews are possible, supporting PC desktop mode, mobile mode, and code viewing mode. Users can view the rendering effect of the code from different device perspectives, ensuring that the code is adapted to multiple device scenarios.

[0069] It also establishes relationships between interface elements and code snippets to support real-time editing and secure rendering of the code corresponding to the code data stream. For example, real-time code editing and syntax highlighting allow users to quickly modify the code. When a user clicks on any interface element in the preview page, the system accurately locates the corresponding code snippet based on the relationships, allowing the user to make targeted modifications and ensuring that the code rendering effect is highly consistent with the original design image.

[0070] As can be seen, through visual fidelity verification, it is possible to support fast switching and scaling adaptation of multiple views on PC desktop and mobile devices during the preview verification stage, along with real-time code viewing, syntax highlighting and online editing functions; by building an isolated rendering environment through iframe sandbox technology, malicious scripts and cross-domain requests are blocked, while online modification and adjustment are supported, which facilitates targeted modifications and visual fidelity verification.

[0071] S105. In response to the update instruction for the rendering result of the code data stream, perform iterative optimization on the complete code corresponding to the code data stream, and archive the version of the iteratively optimized complete code.

[0072] After generating a preview interface by rendering the code data stream in real time, the system can also receive update instructions for the rendered code data stream and, in response to these instructions, perform iterative optimization on the complete code corresponding to the code data stream, achieving a secondary optimization loop. Furthermore, it can archive versions of the iteratively optimized complete code, retaining multiple versions of the complete code.

[0073] To this end, the front-end visual layout generation system can be configured with an interactive iteration module, which can be used to achieve visual verification and dynamic optimization. This involves setting a preview panel on the front end, providing three view modes: PC desktop, mobile phone, and code view, supporting real-time display of generation progress and real-time code editing and preview. When receiving update commands from the user on the preview interface, the system can collect the complete historical record, synchronously upload the new front-end design image, and input new prompts as contextual information. After being sent to the server, AI generates an optimized version of the complete code based on the historical information, the new front-end design image, and the new prompts.

[0074] To perform iterative optimization, in some embodiments, when performing iterative optimization on the complete code corresponding to the code data stream in response to an update instruction for the rendered code data stream result, the update instruction can be obtained first. The update instruction is a user-input instruction for the rendered code data stream result; the update instruction includes supplementary images and supplementary text.

[0075] Then, context information is generated based on the update command, and supplementary code is generated using the target model based on the context information. The context information includes supplementary images and supplementary prompts matched with the supplementary text; correspondingly, the supplementary code is the complete code generated by the target model based on the context information. Therefore, when generating supplementary code, the target model can refer to the supplementary images, supplementary text, and historical information in the context information to make the generated complete code more in line with the user's personalized needs.

[0076] For example, after rendering the code data stream in real time, the front-end visual layout generation system can enter the iteration control phase. The iteration control phase is responsible for managing the entire lifecycle of the code, supporting multiple rounds of iterative optimization and version traceability, ensuring that the process is controllable and traceable.

[0077] During the iteration management phase, optimization requirements can be assessed first. After users view the preview, it can be determined whether the code needs to be updated or optimized. If no optimization is needed, the version archiving phase can be initiated directly. If optimization is needed, the subsequent optimization process can be initiated.

[0078] In the subsequent optimization process, optimization instructions can be processed, that is, the user inputs an update instruction and adds images or text content as context information. At the same time, the front end automatically collects the complete historical modification record of the code from generation to the present and sends it to the backend service.

[0079] The backend service then undergoes a second AI-driven optimization loop. When code optimization needs arise during subsequent iteration and management phases, the backend receives optimization instructions and relevant context. AI then combines historical modification records and supplementary images to generate optimized code. Once optimization is complete, the system re-enters this phase, forming a complete closed loop of "generation, optimization, and regeneration" to ensure continuous code improvement.

[0080] After iteration optimization is completed or when no iteration optimization is needed, the system can archive and manage versions. That is, based on the Git-style version management mechanism, the current version is archived and stored. At the same time, the complete editing history is maintained, allowing users to roll back to any historical point, compare differences between different versions, and have a complete exception feedback mechanism to ensure that the iteration process is traceable and controllable.

[0081] By applying the technical solutions of the above embodiments, the front-end visual layout generation method based on AI large models described in the above embodiments can comprehensively optimize the entire development process by relying on AI intelligent technology. Based on this method, development efficiency can be significantly improved. The front-end visual layout generation process can generate a usable webpage prototype within minutes, quickly outputting visual results to support requirement verification. Furthermore, by automatically generating basic UI code, it reduces repetitive coding work for developers and accelerates the product iteration pace.

[0082] The quality and adaptability of the code generated using the method described above can also be optimized. Since AI-generated code strictly adheres to industry best practices and standards, code consistency is guaranteed. Simultaneously, it can achieve a one-to-one accurate replication of the page's visual effects and design drafts, precisely restoring details such as color, font, and spacing. It also considers browser compatibility and responsive layout requirements, automatically generating multi-terminal adapted code to meet the diverse display needs of the financial industry.

[0083] The method described above offers significant cost-effectiveness advantages, greatly reducing manual coding time in front-end development and lowering labor costs; shortening the design-to-prototype conversion cycle and reducing time costs; and leveraging AI automation capabilities to avoid human coding errors and reduce later debugging and repair costs.

[0084] The method can also optimize the development experience by supporting the generation of multiple versions of code from multiple AI models for comparison and selection, and enabling rapid iteration with intelligent regeneration functionality; it provides multi-device preview and interactive editing capabilities, allowing real-time viewing of multi-terminal effects and online modification and adjustment, ensuring cross-platform compatibility, improving development flexibility and accuracy, and strengthening the digital transformation competitiveness of financial institutions.

[0085] In some embodiments, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, some embodiments of this application also provide a front-end visual layout generation method based on an AI large model. The difference between this method and the above embodiments is that it can employ a source code-based version management mechanism to archive the version of the iteratively optimized complete code, such as... Figure 4 As shown, the method includes: S201. Record the historical modification history of supplementary code; S202, Obtain the stop iteration instruction; S203. In response to the stop iteration command, archive and store supplementary code and historical modification records based on the source code-based version management mechanism.

[0086] When archiving versions, you can first record the historical modifications to the supplementary code and listen for control commands to stop iteration, i.e., stop iteration commands. These stop iteration commands are either user-input commands based on the rendering results of the supplementary code, or they are commands generated based on update commands and preset iteration conditions.

[0087] Upon receiving a stop iteration command, the system can respond by archiving and storing supplementary code and historical modification records based on a source code-style version management mechanism. This allows the front-end visual layout generation system to record the entire editing history and supports version rollback, difference comparison, and operation traceability, ensuring the controllability of code iteration.

[0088] For example, a front-end visual layout generation system can be configured with a history management module. This module uses a Git-style commit model to record the entire editing history, supporting version rollback, difference comparison, and operation traceability, ensuring the controllability of code iteration. After an iteration is completed, the history management module can automatically archive and store the current version, while also supporting version rollback to any historical point and difference comparison between different versions, ensuring the iteration process is traceable and controllable.

[0089] By applying the technical solutions of the above embodiments, the front-end visual layout generation method based on AI large models described in the above embodiments can improve the interaction and iteration capabilities after code generation, and enhance the experience of the entire development process. Through multi-view preview and real-time editing functions, visual verification of code effects can be achieved; based on the update mechanism of historical records and context association, it supports code optimization by combining text commands and image context; and with the historical management function of the Git-style commit model, it enables version rollback, comparison, and traceability.

[0090] In some embodiments, as a specific implementation of the AI-based large-scale model-based front-end visual layout generation method described in the above embodiments, some embodiments of this application also provide an AI-based large-scale model-based front-end visual layout generation system, such as... Figure 5 As shown, the system includes: The front-end module is used to acquire design input data, which includes the target technology stack and the front-end design image; The prompt word matching module is used to match design prompt words based on the design input data. The design prompt words include visual element prompt words matched according to the target technology stack. The visual element prompt words are used to characterize at least one visual element identified from the front-end design image. A code generation module is used to generate a code data stream based on the design prompts using a target model; the code data stream includes multiple code snippets, which are associated with interface elements; the target model is a large language model selected from multiple parallel-running localized deployment models. A real-time rendering module is used to render the code data stream in real time within a pre-built isolated rendering environment; The iterative optimization module is used to perform iterative optimization on the complete code corresponding to the code data stream in response to the update instruction for the rendering result of the code data stream, and to archive the version of the iteratively optimized complete code.

[0091] By collaborating with multiple modules such as the front-end module, prompt word matching module, code generation module, real-time rendering module, and iterative optimization module, a closed-loop process from image uploading to code generation, optimization, and traceability can be achieved, thereby improving the efficiency and standardization of financial front-end development.

[0092] For example, when generating front-end visual layouts based on large AI models, the front-end module can obtain design input data such as front-end design images and selected target technology stacks. Through image optimization, the uploaded front-end design images are automatically processed, detecting image size and quality. If the image exceeds a preset threshold, its size is automatically adjusted and converted to a uniform format to meet the AI ​​model's recognition requirements.

[0093] The prompt word matching module, code generation module, and iterative optimization module can be used to perform AI generation with secondary iterative optimization capabilities. Through parallel processing of multiple AI models, adaptive prompt words are matched based on the target technology stack to generate multiple versions of front-end code variants. The code is then streamed to the front end in real time through the real-time rendering module. Preview verification is then performed, and the generated effect is displayed through a multi-device preview mode, supporting switching between PC and mobile device views and real-time viewing and editing of the code. Secure rendering is achieved based on an iframe sandbox.

[0094] The iterative optimization module, based on Git-style version control, records the entire modification process to support multi-round, conversational incremental updates. It can supplement image context optimization code and archive the version after the iteration is completed. The volume, based on a Git-style version control mechanism, fully records the entire modification trajectory from code generation to editing and optimization. It allows users to supplement images as contextual information, enabling AI to combine historical modification records with newly added image context to generate optimized code, ensuring consistent iteration logic.

[0095] By applying the technical solutions of the above embodiments, the AI-based large model-based front-end visual layout generation system described in the above embodiments can solve the inherent defects in the existing financial front-end development model, such as long style layout time, inconsistent restoration accuracy, difficulty in controlling code quality, and insufficient investment in core links. It reconstructs the front-end code generation process by relying on AI intelligent technology, and achieves a triple improvement in the efficiency of financial front-end page development, restoration accuracy, and code standardization, thus adapting to the core demands of rapid product iteration and compliant controllability under the digital transformation of the financial industry.

[0096] Addressing the needs of financial front-end development for multi-technology stack and multi-scenario adaptation, it can achieve compatibility and adaptation with multiple frameworks and multiple image formats. It supports mainstream front-end frameworks such as React, Vue, Bootstrap, and HTML, as well as image formats such as PNG, JPG, and JPEG for uploading and parsing. It breaks the limitations of a single technology stack and meets the development needs of different front-end applications in financial scenarios. At the same time, by adjusting the image size, compressing the quality, and uniformly converting the format to JPEG, it optimizes image processing efficiency and lays the foundation for subsequent AI analysis and code generation.

[0097] Furthermore, a standardized end-to-end design constructs an efficient and stable front-end and back-end collaboration mechanism. A bidirectional communication channel is established using WebSocket to ensure real-time data transmission; the back-end employs a pipelined approach for layered request processing, standardizing the workflow of WebSocket connection management, parameter validation, prompt creation, and code generation, thereby improving the system's orderliness and maintainability and resolving communication delays and workflow chaos issues in traditional development.

[0098] To ensure the accuracy of visual reproduction and code adaptability of the front-end page, the system can optimize the prompt word construction and multi-version generation mechanism, configure exclusive system prompt words for different technology stacks, and clearly instruct AI to accurately replicate the appearance details of the screenshot, meeting the visual requirements of the financial industry for compliance and seriousness of the page. At the same time, it integrates multiple AI models such as Qwen3-VL-Plus, GLM-4.6V, and Minimax-VL-01 deployed locally, supports the generation of multiple versions of code with customized parameters, provides developers with diversified choices, and improves the flexibility and adaptability of code generation.

[0099] The system can also improve the interaction and iteration capabilities after code generation, enhancing the overall development experience. Through multi-view preview and real-time editing, it enables visual verification of code effects; based on a history and context-based update mechanism, it supports code optimization by combining text commands and image context; and with the Git-style commit model's history management function, it enables version rollback, comparison, and traceability.

[0100] In summary, the system, through its intelligent, standardized, and iterative design, significantly reduces the time spent by financial front-end developers on style layout, allowing them to focus on core business logic construction, performance iteration and optimization, and security system building. The system can achieve a one-to-one accurate replication of the page's visual effects from the design draft, while ensuring the quality, compliance, and standardization of the generated code. This helps financial institutions respond quickly to market dynamics and business needs, significantly improving product iteration efficiency and core competitiveness, and driving the transformation of financial front-end development towards intelligence and efficiency.

[0101] It should be noted that other corresponding descriptions of the functional units involved in the AI-based large model-based front-end visual layout generation system provided in this application embodiment can be found in the corresponding descriptions in the AI-based large model-based front-end visual layout generation method provided in the above embodiments, and will not be repeated here.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

Claims

1. A front-end visual layout generation method based on an AI large model, characterized in that, The method includes: Acquire design input data, which includes the target technology stack and front-end design images; Design prompts are matched based on the design input data, and the design prompts include visual element prompts matched according to the target technology stack; the visual element prompts are used to characterize at least one visual element identified from the front-end design image; The target model is used to generate a code data stream based on the design prompts; the code data stream includes multiple code snippets, which are associated with interface elements; the target model is a large language model selected from multiple parallel-running localized deployment models. The code data stream is rendered in real time within a pre-built isolated rendering environment; In response to an update instruction for the rendering result of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, and version archiving is performed on the iteratively optimized complete code.

2. The method according to claim 1, characterized in that, After obtaining the design input data, the method further includes: A connection request is generated based on the input data; The connection request is sent to the backend service to establish a real-time bidirectional communication connection between the frontend and the backend; the real-time bidirectional communication connection is used to synchronize the system's running progress and execute streaming data transmission. The system sends a setting request to the backend service through the real-time bidirectional communication connection, thereby triggering the backend service to process the setting request in pipeline mode and to perform connection settings and connection exception cleanup in response to the setting request.

3. The method according to claim 1, characterized in that, After obtaining the design input data, the method further includes: Extract image parameters from the front-end design image, wherein the image parameters include at least one of image resolution and image file size; Obtain an image judgment threshold, wherein the image judgment threshold includes at least one of a resolution threshold and a file size threshold corresponding to the image parameters; If the image parameters exceed the image judgment threshold, parameter adjustment is performed on the front-end design image, and the parameter adjustment includes at least one of proportional scaling and lossless compression. The front-end design image with adjusted parameters is converted into a unified format, and redundant metadata in the front-end design image is removed.

4. The method according to claim 1, characterized in that, Based on the design input data, design prompt words are matched, including: Extract the target technology stack and the front-end design image from the design input data; The target technology stack is invoked according to the prompt word database, which includes multiple different versions of prompt word sets; the multiple versions of prompt word sets are used to represent multiple versions of code variants; Identify the visual elements in the front-end design image, wherein the visual elements include at least one of element color, element shape, element classification, and layout parameters between elements; Match the visual element with the prompt words in the prompt word database, and combine the matched prompt words into the design prompt words.

5. The method according to claim 1, characterized in that, The target model is used to generate a code data stream based on the design prompts, including: Obtain model matching information, which includes at least one of the design input data and the current system state; the current system state is used to characterize the model response speed of the locally deployed model. The target model is selected based on the model matching information; Input the design prompts into the target model to obtain the code snippets output by the target model based on the design prompts; The code snippets are combined into the code data stream.

6. The method according to claim 5, characterized in that, Combining the code fragments into the code data stream includes: Iterate through the element prompts in the design prompts to determine the total number of interface elements to be generated; Record the number of code snippets output by the target model; The real-time generation progress of the complete code is calculated by comparing the number of code snippets with the total number of interface elements. The real-time generation progress is bound to the code snippet to encapsulate the real-time generation progress into the code data stream.

7. The method according to claim 1, characterized in that, Rendering the code data stream in real time within a pre-built isolated rendering environment includes: Extract code snippets from the code data stream, and assemble the code snippets into complete code according to the design prompts; The isolated rendering environment is created using an embedded browser window sandbox framework; Add interface elements to the isolated rendering environment according to the complete code, wherein the interface elements are associated with the code snippet; A preview interface is rendered based on the interface elements, and the preview interface is used to present the rendering result of the code data stream.

8. The method according to claim 1, characterized in that, In response to an update instruction for the rendering result of the code data stream, iterative optimization is performed on the complete code corresponding to the code data stream, including: Obtain the update instruction, which is an instruction input by the user regarding the rendering result of the code data stream; the update instruction includes supplementary images and supplementary text; Context information is generated based on the update instruction, and the context information includes the supplementary image and supplementary prompt words matched according to the supplementary text; The target model is used to generate supplementary code based on the context information.

9. The method according to claim 8, characterized in that, The complete code after iterative optimization is version archived, including: Record the historical modification history of the supplementary code; Obtain a stop iteration command, wherein the stop iteration command is a command input by the user for the rendering result corresponding to the supplementary code, or the stop iteration command is a command generated based on the update command and combined with preset iteration judgment conditions; In response to the stop iteration command, the supplementary code and the historical modification records are archived and stored based on the source code-based version management mechanism.

10. A front-end visual layout generation system based on an AI large model, characterized in that, The system includes: The front-end module is used to acquire design input data, which includes the target technology stack and the front-end design image; The prompt word matching module is used to match design prompt words based on the design input data. The design prompt words include visual element prompt words matched according to the target technology stack. The visual element prompt words are used to characterize at least one visual element identified from the front-end design image. A code generation module is used to generate a code data stream based on the design prompts using a target model; the code data stream includes multiple code snippets, which are associated with interface elements; the target model is a large language model selected from multiple parallel-running localized deployment models. A real-time rendering module is used to render the code data stream in real time within a pre-built isolated rendering environment; The iterative optimization module is used to perform iterative optimization on the complete code corresponding to the code data stream in response to the update instruction for the rendering result of the code data stream, and to archive the version of the iteratively optimized complete code.

Images