Methods, apparatus, electronic devices, storage media, and program products for generating front-end code from design drawings.

By dividing the design draft into regions and recognizing the design intent, and combining the design pattern rule base to generate multi-platform adapted page code, the terminal adaptation problem when converting the design draft into front-end code is solved, and the degree of automation and display effect are improved.

CN123114432APending Publication Date: 2026-11-17CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611109920.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-24
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies, when converting design diagrams into front-end code, lack an active understanding of the design intent and cannot adapt to the interaction logic and differentiated design specifications of different terminals, resulting in output results that do not conform to general design principles and user habits.

Method used

By dividing the page design diagram into regions, sub-layout diagrams are generated. The design intent is identified using a pre-built design pattern rule library. The page structure diagram adapted to each target device is generated in combination with the requirements of multiple target devices. Finally, multi-terminal adapted page code is output.

Benefits of technology

It enables proactive identification of design intent and adaptive generation of multi-device differentiated structures, reducing the workload of manual adaptation and adjustment, improving the automation level and code generation efficiency of converting design drafts into front-end code, and ensuring the display effect of pages on multiple devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114432A_ABST
    Figure CN123114432A_ABST
Patent Text Reader

Abstract

This disclosure relates to the field of artificial intelligence, and in particular to a method, apparatus, electronic device, storage medium, and program product for generating front-end code from design diagrams. The method includes: dividing a page design diagram to be processed into regions to obtain a target sub-layout diagram set; determining a design intent description corresponding to the page design diagram to be processed based on a pre-built design pattern rule base, the target sub-layout diagram set, and the page design diagram to be processed; generating a target page structure diagram adapted to each target device based on the target sub-layout diagram set, the page design diagram to be processed, multiple target devices, and the design intent description; and generating page code corresponding to each target device based on the target page structure diagram adapted to each target device and the design intent description. This disclosure can proactively identify design intent and use it to drive the adaptive generation of multi-terminal differentiated structures, abandoning the rudimentary solution of simply scaling the size to adapt to the terminal in the prior art, and is able to adapt to the interaction logic of different terminals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a method, apparatus, electronic device, storage medium, and program product for generating front-end code from design drawings. Background Technology

[0002] In existing technologies, the conversion of design drawings into front-end code is achieved by combining multimodal large language models with hierarchical detection algorithms, or by extracting element features from design drawings through machine learning to identify components. This effectively solves key problems such as loss of details and semantic understanding bias that exist in traditional conversion technologies.

[0003] While the above solutions can improve the accuracy of the design drawings, they lack an active understanding of the design intent of the drawings. Furthermore, they only use a uniform feature scaling method to adjust the interface size for multi-platform compatibility, which cannot actively adapt to the differences in interaction logic and differentiated design specifications of different terminals. They also lack the ability to adapt to multi-screen displays, and the final output results are likely to not conform to general design principles, and the visual effects are difficult to match user habits. Summary of the Invention

[0004] In view of at least one of the above-mentioned technical problems, this disclosure provides a method, apparatus, electronic device, storage medium and program product for generating front-end code from design drawings.

[0005] Firstly, this disclosure provides a method for generating front-end code from design drafts, including: Divide the page design draft to be processed into regions to obtain the target sub-layout atlas; Based on the pre-built design pattern rule library, target sub-layout atlas, and page design diagrams to be processed, the design intent description corresponding to the page design diagrams to be processed is determined. Based on the target sub-layout atlas, the page design diagram to be processed, multiple target devices and design intent descriptions, generate a target page structure diagram adapted to each target device. Based on the target page structure diagram and design intent description for each target device, generate the page code corresponding to each target device.

[0006] Furthermore, according to the method of the first aspect of this disclosure, the page design drawing to be processed is divided into regions to obtain a target sub-layout atlas, including: Based on the visual and spatial features of the page design drawing to be processed, the page design drawing to be processed is divided into regions to generate an initial sub-layout atlas. The initial sub-layout atlas is filtered for noise to generate an intermediate sub-layout atlas; Perform grid alignment on the intermediate sub-layout atlas to obtain the target sub-layout atlas.

[0007] Furthermore, according to the method of the first aspect of this disclosure, the design intent description corresponding to the page design diagram to be processed, determined based on a pre-built design pattern rule base, a target sub-layout graph, and a page design diagram to be processed, includes: The target sub-layout graph set and the page design graph to be processed are input into the neural network model. The design pattern rule base of is used as background knowledge. The neural network model outputs the multimodal features of each target sub-layout in the target sub-layout graph set and its corresponding component type probability. Among them, the multimodal features of are visual features, icon features, text features and layout features. Based on multimodal features, component type probabilities, and a design pattern rule base, the initial design intent of each target sub-layout in the target sub-layout graph is determined. Based on the initial design intent of each target sub-layout, the design pattern rule base, and the design diagram of the page to be processed, a design intent description corresponding to the design diagram of the page to be processed is generated.

[0008] Furthermore, according to the method of the first aspect of this disclosure, based on multimodal features, component type probabilities, and a design pattern rule base, the initial design intent of each target sub-layout in the target sub-layout graph set is determined, including: Based on visual features and component type probabilities, the component type of each target sub-layout is determined. Based on text features, determine the business priority of each target sub-layout; Based on icon features, a design pattern rule library, and the surrounding context of icon features, the semantic tags of icons for each target sub-layout are determined. Based on the layout characteristics of each target sub-layout, determine the spatial functional association of each target sub-layout; Based on the text features and component types, determine the business data area to which each target sub-layout belongs. The component type, business priority, icon semantic labels, spatial function associations, and business data areas of each target sub-layout are used as the initial design intent for each target sub-layout.

[0009] Furthermore, according to the method of the first aspect of this disclosure, based on the initial design intent of each target sub-layout, the design pattern rule base, and the page design diagram to be processed, a design intent description corresponding to the page design diagram to be processed is generated, including: Based on the initial design intent of each sub-layout, generate prompts corresponding to each sub-layout; The design diagram of the page to be processed and the prompt words corresponding to each sub-layout are input into the multimodal large model. With the pre-built design pattern rule library as background knowledge, the multimodal large model outputs a design intent description corresponding to the design diagram of the page to be processed.

[0010] Furthermore, according to the method of the first aspect of this disclosure, based on the target sub-layout atlas, the page design diagram to be processed, multiple target devices, and design intent descriptions, a target page structure diagram adapted to each target device is generated, including: The design intent description is transformed into a baseline page structure diagram corresponding to the page design diagram to be processed. The baseline page structure diagram is scaled based on the logical resolution of the target device and the standard resolution of the page design diagram to be processed. Determine whether the scaled baseline page structure diagram conforms to the interaction specifications of the target device; In response to non-compliant interaction specifications of target devices, the size of the scaled baseline page structure diagram is adjusted. Determine whether the adjusted baseline page structure diagram conforms to the interaction specifications of the target device; In response to non-compliant interaction specifications of target devices, the visual morphology transformation of the resized baseline page structure diagram is performed. Determine whether the baseline page structure diagram after visual transformation conforms to the interaction specifications of the target device; In response to the interaction specifications of non-compliant target devices, the layout of the baseline page structure diagram after visual transformation is reconstructed to generate a target page structure diagram adapted to each target device.

[0011] A second aspect of this disclosure provides an apparatus for generating front-end code from design drawings, comprising: The layout division module is used to divide the page design draft to be processed into regions to obtain the target sub-layout atlas; The intent recognition module is used to determine the design intent description corresponding to the page design diagram to be processed based on the pre-built design pattern rule library, the target sub-layout graph, and the page design diagram to be processed. The multi-device adaptation module is used to generate page code corresponding to each target device based on the target page structure diagram and design intent description for each target device. The multi-device code generation module generates page code corresponding to each target device based on the target page structure diagram and design intent description for each target device.

[0012] Thirdly, this disclosure provides an electronic device, including: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions, causing the electronic device to perform the method as described in any embodiment of the first aspect.

[0013] Fourthly, this disclosure provides a non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a processor, cause the processor to perform the method as described in any embodiment of the first aspect.

[0014] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any embodiment of the first aspect.

[0015] This disclosure provides a method, apparatus, electronic device, storage medium, and program product for generating front-end code from design drafts. The disclosure obtains a sub-layout atlas by partitioning the page design draft, identifies the design intent corresponding to each sub-layout by combining a pre-built design pattern rule library, and then generates page structure diagrams adapted to each target device by combining multi-target device design requirements. Finally, it outputs multi-device adapted page code. It can proactively identify design intent and use this to drive the adaptive generation of multi-device differentiated structures, abandoning the rudimentary solution of simply scaling the size to adapt to the terminal in existing technologies. It can adapt to differences in interaction logic and design specifications of different terminals, reduce the workload of manual adaptation and adjustment, improve the automation level and code generation efficiency of converting design drafts to front-end code, and ensure the display effect of pages on multiple devices.

[0016] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0017] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 A flowchart illustrating a method for generating front-end code from a design drawing, provided in an embodiment of this disclosure; Figure 2 A flowchart illustrating another method for generating front-end code from a design drawing provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating another method for generating front-end code from a design drawing provided in this embodiment of the disclosure; Figure 4 A flowchart illustrating another method for generating front-end code from a design drawing provided in this disclosure. Figure 5 A structural block diagram of an apparatus for generating front-end code from design drawings, provided in an embodiment of this disclosure; Figure 6 A hardware block diagram of an electronic device provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0020] Figure 1 This is a flowchart illustrating a method for generating front-end code from a design drawing, as provided in an embodiment of this disclosure. Figure 1 As shown, the method includes the following steps: Step 101: Divide the page design image to be processed into regions to obtain the target sub-layout image set.

[0021] In one embodiment of this disclosure, the page design diagram to be processed refers to the page rendering to which a webpage needs to be generated. The page design diagram to be processed is divided into multiple independent small blocks according to functional modules, and each independent small block is a sub-layout. After the division is completed, a target sub-layout diagram set containing multiple sub-layouts is obtained.

[0022] Step 102: Based on the pre-built design pattern rule library, target sub-layout atlas, and page design diagram to be processed, determine the target design intent of each target sub-layout.

[0023] In one embodiment of this disclosure, a design pattern rule base is constructed based on commonly used design rules in web page design. The method for constructing this design pattern rule base includes: Based on open-source component libraries and design material libraries, component types and their corresponding multi-dimensional characteristics and typical functions are obtained to create a component type library. Based on common design patterns in page design, the layout, interaction rules, and terminal device adaptation rules of each component in each common design pattern are determined to form page pattern constraint rules. Based on the icon library and component library, build an icon mapping library that includes icon semantics and its corresponding icon content; By integrating the component type library, page pattern constraint rules, and icon mapping library, a design pattern rule library is obtained.

[0024] Specifically, relying on open-source component libraries and design material libraries, various page components are extracted, filtered, and deduplicated in batches. Standardized classifications are then performed according to component functional attributes, dividing components into different major categories and sub-components. Component types include, but are not limited to, input components, interactive components, display components, and layout components. Typical classification examples: input components include text boxes, dropdown lists, radio buttons, and checkboxes; interactive components include buttons, text links, switches, tabs, and pagination controls. Furthermore, based on this, complete multi-dimensional feature attributes are labeled for each component in each category, and core typical functions are defined in a standardized manner. These multi-dimensional features are uniformly divided into four core dimensions: visual features (color, shape, size, built-in icons, font style, border style, shadow effects, etc.), layout features (relative position, fixed size, inner and outer spacing, alignment, arrangement rules, etc.), interactive features (click, hover, drag, scroll, form submission, state switching, and other triggering events and behavioral logic), and responsive features (layout adjustment rules and scaling logic adapted to different screen sizes and terminal devices). The typical functions of components are standardized and defined in conjunction with common business design scenarios. For example, text boxes are used to receive user-defined information input, buttons are used to trigger page jumps, form submissions, status changes, and other operations, and radio buttons / checkboxes are used to enable users to select one or more options. Finally, all component categories, multi-dimensional features, and typical functions are integrated to form a standardized and structured component type library.

[0025] Specifically, based on the universal and classic design patterns and high-frequency page layout paradigms of web and mobile page design, this paper deconstructs the component composition system of various mainstream page design patterns. For each standardized design pattern, the layout specifications, interaction operation rules, and multi-terminal adaptation rules of the combined components are clearly defined, and a unified and reusable page pattern constraint rule system is formed. Among them, the layout rules define the arrangement, combination structure, hierarchical relationship and space ratio of each component in the corresponding page design mode of the overall page (for example, the navigation design mode is divided into horizontal top navigation and vertical side navigation, which is composed of multiple navigation link components arranged in an orderly manner, and is mainly used to realize page route jump); the interaction rules define the standard interaction behavior, triggering logic and applicable business scenarios of each component in the design mode (for example, the form design mode supports interactive behaviors such as submission triggering field validation, abnormal error prompts, and resetting and clearing data, and the button component triggers the corresponding business operation after clicking); the terminal adaptation rules define the adaptive adjustment strategy of each component and the overall page layout under different device terminals and different screen sizes, such as enlarging the button touch area on mobile devices, automatically wrapping form items, collapsing the top navigation into a hamburger menu, and dynamically adapting page spacing and component size to the screen), ultimately forming a complete page mode constraint rule system.

[0026] Specifically, relying on the ICONFONT open-source icon library and the built-in icon collections of mainstream UI component libraries, the entire set of icon resources is collected, cleaned, classified, and organized. It supports matching single icons with multi-dimensional and multi-scenario semantic tags, combining icon form, industry scenarios, functional uses, and interaction states to label each icon with multiple adaptable semantics and standardized business interpretations. This accurately establishes a two-way mapping relationship between a single icon and multiple semantics and scenarios, clarifying the differentiated functional meanings, usage specifications, and visual interpretations of the same icon in different business scenarios, terminal pages, and component combinations. It clarifies the general semantics, scenario-specific semantics, and state semantics of icons, ultimately constructing a semantic icon mapping library that is multi-semantic adaptable, highly adaptable, standardized, and callable.

[0027] Specifically, the three core resource libraries—component type library, page pattern constraint rules, and icon mapping library—are structurally integrated, correlated, validated, unified in rules, and linked in resources. This process connects the logical relationships between component features, page layout and interaction rules, and icon semantic system, forming a comprehensive, standardized, and reusable design pattern rule library that covers component specifications, page paradigms, icon standards, adaptation rules, and interaction logic.

[0028] In one embodiment of this disclosure, a design pattern rule base is used as the judgment standard. Combined with the target sub-layout atlas, the design purpose and functional intent of each sub-layout are inferred and clarified by comparing the component type, icon semantics, layout structure, interaction form, etc., corresponding to each sub-layout. That is, the target design intent.

[0029] Step 103: Based on the target sub-layout atlas, different target devices, and the target design intent of each target sub-layout, generate a target page structure diagram adapted to each target device.

[0030] In one embodiment of this disclosure, a baseline page structure diagram matching the original page design diagram is first constructed based on the target sub-layout diagram and the target design intent of each target sub-layout. Then, considering the screen size, display constraints, and interaction characteristics of each terminal device, each target sub-layout in the baseline page structure diagram is preprocessed, such as scaling, size adjustment, visual shape adjustment, and component rearrangement. A separate set of exclusive target page structure diagrams is generated for each type of target device, and multi-terminal page design drafts adapted to various devices such as mobile phones, tablets, and PCs are output in batches.

[0031] For example, the above-mentioned page structure diagram is a structured data carrier that digitally describes the page's visual elements, layout relationships, and interactive semantics. It is used to uniformly carry the size, style, hierarchical relationship, and target design intent of each sub-layout of the page. The page structure diagram is divided into a baseline page structure diagram and a target page structure diagram. The baseline page structure diagram is converted from the original page design diagram to be processed, and the target page structure diagram is the structured data obtained after adapting and reconstructing based on the baseline page structure diagram and the constraints of the terminal device.

[0032] Step 104: Based on the target page structure diagram adapted to each target device and the target design intent of each corresponding target sub-layout, generate the page code corresponding to each target device.

[0033] In one embodiment of this disclosure, for the target page structure diagram corresponding to each target device, combined with the design intent of each sub-layout previously identified, according to the layout and interaction adaptation rules of different target devices, the subordinate hierarchy of each target sub-layout in the page is sorted from top to bottom and associated with the corresponding target design intent. All sub-layouts in the page are standardized code conversion, and page code adapted to various target devices such as mobile phones, tablets, and PCs is automatically generated.

[0034] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure obtains a sub-layout atlas by partitioning the page design diagram, identifies the design intent corresponding to each sub-layout by combining a pre-built design pattern rule library, generates a page structure diagram adapted to each target device by combining the design requirements of multiple target devices, and finally outputs multi-terminal adapted page code. It can actively identify the design intent and use it to drive the adaptive generation of multi-terminal differentiated structures, abandoning the rudimentary solution of simply scaling the size to adapt to the terminal in the prior art. It can adapt to the differences in interaction logic and design specifications of different terminals, reduce the workload of manual adaptation and adjustment, improve the automation level of converting design diagrams into front-end code, improve code generation efficiency, and ensure the display effect of multi-device pages.

[0035] Figure 2 This is a flowchart illustrating another method for generating front-end code from a design drawing, provided in an embodiment of this disclosure.

[0036] like Figure 2 As shown, the page design to be processed is divided into regions to obtain the target sub-layout atlas, including the following steps: Step 201: Based on the visual and spatial features of the page design drawing to be processed, divide the page design drawing to be processed into regions and generate an initial sub-layout atlas.

[0037] In one embodiment of this disclosure, visual features refer to the color differences between pixels in the page design image to be processed, and spatial features refer to the white space between functional modules in the page design image to be processed. For a gradient background scenario on a webpage, a color cluster diffusion algorithm is used to achieve color dimensional partitioning: An arbitrary base pixel p1=(R1,G1,B1) is selected within the page design image to be processed, and surrounding pixels pn=(Rn,Gn,Bn) are traversed outwards. The color difference between the two is calculated. If the color difference between adjacent pixels is less than a preset color difference threshold, the two pixels are grouped into the same region, thereby generating the first sub-layout image set. Then, dimensional partitioning is completed based on spatial intervals: the design image is converted into a pixel matrix, and the number of non-background color pixels is counted row by row and column by column to construct a projection histogram. Continuous low-lying intervals representing blank areas are identified in the projection histogram. When the width of the blank interval is greater than the average interval width of the page modules, a horizontal or vertical dividing line is generated at the corresponding position to cut the page design image to be processed, resulting in the second sub-layout image set. The dual partitioning results of the first and second sub-layout image sets are merged, and a unified boundary is established to generate the initial sub-layout image set. Among them, the color dimension partition and dimension partition of the page design draft to be processed can be processed in parallel.

[0038] Step 202: Filter noise from the initial sub-layout atlas to generate an intermediate sub-layout atlas.

[0039] In one embodiment of this disclosure, noise filtering includes merging and filtering excessively small text sub-layouts, eliminating and merging decorative dividing lines, identifying and aggregating similar business components, and canceling independent segmentation of embedded redundant sub-layouts. Specifically, merging and filtering excessively small text sub-layouts involves using OCR technology to preprocess features to distinguish between text and non-text areas, eliminating scattered pure text sub-layouts with an area smaller than a preset minimum component size, and merging adjacent contiguous small text sub-layouts. Eliminating and merging decorative dividing lines involves identifying line-like sub-layouts with extreme height-to-width imbalances (narrow height / wide short) that only serve a separating function, determining them as decorative dividing lines, and merging them with adjacent valid sub-layouts to eliminate redundant segmentation units. Aggregating and merging similar business components involves identifying sub-layouts based on the inherent visual and layout characteristics of standard business components such as tables, buttons, and selectors, and merging multiple sub-layouts scattered under the same business component into a single sub-layout. Taking tables as an example, features such as a multi-row, multi-column regular grid, uniform row and column sizes, and differentiated header styles are used to determine whether fragmented sub-layouts within the same grid partition are aggregated. Embedded redundant sub-layout elimination refers to canceling the independent division of an inner sub-layout when a sub-layout is completely nested inside a parent layout, and the remaining areas of the parent layout are of the same color cluster. This avoids meaningless splitting due to scenarios such as inner margins and large-sized buttons. Based on the above noise filtering processing, all sub-layouts in the initial sub-layout atlas are cleaned, merged, and optimized one by one, and various invalid fragmented sub-layouts are removed. After noise filtering is completed, an intermediate sub-layout atlas is output.

[0040] Step 203: Perform grid alignment processing on the intermediate sub-layout atlas to obtain the target sub-layout atlas.

[0041] In one embodiment of this disclosure, the web page interface design follows a grid layout specification. During the rendering process of the design drawing, slight pixel deviations can easily occur due to anti-aliasing effects, resulting in minor misalignments at the boundaries of each sub-layout. By performing grid snapping correction on the boundary coordinates of all sub-layouts in the intermediate sub-layout atlas, the top, bottom, left, and right boundaries of adjacent sub-layouts are uniformly snapped to the standard grid lines, eliminating pixel offset errors, completing the overall layout alignment, and finally obtaining a target sub-layout atlas with regular boundaries and a unified layout.

[0042] In summary, based on the technical solution provided in the embodiments of this disclosure, this disclosure combines a color cluster diffusion algorithm in the color dimension with a projection histogram segmentation mechanism in the spatial dimension. It accurately relies on the color differences and white space intervals of the page design to be processed to achieve intelligent division of the page functional areas, effectively solving the problem of poor adaptability of a single segmentation method to gradient backgrounds and complex module layout scenarios, and obtaining an initial sub-layout atlas of preliminary segmentation. Furthermore, through a multi-dimensional noise filtering strategy, invalid noise blocks generated during the segmentation of the page design to be processed are effectively removed, the problem of excessive layout segmentation is repaired, and the layout structure of business components is standardized, resulting in a noise-reduced intermediate sub-layout atlas. Finally, to address the pixel offset and boundary misalignment problems caused by anti-aliasing of the page design to be processed, grid snapping correction is used to achieve standardized grid alignment of all sub-layouts, ultimately outputting a target sub-layout atlas with regular boundaries, clear hierarchy, complete components, and standardized layout.

[0043] Figure 3 This is a flowchart illustrating another method for generating front-end code from a design drawing, provided in an embodiment of this disclosure.

[0044] like Figure 3 As shown, based on the pre-built design pattern rule base, the target sub-layout diagram, and the page design diagram to be processed, the design intent description corresponding to the page design diagram to be processed is determined, specifically including the following steps: Step 301: Input the target sub-layout graph set and the page design graph to be processed into the neural network model. Using the design pattern rule base as background knowledge, obtain the multimodal features of each target sub-layout in the target sub-layout graph set output by the neural network model and its corresponding component type probability. The multimodal features include visual features, icon features, text features and layout features.

[0045] In one embodiment of this disclosure, the target sub-layout atlas and the page design to be processed are... Figure 1The model uses machine learning neural network models such as Convolutional Neural Network (CNN) and Fully Convolutional Network (FCN) as inputs. Throughout the model inference process, it calls a pre-built design pattern rule base as prior background knowledge. The design pattern rule base stores the visual, interactive, layout, and icon matching standards of various components, providing a basis for the model to identify blocks and extract features. After the model traverses each independent sub-layout in the sub-layout atlas, it outputs two types of core data simultaneously. The first type is multimodal features covering four dimensions: visual features representing the shape, size, border style, and color distribution of the sub-layout; icon features matching icon semantic labels; text features such as text size, color, and content obtained through OCR parsing; and layout features recording the coordinate position of the sub-layout in the design diagram of the page to be processed. The second type is the component type probability vector corresponding to each sub-layout, which quantifies the probability score of the sub-layout belonging to various front-end components such as buttons, input boxes, tables, and text.

[0046] Step 302: Based on multimodal features, component type probabilities, and design pattern rule base, determine the initial design intent of each target sub-layout in the target sub-layout graph.

[0047] In one embodiment of this disclosure, the initial design intent of each target sub-layout in the target sub-layout graph is determined based on multimodal features, component type probabilities, and a design pattern rule base, including: Based on visual features and component type probabilities, the component type of each target sub-layout is determined; Based on text features, determine the business priority of each target sub-layout; Based on icon features, design pattern rule base, and the surrounding context of icon features, determine the icon semantic label for each target sub-layout; Based on the layout characteristics of each target sub-layout, determine the spatial functional associations of each target sub-layout; Based on text features and component types, determine the business data area to which each target sub-layout belongs; The component type, business priority, icon semantic labels, spatial function associations, and business data areas of each target sub-layout are used as the initial design intent for each target sub-layout.

[0048] Specifically, the visual features extracted from the sub-layouts are used as input to the neural network, and high-confidence component annotations are selected based on the component type probabilities output by the model. For each sub-layout, its visual features such as shape, size, color, and border are fed into neural network models such as CNN and FCN. The model calculates and outputs a component type probability vector based on the visual features. Each value in the vector represents the probability of the sub-layout being identified as a component of a certain type based on visual features. The probability of each component type is predicted by traversing all component types within the vector. If the probability value corresponding to a certain component type is greater than a preset threshold T, then that component type is marked as a high-confidence valid label, thus completing the component type determination for the sub-layout. The model references a design pattern rule base when predicting the probabilities of various components. The training process of the above neural network model is a mature solution already available in the prior art, and this disclosure does not limit it.

[0049] Specifically, in text visual feature analysis, the font size of title text in page design drafts is generally larger than that of body text, and this characteristic can be used to distinguish text levels. The font size S corresponding to each character is extracted using OCR. t If S t Greater than the set font size threshold S h1 The corresponding text is marked as a title. Secondly, high-contrast, high-saturation text colors are often used to highlight key business information. The color contrast difference between the text and the background area is calculated, and the pixel percentage of the highlighted accent color in all text colors within the block is statistically analyzed. Combined with the aforementioned text font size hierarchy results, the information weight of a single text segment is obtained. Finally, the weights of all text within the target sub-layout are aggregated for a comprehensive calculation. Based on the two text features of font size and color, the business priority corresponding to each target sub-layout is determined.

[0050] Specifically, the extracted sub-layout icon features are first matched with the icon semantic tags pre-stored in the design pattern rule base to obtain the basic semantic meaning of the icon. Since the same icon may have multiple meanings, it is also necessary to combine the contextual information such as the text, components, and spatial position of the surrounding sub-layouts to comprehensively deduce, correct, and determine the final accurate semantic tag of the icon. For example, for a magnifying glass icon, basic matching can only determine it as a magnifying icon; if there is an input box context adjacent to the icon, it is accurately labeled as a "search icon"; if the icon is attached to the lower right corner of an image block, it is labeled as an "image magnifying icon".

[0051] Specifically, based on the layout characteristics of each sub-layout, the relative positions and / or layout relationships between them are analyzed to infer their functional relationships. For example, buttons are usually located below forms, while navigation bars are located at the top of the page. Relationships can be inferred using the coordinates of one sub-layout with the coordinates of another. For example, a relative position threshold D can be set. min If the distance between two sub-layouts is less than this value, it is inferred that they have some kind of functional relationship. Using a spatial distribution model, combined with the coordinates of the sub-layouts and component types, the layout relationships between the sub-layouts are inferred. For example, the top area usually contains navigation or titles, and the left area contains menus, etc. The spatial functional relationships between the sub-layouts are inferred by combining their relative positions and / or layout relationships.

[0052] Specifically, by combining the text content, layout (i.e., position), and component type of the text features corresponding to each sub-layout, the business data areas of the page are identified and divided. For example, in an e-commerce website, the product area typically consists of multiple cards, each containing images, prices, and product descriptions. The relationship between text content and layout is analyzed through a model to automatically identify and label business data areas. For instance, product names, prices, and descriptions are usually organized within one area; text analysis and layout reasoning determine this area as the "product display area." This achieves the division of the business data areas to which each target sub-layout belongs.

[0053] Specifically, by integrating the above multi-dimensional analysis results, the initial design intent of each target sub-layout is generated: including the component type identified by the visual model, the business priority calculated by text features, the semantic labels of icons corrected by context, the functional association of the sub-layout space based on coordinate position reasoning, and the business data area divided according to text and layout features.

[0054] Step 303: Based on the initial design intent, design pattern rule base, and page design diagram to be processed for each target sub-layout, generate a design intent description corresponding to the page design diagram to be processed.

[0055] In one embodiment of this disclosure, based on the initial design intent of each target sub-layout, the design pattern rule base, and the design diagram of the page to be processed, a design intent description corresponding to the design diagram of the page to be processed is generated, including: Based on the initial design intent of each sub-layout, generate corresponding prompts for each sub-layout; Input the page design diagram to be processed and the prompts corresponding to each sub-layout into the multimodal large model. Using the pre-built design pattern rule library as background knowledge, the multimodal large model outputs a design intent description corresponding to the page design diagram to be processed.

[0056] Specifically, based on the initial design intent obtained from integrating each target sub-layout (including component type, business priority, icon semantic labels, spatial functional associations, and multi-dimensional information of business data areas), structured prompts are generated for each sub-layout to adapt to the large model's inference input. These prompts use a standardized structured format, fully carrying all visual, textual, spatial, and business-related feature information of the sub-layout, allowing the large model to perform subsequent page design pattern parsing, rule matching, and other inference operations. The original page design image to be parsed and the structured prompts for each sub-layout are fed into the multimodal large model. During inference, a pre-built design pattern rule base is injected as external background knowledge into the model, providing standardized component specifications, layout constraints, icon semantics, and business area determination criteria. The model integrates image visual information, multi-dimensional intent prompts for sub-layouts, and prior knowledge from the rule base for joint inference, ultimately outputting a complete design intent description corresponding to the entire page design image. This design intent description includes overall textual information of the page design image to be processed, encompassing the overall page positioning, functional layout of each area, functional priority and interaction logic of all components (i.e., sub-layouts), and multi-platform adaptation optimization strategies. For example... The specific design intent description includes the following: Overall description of the page: including page type, the function of each area and its layout.

[0057] Component Function Description: The functional priority and interaction logic of each component.

[0058] Prioritization and optimization strategies: Which components need to be highlighted, which can be collapsed, and possible optimization strategies.

[0059] Furthermore, based on the design intent description output in step 303, the design intent description is iteratively optimized to determine the optimized design intent description.

[0060] Specifically, based on the design intent description obtained in step 303, the initial design intent of all target sub-layouts is reverse-verified to complete iterative optimization. This process is divided into two layers: automated verification and manual correction. First, the initial design intent of each sub-layout is verified for compliance according to the design interaction specifications, filtering out logical contradictions and identifying erroneous annotations. For example, if a sub-layout is marked as a button, but the block size is large and contains multiple layers of text and image sub-layouts, which conflicts with the standard design specifications for buttons, then the annotation is determined to be an abnormal annotation. After automated verification, for erroneous annotation cases that are not identified and cannot be automatically corrected, a manual feedback mechanism is introduced to complete the correction. Based on the verification and correction results, the initial design intent of each sub-layout is updated, and the corrected sub-layout intent is integrated with the design intent description of the original page design drawing to be processed, ultimately forming a logically consistent and verified complete design intent description.

[0061] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure extracts the multimodal features of each target sub-layout through a neural network model and outputs the component type probability. Combined with a design pattern rule base, it generates the initial design intent of each target sub-layout from multiple dimensions such as component type, business priority, icon semantics, spatial association, and business area. Then, the initial intent of the target sub-layout is converted into structured prompt words and sent to a multimodal large model to generate a global design intent description covering the entire page, component interaction, and multi-terminal adaptation strategy. Finally, based on the design intent description, reverse automatic verification of annotation anomalies is performed and supplemented by manual error correction to obtain an accurate design intent description. The above method can solve the defects of traditional graph-generated code solutions that can only identify component appearance, cannot automatically mine design interaction and business intent, and are prone to annotation errors due to single-local recognition.

[0062] Figure 4 This is a flowchart illustrating another method for generating front-end code from a design drawing, provided in an embodiment of this disclosure.

[0063] like Figure 4 As shown, based on the target sub-layout atlas, the page design diagram to be processed, multiple target devices, and design intent descriptions, a target page structure diagram adapted to each target device is generated. This process includes the following steps: Step 401: Transform the design intent description into a baseline page structure diagram corresponding to the page design diagram to be processed.

[0064] In one embodiment of this disclosure, a hierarchical component tree is constructed based on the design intent description as a baseline page structure diagram; each node in the component tree corresponds to a component (i.e., a target sub-layout, such as a button, navigation bar, table, etc.), and each node is bound to the design attributes of the corresponding component, including color, size, interaction logic, etc.

[0065] For example, the above design intent description is a complete design intent description after iterative optimization.

[0066] Step 402: Scale the baseline page structure diagram based on the logical resolution of the target device and the standard resolution of the page design diagram to be processed.

[0067] In one embodiment of this disclosure, the logical resolution corresponding to the target device and the preset standard resolution of the page design drawing to be processed are obtained, the size scaling ratio between the two is calculated, and then the scaling adaptation processing of each target sub-layout in the reference page structure drawing is uniformly performed according to the scaling ratio, so that the size reference of each target sub-layout matches the display specifications of the target device.

[0068] Step 403: Determine whether the scaled baseline page structure diagram conforms to the interaction specifications of the target device.

[0069] In one embodiment of this disclosure, different terminal devices (i.e., target terminals) are equipped with independent human-computer interaction specifications. The target sub-layouts in the baseline page structure diagram that are only scaled proportionally are prone to non-compliance with the specifications. Therefore, it is necessary to verify each item in the scaled target sub-layout. Taking a touch screen mobile terminal as an example, the interactive buttons must meet the minimum touch size specification (minimum 44px). Traverse all interactive elements in the target sub-layout and verify whether their size meets the specification threshold.

[0070] Step 404: In response to the non-compliance with the interaction specifications of the target device, adjust the size of the scaled baseline page structure diagram.

[0071] In one embodiment of this disclosure, the size of sub-layouts that fail the verification is adjusted in different dimensions: if the size of interactive controls such as buttons is lower than the minimum touch threshold, the size of the controls is enlarged to the standard; considering the one-handed operation habits of mobile devices, if the spacing between components is less than the preset spacing threshold, the spacing between components is uniformly widened to the compliant range; for text-type target sub-layouts, excessively long content is adaptively processed according to the device's visible width, and text is prevented from being clipped by the page boundary by rendering automatic line wrapping or ellipses at the end.

[0072] Step 405: Determine whether the adjusted baseline page structure diagram meets the interaction specifications of the target device.

[0073] In one embodiment of this disclosure, the target sub-layout in the base page structure diagram after size adjustment is verified again to determine whether the target sub-layout after size adjustment meets the interaction specification requirements of the target device.

[0074] Step 406: In response to the non-compliance with the interaction specifications of the target device, perform visual transformation processing on the base page structure diagram after size adjustment.

[0075] In one embodiment of this disclosure, if the target sub-layout in the base page structure diagram after size adjustment still does not meet the interaction specification requirements of the target device, a visual morphological transformation process is performed on the target sub-layout after size adjustment, including: 1. Concise copywriting: For example, “User Account Settings” → “Account Settings”, using OCR text weighting (bold / color) to retain the core semantics; 2. Convert text buttons to icon buttons based on their function labels, such as Settings → Gear icon, Search → Magnifying glass icon.

[0076] Step 407: Determine whether the baseline page structure diagram after visual transformation meets the interaction specifications of the target device.

[0077] In one embodiment of this disclosure, the target sub-layout in the baseline page structure diagram after visual morphology transformation is verified again to determine whether the target sub-layout after visual morphology transformation conforms to the interaction specification requirements of the target device.

[0078] Step 408: In response to the interaction specifications that do not meet the requirements of the target device, the layout of the baseline page structure diagram after the visual transformation is reconstructed to generate a target page structure diagram adapted to different target devices.

[0079] In one embodiment of this disclosure, if the target sub-layout in the baseline page structure diagram after visual morphology transformation still does not meet the interaction specifications of the target device, the target sub-layout after visual morphology transformation is restructured to generate a target page structure diagram adapted for different target devices. The layout reconstruction includes the following steps: Step 1: Determine information priority: Mark the functional priority of components according to the target design intent corresponding to each target sub-layout; divide into high-priority core components and low-priority secondary components. High-priority core components must be continuously and explicitly displayed on the page and must not be collapsed or stored. For example, core operation controls such as submit buttons and save buttons, as well as key business information that needs to be displayed in real time, such as order status and user account balance.

[0080] Step 2: Strengthen core functions based on priority: Take the high-priority components marked in Step 1 as objects and retain the visual features such as bold text and exclusive highlight color. For the core components that have completed the visual transformation in the previous step, re-verify the correlation and matching degree between the components and the transformed form. If the text is ambiguous or the icon correlation is lower than the preset threshold, restore the original display form appropriately according to the component priority.

[0081] Step 3: Prioritize and Fold Secondary Functions: For the low-priority secondary information identified in Step 1, fold or hide it based on the available space on the terminal screen to simplify page content and prevent information overload. The detailed processing methods are as follows: Button Collapse: Low-priority operation buttons and text links such as "View More" and "Help Center" are uniformly stored in a collapse container or sidebar, and can be expanded to view by triggering a control; Hide at the bottom: Low-priority content such as help documents, copyright notices, and contact information are placed at the bottom of the page or carried by pop-ups or drawer menus; Area Collapse: By default, secondary business modules such as shopping cart details only display an overview. The complete details are stored in a collapsible panel, which can be expanded by clicking. Priority-based folding: Based on the weight levels marked in step 1, low-priority text modules such as FAQ and comment details are folded and stored uniformly.

[0082] Priority-based aggregation of similar layouts: Combining the information hierarchy distinction in step 1, aggregate components and layout modules with similar functions within the page to eliminate redundant elements and blank areas; dynamic aggregation in multi-platform scenarios can improve page cleanliness and operational efficiency. The aggregation strategies are detailed below: Component aggregation: Low-priority operation buttons such as edit, delete, and update are integrated into the same operation bar, and function reuse is achieved through drop-down pop-ups and state switching to avoid independent space occupation; high-priority operation buttons are displayed independently. Merging similar layouts: The two types of data display modules, tables and cards, are rendered in a unified manner to reduce the number of component types and compress the page space occupied. The layout of core business data is not merged or compressed. Duplicate content consolidation: Similar secondary business content such as customer information and order information is displayed using tabs or accordion-style collapsible panels to prevent duplicate page rendering; List card conversion: Batch data such as products and contacts can be converted and stacked on mobile devices to improve space utilization while keeping core product information fully displayed; Dynamic adaptive aggregation: Aggregation rules are dynamically adjusted based on screen size and information priority. On PC, high and low priority components are displayed independently, while on mobile, only low priority components are aggregated, and high priority components remain fully exposed.

[0083] Throughout all the layout adjustments and restructuring mentioned above, it is crucial to ensure the clarity of user interaction logic. Layout changes should not affect the user's cognitive path and operational habits, as detailed below: Clearly define the hierarchical structure: Maintain a clear hierarchy among functional modules. For example, always place the navigation bar at the top of the page, and always place important function buttons in a prominent position in the view, without changing their interactive functions due to layout adjustments.

[0084] Adapt to user habits: Based on the information marked in the design intent, layout adjustments should conform to the habits of end-device users. For example, mobile users are accustomed to placing the search box at the top of the screen, not at the bottom; the "menu button" commonly used on mobile devices should be placed at the top or upper left corner of the page.

[0085] In summary, based on the technical solutions provided in the embodiments of this disclosure, this disclosure achieves accurate adaptive adaptation of general standard resolution page designs to various terminal devices by adopting a multi-layered progressive compliance adaptation strategy that includes scaling, size fine-tuning, visual form transformation, and priority-driven layout reconstruction, coupled with a multi-round interaction specification verification mechanism. This solution effectively overcomes the limitations of existing technologies that rely solely on simple adaptation methods such as prompt word feature scaling, flexible box models, and basic adaptive layouts. It eliminates the cumbersome operation of requiring designers to output design drafts separately for each terminal. Relying on a multi-level adaptation strategy involving size adjustment, form transformation, and structural reconstruction, and constrained by terminal design principles, it can proactively complete intelligent multi-terminal adjustments to web pages. Under the premise of strictly adhering to the interaction specifications of each device, fully preserving core business functions and visual hierarchy, and without changing user operating habits and cognitive logic, it properly solves common problems in traditional adaptation solutions such as non-compliant control sizes, abnormal text display, redundant page layouts, and low screen space utilization. This significantly improves the display standardization, interactive usability, and visual cleanliness of multi-terminal pages, enabling efficient cross-terminal reuse of design resources and comprehensively optimizing the user browsing and operating experience.

[0086] The target page structure diagram generated after the multi-level progressive adaptation processing of steps 401 to 408 above contains embedded storage of a complete set of terminal adaptation information for the corresponding target device in each component node, including resolution scaling ratio, compliant size of interactive controls, text display rules, visual form transformation rules, layout folding and aggregation strategies based on information priority, etc. The terminal adaptation information is the inherent built-in data of the target page structure diagram.

[0087] Furthermore, based on the target page structure diagram and design intent description adapted to each target device, the page code corresponding to each target device is generated, including: Based on the target page structure diagram and design intent description adapted to each target device, the design intent description is associated with each node in the target page structure diagram to obtain the final target page structure diagram of each target device. Convert the final target page structure diagram into page code.

[0088] The target page structure diagram adapted for each target device is a nested component tree structure; each node in the component tree corresponds to a component (such as a button, text box, table, image, etc.), and the page hierarchy is expressed by the nesting relationship of child nodes. For example, the top navigation bar is the root node, the navigation menu item is its child node, and the sub-components under the menu item are grandchild nodes.

[0089] After the design intent description is associated with each component node, the annotation information carried by the node includes: 1. Component function description, such as "navigation button, used to jump to the product page"; 2. Interactive behavior information, such as click redirection onClick="redirect to product page", form submission onSubmit="call user registration interface"; Meanwhile, terminal interaction design constraints (such as "minimum button height requirement of 44px"), component width and height, color and other style information are natively stored in the adapted target page structure diagram. Therefore, the final target page structure diagram carries the above two types of information (such as the minimum touch height of the button of 44px).

[0090] After obtaining the final target page structure diagram, the number of components in the diagram is converted into code for the target technology stack (Vue, etc.). The annotated interaction logic is then converted into methods using programming languages ​​such as JavaScript and TypeScript. Code is recursively generated according to the page hierarchy, and the generated interaction logic is inserted sequentially.

[0091] Generate the HTML structure of components: Traverse each node in the component tree and generate the corresponding HTML structure. For each component, generate appropriate tags (e.g., buttons, forms, text boxes, navigation bars, etc.) based on its type. <button>, <input> , <nav>(etc.), and can also be adapted to target component library components (such as iview, etc.). Each component generates the necessary HTML attributes (such as onClick, onSubmit) according to the design intent annotation to ensure that the interactive functions are consistent with the design intent.

[0092] Add styles: Generate CSS styles based on style information.

[0093] Generate interaction logic: Generate JS or TS code to convert the interactive behaviors of each component (such as button clicks, form submissions, etc.) into corresponding methods.

[0094] Data acquisition logic is generated by mapping page data to backend interfaces based on the business data relationships in the large model inference, and then encapsulating the interface logic. This way, if the interface implementation deviates from expectations, adjustments only need to be made within the encapsulation, without modifying the core logic of the frontend page.

[0095] The code for each component is recursively generated based on the nesting relationship of the component tree, ultimately resulting in the page code.

[0096] Furthermore, the page code is optimized to obtain the optimized page code.

[0097] Specifically, the rendering effect of the generated page code is compared with the original design to further correct the visual fidelity. Automated testing is performed to verify whether button clicks trigger the intended functionality. Similar code is extracted into common components, business logic and view code are separated, and coding style corrections are made.

[0098] Compare the rendered results with the design: Render the generated page code to a browser or development environment and check the visual effect of the page to see how well it matches the original design. If discrepancies are found between the rendered results and the design, adjust CSS styles, layout structure, etc., to ensure visual fidelity.

[0099] Functional Validation and Interaction Testing: Use an AI-powered automated testing framework (such as mcp-playwright) to perform functional testing on the generated code, ensuring that all interactive logic (such as button clicks, form submissions, etc.) functions correctly as designed. For each interactive element such as buttons, forms, and links, simulate user actions to verify whether the correct functionality is triggered. For example, when the "Search" button is clicked, does it trigger the search method?

[0100] Common component extraction: Components that are used multiple times can be extracted into a common component library.

[0101] Maintain code clarity and modularity to improve readability and maintainability. Perform coding style checks on the generated code to ensure it conforms to the team's coding standards.

[0102] This disclosure also provides an apparatus for generating front-end code from design diagrams. Figure 5 A structural block diagram of an apparatus for generating front-end code from design drawings, as provided in this disclosure embodiment, is shown below. Figure 5 As shown, the device 500 for generating front-end code from the design drawing includes: a layout division module 501, an intent recognition module 502, a multi-terminal adaptation module 503, and a multi-terminal code generation module 504.

[0103] In one exemplary embodiment, the layout division module 501 is used to divide the page design drawing to be processed into regions to obtain the target sub-layout atlas.

[0104] In one exemplary embodiment, the intent recognition module 502 is used to determine the design intent description corresponding to the page design diagram to be processed based on a pre-built design pattern rule base, the target sub-layout diagram, and the page design diagram to be processed.

[0105] In one exemplary embodiment, the multi-device adaptation module 503 is used to generate a target page structure diagram adapted to each target device based on the target sub-layout diagram, the page design diagram to be processed, multiple target devices, and design intent description.

[0106] In one exemplary embodiment, the multi-device code generation module 504 is used to generate page code corresponding to each target device based on the target page structure diagram and design intent description adapted to each target device.

[0107] Figure 6 This is a hardware block diagram of an electronic device provided according to an embodiment of the present disclosure. The electronic device 600 according to an embodiment of the present disclosure includes at least a processor; and a memory for storing computer-readable instructions. When the computer-readable instructions are loaded and executed by the processor, the processor performs a method for generating front-end code from a design drawing as described in any of the preceding embodiments of the present disclosure.

[0108] Figure 6 The illustrated electronic device 600 specifically includes a central processing unit (CPU) 601, a graphics processing unit (GPU) 602, and a memory 603. These units are interconnected via a bus 604. The CPU 601 and / or GPU 602 can function as the aforementioned processor, and the memory 603 can function as the aforementioned memory storing computer-readable instructions. Furthermore, the electronic device 600 may also include a communication unit 605, a storage unit 606, an output unit 607, an input unit 608, and an external device 609, all of which are also connected to the bus 604.

[0109] Figure 7 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this disclosure. (As shown...) Figure 7 As shown, a computer-readable storage medium 700 according to an embodiment of the present disclosure stores computer-readable instructions 701 thereon. When the computer-readable instructions 701 are executed by a processor, a method for generating front-end code from a design drawing according to any embodiment of the present disclosure described above with reference to the accompanying drawings is performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0110] This disclosure further provides a computer program product, including a computer program that, when executed by a processor, implements the method for generating front-end code from design drawings as described in any of the preceding embodiments of this disclosure.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0112] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0113] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0114] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0115] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0116] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0117] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0118] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.< / nav> < / button>

Claims

1. A method for generating front-end code from design drafts, characterized in that, include: Divide the page design draft to be processed into regions to obtain the target sub-layout atlas; Based on the pre-built design pattern rule base, the target sub-layout diagram, and the page design diagram to be processed, the design intent description corresponding to the page design diagram to be processed is determined. Based on the target sub-layout atlas, the page design diagram to be processed, multiple target devices and design intent descriptions, a target page structure diagram adapted to each target device is generated. Based on the target page structure diagram and design intent description for each target device, generate the page code corresponding to each target device.

2. The method for generating front-end code from design drawings according to claim 1, characterized in that, The process of dividing the page design image to be processed into regions to obtain the target sub-layout image set includes: Based on the visual and spatial features of the page design drawing to be processed, the page design drawing to be processed is divided into regions to generate an initial sub-layout atlas; The initial sub-layout atlas is subjected to noise filtering to generate an intermediate sub-layout atlas; The intermediate sub-layout atlas is subjected to grid alignment processing to obtain the target sub-layout atlas.

3. The method for generating front-end code from design drawings according to claim 1, characterized in that, The process of determining the design intent description corresponding to the page design diagram to be processed, based on the pre-built design pattern rule base, the target sub-layout atlas, and the page design diagram to be processed, includes: The target sub-layout graph set and the page design graph to be processed are input into the neural network model. Using the design pattern rule base as background knowledge, the multimodal features of each target sub-layout in the target sub-layout graph set output by the neural network model and its corresponding component type probability are obtained. The multimodal features include visual features, icon features, text features and layout features. Based on the multimodal features, the component type probabilities, and the design pattern rule base, the initial design intent of each target sub-layout in the target sub-layout graph set is determined; Based on the initial design intent of each target sub-layout, the design pattern rule base, and the page design diagram to be processed, a design intent description corresponding to the page design diagram to be processed is generated.

4. The method for generating front-end code from design drawings according to claim 3, characterized in that, The step of determining the initial design intent of each target sub-layout in the target sub-layout graph set based on the multimodal features, the component type probabilities, and the design pattern rule base includes: Based on the visual features and the component type probability, the component type of each target sub-layout is determined; Based on the text features, determine the business priority of each target sub-layout; Based on the icon features, the design pattern rule base, and the surrounding context of the icon features, determine the icon semantic label for each target sub-layout; Based on the layout features of each target sub-layout, determine the spatial functional association of each target sub-layout; Based on the text features and the component type, determine the business data region to which each target sub-layout belongs; The component type, business priority, icon semantic label, spatial function association, and business data area of ​​each target sub-layout are taken as the initial design intent corresponding to each target sub-layout.

5. The method for generating front-end code from design drawings according to claim 4, characterized in that, The process of generating a design intent description corresponding to the page design diagram to be processed, based on the initial design intent of each target sub-layout, the design pattern rule base, and the page design diagram to be processed, includes: Based on the initial design intent of each sub-layout, generate prompt words corresponding to each sub-layout; The design diagram of the page to be processed and the prompt words corresponding to each of the sub-layouts are input into the multimodal large model. With the pre-built design pattern rule library as background knowledge, the design intent description corresponding to the design diagram of the page to be processed is output by the multimodal large model.

6. The method for generating front-end code from design drawings according to claim 5, characterized in that, The step of generating a target page structure diagram adapted to each target device based on the target sub-layout atlas, the page design diagram to be processed, multiple target devices, and design intent descriptions includes: The design intent description is transformed into a baseline page structure diagram corresponding to the page design diagram to be processed; The baseline page structure diagram is scaled based on the logical resolution of the target device and the standard resolution of the page design diagram to be processed. Determine whether the scaled baseline page structure diagram conforms to the interaction specifications of the target device; In response to non-compliance with the interaction specifications of the target device, the size of the scaled baseline page structure diagram is adjusted. Determine whether the adjusted baseline page structure diagram conforms to the interaction specifications of the target device; In response to non-compliance with the interaction specifications of the target device, the visual morphology transformation process is performed on the adjusted baseline page structure diagram. Determine whether the baseline page structure diagram after visual transformation conforms to the interaction specifications of the target device; In response to non-compliance with the interaction specifications of the target device, the layout of the baseline page structure diagram after visual transformation is reconstructed to generate a target page structure diagram adapted to each target device.

7. An apparatus for generating front-end code from design drawings, characterized in that, include: The layout division module is used to divide the page design draft to be processed into regions to obtain the target sub-layout atlas; The intent recognition module is used to determine the design intent description corresponding to the page design diagram to be processed based on the pre-built design pattern rule library, the target sub-layout diagram, and the page design diagram to be processed. The multi-device adaptation module is used to generate page code corresponding to each target device based on the target page structure diagram and design intent description for each target device. The multi-device code generation module generates page code corresponding to each target device based on the target page structure diagram and design intent description for each target device.

8. An electronic device, characterized in that, include: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the electronic device to perform the method of generating front-end code from a design drawing as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium for storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by a processor, the processor performs the method of generating front-end code from a design drawing as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The invention includes a computer program that, when executed by a processor, implements the method for generating front-end code from design drawings as described in any one of claims 1 to 6.