Method and device for generating front-end component library, storage medium and program product

Abstract

The present application relates to the technical field of computers, and more particularly to a method for generating a front-end component library, a computer device implementing the method, a computer readable storage medium and a computer program product. The method comprises: identifying and extracting a component region from a received design file; performing multi-modal feature extraction on the component region to obtain visual features, text features and structural features of the component; performing dynamic weighted fusion on the visual features, text features and structural features of the component to obtain a fused feature vector, and classifying the component based on the fused feature vector; performing style feature clustering and parameter fusion on components of the same type to generate a style template representing the overall style performance of the components of the type; and integrating the style template with a structural template of the components of the corresponding type, and incorporating the integrated style template into the front-end component library.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to a method for generating a front-end component library, a computer device for implementing the method, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In current enterprise-level front-end development practices, development based on standardized component libraries has become the mainstream choice for improving development efficiency and ensuring product consistency. These component libraries typically pre-define a unified set of interaction logic and visual styles to meet the high design standardization requirements of scenarios such as back-end management systems. However, with the deepening of digital transformation, especially in C-end (client-side) businesses targeting a broad consumer base, such as government consumer subsidy platforms and online marketing campaign pages, the demand for personalized and localized visual design is becoming increasingly prominent, and business iteration is extremely rapid. The inherent and relatively rigid style system of standardized component libraries is difficult to flexibly adapt to such diverse and ever-changing visual solutions, leading front-end developers to often need to invest a significant amount of work in deeply overriding or even refactoring component styles when faced with new design drafts.

[0003] In recent years, research has attempted to introduce artificial intelligence (AI) technology into front-end development processes to seek automation solutions. However, these existing solutions mostly focus on the overall generation of single pages or the functional identification of basic components, and their intelligent applications have not yet effectively extended to the deeper pain point of intelligent parsing and adaptation of component visual styles. Specifically, when dealing with the identification of complex components composed of multiple basic elements, existing methods often lack the ability to comprehensively analyze and integrate multi-dimensional features, easily leading to misjudgments. More importantly, for the numerous style variations derived from the same type of component in different business scenarios, existing technologies cannot automatically accumulate reusable style themes. This makes the generated component library difficult to manage and lacks the ability to continuously evolve, thus limiting its practical application value in complex and ever-changing consumer-facing products.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To address or at least mitigate one or more of the above problems, embodiments of this application provide a method for generating a front-end component library, a computer device implementing the method, a computer-readable storage medium, and a computer program product, which enables intelligent generation from design drafts to a reusable front-end component library, significantly improving the development efficiency and style adaptation capabilities of the component library.

[0006] According to a first aspect of this application, a method for generating a front-end component library is provided. The method includes the following steps: identifying and extracting component regions from a received design file; performing multimodal feature extraction on the component regions to obtain visual features, text features, and structural features of the components; dynamically weighting and fusing the visual features, text features, and structural features of the components to obtain a fused feature vector, and classifying the components based on the fused feature vector; performing style feature clustering and parameter fusion on components of the same type to generate a style template representing the overall style performance of components of that type; and integrating the style template with the structural template of the corresponding type of components, and incorporating the integrated style template into the front-end component library.

[0007] As an alternative or supplement to the above solutions, a method for generating a front-end component library according to an embodiment of this application further includes: identifying and extracting component regions from a received design file, including: parsing the design file using an object detection model to generate multiple component candidate boxes; calculating the confidence level of the component candidate boxes based on their existence probability and classification accuracy; and filtering out valid candidate boxes according to a preset confidence level threshold.

[0008] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the target detection model adopts the YOLOv8 architecture, is pre-trained using public datasets in the user interface domain, and then fine-tuned using business-specific user interface design graph datasets to enhance the model's ability to detect business-specific components, irregularly shaped components, and small-sized components.

[0009] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, the confidence level of the component candidate box is the product of the existence probability and the maximum classification accuracy, wherein the existence probability represents the probability that a valid component exists in the candidate box, and the maximum classification accuracy represents the maximum value among the probabilities of the candidate box belonging to each preset component type.

[0010] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, identifying and extracting component regions from the received design file further includes: extracting edge pixels of the effective candidate boxes using an edge detection algorithm to determine the component shape; selecting a corresponding contour fitting method according to the component shape to correct the positioning deviation of the effective candidate boxes; and updating the boundary coordinates of the effective candidate boxes based on the correction result.

[0011] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the corresponding contour fitting method is selected according to the component shape, including: for regular-shaped components, the minimum bounding rectangle algorithm is used to correct their bounding boxes; for irregular-shaped components, the edge curves of the bounding boxes are fitted using Bézier curves.

[0012] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, identifying and extracting component regions from the received design file further includes: cropping independent component region images according to the updated boundary coordinates, wherein, for nested components, independent local images of the parent component and the child component are extracted respectively.

[0013] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, identifying and extracting component regions from the received design file further includes: standardizing the component region image by proportional scaling and whitespace filling; and / or determining the structured data of the component region, wherein the structured data includes one or more of the following: component index, bounding box coordinates, preliminary prediction type, image path, confidence level, sub-component list, sub-component index, and coordinate offset of the sub-component relative to the parent component.

[0014] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, the extraction of visual features includes: extracting the basic visual features of the component, including: obtaining the dominant color value through K-means clustering algorithm, calculating the mean and variance of the color mode through color space conversion, and calculating the aspect ratio and contour complexity of the component; and extracting the higher-order visual features of the component, including: first performing feature extraction through a convolutional neural network model, and then obtaining a higher-order visual feature vector of a preset dimension through principal component analysis.

[0015] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, the extraction of text features includes: identifying and extracting text content from the component region using an optical character recognition algorithm; and inputting the extracted text content into a pre-trained language model for semantic encoding to generate a text semantic vector.

[0016] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, the extraction of structural features includes: extracting the positional features of the component, including its normalized coordinates in the design file, the deviation of the center point coordinates relative to the center of the screen, the functional area it belongs to based on its vertical position, and the screen ratio it has based on its horizontal position; extracting the hierarchical relationship features of the component, including the type of its parent component, its hierarchical depth in the component tree, the number and type composition of its child components, and the arrangement direction of its child components; and extracting the adjacency relationship features of the component, including the types of the preceding and succeeding components determined by Euclidean distance calculation, and the minimum nearest neighbor distance.

[0017] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, the visual features, text features, and structural features of the components are dynamically weighted and fused to obtain a fused feature vector. This includes: assigning basic weights to the visual features, text features, and structural features based on the preliminary predicted type of the components; performing weighted processing on the visual features, text features, and structural features, and concatenating the weighted feature vectors into a combined feature vector; and sequentially performing principal component analysis dimensionality reduction and norm normalization processing on the combined feature vector to obtain a fused feature vector for component classification.

[0018] As an alternative or supplement to the above solutions, in a method for generating a front-end component library according to an embodiment of this application, classifying the components based on the fused feature vector includes: inputting the fused feature vector into a pre-trained classification model to obtain the predicted probability that the component belongs to each predefined type; and taking the type with the highest predicted probability as the final classification result of the component.

[0019] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the multimodal feature extraction further includes the extraction of style features, wherein the style features include one or more of the following: basic color features, including background color, text color, and border color; border features, including border width, border style, and border rounded corners; shadow features, including shadow color, shadow offset, shadow blur radius, and shadow spread radius; layout spacing features, including inner margin, outer margin, and width and height dimensions; and text style features, including font size, font weight, text alignment, text spacing, line height, and font style.

[0020] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the style feature clustering is performed using a density-based clustering algorithm. The process includes: determining the style feature differences between any two components based on a weighted Euclidean distance algorithm; and performing density clustering on a set of components belonging to the same fine-grained type according to preset neighborhood radius and minimum sample number parameters, thereby grouping components with similar styles into the same cluster.

[0021] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, before performing style feature clustering, the method further includes a step of standardizing the style features. The standardization process includes: for numerical style features, normalizing them by dividing the feature value by the maximum possible value; for color style features, converting the color value to a hue, saturation, and brightness color space and normalizing it; and for discrete style features, converting the text description into a numerical vector using one-hot encoding.

[0022] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the rules for parameter fusion include: taking the median of numerical parameters as the fusion result; taking the average value of color parameters in the hue, saturation, brightness, and color space as the fusion result; and taking the value with the highest frequency of occurrence as the fusion result for discrete parameters.

[0023] As an alternative or supplement to the above solutions, in the method for generating a front-end component library according to an embodiment of this application, the integration of the style template with the structural template of the corresponding type of component is achieved through template engine technology. The integration process includes: injecting the style parameters after fusion in the style template into the corresponding placeholders of the predefined structural template of the corresponding type, thereby generating a front-end component code file with specific visual styles that can be reused on multiple platforms.

[0024] According to a second aspect of this application, a computer device is provided, comprising: a memory; a processor; and a computer program stored on the memory and executable on the processor, wherein execution of the computer program causes any one of the methods for generating a front-end component library according to a first aspect of this application to be performed.

[0025] According to a third aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including instructions that, when executed, perform any one of the methods for generating a front-end component library according to a first aspect of this application.

[0026] According to a fourth aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any one of the methods for generating a front-end component library according to a first aspect of this application.

[0027] The front-end component library generation scheme according to one or more embodiments of this application significantly improves the generation efficiency and intelligence level of the front-end component library by constructing a complete automated process. Its core effect is primarily reflected in the accuracy of component recognition: the scheme dynamically weights and fuses visual, textual, and structural features extracted from component regions, adaptively and comprehensively evaluating the characteristics of components across different dimensions. This dynamically weighted fusion strategy effectively overcomes the limitations of single feature information, significantly improving the accuracy and robustness of component type recognition, and laying a solid foundation for generating high-quality component libraries.

[0028] Building upon this foundation, the front-end component library generation scheme according to one or more embodiments of this application further achieves the systematic aggregation and standardization of component styles. By clustering and parameter fusion of style features of similar components, style templates representing their overall style expression can be automatically summarized and extracted. This process transforms the originally scattered and diverse component styles into unified, standardized, and reusable design resources. Finally, by integrating the style templates with the corresponding component structure templates, a component library that can be directly used in development is directly output. This process fundamentally upgrades the component library construction mode from traditional manual adaptation and repetitive coding to efficient intelligent generation, greatly reducing development costs and time. Attached Figure Description

[0029] The above and / or other aspects and advantages of this application will become clearer and more readily understood from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are denoted by the same reference numerals. In the drawings: Figure 1 This is a schematic flowchart of a method 10 for generating a front-end component library according to one or more embodiments of this application; Figure 2 A schematic flowchart illustrating a method 20 for generating a front-end component library according to one or more embodiments of this application; and Figure 3 This is a schematic block diagram of a computer device 30 according to one or more embodiments of this application. Detailed Implementation

[0030] The following detailed description is merely exemplary in nature and is not intended to limit the disclosed technology or its application and use. Furthermore, it is not intended to be bound by any express or implied theory presented in the foregoing technical fields, background art, or the following detailed description.

[0031] In the following detailed description of the embodiments, numerous specific details are set forth in order to provide a more thorough understanding of the disclosed technology. However, it will be apparent to those skilled in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0032] Terms such as "comprising" and "including" indicate that, in addition to the units and steps that are directly and explicitly described in the specification, the technical solution of this application does not exclude the presence of other units and steps that are not directly or explicitly described. Terms such as "first" and "second" do not indicate the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0033] Furthermore, it should be noted that the user information and data involved in one or more embodiments of this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] The front-end component library generation solution proposed in this application can be widely applied to scenarios requiring rapid development and iteration of user interfaces, especially in consumer-facing front-end product development. For example, when developing digital platforms that implement policies in various regions (such as consumer subsidy systems and trade-in activity pages), traditional standardized component libraries struggle to adapt to the diverse visual design needs and tight launch cycles of different provinces and cities. This solution can take various design drafts as input and, through automated component recognition, feature fusion, and style aggregation, quickly generate customized component libraries that match local design styles. This ensures a high degree of fidelity in the visual effect of the interface while significantly improving the responsiveness of front-end development and project delivery efficiency.

[0035] In the following, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings.

[0036] Please refer to the attached diagram below. Figure 1 This is a schematic flowchart of a method 10 for generating a front-end component library according to one or more embodiments of this application.

[0037] like Figure 1 As shown, in step 101, component regions are identified and extracted from the received design file.

[0038] This step aims to identify various interface elements in the design draft, such as basic components or complex nested components, by establishing a component extraction process, and transform them into standardized data that can be used for subsequent feature analysis. A reasonable component region extraction process can not only improve the efficiency of component extraction, but also lay a reliable data foundation for subsequent multimodal feature fusion and style aggregation. Step 101 may involve collaborative processing of multiple stages to ensure accurate identification and efficient extraction of component regions.

[0039] Optionally, step 101 includes a coarse detection stage. This stage first uses an object detection model to perform preliminary parsing of the design file, generating multiple component candidate boxes. For example, the object detection model can employ a neural network model based on the YOLOv8 architecture. This model can be pre-trained using publicly available datasets in the User Interface (UI) domain to learn feature representations of general UI components. Furthermore, the pre-trained model can be fine-tuned using a business-specific UI design graph dataset to enhance its detection capabilities for business-specific components, irregularly shaped components, and small-sized components, thereby improving the model's adaptability and recognition accuracy in specific business environments. During feature extraction, the UI design file (e.g., a design graph) can be input into the trained object detection model. Its backbone network extracts multi-scale features from the design file, and the detection head predicts each grid cell in the feature map, generating multiple component candidate boxes. For example, each candidate bounding box may include the following information: bounding box coordinates (the absolute coordinates of the candidate box in the design file, such as (x1, y1, x2, y2)), classification accuracy P(exist) (representing the probability that the candidate box belongs to a preset component type (e.g., a button)), and existence probability P(class) (representing the probability that a valid component exists within the candidate box, with a value ranging from 0 to 1). Based on this, confidence calculation and candidate box filtering can be further performed. Specifically, the confidence of each candidate box can be calculated based on the maximum value between the existence probability and the classification accuracy (i.e., the maximum classification accuracy), and the calculation formula is as follows: confidence = P(exist) × Max(P(class)) Next, candidate boxes with confidence levels below a preset confidence threshold (e.g., 0.6) can be eliminated, ultimately outputting a list of valid candidate boxes. This filtering mechanism helps eliminate interference, improving the accuracy and reliability of subsequent component region extraction, thereby providing high-quality input data for multimodal feature fusion and style aggregation.

[0040] Optionally, step 101 includes an edge and contour optimization stage. In this stage, the valid candidate boxes output from the coarse detection stage can be further processed to improve the accuracy of component boundary localization. Specifically, firstly, an edge detection algorithm (e.g., the Canny operator) can be used to extract the edge pixels of the image region corresponding to each candidate box, obtaining the true edge features of the component and thus determining its morphological features. Subsequently, a contour fitting method can be adaptively selected based on the component's morphological features to correct potential localization errors from the coarse detection stage. For example, for regularly shaped components, the minimum bounding rectangle algorithm can be used for geometric correction of their bounding boxes; for irregularly shaped components, Bézier curves can be used to fit their edge contours. Next, the boundary coordinates of the candidate boxes can be updated based on the corrected contour fitting results. Furthermore, for composite components, the offsets of their child components relative to their parent components in the x-axis and y-axis directions can be calculated. Finally, an optimized bounding box list containing accurate boundary coordinates and hierarchical offset relationships can be output. This optimization process effectively overcomes the boundary deviation problem that may exist in the coarse detection stage, laying the foundation for accurate component region extraction in the subsequent stages.

[0041] Optionally, step 101 includes a region extraction and standardization stage. In this stage, the component bounding boxes, after edge and contour optimization, undergo final processing to generate standardized component data with uniform specifications and complete information, providing structured input for subsequent feature extraction and style aggregation. This stage first involves cropping the identified independent component regions based on the corrected boundary coordinates. For example, during cropping, a preset width of redundant edge regions (e.g., 2 to 3 pixels) can be retained to prevent the loss of edge visual features such as component shadows and outer glows. For nested components (i.e., composite components), the relationship between parent and child components can be recorded, and their independent local images can be extracted separately, ensuring the integrity of the component hierarchy is preserved. Subsequently, all extracted local component images can be standardized by proportional scaling combined with whitespace filling, uniformly converting them to a preset size (e.g., 224×224 pixels) image format (e.g., PNG format) to eliminate differences in size and proportion between the original images, ensuring consistency of input data in subsequent processing flows. Ultimately, the output consists of structured data containing one or more of the following key information: component index, bounding box coordinates (i.e., the coordinates of the component's bounding box in the original design drawing), preliminary prediction type based on coarse detection, image path of the standardized component image, confidence score of the component detection, and a list of sub-components for the composite component. The sub-component list may further include the sub-component index and its coordinate offset relative to the parent component. This structured data not only fully preserves the visual information of the components and their spatial location and hierarchical relationship in the original design, but also provides a standardized and traceable data foundation for subsequent multimodal feature analysis and clustering analysis. This stage of processing effectively improves the standardization and consistency of the component data format, ensuring the automation and reliability of the entire component library generation process.

[0042] In step 103, multimodal feature extraction is performed on the component region to obtain the component's visual features, text features, and structural features.

[0043] Multimodal feature extraction refers to the parallel extraction of different types and dimensions of feature information from the source data of the same component, and the construction of a more comprehensive and accurate digital representation of the component by comprehensively analyzing these complementary information. Multimodal feature extraction (including at least visual, textual, and structural features) effectively overcomes the shortcomings of isolated or limited representational capabilities of single feature information. Visual features ensure a detailed depiction of the component's appearance, textual features provide a semantic understanding of its functional intent, and structural features reflect the component's contextual relationships and interaction logic within its layout. This synergistic effect of multi-dimensional features significantly improves the accuracy and robustness of subsequent component type identification, especially for diverse and complex C-end interface components. It effectively distinguishes visually similar but functionally different, or functionally similar but with different layouts, laying a solid feature foundation for building a highly usable component library.

[0044] In one or more embodiments of this application, the extraction of visual features includes extracting basic visual features and high-order visual features of the component respectively. The basic visual features include color features and morphological features: the dominant RGB value of the component can be determined using a K-means clustering algorithm; after converting the image from the RGB color space to the HSL color space, the mean and variance of hue (H), saturation (S), and brightness (L) are calculated to describe the color distribution characteristics; the aspect ratio (original component width / original component height) of the component is calculated to characterize its basic shape; and the contour complexity can be quantified by calculating the ratio of edge pixels to the total number of pixels. Furthermore, a pre-trained convolutional neural network model (such as ResNet-18) can be used to extract the high-order visual features of the component, and then principal component analysis (PAC) technology can be used to reduce the dimensionality of this high-dimensional feature vector to a preset dimension (such as 128 dimensions), thereby forming a comprehensive visual feature dictionary that integrates basic attributes and high-level semantic information.

[0045] In one or more embodiments of this application, during the text feature extraction process, optical character recognition (OCR) technology can be used to identify and extract all text content from the component image. Subsequently, the extracted raw text can be input into a pre-trained language model (such as BERT) for deep semantic encoding. This process transforms the text content into a high-dimensional semantic vector (e.g., 768-dimensional), which effectively captures the contextual relationships and functional intent between words, thereby elevating the text information in the component from surface characters to a feature representation with rich semantics, providing a key basis for distinguishing functionally similar components (such as a "Submit" button and a "Cancel" button).

[0046] In one or more embodiments of this application, during the structural feature extraction process, the spatial and topological attributes of a component in the overall user interface can be extracted based on the component's coordinate information and hierarchical relationship data in the original design drawing. Specifically, the structural features of a component may include positional features, hierarchical relationship features, and adjacency relationship features.

[0047] Optionally, the extraction of position features may include: normalizing the absolute coordinates of the component bounding box to the interface size to obtain its absolute position in the design file (normalized coordinates); calculating the deviation of its center point coordinates from the center of the UI screen; determining the functional area it belongs to based on its vertical position (such as the top navigation area, the middle content area, and the bottom operation area); and calculating the proportion of its horizontal width to the entire interface width (screen ratio).

[0048] Optionally, the extraction of hierarchical relationship features relies on the list of sub-components for the composite component recorded in the structured data. Specifically, the hierarchical relationship features of the component may include: the type of the parent component, the hierarchical depth of the component in the component tree, the number and type composition of the sub-components, and the arrangement direction of the sub-components determined by calculating the variance of the coordinates of the center points of the sub-components. For example, the arrangement direction of the sub-components can be determined as follows: when the variance of the ordinate of the center point of the sub-component is less than a preset first threshold, it is determined to be horizontally arranged; when the variance of the abscissa is less than a preset second threshold, it is determined to be vertically arranged; when the variance of the ordinate is not less than the first threshold and the variance of the abscissa is not less than the second threshold, it is determined to be grid arrangement. This statistical feature-based judgment method can effectively identify the internal layout structure of the composite component.

[0049] Optionally, the extraction of adjacency features can be achieved by calculating the spatial distance between components. In one embodiment, the Euclidean distance between the center points of the current component and other components can be calculated to determine the preceding (e.g., above, left) and succeeding (e.g., below, right) adjacent components with the smallest distance, and their component types and minimum proximity distance values ​​can be recorded to depict the contextual association of components in the layout. For example, the Euclidean distance d between the center points of the current component and other components can be calculated as follows: Where cx_A and cy_A represent the x and y coordinates of the center point of the current component, respectively, and cx_B and cy_B represent the x and y coordinates of the center points of other candidate components.

[0050] In one or more embodiments of this application, the multimodal features extracted in step 103 may include component style features in addition to visual, textual, and structural features. Style features aim to comprehensively characterize the visual representation of the component and serve as the direct basis for subsequent style clustering to generate reusable style templates. Optionally, style feature extraction can be performed on component images after region extraction and standardization. For example, the extracted style features may cover one or more of the following visual attributes: 1) Basic color features, including the component's background color, the text color contained within the component, and the component's border color; these color information collectively constitute the component's basic visual tone; 2) Border features, used to describe the visual attributes of the component's border, which may include the border's line width, border style (such as solid line, dashed line, dotted line, etc.), and the radius of the border's vertices (border rounded corners); 3) Shadow features, i.e., the shadow effect parameters that the component may have, including the shadow's color (shadow color), the shadow's offset relative to the component's main body (shadow offset), and the shadow modulus. The blur radius and shadow expansion radius together define the three-dimensionality and layering of the component; 4) Layout spacing features, used to characterize the spatial occupation and internal layout of the component, including the inner margin between the component content and the border, the outer margin between the component and other external elements, and the width and height of the component itself; 5) Text style features, including the font size, font weight (such as normal, bold), text alignment (such as left alignment, center alignment, right alignment), character spacing, line height between text lines, and specific font styles (such as regular, italic, etc.) of the text elements contained in the component. These features ensure the consistency of the visual style of the text part.

[0051] In step 105, the visual features, text features, and structural features of the component are dynamically weighted and fused to obtain a fused feature vector, and the component is classified based on the fused feature vector.

[0052] This step is the core of achieving accurate component identification. Its purpose is to effectively integrate heterogeneous features from different modalities into a unified, highly discriminative feature representation through an adaptive feature fusion strategy, thereby overcoming the limitations of a single feature perspective. This classification mechanism based on multimodal dynamic weighted fusion significantly improves the accuracy and robustness of component type identification in complex scenarios.

[0053] In one or more embodiments of this application, before weighted fusion, the feature vectors can be standardized. Specifically, the data types of the visual features (e.g., 133-dimensional visual feature vectors), text features (e.g., 768-dimensional text semantic vectors), and structural features (e.g., 13-dimensional structural feature vectors) extracted in step 103 can be standardized and converted to ensure that the storage format of each vector in memory is consistent, providing a basis for subsequent vector operations. For example, the word embedding value in the text semantic vector can be converted from 16-bit precision to 32-bit precision to maintain consistency with the numerical precision of the visual feature vectors.

[0054] The dynamic weight allocation stage then begins, employing a hierarchical weighting strategy. First, basic weights are assigned to different modalities based on the initial predicted type of the component. For example, for functional components such as buttons and input boxes, given that their text labels typically carry clear functional semantics, text features are given a higher weight (e.g., 0.4), while visual and structural features are relatively balanced (e.g., 0.3 each). For form-oriented components such as icons and cards, their visual appearance is crucial, so the weight of visual features is correspondingly increased (e.g., visual and text features are both 0.4, and structural features are 0.2). For components emphasizing overall layout, such as navigation bars and grids, structural features are given the highest weight (e.g., structural features 0.5, visual features 0.3, and text features 0.2) to better capture their spatial arrangement characteristics. Furthermore, within each modality, secondary weight allocation can be performed based on the importance of sub-features. For example, in visual features, the weights of high-order visual features extracted by convolutional neural network models (such as a weight of 0.7 for complex patterns representing texture and shape) can be higher than those of basic visual features calculated by traditional algorithms (such as a weight of 0.3 for dominant color and aspect ratio); textual features are generally represented by semantic vectors; in structural features, the weights of hierarchical relationships and adjacency relationships (each 0.4) are greater than the weight of positional features (0.2). This fine-grained dynamic weighting mechanism ensures that the fusion process can adaptively highlight the feature information that is most discriminative for classifying the current component.

[0055] After weighting, the weighted modal features can be concatenated into a combined feature vector (e.g., 133 + 768 + 13 = 914 dimensions) in a certain order (e.g., visual-textual-structural). To avoid the curse of dimensionality and retain key discriminative information, the vector can be further reduced in dimensionality using the PCA algorithm (e.g., to 256 dimensions), for example, by retaining the feature directions with the highest variance contribution rate through eigenvalue decomposition. Subsequently, the dimensionality-reduced vector can be normalized, such as by L2 normalization, to ensure that the modulus is 1. For example, suppose the dimensionality-reduced vector is f = [f1, f2, ..., f n ], calculate its modulus ‖f‖ as: The final normalized vector f_norm is as follows: Finally, we move to the precise component classification stage. In this stage, a hierarchical classification system combined with a deep learning model can be used to determine the final component type. For example, a tree-structured two-level classification system can be constructed first, where the first-level type represents the basic functional category of the component, and the second-level type further describes the specific manifestation under that functional category. Taking the most common interactive elements in graphical user interfaces as an example: "Button" as a first-level type can be further subdivided into "Base Button," "Text Button," and "Icon Button"; similarly, "Input" as a first-level type can be subdivided into "Text Input" and "Select Input." This fine-grained classification hierarchy can accurately distinguish component variants that are visually similar but have subtle differences in interaction logic or usage scenarios, laying the foundation for the subsequent generation of a highly usable component library.

[0056] In terms of model architecture selection, MobileNet can be used as the backbone network for feature extraction, followed by fully connected layers to form a complete classification model. MobileNet is a lightweight neural network based on deep separable convolutions, which significantly reduces the number of parameters while maintaining high feature extraction capabilities, making it very suitable for the need for efficient classification of a large number of design components in this scenario. Specifically, the feature vectors obtained after multimodal fusion and dimensionality reduction are input into the pre-trained classification model. This model first performs further nonlinear transformation and feature abstraction on the features through the MobileNet network, then maps the high-order features to the dimensional space of predefined classification labels through fully connected layers, and finally outputs the probability distribution of the component belonging to each secondary type through the Softmax activation function. The system can apply the maximum likelihood criterion to select the type with the highest probability value as the final classification result for the component. This deep learning-based classification method can automatically learn the complex mapping relationship between multimodal features and component types. In particular, it can effectively distinguish components that are similar in a single visual feature but are distinguishable when combined with text semantics or layout context (such as a button that only contains the text "search" and a button that contains both an icon and the text "search"), thereby significantly improving the accuracy and robustness of component recognition in complex UI interfaces.

[0057] In step 107, style feature clustering and parameter fusion are performed on components of the same type to generate style templates that represent the overall style performance of components of that type.

[0058] This step aims to solve the long-standing problem of style fragmentation in front-end development through intelligent algorithms. Specifically, in traditional development models, components of the same type (such as buttons) created by different designers or by the same designer at different times often exhibit numerous subtle or significant style differences (such as corner radius, primary color value, shadow effects, etc.), leading to difficulties in component library maintenance and inconsistent visual styles. Step 107 aims to automatically identify and classify these style variations through clustering algorithms, and then extract the core style features of each variation through parameter fusion, ultimately generating a limited number of highly representative standard style templates. This not only achieves the standardized accumulation of design assets but also endows the component library with an "intelligent adaptive" capability—it can automatically absorb and integrate new design styles, rather than relying on manual creation of new component variations to meet design needs. This greatly improves the maintainability and scalability of the component library while maintaining visual diversity, providing a sustainable style management solution for long-term iterations of large projects.

[0059] In one or more embodiments according to this application, the input style features can first be preprocessed by standardization, which is the basis for ensuring that the subsequent clustering algorithm processes each feature fairly. Since different style features have different dimensions and value ranges, direct calculation will lead to some features dominating the clustering results. Therefore, different standardization strategies can be adopted for the three main feature types: numerical style features (such as rounded corners and inner margins) can be normalized by dividing by their theoretical maximum value, such as linearly mapping the rounded corner value from the 0-20 pixel range to the 0-1 interval; color style features can be converted from the RGB color space to the HSL space, where the hue H is cyclically normalized from 0-360° to the 0-1 interval, and the saturation S and brightness L maintain their original 0-1 value range; discrete style features (such as border style and text alignment) can be converted into numerical vectors through one-hot encoding technology.

[0060] In one or more embodiments according to this application, the standardized data can be further weighted. Specifically, a dynamic weighting strategy based on component type can be adopted. Unlike fixed weight schemes, this method fully considers the relative importance of different style features to the visual performance of various components. Taking button components as an example, their basic color feature is given a high weight of 0.3, the weight of border features and layout spacing features is 0.2, and the weight of shadow features and text style features is 0.15, because the background color and text color of the button are its most prominent visual identifiers; while in input box components, the weight of border features can be increased to 0.3, reflecting the importance of border style for input box recognition. This differentiated weighting configuration ensures that the clustering process can focus on the visual features that are most distinguishable for a specific component type, thereby producing clustering results that are more in line with design intuition.

[0061] In the specific style feature clustering process, density-based clustering algorithms can be used to automatically group similar component styles. The following section will use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm as an example to introduce the clustering process. It should be noted that in other optional embodiments, other clustering algorithms such as K-means, hierarchical clustering, or Gaussian Mixture Model can also be selected according to specific needs.

[0062] Specifically, the density-based DBSCAN algorithm's core advantage lies in its ability to automatically identify clusters of arbitrary shapes without requiring a pre-defined number of clusters, and to effectively detect and filter noise points (i.e., style outliers). The algorithm's execution can be summarized in the following key steps: determining the style feature differences between any two components based on a weighted Euclidean distance algorithm; and performing density clustering on a set of components belonging to the same fine-grained type based on pre-defined neighborhood radius and minimum sample number parameters, thus grouping components with similar styles into the same cluster. Specifically, two important parameters need to be set first—neighborhood radius ε and minimum sample number minPts. Taking button component style clustering as an example, ε can be set to 0.15 in the weighted Euclidean distance space, and minPts can be set to 3. The algorithm traverses all sample points (i.e., the style feature vectors of each button). For the current point, it calculates its weighted Euclidean distance to all other points in the dataset (according to the weights assigned above), finding all points falling within its ε-neighborhood. If the number of points in the neighborhood reaches or exceeds minPts, then the point is marked as a core point and a new cluster is created; otherwise, it is temporarily marked as a boundary point or noise point. Then, starting from the core point, all points that are density-reachable (i.e., points connected by a series of core points) are recursively grouped into the same cluster. This process is repeated until all points have been visited. For example, when clustering the styles of 50 "basic buttons," the algorithm might find that most buttons are concentrated in three high-density regions—one group has a blue main color and 4px rounded corners (cluster A), one group has a green main color and 8px rounded corners (cluster B), and another group has a minimalist style with no shadows and right angles (cluster C). At the same time, there may be individual buttons with unique styles, such as a button with rounded corners as high as 20px. Because its style is more than ε away from the core point of any cluster and it lacks enough similar samples around it, it is correctly identified as a noise point and excluded. Ultimately, DBSCAN will automatically output three valid pattern groups: Cluster A, Cluster B, and Cluster C, without requiring manual specification of clustering into three categories.

[0063] After successful style clustering, the parameter fusion stage begins, where the scattered style features within each cluster are transformed into style templates representing the overall style of components of that type. During fusion, different fusion strategies can be set for different parameter types. For example, numerical parameters (such as rounded corners and margins) can use the median as the fusion rule, which effectively resists the interference of extreme values. For instance, if the rounded corner values ​​of 9 buttons in a cluster are [4, 4, 4, 4, 4, 4, 4, 20], using the median of 4 better represents the mainstream style of that cluster, rather than the arithmetic mean of 5.3, which is easily affected by the extreme value of 20. Color parameters can be fused in the HSL color space. When calculating the mean, a cyclic mean algorithm can be used for hue H. For example, when calculating the mean of red (H=0°) and magenta (H=300°), a simple arithmetic mean is 150° (cyan), while the cyclic mean correctly calculates 330° (also magenta), which conforms to the cyclic characteristics of the color wheel. Discrete parameters (such as border style and alignment) can be represented by the mode, which is the value that appears most frequently within a cluster as the fusion result. For example, if 7 out of 10 buttons in a cluster have "solid lines", 2 have "dashed lines", and 1 has "dotted lines", then the fusion result will be "solid lines".

[0064] In step 109, the style template is integrated with the structural template of the corresponding type of component, and the integrated style template is included in the front-end component library.

[0065] This step aims to automate the transformation of design systems into code assets. Through this process, abstract style parameters obtained through clustering and fusion are combined with the structural definitions of specific component types to generate component code that can be directly used in development. This automated integration mechanism not only significantly improves the efficiency of component library construction but, more importantly, ensures the accurate implementation of design specifications, avoiding deviations that may occur in traditional development processes due to manually writing styles.

[0066] In one or more embodiments of this application, the integration of style templates with the structural templates of corresponding component types is achieved through template engine technology. The specific integration process may include: injecting the merged style parameters from the style template into the corresponding placeholders of a predefined structural template of the corresponding type, thereby generating a front-end component code file with specific visual styles that is reusable across multiple platforms. Specifically, corresponding structural templates can be predefined for each component type (such as buttons, input boxes, etc.). These templates contain the HTML structure, basic interaction logic, and style placeholders for that type of component. The template engine parses the parameter set in the style template and maps it to the corresponding placeholders in the structural template, completing the style injection and code generation. This template engine-based integration scheme not only ensures the standardization and consistency of component code but also provides a technical foundation for dynamic updates and version management of the component library. When design specifications change, only the style template parameters need to be updated and the template rendering process re-executed to quickly generate a new version of the component library, significantly reducing the maintenance cost of the component library. The following is for reference. Figure 2 , Figure 2 This is a schematic flowchart of a method 20 for generating a front-end component library according to one or more embodiments of this application.

[0067] exist Figure 2 In this process, steps 201-203 are the coarse detection stage, steps 205-209 are the edge and contour optimization stage, steps 211-215 are the region extraction and standardization stage, step 217 is the multimodal feature extraction stage, steps 219-223 are the feature fusion and classification stage, steps 225-229 are the style aggregation and parameter fusion stage, and step 231 is the component library generation stage.

[0068] The coarse detection stage includes: in step 201, parsing the received design file using the target detection model to generate multiple component candidate boxes; and in step 203, calculating the confidence level of the component candidate boxes based on their existence probability and classification accuracy, and filtering out valid candidate boxes according to a preset confidence level threshold.

[0069] The edge and contour optimization stage includes: in step 205, using an edge detection algorithm to extract edge pixels of valid candidate boxes to determine the component shape; in step 207, selecting the corresponding contour fitting method according to the component shape to correct the positioning deviation of the valid candidate boxes; and in step 209, updating the boundary coordinates of the valid candidate boxes based on the correction results.

[0070] The region extraction and standardization stage includes: in step 211, cropping individual component region images based on the updated boundary coordinates; in step 213, standardizing the component region images by proportional scaling and whitespace padding; and in step 215, determining the structured data of the component regions. For example, the structured data includes one or more of the following: component index, bounding box coordinates, preliminary prediction type, image path, confidence score, list of sub-components, sub-component index, and coordinate offset of the sub-component relative to the parent component.

[0071] The multimodal feature extraction stage includes: in step 217, multimodal feature extraction is performed on the component region to obtain the component's visual features, text features, structural features, and style features.

[0072] The feature fusion and classification stage includes: in step 219, the visual features, text features and structural features are weighted and the weighted feature vectors are concatenated into a combined feature vector; in step 221, the combined feature vector is subjected to principal component analysis for dimensionality reduction and norm normalization in sequence; in step 223, the fused feature vector is input into the pre-trained classification model to obtain the final classification result of the component.

[0073] The style aggregation and parameter fusion stage includes: in step 225, the style features are standardized; in step 227, the style features of the same type of components are clustered based on DBSCAN; in step 229, the parameters of each cluster are fused to obtain a style template that represents the overall style performance of the component of that type.

[0074] The component library generation stage includes: in step 231, integrating the style template with the structural template of the corresponding type of component, and incorporating the integrated style template into the front-end component library.

[0075] Figure 3 This is a schematic block diagram of a computer device 30 according to one or more embodiments of this application. The computer device 30 includes a memory 310, a processor 320, and a computer program 330 stored on the memory 310 and executable on the processor 320, the execution of which causes the methods 10 or 20 described above to be performed.

[0076] Additionally, as described above, this application can also be implemented as a computer-readable storage medium storing information for making such... Figure 1The program that executes the process of method 10 or 20 shown. Here, computer-readable storage media can be various types of computer-readable storage media, such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memory (e.g., ROM, non-volatile memory, etc.), and tapes (e.g., magnetic tapes, cassette tapes, etc.).

[0077] This application can also be implemented as a computer program product, comprising a computer program that, when executed by a processor, implements, as described above. Figure 1 The steps of method 10 or 20 are shown.

[0078] Where applicable, the various embodiments provided in this application may be implemented using hardware, software, or a combination of hardware and software. Furthermore, where applicable, without departing from the scope of this application, the various hardware and / or software components described herein may be combined into composite components comprising software, hardware, and / or both. Where applicable, without departing from the scope of this application, the various hardware and / or software components described herein may be divided into sub-components comprising software, hardware, or both. Additionally, where applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.

[0079] The software (such as program code and / or data) according to this application may be stored on one or more computer-readable storage media. It is also contemplated that the software identified herein may be implemented using one or more networked and / or otherwise general-purpose or special-purpose computers and / or computer systems. Where applicable, the order of the various steps described herein may be changed, combined into compound steps, and / or divided into sub-steps to provide the features described herein.

[0080] The embodiments and examples presented herein are provided to best illustrate embodiments of this application and its particular applications, thereby enabling those skilled in the art to implement and use this application. However, those skilled in the art will understand that the above description and examples are provided for ease of illustration and example only. The descriptions presented are not intended to cover all aspects of this application or to limit this application to the precise forms disclosed.

Claims

1. A method for generating a front-end component library, characterized in that, The method includes the following steps: Identify and extract component regions from the received design files; Multimodal feature extraction is performed on the component region to obtain the component's visual features, text features, and structural features; The visual features, text features, and structural features of the component are dynamically weighted and fused to obtain a fused feature vector, and the component is classified based on the fused feature vector; Clustering and parameter fusion of style features for components of the same type to generate style templates that represent the overall style performance of components of that type; and The style template is integrated with the structural template of the corresponding type of component, and the integrated style template is included in the front-end component library.

2. The method according to claim 1, characterized in that, Identify and extract component regions from the received design files, including: The design file is parsed using an object detection model to generate multiple component candidate boxes; The confidence level is calculated based on the existence probability and classification accuracy of the candidate bounding boxes; and Valid candidate boxes are selected based on a preset confidence threshold.

3. The method according to claim 2, characterized in that, The object detection model adopts the YOLOv8 architecture and is pre-trained using public datasets in the user interface domain. It is then fine-tuned using business-specific user interface design graph datasets to enhance the model's ability to detect business-specific components, irregularly shaped components, and small-sized components.

4. The method according to claim 2, characterized in that, The confidence level of the component candidate box is the product of the existence probability and the maximum classification accuracy, wherein the existence probability represents the probability that a valid component exists in the candidate box, and the maximum classification accuracy represents the maximum value among the probabilities of the candidate box belonging to each preset component type.

5. The method according to claim 2, characterized in that, Identifying and extracting component regions from the received design files also includes: Edge pixels of the valid candidate boxes are extracted using an edge detection algorithm to determine the component shape; Based on the component shape, a corresponding contour fitting method is selected to correct the positioning deviation of the effective candidate boxes; and The boundary coordinates of the valid candidate boxes are updated based on the correction results.

6. The method according to claim 5, characterized in that, Select the appropriate contour fitting method based on the component shape, including: For regular-shaped components, the minimum bounding rectangle algorithm is used to correct their bounding boxes; For irregularly shaped components, Bézier curves are used to fit the edge curves of the bounding box.

7. The method according to claim 5, characterized in that, Identifying and extracting component regions from the received design files also includes: Based on the updated boundary coordinates, independent component region images are cropped. For nested components, independent local images of the parent and child components are extracted separately.

8. The method according to claim 7, characterized in that, Identifying and extracting component regions from the received design files also includes: The component region image is normalized by proportional scaling and whitespace padding; and / or The structured data of the component region is determined, and the structured data includes one or more of the following: component index, bounding box coordinates, preliminary prediction type, image path, confidence score, sub-component list, sub-component index, and coordinate offset of the sub-component relative to the parent component.

9. The method according to claim 1, characterized in that, The extraction of the visual features includes: Extracting the basic visual features of the components includes: obtaining the dominant color value through K-means clustering; calculating the mean and variance of the color mode through color space conversion; and calculating the aspect ratio and contour complexity of the components. Extracting the high-order visual features of the components includes: firstly, performing feature extraction using a convolutional neural network model, and then using principal component analysis to reduce the dimensionality to obtain a high-order visual feature vector of a preset dimension.

10. The method according to claim 1, characterized in that, The extraction of the text features includes: The text content is identified and extracted from the component area using an optical character recognition algorithm; The extracted text content is input into a pre-trained language model for semantic encoding to generate text semantic vectors.

11. The method according to claim 1, characterized in that, The extraction of the structural features includes: Extract the positional features of the component, including its normalized coordinates in the design file, the deviation of the center point coordinates from the center of the screen, the functional area it belongs to based on its vertical position, and the screen ratio it has based on its horizontal position. Extract the hierarchical relationship features of the components, including the type of its parent component, its hierarchical depth in the component tree, the number and type composition of its child components, and the arrangement direction of its child components; The adjacency relationship features of the components are extracted, including the types of the preceding and succeeding components determined by Euclidean distance calculation, and the minimum nearest neighbor distance.

12. The method according to claim 1, characterized in that, The visual features, text features, and structural features of the components are dynamically weighted and fused to obtain a fused feature vector, including: Assign basic weights to the visual features, text features, and structural features based on the initial prediction type of the components; The visual features, text features, and structural features are weighted, and the weighted feature vectors are concatenated into a combined feature vector; and Principal component analysis and norm normalization are sequentially performed on the combined feature vectors to obtain fused feature vectors for component classification.

13. The method according to claim 1, characterized in that, Classifying the components based on the fused feature vector includes: The fused feature vector is input into a pre-trained classification model to obtain the predicted probability that the component belongs to each predefined type; and The type with the highest predicted probability is taken as the final classification result of the component.

14. The method according to claim 1, characterized in that, The multimodal feature extraction also includes the extraction of style features, wherein the style features include one or more of the following: Basic color features include background color, text color, and border color; Border features include border width, border style, and border rounded corners; Shadow features include shadow color, shadow offset, shadow blur radius, and shadow spread radius; Layout spacing features, including inner margins, outer margins, and width and height dimensions; Text style features include font size, font weight, text alignment, text spacing, line height, and font style.

15. The method according to claim 1, characterized in that, The pattern feature clustering is performed using a density-based clustering algorithm, and the process includes: The style feature differences between any two components are determined based on the weighted Euclidean distance algorithm; Based on preset neighborhood radius and minimum sample number parameters, density clustering is performed on component sets belonging to the same fine-grained type, thereby grouping components with similar styles into the same cluster.

16. The method according to claim 15, characterized in that, Before performing style feature clustering, the method further includes a step of standardizing the style features, the standardization process including: For numerical style features, normalization is performed by dividing the feature value by the maximum possible value. For color-type style features, the color values ​​are converted to the hue, saturation, and brightness color space and then normalized. For discrete style features, one-hot encoding is used to convert the text description into a numerical vector.

17. The method according to claim 1, characterized in that, The rules for parameter fusion include: The median of numerical parameters is used as the fusion result; The average value of color parameters in the hue, saturation, and brightness color spaces is used as the fusion result. The most frequently occurring value of the discrete parameter is taken as the fusion result.

18. The method according to claim 1, characterized in that, The integration of the style template with the structural template of the corresponding component type is achieved through template engine technology. The integration process includes: The style parameters, after being integrated from the style template, are injected into the corresponding placeholders of the predefined structural template of the corresponding type, thereby generating a front-end component code file with specific visual styles that can be reused on multiple platforms.

19. A computer device, characterized in that, It includes: a memory; a processor; and a computer program stored in the memory and executable on the processor, the execution of which causes the following operations: Identify and extract component regions from the received design files; Multimodal feature extraction is performed on the component region to obtain the component's visual features, text features, and structural features; The visual features, text features, and structural features of the component are dynamically weighted and fused to obtain a fused feature vector, and the component is classified based on the fused feature vector; Style features are clustered and parameters are fused for components of the same type to generate style templates that represent the overall style performance of components of that type. as well as The style template is integrated with the structural template of the corresponding type of component, and the integrated style template is included in the front-end component library.

20. The computer device according to claim 19, characterized in that, Identify and extract component regions from the received design files, including: The design file is parsed using an object detection model to generate multiple component candidate boxes; The confidence level is calculated based on the existence probability and classification accuracy of the candidate bounding boxes; and Valid candidate boxes are selected based on a preset confidence threshold.

21. The computer device according to claim 20, characterized in that, The object detection model adopts the YOLOv8 architecture and is pre-trained using public datasets in the user interface domain. It is then fine-tuned using business-specific user interface design graph datasets to enhance the model's ability to detect business-specific components, irregularly shaped components, and small-sized components. And / or The confidence level of the component candidate box is the product of the existence probability and the maximum classification accuracy, wherein the existence probability represents the probability that a valid component exists in the candidate box, and the maximum classification accuracy represents the maximum value among the probabilities of the candidate box belonging to each preset component type.

22. The computer device according to claim 20, characterized in that, Identifying and extracting component regions from the received design files also includes: Edge pixels of the valid candidate boxes are extracted using an edge detection algorithm to determine the component shape; Based on the component shape, a corresponding contour fitting method is selected to correct the positioning deviation of the effective candidate boxes; and The boundary coordinates of the valid candidate boxes are updated based on the correction results.

23. The computer device according to claim 22, characterized in that, Select the appropriate contour fitting method based on the component shape, including: For regular-shaped components, the minimum bounding rectangle algorithm is used to correct their bounding boxes; For irregularly shaped components, Bézier curves are used to fit the edge curves of the bounding box.

24. The computer device according to claim 22, characterized in that, Identifying and extracting component regions from the received design files also includes: Based on the updated boundary coordinates, independent component region images are cropped. For nested components, independent local images of the parent and child components are extracted separately.

25. The computer device according to claim 24, characterized in that, Identifying and extracting component regions from the received design files also includes: The component region image is normalized by proportional scaling and whitespace padding; and / or The structured data of the component region is determined, and the structured data includes one or more of the following: component index, bounding box coordinates, preliminary prediction type, image path, confidence score, sub-component list, sub-component index, and coordinate offset of the sub-component relative to the parent component.

26. The computer device according to claim 19, characterized in that, The extraction of the visual features includes: Extracting the basic visual features of the components includes: obtaining the dominant color value through K-means clustering; calculating the mean and variance of the color mode through color space conversion; and calculating the aspect ratio and contour complexity of the components. Extracting the high-order visual features of the components includes: firstly, performing feature extraction using a convolutional neural network model, and then using principal component analysis to reduce the dimensionality to obtain a high-order visual feature vector of a preset dimension.

27. The computer device according to claim 19, characterized in that, The extraction of the text features includes: The text content is identified and extracted from the component area using an optical character recognition algorithm; The extracted text content is input into a pre-trained language model for semantic encoding to generate text semantic vectors.

28. The computer device according to claim 19, characterized in that, The extraction of the structural features includes: Extract the positional features of the component, including its normalized coordinates in the design file, the deviation of the center point coordinates from the center of the screen, the functional area it belongs to based on its vertical position, and the screen ratio it has based on its horizontal position. Extract the hierarchical relationship features of the components, including the type of its parent component, its hierarchical depth in the component tree, the number and type composition of its child components, and the arrangement direction of its child components; The adjacency relationship features of the components are extracted, including the types of the preceding and succeeding components determined by Euclidean distance calculation, and the minimum nearest neighbor distance.

29. The computer device according to claim 19, characterized in that, The visual features, text features, and structural features of the components are dynamically weighted and fused to obtain a fused feature vector, including: Assign basic weights to the visual features, text features, and structural features based on the initial prediction type of the components; The visual features, text features, and structural features are weighted, and the weighted feature vectors are concatenated into a combined feature vector; and Principal component analysis and norm normalization are sequentially performed on the combined feature vectors to obtain fused feature vectors for component classification.

30. The computer device according to claim 19, characterized in that, Classifying the components based on the fused feature vector includes: The fused feature vector is input into a pre-trained classification model to obtain the predicted probability that the component belongs to each predefined type; and The type with the highest predicted probability is taken as the final classification result of the component.

31. The computer device according to claim 19, characterized in that, The multimodal feature extraction also includes the extraction of style features, wherein the style features include one or more of the following: Basic color features include background color, text color, and border color; Border features include border width, border style, and border rounded corners; Shadow features include shadow color, shadow offset, shadow blur radius, and shadow spread radius; Layout spacing features, including inner margins, outer margins, and width and height dimensions; Text style features include font size, font weight, text alignment, text spacing, line height, and font style.

32. The computer device according to claim 19, characterized in that, The pattern feature clustering is performed using a density-based clustering algorithm, and the process includes: The style feature differences between any two components are determined based on the weighted Euclidean distance algorithm; Based on preset neighborhood radius and minimum sample number parameters, density clustering is performed on component sets belonging to the same fine-grained type, thereby grouping components with similar styles into the same cluster.

33. The computer device according to claim 32, characterized in that, The execution of the computer program also results in the following operation: a step of standardizing the style features before performing style feature clustering, the standardization process including: For numerical style features, normalization is performed by dividing the feature value by the maximum possible value. For color-type style features, the color values ​​are converted to the hue, saturation, and brightness color space and then normalized. For discrete style features, one-hot encoding is used to convert the text description into a numerical vector.

34. The computer device according to claim 19, characterized in that, The rules for parameter fusion include: The median of numerical parameters is used as the fusion result; The average value of color parameters in the hue, saturation, and brightness color spaces is used as the fusion result. The most frequently occurring value of the discrete parameter is taken as the fusion result.

35. The computer device according to claim 19, characterized in that, The integration of the style template with the structural template of the corresponding component type is achieved through template engine technology. The integration process includes: The style parameters, after being integrated from the style template, are injected into the corresponding placeholders of the predefined structural template of the corresponding type, thereby generating a front-end component code file with specific visual styles that can be reused on multiple platforms.

36. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed, perform the method for generating a front-end component library according to any one of claims 1-18.

37. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method for generating a front-end component library according to any one of claims 1-18.

Images