Front-end page code generation method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a front-end page code generation method and device, electronic equipment and a medium. A specific embodiment of the method comprises the steps of performing multi-source data fusion on a multi-source front-end page design draft set and user historical page browsing behavior information to obtain front-end page fusion design information; performing element visual semantic extraction on the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set; constructing a page behavior semantic knowledge graph; generating an initial front-end page code; code interaction adjustment is carried out after code adjustment is carried out on the initial front-end page code, and a front-end interaction code is obtained; performing multi-dimensional page evaluation on the front-end page to obtain a page multi-dimensional evaluation value set; and performing code dynamic adjustment and optimization on the front-end interaction code. According to the embodiment, the front-end page code can be automatically and accurately generated, the code generation efficiency and adaptability are improved, the page error with a design draft is reduced, and the user interaction experience feeling is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and particularly to methods, apparatuses, electronic devices, and media for generating front-end page code. Background Art

[0002] The front-end page is a page for realizing the interaction between users and the system. With the development of computer technologies, the demand for front-end pages is increasing. The generation of front-end pages is a page obtained by technical personnel writing code for the obtained front-end page design drafts. However, in the generation process, there are problems of low coding efficiency and a large difference between the generated front-end page and the design draft. For the generation of front-end page code, the commonly used method is: using a template-based code generation framework to generate static front-end code for multi-source front-end page design drafts. Then, through technical personnel, manual intervention and business logic implementation are performed on the static front-end code to generate front-end page code.

[0003] However, it is found in practice that when generating front-end page code in the above manner, there is often the following technical problem 1: Since the template-based code generation framework generates front-end page code based on existing front-end page design templates, it cannot generate complex images, animations, and page interaction effects, and only extracts elements and copies styles of the front-end page, and cannot understand the deep semantic information of the business logic behind the elements. At the same time, manual implementation of business logic takes a lot of time, resulting in low efficiency of front-end page code generation, low code accuracy and quality, and a large gap between the front-end page corresponding to the design draft, thereby reducing the user's page experience.

[0004] The above information disclosed in this background art section is only used to enhance the understanding of the background of the concept of the present disclosure. Therefore, it may include information on prior art that is not known to those of ordinary skill in the art. Summary of the Invention

[0005] This summary part of the present disclosure is used to briefly introduce concepts, which will be described in detail in the following detailed implementation part. This summary part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure propose methods, apparatuses, electronic devices, and media for generating front-end page code to solve one or more of the technical problems mentioned in the above background art section.

[0007] First aspect, some embodiments of the present disclosure provide a method for generating front-end page code, including: obtaining a multi-source front-end page design draft set and user historical page browsing behavior information; performing multi-source data fusion on the multi-source front-end page design draft set and the user historical page browsing behavior information to obtain front-end page fusion design information; performing element visual semantic extraction on the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set; constructing a semantic graph of page behavior knowledge based on the page text element information set and the page design visual element information set, the front-end page fusion design information, and a preset page design knowledge base to obtain a page behavior semantic knowledge graph; generating initial front-end page code based on the page behavior semantic knowledge graph and a preset front-end page template knowledge base; adjusting the initial front-end page code according to reinforcement learning and the page behavior semantic knowledge graph to obtain adjusted front-end code; performing code interaction adjustment on the adjusted front-end code according to currently obtained real-time user interaction behavior information to obtain front-end interaction code; performing multi-dimensional page evaluation on the front-end page corresponding to the front-end interaction code to obtain multi-dimensional page evaluation values; and dynamically adjusting and optimizing the front-end interaction code according to the multi-dimensional page evaluation values.

[0008] Second aspect, some embodiments of the present disclosure provide a front-end page code generation device, including: an obtaining unit configured to obtain a multi-source front-end page design draft set and user historical page browsing behavior information; a multi-source data fusion unit configured to perform multi-source data fusion on the multi-source front-end page design draft set and the user historical page browsing behavior information to obtain front-end page fusion design information; an element visual semantic extraction unit configured to perform element visual semantic extraction on the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set; a semantic graph construction unit configured to construct a semantic graph of page behavior knowledge based on the page text element information set, the page design visual element information set, the front-end page fusion design information, and a preset page design knowledge base to obtain a page behavior semantic knowledge graph; a generation unit configured to generate initial front-end page code based on the page behavior semantic knowledge graph and a preset front-end page template knowledge base; a code adjustment unit configured to adjust the initial front-end page code according to reinforcement learning and the page behavior semantic knowledge graph to obtain adjusted front-end code; a code interaction adjustment unit configured to perform code interaction adjustment on the adjusted front-end code according to currently obtained real-time user interaction behavior information to obtain front-end interaction code; a multi-dimensional page evaluation unit configured to perform multi-dimensional page evaluation on the front-end page corresponding to the front-end interaction code to obtain multi-dimensional page evaluation values; and a code dynamic adjustment and optimization unit configured to dynamically adjust and optimize the front-end interaction code according to the multi-dimensional page evaluation values.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0011] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: The front-end page code generation method of some embodiments of the present disclosure can automatically and accurately generate front-end page code, improve the efficiency and adaptability of the generated code, reduce the page error with the design draft, and improve the user interaction experience. Specifically, the reasons for the relatively low efficiency of the relevant front-end page code generation, the relatively low code accuracy and quality, and the large gap between the front-end page corresponding to the design draft, which in turn reduces the user's page experience, are as follows: Since the template-based code generation framework generates front-end page code based on the existing front-end page design templates, it is difficult to generate complex images, animations, and page interaction effects, and only extracts elements and copies styles from the front-end page, unable to understand the deep semantic information of the business logic behind the elements. At the same time, manual implementation of business logic requires a lot of time, resulting in relatively low efficiency of front-end page code generation, relatively low code accuracy and quality, a large gap between the front-end page corresponding to the design draft, and thus reducing the user's page experience. Based on this, the front-end page code generation method of some embodiments of the present disclosure can first, obtain a multi-source front-end page design draft set and user historical page browsing behavior information. Here, the multi-source front-end page design draft set and user historical page browsing behavior information can be used for subsequent multi-source data fusion. Secondly, perform multi-source data fusion on the above-mentioned multi-source front-end page design draft set and the above-mentioned user historical page browsing behavior information to obtain front-end page fusion design information. Here, by the mutual complementarity of the multi-source front-end page design draft set and user historical page browsing behavior information, the comprehensiveness and data quality of the front-end page fusion design information are improved. Thirdly, perform element visual semantic extraction on the above-mentioned multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set. Here, the accuracy of the visual and semantic feature extraction of the elements is improved to more accurately and comprehensively understand the design intention of the front-end page design draft and the logical relationship between the design visuals. Subsequently, according to the above-mentioned page text element information set and the above-mentioned page design visual element information set, construct a semantic graph of page behavior semantics for the above-mentioned front-end page fusion design information and a preset page design knowledge base to obtain a page behavior semantic knowledge graph. Here, the preset page design knowledge base includes a large amount of front-end development knowledge, front-end page development knowledge, and business implementation logic information. Associating with the existing page design knowledge base can improve the adaptability between the front-end page and the business requirements and improve the interactivity of the front-end page. After that, generate the initial front-end page code according to the above-mentioned page behavior semantic knowledge graph and the preset front-end page template knowledge base. Here, the preset front-end page template knowledge base includes a large amount of front-end code templates and code generation rule information, which can reduce the redundancy of the front-end page code, improve the reusability and scalability of the code, and improve the efficiency of front-end page code generation. Then, according to reinforcement learning and the above-mentioned page behavior semantic knowledge graph, adjust the above-mentioned initial front-end page code to obtain the adjusted front-end code.Here, the page behavior semantic knowledge graph can serve as a prior knowledge base for reinforcement learning. Through dynamic interaction with the page behavior semantic knowledge graph environment, reinforcement learning can autonomously optimize the code generation path, adapt to different business requirements and front-end frameworks. Moreover, the powerful semantic understanding and reward mechanism of reinforcement learning can cover the entire business process corresponding to the front-end page, improving the complex interaction functions of the page. Then, based on the currently obtained real-time user interaction behavior information, code interaction adjustment is performed on the adjusted front-end code above to obtain front-end interaction code. Here, the responsiveness of the front-end page code can be improved. By combining multiple front-end page code generation methods, the quality, performance, and accuracy of the front-end page code can be improved more comprehensively, thereby enhancing the user interaction experience. Then, a multi-dimensional page evaluation is performed on the front-end page corresponding to the above front-end interaction code to obtain multi-dimensional page evaluation values. Here, the multi-dimensional page evaluation can effectively and comprehensively grasp the quality of the front-end page and the page code, facilitating subsequent real-time dynamic optimization. Finally, based on the above multi-dimensional page evaluation values, dynamic adjustment and optimization of the above front-end interaction code are performed. Here, through the feedback information of the evaluation values for backpropagation and iterative adjustment, targeted code adjustment can be carried out, improving the efficiency and pertinence of dynamic adjustment and optimization. Thus, it can be seen that the front-end page code generation method provides a more comprehensive and accurate data basis for code generation by integrating multi-source front-end page design drafts and user browsing behaviors, improving the quality and adaptability of code generation; page design element extraction and knowledge graph generation can better grasp the design intent and business logic, generating front-end code that better meets the requirements; the combination of multiple code generation methods can give full play to the advantages of various methods, improving the efficiency and flexibility of code generation; multi-dimensional code evaluation and continuous optimization based on the evaluation values can improve the quality of the front-end page and the user experience, as well as improve the restoration degree of the design draft. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In conjunction with the accompanying drawings and with reference to the following detailed embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of the front-end page code generation method according to the present disclosure; Figure 2 is a schematic diagram of a page for dynamically adjusting and optimizing a front-end page through code in some embodiments of the front-end page code generation method according to the present disclosure; Figure 3 is a comparison schematic diagram of the generated front-end pages in some embodiments of the front-end page code generation method according to the present disclosure; Figure 4 is a schematic diagram comparing the generated front - end page and page code in multiple evaluation dimensions in some embodiments of the front - end page code generation method according to the present disclosure; Figure 5 is a schematic structural diagram of some embodiments of the front - end page code generation device according to the present disclosure; Figure 6 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed implementation manners

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0015] In addition, it should be noted that for the sake of convenience of description, only parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0017] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present disclosure will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0020] Figure 1 Flow 100 of some embodiments of the front - end page code generation method according to the present disclosure is shown. The front - end page code generation method includes the following steps: Step 101, obtain a multi - source front - end page design draft set and user historical page browsing behavior information.

[0021] In some embodiments, the execution subject (e.g., an electronic device) of the above-mentioned front-end page code generation method can obtain a multi-source front-end page design draft set and user historical page browsing behavior information through a wired connection method or a wireless connection method. Among them, the front-end page design drafts in the above-mentioned multi-source front-end page design draft set can be drafts that show the relationships of various elements included in the front-end page in the form of images and come from different data sources. The above-mentioned multi-source front-end page design draft set can include, but is not limited to, at least one of the following: crawler documents for design material websites (e.g., Dribbble website, Behance website), project case documents of front-end open source communities (e.g., Open Source China, SegmentFault community), and design drafts of front-end pages corresponding to different design software. The above-mentioned user historical page browsing behavior information can be the behavior information of the user browsing the front-end page collected before the current time. The above-mentioned user historical page browsing behavior information can include, but is not limited to, at least one of the following: the scrolling distance within the page, the hovering time of page elements, and the operation order of page elements. In practice, the above-mentioned execution subject can use distributed crawler technology to collect the front-end page design drafts from different sources at different frequencies to obtain a multi-source front-end page design draft. For example, the design material website can be set to obtain data at 2:00 am on Tuesdays and Fridays. The front-end open source community uses a combination of real-time collection and full-volume collection at 1:00 am every day for data acquisition. Use a user behavior recording tool or a browser plugin to obtain data on user historical page browsing behavior information. For example, the above-mentioned user behavior recording tool can be the Hotjar website behavior analysis tool. The above-mentioned browser plugin can be Mouseflow.

[0022] Step 102, perform multi-source data fusion on the above-mentioned multi-source front-end page design draft set and the above-mentioned user historical page browsing behavior information to obtain front-end page fusion design information.

[0023] In some embodiments, the above-mentioned execution subject can perform multi-source data fusion on the above-mentioned multi-source front-end page design draft set and the above-mentioned user historical page browsing behavior information to obtain front-end page fusion design information. Among them, the above-mentioned front-end page fusion design information can be information that combines the styles, layouts, interaction logics, semantic information of the design elements of the front-end page design draft and the information of various elements that the user behavior pays attention to. The above-mentioned style can be the red, green, and blue colors, font name, and size of the design element. The above-mentioned layout can be the coordinate position, width, and height of the design element. The above-mentioned interaction logic can be the click event of the button and the page jump logic.

[0024] In some optional implementation manners of some embodiments, the above-mentioned performing multi-source data fusion on the above-mentioned multi-source front-end page design draft set and the above-mentioned user historical page browsing behavior information to obtain front-end page fusion design information may include the following steps: First step, perform format conversion on the above-mentioned multi-source front-end page design draft set to obtain a converted multi-source front-end page design draft set. Among them, the front-end page design drafts in the above-mentioned converted multi-source front-end page design draft set can be design drafts in XML (Extensible Markup Language)-like format. "like" is a parameter for fuzzy matching in XML format documents by cooperating with placeholder or functions. In practice, the above-mentioned execution entity can first perform file parsing on the multi-source front-end page design draft set to obtain a parsed file set. Secondly, extract design elements from the parsed file set to obtain a design element set. Then, construct a rule engine for the preset format conversion rule information set and format conversion template to obtain a format conversion rule engine. Among them, the above-mentioned preset format conversion rule information set and format conversion template can be pre-set rule information. For example, the above-mentioned preset format conversion rule information set and format conversion template can include: syntax conversion, structure mapping conversion, general label attribute template, responsive design template, element mapping template. Finally, input the above-mentioned design element set into the format conversion rule engine to obtain a converted multi-source front-end page design draft. The above-mentioned syntax conversion can include at least one of the following: converting the proprietary syntax of design software into a general page description, unit standardization, and attribute mapping. The above-mentioned structure mapping conversion can include at least one of the following: hierarchical relationship conversion, template matching conversion.

[0025] Second step, perform data preprocessing on the above-mentioned user historical page browsing behavior information to obtain preprocessed user historical page browsing behavior information.

[0026] Third step, construct graph structures for the above-mentioned converted multi-source front-end page design draft set and the above-mentioned preprocessed user historical page browsing behavior information to obtain a multi-source front-end page design structure graph set and a user browsing behavior structure graph. Among them, the multi-source front-end page design structure graphs in the above-mentioned multi-source front-end page design structure graph set can be graphs including design element nodes, code nodes, association edges between design element nodes and code nodes, association edges between page interaction event nodes and code nodes, and parent-child association edges of the code DOM (Document Object Model) tree. The above-mentioned user browsing behavior structure graph can be a directed acyclic graph with browsing behavior events of user browsing behavior information as nodes and the dependency relationships between various browsing behavior events as edges.

[0027] Fourth, perform data fusion on the above multi-source front-end page design structure atlas and the above user browsing behavior structure diagram to obtain a multi-source fusion node feature vector set. Among them, the multi-source fusion node feature vectors in the multi-source fusion node feature vector set can represent the fused user browsing behavior information, page element semantic information, page visual information, and the association relationships between each node. For example, the above association relationship can be the association relationship between the high click frequency of any button in the user behavior data and the style and interaction logic of the button. In practice, the above execution entity can input the above multi-source front-end page design structure atlas and the above user browsing behavior structure diagram into the HAN (Heterogeneous Graph Attention Network) model to perform data fusion at the feature level and obtain a multi-source fusion node feature vector set.

[0028] Fifth, input the above multi-source fusion node feature vector set into the page design anomaly recognition model to obtain a page design anomaly information set. Among them, the page design anomaly information in the page design anomaly information set can be syntax errors in the code, abnormal sizes of page elements, and information indicating conflicts in the interaction logic between elements. The page design anomaly recognition model can be a neural network model that performs data reconstruction on the input multi-source fusion node feature vector set to identify anomaly information. For example, the page design anomaly recognition model can be a variational autoencoder model.

[0029] Sixth, perform multi-dimensional abnormal data repair processing on the above page design anomaly information set to obtain the repaired page design information as the front-end page fusion design information. Among them, the above repaired page design information can be the information obtained after correcting or deleting the abnormal information in the above page design anomaly information set. In practice, the above execution entity can first perform abnormal type classification processing on the above page design anomaly information set to obtain a set of page anomaly information groups. Secondly, in response to determining that the page anomaly information is the abnormal information of the code semantic anomaly type, use the syntax rule library and similar case learning to perform abnormal data repair processing to obtain the repaired page design information. Among them, the above syntax rule library can be a database storing various correct page code syntaxes. Then, in response to determining that the page anomaly information is the abnormal information of the page element size anomaly type, use the preset page design specification knowledge base to perform abnormal data repair processing to obtain the repaired page design information. Among them, the above preset page design specification knowledge base can be a knowledge base formed by prior knowledge related to page design that is preset in advance. Finally, in response to determining that the page anomaly information is the abnormal information of the interaction logic anomaly type, use the preset business process knowledge base and the user's historical page browsing behavior information to perform abnormal data repair to obtain the repaired page design information. Among them, the above preset business process knowledge base can be a knowledge base of preset business execution logic information.

[0030] Step 103: Extract the visual semantics of elements from the above multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set.

[0031] In some embodiments, the above execution subject may extract the visual semantics of elements from the above multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set. Among them, the page text element information in the above page text element information set may be the text content, font format, semantic information, and information on the logical structure of the text on the page of the page element. The page design visual element information in the above page design visual element information set may be information such as colors, shapes, textures, element style features (e.g., minimalist style, retro style, sense of technology), and emotional features (e.g., friendly, professional, lively) extracted from the front-end page design draft. It should be noted that since the element style features and emotional features are somewhat subjective, a standardized annotation system can be established to reduce the source of subjective deviation, the model can be optimized to enhance the robustness to fuzzy features, domain knowledge constraints can be introduced to narrow the subjective judgment space, and an interactive iterative optimization of a human-machine collaborative closed loop can be established. The above establishment of a standardized annotation system may be multi-expert collaborative annotation and noise filtering. The above optimization of the model may include: feature learning adversarial training with uncertainty perception to enhance generalization, and multi-model integration and evidence fusion. The above introduction of domain knowledge constraints may be the embedding of an existing design specification knowledge graph and dynamic context-aware classification. The establishment of an interactive iterative optimization of a human-machine collaborative closed loop may be reinforcement learning and continuous optimization.

[0032] In some optional implementation manners of some embodiments, the above extraction of text visual features from the above multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set may include the following steps: The first step: Detect the text areas in each multi-source front-end page design draft in the above multi-source front-end page design draft set to generate a page text area group and obtain a page text area group set. Among them, the page text area in the above page text area group may be the page area containing text in the front-end page design draft. The above text area detection may be text area detection using CRAFT (Character Region Awareness for Text Detection).

[0033] In the second step, input the above page text area set into the page text recognition model to obtain a page text recognition information set. Among them, the page text recognition information in the above page text recognition information set can be the text characters within the page text area. The above page text recognition model can be a deep neural network model for text recognition of the input page text area set. The above page text recognition model can include: an improved ResNet-34 model, a bidirectional long short-term memory neural network model, and a CTC (Connectionist Temporal Classification) model. The above improved ResNet-34 model can be a model that removes the downsampling operation of stage4 in the ResNet-34 backbone network and inserts a channel spatial attention module after stage3.

[0034] In the third step, extract the font style features of the above page text area set to obtain a text font style feature vector set. Among them, the text font style feature vectors in the above text font style feature vector set can represent the font name and font size feature information of the text font. In practice, the above execution entity can first input the above page text area set into the DeepFont (a font detection model using deep learning) model to obtain a text font classification information set. Then, input the above text font classification information set into the FontMagic font parsing library to obtain a text font style feature vector set.

[0035] Fourth step, perform semantic and syntactic extraction on the above page text recognition information set to obtain a page text semantic feature vector set. Among them, the page text semantic feature vectors in the above page text semantic feature vector set can represent the semantic information, syntactic information, and context association information of the page text. In practice, the above execution entity can first perform text preprocessing on the above page text recognition information set to obtain a preprocessed page text recognition information set. The above text preprocessing can include but is not limited to at least one of the following: text character repair, word segmentation, stop word removal, and part-of-speech tagging. Secondly, perform dependency syntactic analysis on the above preprocessed page text recognition information set to obtain a page text dependency tree set. Among them, the page text dependency trees in the above page text dependency tree set can be tree-shaped structures that display the syntactic structures of each word segment in the preprocessed page text recognition information in the form of trees. Then, generate a page text rhetorical structure tree set according to the above page text dependency tree set. Among them, the page text rhetorical structure trees in the above page text rhetorical structure tree set can be tree-shaped structures that annotate the semantic roles of each word segment in the above page text dependency trees, clarifying the action executor and object. The above page text rhetorical structure tree can be a tree-shaped structure that corrects the dependency relationship and infers the cross-sentence logical relationship for the ambiguous sentences in the above page text dependency tree through the provided semantic roles. As an example, the above execution entity can first perform predicate recognition on the above page text dependency tree set to obtain a page text predicate set. Secondly, determine multiple predicate dependency paths for each page text predicate in the above page text predicate set to obtain a predicate dependency path set. Then, perform feature extraction on the predicate dependency path set to obtain a predicate dependency path feature vector set. Among them, the predicate dependency path feature vectors in the predicate dependency path feature vector set can represent the path dependency relationship, path length, part-of-speech, word form, and semantics of the path core node. Finally, input the above predicate dependency path feature vector set into a support vector machine to determine the argument roles of each node in the predicate dependency path set to obtain an argument role set. Finally, add the argument role set to the above page text dependency tree set to obtain a page text rhetorical structure tree set. Finally, perform semantic and syntactic extraction on the above page text rhetorical structure tree set to obtain a page text semantic feature vector set. Among them, the page text semantic feature vectors in the above page text semantic feature vector set can represent the semantic information, syntactic information, and context association information of the page text. The above semantic and syntactic extraction can be performed using a Tree-LSTM (Tree-Structured Long Short Term Memory) model for semantic and syntactic extraction.

[0036] Step 5: Input the above page text area set, the above text font style feature vector set, and the above page text semantic feature vector set into the text logical topology model to obtain a text logical layout feature vector set. Among them, the text logical layout feature vectors in the above text logical layout feature vector set can represent the hierarchical logical relationship and layout type among the page text areas included in the page text area set. The above text logical topology model can be a neural network model that determines the layout relationship among the input page text area sets through the input text font style feature vector set and page text semantic feature vector set. The above text logical topology model can be a GraphSAGE (Graph Sample and Aggregate) model including a graph attention network. In practice, the above execution entity can first splice the above text font style feature vector set and the above page text semantic feature vector set to obtain a spliced feature vector set, and use each text element corresponding to the above page text area set as a graph node, and the spatial position relationship and semantic relationship between text elements as the associated edges between graph nodes to obtain a page text graph structure. Then, input the above page text graph structure into the GraphSAGE model to obtain a page node feature vector set. Among them, the above GraphSAGE model can be a model including 3 aggregation layers, a non-linear transformation layer, a mean aggregation function, and a ReLU (Rectified Linear Unit) activation function. Finally, input the above page node feature vector set into the graph attention network to obtain a text logical layout feature vector set.

[0037] Step 6: Determine the above text font style feature vector set, the above page text semantic feature vector set, and the above text logical layout feature vector set as the page text element information set, and input the above multi-source front-end page design draft set into the page design visual feature extraction model to obtain a page design visual element information set. Among them, the above page design visual feature extraction model can be a neural network model that extracts visual information features of elements in the input multi-source front-end page design draft set. The above page design visual feature extraction model can be a Swin Transformer (Shifted Window Transformer) model.

[0038] In the process of adopting technical solutions to solve the above technical problem 1, there is often another technical problem 2 as follows: In the process of extracting visual features from a multi-source front-end page design draft set, the extracted visual information is too superficial, and only information such as color information, geometric shapes, and textures can be extracted, resulting in an incomplete understanding of the deep semantics of the multi-source front-end page design draft set, resulting in a low accuracy of the generated front-end page code, a large difference in pages, and a reduced user experience. For the above technical problem 2, the conventional solution is generally: Use a convolutional neural network model to perform visual extraction on the single features and shallow visual features in the multi-source front-end page design draft set to obtain a page design visual element information set. However, the above conventional solution still has the following problems: Since only single features are extracted, the extraction of multi-scale visual feature information and depth visual information is ignored, resulting in incomplete visual information extraction from the multi-source front-end page design draft set, resulting in a low accuracy rate of the generated front-end page code, a low generation efficiency, a long code generation time, and a reduced user experience. Considering the disadvantages of the above conventional solution and also combining the advantages / technical status of the deep neural network model and visual feature extraction technology owned by our research institute partners in this field, we have decided to adopt the following solution: Optionally, the above step of inputting the multi-source front-end page design draft set into the page design visual feature extraction model to obtain a page design visual element information set may include the following steps: The first step is to perform image block segmentation on each multi-source front-end page design draft in the multi-source front-end page design draft set to generate a page design image block sequence and obtain a page design image block sequence set. Among them, the page design image blocks in the above page design image block sequence can be pixel blocks with a size of 4*4.

[0039] In the second step, input the above page design image block sequence set into the first window-shifted self-attention stage network included in the above page design visual feature extraction model to obtain the first page visual feature map set. Among them, the above page design visual feature extraction model further includes: a second window-shifted self-attention stage network, a third window-shifted self-attention stage network, and a fourth window-shifted self-attention stage network. Among them, the above page design visual feature extraction model can be a deep neural network model for extracting visual features from the input page design image block sequence set. The above first window-shifted self-attention stage network can be a deep neural network model that linearly embeds the above page design image block sequence set and then inputs it into a model network with a window self-attention mechanism (W-MSA) and a model network with a shifted window self-attention mechanism (SW-MSA). The above model network with a window self-attention mechanism can be a network including a linear layer, a window self-attention mechanism layer, a residual splicing layer, a linear layer, a multi-layer perceptron layer, and a residual splicing layer. The above model network with a shifted window self-attention mechanism can replace the window self-attention mechanism with a shifted window self-attention mechanism. The above second window-shifted self-attention stage network can be a model network that only changes the linear layer in the above first window-shifted self-attention stage network to a block feature map splicing layer to splice the block feature map group set output by the first window-shifted self-attention stage network. The above third window-shifted self-attention stage network can be a model network that only changes the two network models with a self-attention mechanism in the above second window-shifted self-attention stage network to six. The above fourth window-shifted self-attention stage network can be a model network with the same model as the above second window-shifted self-attention stage network but different input and output. The first page visual feature map in the above first page visual feature map set can represent low-order information such as the lines, HSV (Hue Saturation Value) color space, and low-frequency components of texture in the multi-source front-end page design draft set.

[0040] In the third step, input the above first page visual feature map set into the above second window-shifted self-attention stage network to obtain the second page visual feature map set. Among them, the second page visual feature map in the above second page visual feature map set can represent the shape geometric moments of the design draft and the feature information of the middle-order texture.

[0041] In the fourth step, input the above second page visual feature map set into the above third window-shifted self-attention stage network to obtain the third page visual feature map set. Among them, the third page visual feature map in the above third page visual feature map set can represent the middle-order semantic feature information of the regional layout and the color distribution of the design draft.

[0042] Step 5: Input the above-mentioned visual feature atlas of the third page into the above-mentioned fourth window offset self-attention stage network to obtain the visual feature atlas of the fourth page. Among them, the visual feature maps of the fourth page in the above-mentioned visual feature atlas of the fourth page can represent high-level information such as global layout features, context features, and complex layout information of element nesting.

[0043] Step 6: Perform multi-feature fusion on the above-mentioned visual feature atlas of the first page, the above-mentioned visual feature atlas of the second page, the above-mentioned visual feature atlas of the third page, and the above-mentioned visual feature atlas of the fourth page to obtain the fused visual feature atlas of the page. Among them, the above-mentioned multi-feature fusion can be feature fusion using BiFPN (Bidirectional Feature Pyramid Network).

[0044] Step 7: Extract convolutional features from the above-mentioned visual feature atlas of the first page to obtain the page color space feature atlas. Among them, the page color space feature maps in the above-mentioned page color space feature atlas can represent the distribution probability of the color space. The above-mentioned convolutional feature extraction can be feature extraction using a convolutional layer with a convolutional kernel of 3*3.

[0045] Step 8: Determine the gray-level co-occurrence matrix of the above-mentioned visual feature map of the first page as the page texture feature atlas. Among them, the page texture feature maps in the above-mentioned page texture feature atlas can represent a two-dimensional matrix formed by multiple statistics such as texture contrast, energy, and entropy.

[0046] Step 9: Determine the spatial image visual moments of the above-mentioned visual feature atlas of the third page as the page geometric contour feature atlas. Among them, the page geometric contour feature maps in the above-mentioned page geometric contour feature atlas can represent feature information with translational, rotational, and scale invariance, and can quantify the shape, size, and position information of the design draft. The above-mentioned spatial image visual moments can be Hu invariant moments.

[0047] Step 10: Perform global average pooling on the above-mentioned visual feature atlas of the fourth page to obtain the page global visual style feature atlas. Among them, the page global visual style feature maps in the above-mentioned page global visual style feature atlas can represent information about the overall color scheme and layout style of the page. The above-mentioned global average pooling can be pooling using a global average pooling layer.

[0048] Step 11: Perform max pooling on the above-mentioned visual feature atlas of the first page to obtain the page local visual style feature atlas. Among them, the page local visual style feature maps in the above-mentioned page local visual style feature atlas can represent information about element-level font styles and geometric shapes. The above-mentioned max pooling can be pooling using a max pooling layer.

[0049] Step 12: Perform feature fusion on the above-mentioned global visual style feature atlas of the page and the above-mentioned local visual style feature atlas of the page to obtain a visual style feature atlas of the page. Among them, the visual style feature maps in the visual style feature atlas of the page can represent the style feature information of the page elements in the design draft. The above-mentioned style feature information may include, but is not limited to, at least one of the following: a minimalist style with simple lines and a single color tone, a retro style with specific color combinations and font styles, and a technological style reflected by a metallic texture and a blue color tone.

[0050] Step 13: Input the above-mentioned page geometric contour feature atlas, the above-mentioned page texture feature atlas, the above-mentioned page color space feature atlas, and the above-mentioned fused page visual feature atlas into the cross-attention mechanism layer to obtain a page visual emotion feature atlas. Among them, the page visual emotion feature maps in the page visual emotion feature atlas can represent the emotion information conveyed by the design draft to the user. The above-mentioned emotion information may include, but is not limited to, at least one of the following: friendliness information, lively emotion information, professional emotion information.

[0051] Step 14: Determine the above-mentioned page visual style feature atlas, the above-mentioned page visual emotion feature atlas, the above-mentioned page color space feature atlas, the above-mentioned page texture feature atlas, and the above-mentioned page geometric contour feature atlas as the page design visual element information set.

[0052] The above technical solution combines step "step 107" and its related content as an inventive point of an embodiment of the present disclosure, and solves the second technical problem mentioned in the background technology, "Since only single features are extracted, the extraction of multi-scale visual feature information and depth visual information is ignored, resulting in incomplete extraction of visual information from the multi-source front-end page design draft set, leading to low accuracy of the generated front-end page code, low generation efficiency, long code generation time, and reduced user experience." To achieve this effect, the present disclosure first divides the multi-source front-end page design draft set into image blocks and inputs them into the page design visual feature extraction model for multi-scale feature extraction and attention mechanism, which can improve the multi-scale information and cross-level interaction of the extracted visual features. Secondly, traditional visual features of the design draft are extracted. Then, high-level visual features of style and emotion are extracted from the design draft, which can improve the comprehensiveness of the extracted visual information, and by assigning different weights to different visual features through the weight values of the cross-attention mechanism, the amount of computation and system load can be reduced, and the comprehensive and in-depth extraction of visual element information of the design draft can be improved. Finally, by extracting comprehensive and in-depth visual features and generating front-end page code, the accuracy and generation efficiency of the generated front-end page code can be improved, the code generation time can be reduced, and the user experience can be improved.

[0053] Step 104: Construct a semantic graph for the above front-end page fusion design information and the preset page design knowledge base based on the above page text element information set and the above page design visual element information set, to obtain a page behavior semantic knowledge graph.

[0054] In some embodiments, the above execution entity may construct a semantic graph for the above front-end page fusion design information and the preset page design knowledge base based on the above page text element information set and the above page design visual element information set, to obtain a page behavior semantic knowledge graph. Among them, the above page behavior semantic knowledge graph may be a directed weighted graph formed by integrating the existing front-end page design rule information in the preset page design knowledge base into the knowledge graph corresponding to the front-end page fusion design information. The weight in the above directed weighted graph may be a weight value determined by the above user historical page browsing behavior information. The above preset page design knowledge base may be a database formed by pre-collecting existing front-end development knowledge, front-end page design specifications, and business implementation logic information behind the elements in the front-end page related to page design.

[0055] As an example, the above execution entity may first input the above page text element information set and the above page design visual element information set into the ArangoDB graph database to obtain a front-end page knowledge graph. Secondly, determine the number of views of each element in the user historical page browsing behavior information in the above front-end page fusion design information, and determine the ratio to the total number of views as the weight value of each node in the front-end page knowledge graph, to obtain a front-end page browsing knowledge graph. Then, screen out the page design information set related to the above page text element information set and the above page design visual element information set from the above preset page design knowledge base. Finally, input the above page design information set into the above front-end page browsing knowledge graph to obtain a page behavior semantic knowledge graph.

[0056] In some optional implementation manners of some embodiments, the above constructing a semantic graph for the above front-end page fusion design information and the preset page design knowledge base based on the above page text element information set and the above page design visual element information set, to obtain a page behavior semantic knowledge graph, may include the following steps: Step 1: Determine the page design knowledge graph schema layer of the above page behavior semantic knowledge graph. The page design knowledge graph schema layer includes: a node type set and a node relationship type set. The page design knowledge graph schema layer can be an initial determination of the entity ontology, class hierarchy, relationship type, and data schema in the page behavior semantic knowledge graph, so as to ensure the consistency and integrity of data through the input of page text element information, page design visual element information set, and preset page design knowledge base. The class hierarchy can be an inheritance relationship between entity ontologies in a node type, that is, a hierarchical structure in which a subclass inherits the attributes of a superclass. The data schema can be a schema that defines the structure and rules of data storage.

[0057] Step 2: Extract triples from the above preset page design knowledge base to obtain a page design triple set. The page design triples in the page design triple set can be in the form of triples of head entity, entity relationship, and tail entity, which are used to display the association relationships between front-end page element entities.

[0058] Step 3: Extract semantic information from the above front-end page fusion design information according to the above page text element information set and the above page design visual element information to obtain a page text visual semantic information set. The page text visual semantic information in the page text visual semantic information set can be the semantic information, hierarchical relationship, and position relationship information of page elements.

[0059] As an example, the above execution subject can extract the page text visual semantic information set by performing text matching between the above page text element information set and the above page design visual element information and the above front-end page fusion design information.

[0060] Step 4: Extract the dynamic and static association relationships of design elements from the user browsing behavior information obtained in real time to obtain a design element dynamic and static association information set. The design element dynamic and static association information in the design element dynamic and static association information set can be dynamic information such as the business logic and data flow relationship behind page elements determined by user browsing behavior information, and static information such as the click hierarchy relationship, position relationship, and click times of page elements in user behavior. The above user browsing behavior information can be the behavior information of users operating the front-end page collected every 15 minutes.

[0061] Step 5: Perform heterogeneity detection on the above page design triple set, the above page text visual semantic information set, and the above design element dynamic and static association information set to obtain a data heterogeneity value. Among them, the above data heterogeneity value can characterize the degree of difference between the above page design triple set, the above page text visual semantic information set, and the above design element dynamic and static association information set. In practice, the above execution subject can first represent the above page design triple set, the above page text visual semantic information set, and the above design element dynamic and static association information set as feature vectors to obtain a page design feature vector group set, a text visual feature vector set, and a dynamic and static association feature vector set. Then, through a data heterogeneity detection function, perform standardization on the above page design feature vector group set, the above text visual feature vector set, and the above dynamic and static association feature vector set and then perform heterogeneity detection to obtain a data heterogeneity value. Among them, the above data heterogeneity detection function can be 。 represents the data heterogeneity detection function, represents the total sum of squared differences of the above page design feature vector group set, the above text visual feature vector set, and the above dynamic and static association feature vector set after standardization. represents the degrees of freedom related to the number of data categories, which is 2 in the current scenario.

[0062] Step 6, in response to determining that the above data heterogeneity value meets the preset heterogeneity condition, input the above page design triple set, the above page text visual semantic information set, and the above design element dynamic-static association information set into the above page design knowledge graph schema layer to obtain a page behavior semantic knowledge graph. Among them, the above preset heterogeneity condition can be a condition that is preset and used to determine the execution relationship between the above data heterogeneity value and subsequent steps. The above preset heterogeneity condition can be a condition that the data heterogeneity value is between 40% and 60%. It should be noted that when the data heterogeneity value is greater than 60%, the data heterogeneity is strong, and the above page design triple set, the above page text visual semantic information set, and the above design element dynamic-static association information set are preprocessed and grouped again. In practice, the above execution subject can first, in response to determining that the above data heterogeneity value meets the preset heterogeneity condition, input the above page design triple set into the above page design knowledge graph schema layer to obtain an initial page design knowledge graph. Secondly, perform text matching through the above page text element information set, the above page design visual element information, and the above initial page design knowledge graph to determine the input storage location information set. Then, according to the above storage location information set, input the above page text visual semantic information set into the above initial page design knowledge graph to obtain a page design semantic knowledge graph. Among them, the above page design semantic graph can be a graph that integrates the prior knowledge of existing page designs and the knowledge of front-end page designs of multi-source front-end page design drafts. Finally, input the above design element dynamic-static association information set into the above page design semantic knowledge graph to perform real-time dynamic association update on the page design semantic knowledge graph to obtain a page behavior semantic knowledge graph.

[0063] Step 105, generate initial front-end page code according to the above page behavior semantic knowledge graph and the preset front-end page template knowledge base.

[0064] In some embodiments, the above-mentioned execution entity may generate initial front-end page code according to the above-mentioned page behavior semantic knowledge graph and the preset front-end page template knowledge base. Among them, the above-mentioned preset front-end page template knowledge base may be a knowledge base collected for each element of the front-end page and the corresponding element code. The above-mentioned preset front-end page template knowledge base may include: a front-end code template library and a code generation rule set. The above-mentioned front-end code template library may include: a variety of common page layout templates and a variety of components. The above-mentioned page layout templates may include, but are not limited to, at least one of the following: responsive layout, waterfall layout, grid layout, card layout. The above-mentioned components may include, but are not limited to, at least one of the following: navigation bar, form component, carousel component, pop-up window component. The above-mentioned initialized front-end page code may be page code obtained by code reuse and extension of front-end page elements through the preset front-end page template knowledge base.

[0065] As an example, the above-mentioned execution entity may first use the CART (Classification and Regression Tree) decision tree selection algorithm to screen out a set of front-end page element templates corresponding to the page behavior semantic knowledge graph from the above-mentioned preset front-end page template knowledge base. Then, code adjustment is performed on the above-mentioned set of front-end page element templates to better conform to the front-end page corresponding to the above-mentioned multi-source front-end page design draft set, resulting in initial front-end page code.

[0066] In the process of adopting technical solutions to solve the above-mentioned technical problem 1, there is often the following technical problem 3: How to quickly and accurately select component templates that conform to the multi-source front-end page design draft from a large number of existing front-end page component templates, and accurately extract the complex nested constraints between component templates. For the above-mentioned technical problem 3, the conventional solution is generally: through a low-code platform, generate initial front-end page code according to the page behavior semantic knowledge graph and the preset front-end page template knowledge base. However, the above-mentioned conventional solution still has the following problems: Since the low-code platform usually performs page component matching through string replacement or fixed matching rules, it cannot understand the semantic information in the multi-source front-end page design draft and the page behavior semantic knowledge graph, as well as the complex combination and nested relationship of multiple front-end page component codes for one component in the multi-source front-end page design draft. The low-code platform cannot model and process the page structure and the nested constraint relationship between components, resulting in the mismatch between the generated initial front-end page code and the multi-source front-end page design draft, low code quality, the generated front-end page not meeting the user's needs, and reducing the user's page experience. Considering the disadvantages of the above-mentioned conventional solution and also combining the technical advantages / technical status of generating front-end page code based on page component code templates and preset code generation rules owned by our research institute partners in this field, we have decided to adopt the following solution: In some alternative implementations of some embodiments, generating the initial front-end page code according to the above-mentioned page behavior semantic knowledge graph and the preset front-end page template knowledge base may include the following steps: First, determine the template attribute selection metric of each front-end code template in the front-end code template set in the above-mentioned preset front-end page template knowledge base and the template mutual information set between the front-end code templates, and obtain a template attribute selection metric set. Among them, the template selection metric in the template attribute selection metric set represents the information gain of the above-mentioned front-end code template, that is, it evaluates the effectiveness of the front-end code template set for template selection. The determination can be made using the information gain algorithm. The front-end code templates in the above-mentioned front-end code template set can be the element code templates of existing front-end page elements. For example, the front-end code template can be the code template of a form. The template mutual information in the above-mentioned template mutual information set can represent the relevance of the front-end code templates.

[0067] Second, construct a decision tree for the above-mentioned front-end code template set according to the above-mentioned template attribute selection metric set and the template mutual information set, and obtain a template decision tree. Among them, the template decision tree can be a tree structure formed by the above-mentioned front-end code template set. The leaf nodes in the above-mentioned template decision tree can be front-end code templates. As an example, the above-mentioned execution subject can use the C4.5 (Classification and Regression Trees, version 4.5 of classification and regression trees) decision tree algorithm to construct a decision tree for the above-mentioned front-end code template set according to the above-mentioned template attribute selection metric set, and obtain a template decision tree.

[0068] Third, extract the node feature vectors of the above-mentioned page behavior semantic knowledge graph to obtain a page node feature vector set. Among them, the page node feature vectors in the above-mentioned page node feature vector set can represent information such as the attribute information, semantic information, syntactic information, and positional layout association of the nodes. The above-mentioned node feature vector extraction can be performed using a graph neural network model for feature vector extraction.

[0069] Fourth, perform template matching processing on the above-mentioned page behavior semantic knowledge graph and the preset front-end page template knowledge base according to the above-mentioned template decision tree and the above-mentioned page node feature vector set, and obtain a page node decision path group set. Among them, the page node decision paths in the above-mentioned page node decision path group set can be the selection paths of at least one front-end code template corresponding to each node in the above-mentioned page behavior semantic knowledge graph. As an example, the above-mentioned execution subject can use the random forest algorithm to perform template matching processing on the page behavior semantic knowledge graph and the preset front-end page template knowledge base according to the template decision tree and the page node feature vector set, and obtain a page node decision path group set.

[0070] Step 5: Sort the page node decision path groups in the above page node decision path group set to obtain a page node decision path sequence set. Among them, the page node decision path sequences in the above page node decision path sequence set can be sorted according to the coincidence degree between the front-end code template and the nodes in the above page behavior semantic knowledge graph, the rule priority in the code generation rule information included in the above preset front-end page template knowledge base, and the historical template usage rate.

[0071] Step 6: Filter the above page node decision path sequence set to obtain a target page node decision path set. Among them, the target page node decision paths in the above target page node decision path set can be the optimal first preset number of decision paths corresponding to the nodes in each page behavior semantic knowledge graph. The above filtering can be performed using the Monte Carlo tree search algorithm. The above preset number can be a pre-set number. For example, the above preset number can be 10.

[0072] Step 7: Determine the front-end code templates corresponding to each leaf node included in the above target page node decision path set as the target front-end code template set of the above page behavior semantic knowledge graph.

[0073] Step 8: According to the combined nesting relationship between the nodes included in the above page behavior semantic knowledge graph, perform template combination nesting on the above target front-end code template set to obtain a template combination nested page code.

[0074] Step 9: According to the code generation rule information set in the above preset front-end page template knowledge base, perform parameter optimization on the above template combination nested page code to obtain an initial front-end page code. Among them, the code generation rule information in the code generation rule information set can be rule information for personalized adjustment and parameter configuration of the front-end code template according to the personalized needs of the front-end page corresponding to the multi-source front-end page design draft set. For example, the code generation rule information can be rule information for personalized adjustment according to the form element types and quantities in the multi-source front-end page design draft and in accordance with the element styles and verification logics in the multi-source front-end page design draft. Parameter optimization can be content matching between the template combination nested page code and the content of the multi-source front-end page design draft to optimize to better meet the requirements of the multi-source front-end page design draft.

[0075] The above technical solution, combined with the steps "Step 106 to Step 109" and their related content, is an inventive point of an embodiment of the present disclosure, which solves the third technical problem mentioned in the background art: "Since low-code platforms usually perform page component matching through string replacement or fixed matching rules, they cannot understand the semantic information in multi-source front-end page design drafts and page behavior semantic knowledge graphs, as well as the complex combinations and nested relationships of multiple front-end page code components for one component in multi-source front-end page design drafts. Low-code platforms cannot model and process the page structure and nested constraint relationships between components, resulting in a mismatch between the generated initial front-end page code and multi-source front-end page design drafts, low code quality, the generated front-end pages not meeting user needs, and a reduced user page experience." If the above factors are solved, the effect of improving in-vehicle data accuracy can be achieved. To achieve this effect, the present disclosure first improves the accuracy of decision tree construction based on a preset front-end page template through a template attribute selection metric set and a template mutual information set. Then, by extracting the visual and semantic features of the page behavior semantic knowledge graph and using a template decision tree, the page node decision path corresponding to the multi-source front-end page design draft is determined, sorted, and filtered. At the same time, through semantic and visual feature information, the accuracy and efficiency of front-end page template selection can be improved. After that, template combination nesting is performed on the target front-end code template set, which can accurately extract the complex nested constraints between component templates, improve the component nested relationship and code representation performance of the template combination nested page code representing the multi-source front-end page design draft. Finally, through detailed optimization of code parameters based on the detailed content of the multi-source front-end page design draft and code adjustment through the page multi-dimensional evaluation numerical set, the performance of the front-end page code can be improved, the matching degree between the generated front page and the design draft can be increased, thereby meeting user needs and improving the user experience. By selecting the optimal front-end code template through a decision tree, components that have been performance-optimized and reusable can be effectively utilized, redundant code can be reduced, and the loading time and memory occupancy of the generated front-end page can be reduced.

[0076] Step 106, according to reinforcement learning and the page behavior semantic knowledge graph, perform code adjustment on the initial front-end page code to obtain the adjusted front-end code.

[0077] In some embodiments, the above-mentioned execution entity may adjust the initial front-end page code according to reinforcement learning and the above-mentioned page behavior semantic knowledge graph to obtain the adjusted front-end code. Among them, the above-mentioned adjusted front-end code may be the front-end code obtained by optimally screening the initial front-end page code and fusing it with the enhanced front-end code generated by the agent in reinforcement learning through the input page behavior semantic knowledge graph. The above-mentioned reinforcement learning may be reinforcement learning including the Double DQN (Double Deep Q-Network) and the prioritized experience replay mechanism algorithm. The state representation of the above-mentioned reinforcement learning may be: the code structure features of the abstract syntax tree corresponding to the initial front-end page code, the graph embedding vector of the semantic association of page elements in the page behavior semantic knowledge graph, and the multi-dimensional state vector of the visual difference features of multi-source design drafts. The above-mentioned semantic association may be a business logic relationship or a user interaction path. The above-mentioned visual difference features may include, but are not limited to, at least one of the following: element position offset, font style attribute difference. The action space of reinforcement learning includes a discrete action space and a continuous action space. The above-mentioned discrete action space may include: layout adjustment, style optimization, and interaction enhancement. The above-mentioned continuous action space may be an action space that maps style values to the [0, 1] interval through normalization processing and supports gradient optimization and fine-grained adjustment. The reward function of the above-mentioned reinforcement learning may include: an immediate reward function based on visual restoration, interaction fluency, performance score, and code complexity, and a long-term reward function that is discounted and accumulated using the Bellman equation. The structure network of the above-mentioned reinforcement learning adopts the architecture of an Actor network and a Critic network. The reinforcement learning agent uses the "performance score" as the reward and autonomously explores a better code structure (such as asynchronously loading non-first-screen components, merging CSS / JS requests, etc.) by interacting with the page behavior semantic knowledge graph environment, thereby significantly improving performance.

[0078] As an example, the above-mentioned page behavior semantic knowledge graph and the above-mentioned initial front-end page code are input into the above-mentioned reinforcement learning to obtain the adjusted front-end code.

[0079] Step 107, according to the currently obtained real-time user interaction behavior information, perform code interaction adjustment on the above-mentioned adjusted front-end code to obtain the front-end interaction code.

[0080] In some embodiments, the above-mentioned execution entity may perform code interaction adjustment on the above-mentioned adjusted front-end code according to the currently obtained current user interaction behavior information to obtain front-end interaction code. Among them, the above-mentioned front-end interaction code may be code that adds the interaction experience between the user and the page on the basis of the above-mentioned adjusted front-end code and optimizes the above-mentioned adjusted front-end code. The above-mentioned current user interaction behavior information may be the behavior information of the user adjusting the style, code structure, and business logic behind the components of the elements in the front-end page. The above-mentioned current user interaction behavior information may include, but is not limited to, at least one of the following: modifying the style parameters of the element, adjusting the code structure, and dynamically adjusting the code generation strategy in an event-driven manner. For example, when it is found that the style of a certain button does not match the design draft, after directly modifying the color and size parameters of the button on the interface, the front-end framework automatically regenerates the relevant code within 0.5 seconds.

[0081] As an example, the above-mentioned execution entity may use the React user interface framework that supports an interactive code generation interface to perform code interaction adjustment on the above-mentioned adjusted front-end code according to the currently obtained current user interaction behavior information to obtain front-end interaction code.

[0082] Step 108, perform multi-dimensional page evaluation on the front-end page corresponding to the above-mentioned front-end interaction code to obtain a multi-dimensional page evaluation value set for the page.

[0083] In some embodiments, the above-mentioned execution entity may perform multi-dimensional page evaluation on the front-end page corresponding to the above-mentioned front-end interaction code to obtain a multi-dimensional page evaluation value set for the page. Among them, the above-mentioned multi-dimensional page evaluation value set may be the result obtained by evaluating the front-end page rendered by the above-mentioned front-end interaction code and the front-end interaction code in multiple dimensions. The above-mentioned multiple dimensions may include, but are not limited to, at least one of the following: front-end page similarity, code performance, and cost interaction performance. The larger the value of the above-mentioned multi-dimensional page evaluation value, the better the performance of the front-end interaction code and the closer the front-end page rendered to the front-end design page in the above-mentioned multi-source front-end page design draft set.

[0084] In some optional implementation manners of some embodiments, the above-mentioned performing multi-dimensional page evaluation on the front-end page corresponding to the above-mentioned front-end interaction code to obtain a multi-dimensional page evaluation value set for the page may include the following steps: First step, determine the Hamming distance of the image pixels between the front-end page corresponding to the above-mentioned front-end interaction code and the front-end design page corresponding to the above-mentioned multi-source front-end page design draft set. Among them, the Hamming distance of the image pixels may represent the degree of difference between the front-end page and the front-end design page. The determination may be performed using the DHash (Difference Hash) algorithm.

[0085] Step 2: Determine the image structure similarity value between the above front-end page and the above front-end design page. The image structure similarity value can measure the similarity between the front-end page and the front-end design page. In practice, the above execution entity can first input the above front-end page and the above front-end design page into the ResNet-101 model to obtain the first front-end page feature vector and the second front-end page feature vector. Then, determine the image structure similarity value (SSIM, Structural Similarity) between the first front-end page feature vector and the second front-end page feature vector.

[0086] Step 3: Determine the page visual evaluation value based on the above image pixel Hamming distance and the above image structure similarity value. The page visual evaluation value can represent the appearance difference degree between the front-end page and the front-end design page in terms of appearance vision.

[0087] As an example, the above execution entity can perform weighted summation on the above image pixel Hamming distance and the above image structure similarity value to obtain the page visual evaluation value.

[0088] Step 4: Perform interactivity detection on the above front-end page to obtain the page interactivity evaluation value. The page interactivity evaluation value can represent the interaction performance and page response performance of the front-end page. The above interactivity detection can be performed through the Selenium automated testing tool and interaction test scripts.

[0089] Step 5: Perform code performance detection on the above front-end page to obtain the page code performance evaluation value. The page performance evaluation value can represent the evaluation of the memory leakage situation, resource loading situation, rendering jank, etc. of the code. The above code performance detection can be performed through a performance analysis tool to evaluate the performance of the above front-end interaction code. The performance analysis tool can be one of the following: Lighthouse tool, Chrome DevTools tool.

[0090] Step 6: Perform code quality evaluation on the above front-end interaction code to obtain the page code quality evaluation value. The page code quality evaluation value can represent the complexity, code duplication rate, and code maintainability of the above front-end interaction code.

[0091] Step 7: Perform sentiment analysis on the obtained user comment information set to obtain a user sentiment tendency information set. Among them, the user sentiment tendency information in the above user sentiment tendency information set can represent the positive, neutral, and negative sentiment tendencies of users towards the front-end page corresponding to the front-end interaction code. The above user evaluation information set can be information on the satisfaction feedback of test users on the generated front-end page collected through online questionnaires and user behavior analysis. The above online questionnaire can be a questionnaire covering 20 questions regarding the aesthetics, usability, functionality, and loading speed of the page.

[0092] Step 8: Determine the user satisfaction evaluation value of the above front-end page according to the above user sentiment tendency information set. Among them, the above user satisfaction evaluation value can be the ratio of the number of tendency information with a positive tendency in the above user sentiment tendency information set to the number of user sentiment tendency information included in the above user sentiment tendency information set. As an example, the above execution entity can first determine the number of at least one user sentiment tendency information with a positive tendency in the above user sentiment tendency information set as the positive sentiment tendency number. Then, determine the number of user sentiment tendency information included in the above user sentiment tendency information set as the sentiment tendency number. Finally, determine the ratio of the above positive sentiment tendency number to the above sentiment tendency number as the user satisfaction evaluation value.

[0093] Step 9: Determine the page multi-dimensional evaluation value set by the above page visual evaluation value, the above page interactivity evaluation value, the above page code performance evaluation value, the above page code quality evaluation value, and the above user satisfaction evaluation value.

[0094] Step 109: Dynamically adjust and optimize the above front-end interaction code according to the above page multi-dimensional evaluation value set.

[0095] In some embodiments, the above execution entity can dynamically adjust and optimize the above front-end interaction code according to the above page multi-dimensional evaluation value set. Among them, the above code dynamic adjustment and optimization can include but are not limited to at least one of the following: the reward weight of reinforcement learning, the adjustment of page layout, the optimization of interaction processes, and the addition of new front-end code templates. As Figure 2 shown, Figure 2 shows a schematic diagram of dynamically adjusting and optimizing the front-end page through code. As Figure 3 shown, it shows a comparison schematic diagram between the code generation framework based on templates in the prior art and the front-end page generated by the present disclosure based on the multi-source front-end page design draft set. As Figure 4 shown, it shows a comparison schematic diagram of the front-end page and code quality in terms of page loading time, memory occupancy, and code maintainability index in the page multi-dimensional evaluation dimension between the prior art and the present disclosure.

[0096] As an example, the above-mentioned execution entity can dynamically adjust and optimize the front-end interaction code in response to determining that any page dimension evaluation value in the above-mentioned page multi-dimensional evaluation value set does not meet the corresponding evaluation condition set. The above-mentioned evaluation condition set can include, but is not limited to, at least one of the following: the page visual evaluation value is greater than or equal to 0.85, the code complexity in the page code quality evaluation value is less than or equal to 8, the code maintainability in the page code quality evaluation value reaches level A or above, and the user satisfaction evaluation value is greater than or equal to 0.8.

[0097] Further reference Figure 5 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a front-end page code generation device. These device embodiments correspond to Figure 1 the method embodiments shown, and the front-end page code generation device can be specifically applied to various electronic devices.

[0098] As Figure 5 shown, a front-end page code generation device 500 includes: an acquisition unit 501, a multi-source data fusion unit 502, an element visual semantics extraction unit 503, a semantic graph construction unit 504, a generation unit 505, a code adjustment unit 506, a code interaction adjustment unit 507, a multi-dimensional page evaluation unit 508, and a code dynamic adjustment and optimization unit 509.

[0099] It can be understood that the units described in the front-end page code generation device 500 correspond to the respective steps in the method described with reference to Figure 1 . Therefore, the operations, features, and beneficial effects described above for the method also apply to the front-end page code generation device 500 and the units included therein, and will not be repeated here.

[0100] Next, reference is made to Figure 6 , which shows a schematic structural diagram of an electronic device (for example, an electronic device) 600 suitable for implementing some embodiments of the present disclosure. Figure 6 The electronic device shown is only an example and should not impose any limitations on the functions and usage scopes of the embodiments of the present disclosure.

[0101] As Figure 6As shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to a program stored in the read-only memory (ROM) 602 or a program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0102] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 the electronic device 600 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 6 Each block shown in the figure may represent a device or, as needed, multiple devices.

[0103] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are executed.

[0104] It should be noted that in some embodiments of the present disclosure, the above-mentioned computer-readable medium may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0105] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0106] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist separately without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to perform the content of the above-mentioned steps 101 - step 109.

[0107] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0109] The units described in some embodiments of the present disclosure may be implemented in software or in hardware. The described units may also be provided in a processor. For example, it may be described as: a processor includes an acquisition unit, a multi-source data fusion unit, an element visual semantic extraction unit, a semantic graph construction unit, a generation unit, a code adjustment unit, a code interaction adjustment unit, a multi-dimensional page evaluation unit, and a code dynamic adjustment and optimization unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the acquisition unit may also be described as "the unit for acquiring a multi-source front-end page design draft set and user historical page browsing behavior information".

[0110] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0111] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, technical solutions formed by mutually replacing the above features with technical features having similar functions (but not limited to) disclosed in the embodiments of the present disclosure.

Claims

1. A method for generating front-end page code, characterized in that, Including: Obtain a multi-source front-end page design draft set and user historical page browsing behavior information; Perform multi-source data fusion on the multi-source front-end page design draft set and the user historical page browsing behavior information to obtain front-end page fusion design information; Extract element visual semantics from the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set; Construct a semantic graph of page behavior knowledge based on the page text element information set and the page design visual element information set for the front-end page fusion design information and a preset page design knowledge base; Generate initial front-end page code based on the page behavior semantic knowledge graph and a preset front-end page template knowledge base; Adjust the initial front-end page code according to reinforcement learning and the page behavior semantic knowledge graph to obtain adjusted front-end code; Perform code interaction adjustment on the adjusted front-end code according to currently obtained real-time user interaction behavior information to obtain front-end interaction code; Conduct multi-dimensional page evaluation on the front-end page corresponding to the front-end interaction code to obtain a multi-dimensional page evaluation value set; Dynamically adjust and optimize the front-end interaction code according to the multi-dimensional page evaluation value set.

2. The method according to claim 1, wherein The performing multi-source data fusion on the multi-source front-end page design draft set and the user historical page browsing behavior information to obtain front-end page fusion design information includes: Convert the format of the multi-source front-end page design draft set to obtain a converted multi-source front-end page design draft set; Perform data preprocessing on the user historical page browsing behavior information to obtain preprocessed user historical page browsing behavior information; Construct a graph structure for the converted multi-source front-end page design draft set and the preprocessed user historical page browsing behavior information to obtain a multi-source front-end page design structure graph set and a user browsing behavior structure graph; Perform data fusion on the multi-source front-end page design structure graph set and the user browsing behavior structure graph to obtain a multi-source fusion node feature vector set; Input the multi-source fusion node feature vector set into a page design anomaly recognition model to obtain a page design anomaly information set; Perform multi-dimensional anomaly data repair processing on the page design anomaly information set to obtain repaired page design information as the front-end page fusion design information.

3. The method according to claim 1, wherein The constructing a semantic graph of page behavior knowledge based on the page text element information set and the page design visual element information set for the front-end page fusion design information and a preset page design knowledge base includes: Determine the page design knowledge graph schema layer of the page behavior semantic knowledge graph; Extract triples from the preset page design knowledge base to obtain a page design triple set; Extract semantic information from the front-end page fusion design information according to the page text element information set and the page design visual element information to obtain a page text visual semantic information set; Extract the dynamic and static association relationship of elements from currently obtained real-time user browsing behavior information to obtain a design element dynamic and static association information set; Perform heterogeneity detection on the page design triple set, the page text visual semantic information set, and the design element dynamic and static association information set to obtain a data heterogeneity value; In response to determining that the data heterogeneity value meets a preset heterogeneity condition, input the page design triple set, the page text visual semantic information set, and the design element dynamic and static association information set into the page design knowledge graph schema layer according to the page text element information set and the page design visual element information to obtain a page behavior semantic knowledge graph.

4. The method according to claim 1, wherein The element visual semantic extraction of the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set includes: Perform text area detection on each multi-source front-end page design draft in the multi-source front-end page design draft set to generate a page text area group and obtain a page text area group set; Input the page text area group set into a page text recognition model to obtain a page text recognition information group set; Extract font style features from the page text area group set to obtain a text font style feature vector group set; Perform semantic and syntactic extraction on the page text recognition information group set to obtain a page text semantic feature vector group set; Input the page text area group set, the text font style feature vector group set, and the page text semantic feature vector group set into a text logical topology model to obtain a text logical layout feature vector set; Determine the text font style feature vector group set, the page text semantic feature vector group set, and the text logical layout feature vector set as the page text element information set, and input the multi-source front-end page design draft set into a page design visual feature extraction model to obtain a page design visual element information set.

5. The method according to claim 1, wherein The multi-dimensional page evaluation of the front-end page corresponding to the front-end interaction code to obtain a page multi-dimensional evaluation value set includes: Determine the image pixel Hamming distance between the front-end page corresponding to the front-end interaction code and the front-end design page corresponding to the multi-source front-end page design draft set; Determine the image structure similarity value between the front-end page and the front-end design page; Determine a page visual evaluation value according to the image pixel Hamming distance and the image structure similarity value; Perform interactivity detection on the front-end page to obtain a page interactivity evaluation value; Perform code performance detection on the front-end page to obtain a page code performance evaluation value; Perform code quality evaluation on the front-end interaction code to obtain a page code quality evaluation value; Perform sentiment analysis on the obtained user comment information set to obtain a user sentiment tendency information set; Determine the user satisfaction evaluation value of the front-end page according to the user sentiment tendency information set; Perform multi-feature fusion on the page visual evaluation value, the page interactivity evaluation value, the page code performance evaluation value, the page code quality evaluation value, and the user satisfaction evaluation value, and determine it as the page multi-dimensional evaluation value set.

6. A front-end page code generation device, characterized in that, Including: An acquisition unit that acquires a multi-source front-end page design draft set and user historical page browsing behavior information; A multi-source data fusion unit that performs multi-source data fusion on the multi-source front-end page design draft set and the user's historical page browsing behavior information to obtain front-end page fusion design information; An element visual semantics unit that extracts element visual semantics from the multi-source front-end page design draft set to obtain a page text element information set and a page design visual element information set; A semantic graph construction unit that constructs a semantic graph of page behavior knowledge based on the page text element information set and the page design visual element information set for the front-end page fusion design information and a preset page design knowledge base; A generation unit that generates initial front-end page code based on the page behavior semantic knowledge graph and a preset front-end page template knowledge base; A code adjustment unit that adjusts the initial front-end page code according to reinforcement learning and the page behavior semantic knowledge graph to obtain adjusted front-end code; A code interaction adjustment unit that performs code interaction adjustment on the adjusted front-end code according to currently obtained real-time user interaction behavior information to obtain front-end interaction code; A multi-dimensional page evaluation unit that performs multi-dimensional page evaluation on the front-end page corresponding to the front-end interaction code to obtain multi-dimensional page evaluation values; A code dynamic adjustment and optimization unit that dynamically adjusts and optimizes the front-end interaction code according to the multi-dimensional page evaluation values.

7. An electronic device, characterized in that, Including: One or more processors; A storage device on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-5.

8. A computer-readable medium, characterized in that, On which a computer program is stored, wherein when the computer program is executed by a processor, the method according to any one of claims 1-5 is implemented.

Images