Webpage code generation method, electronic equipment, readable medium and program product

Abstract

The invention provides a webpage code generation method, electronic equipment, a computer readable medium and a computer program product, and the webpage code generation method comprises the steps: carrying out the preprocessing of a dynamic picture, and generating a frame sequence of the dynamic picture; frame page component features are extracted according to the frame sequence, a dynamic feature sequence is generated, and the frame page component features comprise layout structure features, static component features and dynamic interaction features; performing dynamic interaction feature fusion on the frame page component features according to the dynamic feature sequence to generate a fusion feature sequence; and inputting the fused feature sequence into a code generation model to generate a webpage code. According to the method and the device, the frame sequence of the dynamic picture can be automatically converted into the feature sequence, the webpage code is generated, end-to-end conversion from the dynamic picture to the code is realized, the development efficiency of the webpage is greatly improved, and the technical threshold of webpage development is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method for generating web page code, an electronic device, a computer-readable medium, and a computer program product. Background Technology

[0002] In the software development process, web design and front-end development are usually treated as two independent stages, completed by different roles. Because these two stages are progressive in the process, front-end development can only begin after the web prototype design is completed, resulting in a long overall development cycle and low web development efficiency.

[0003] To improve the efficiency of web development and lower the technical barrier to entry, it is necessary to automatically convert web prototype design images into web code. However, currently, most methods rely on static web images, converting their fixed styles into code, which cannot directly generate interactive code. Therefore, the efficiency of web development needs further improvement. Summary of the Invention

[0004] This disclosure provides a webpage code generation method, an electronic device, a computer-readable medium, and a computer program product, aiming to solve the problem of low webpage development efficiency due to the inability to directly generate interactive webpage code from dynamic webpage images.

[0005] In a first aspect, embodiments of this disclosure provide a webpage code generation method, comprising: preprocessing an animated image to generate a frame sequence of the animated image, the frame sequence being composed of multiple frame pages; extracting frame page component features based on the frame sequence to generate a dynamic feature sequence, wherein the frame page component features include layout structure features, static component features, and dynamic interaction features, and the dynamic feature sequence characterizes the interactive change information of webpage components in the animated image; fusing dynamic interaction features of the frame page component features based on the dynamic feature sequence to generate a fused feature sequence; and inputting the fused feature sequence into a code generation model to generate webpage code, the webpage code including static style code, dynamic interaction code, and style change code.

[0006] In some embodiments, extracting frame page component features based on the frame sequence to generate a dynamic feature sequence includes: dividing the layout structure of a webpage based on the frame sequence, identifying functional area information in the webpage, and obtaining the layout structure features, wherein the functional area information includes function, position, and range; identifying and locating frame page components in the frame sequence, marking the bounding boxes of webpage components in the frame pages and determining the categories of the webpage components, and obtaining static component features; determining the association relationship between webpage components in different frame pages based on the layout structure features and the static component features, determining the dynamic interaction features of the webpage components, and generating the dynamic feature sequence.

[0007] In some embodiments, determining the association relationship between web page components in different frame pages, identifying the dynamic interaction features of the web page components, and generating the dynamic feature sequence includes: matching and tracking web page components in different frame pages to determine the association relationship between web page components in different frame pages, and identifying web page components with association relationships in different frame pages as the same web page component; for the same web page component in different frame pages, performing position change judgment, shape change judgment, and content change judgment to obtain dynamic interaction change information of the web page component; generating a dynamic feature vector of the web page component based on the association relationship and the change information; and generating the dynamic feature sequence based on the dynamic feature vector.

[0008] In some embodiments, for the same webpage component in different frame pages, position change judgment, shape change judgment, and content change judgment are performed to obtain dynamic interaction change information of the webpage component, including: calculating the Euclidean distance between the center coordinates of the bounding boxes of the same webpage component in adjacent frame pages, and determining that the position of the webpage component has changed when the Euclidean distance changes; calculating the relative rate of change of the bounding box area of ​​the same webpage component in different frame pages, and determining that the shape of the webpage component has changed when the relative rate of change is not 0; calculating the structural similarity index of the same webpage component in different frame pages, and determining that the content of the webpage component has changed when the structural similarity index is less than a preset threshold.

[0009] In some embodiments, the frame page component features are dynamically interacted and fused according to the dynamic feature sequence to generate a fused feature sequence, including: standardizing the layout structure features, the static component features, and the dynamic interaction features to unify feature dimensions and formats; and performing feature fusion on the layout structure features, the static component features, and the dynamic interaction features based on a three-layer attention mechanism to generate the fused feature sequence.

[0010] In some embodiments, the layout structure features, the static component features, and the dynamic interaction features are standardized to unify feature dimensions and formats, including: vectorizing the static component features to generate static feature vectors; vectorizing the layout structure features to generate layout feature vectors; and using the dynamic feature sequence as a dynamic feature vector representing the dynamic interaction features.

[0011] In some embodiments, feature fusion is performed on the layout structure features, static component features, and dynamic interaction features based on a three-layer attention mechanism to generate the fused feature sequence. This includes: matching the position and category of the static component with the layout structure based on the static feature vector and the layout feature vector to achieve spatial attention fusion and generate a spatial fused feature sequence, where the spatial fused feature sequence represents the layout features of the static component; determining the temporal correlation of the dynamic interaction features of different frame pages based on the dynamic feature sequence, performing temporal attention fusion on the dynamic interaction features of the different frame pages based on the temporal correlation to generate a temporal fused feature sequence, where the temporal fused feature sequence represents the dynamic temporal features; and assigning matching weights between the position and category of the static component and the layout structure based on the dynamic component features of the component after temporal attention fusion to achieve dynamic-static attention fusion and generate a multimodal fused feature sequence.

[0012] In some embodiments, the method of inputting the fused feature sequence into the code generation model to generate web page code includes: inputting the static component layout features and the dynamic temporal features in the multimodal fused feature sequence into an encoder to generate a global context vector; generating the static style code, the dynamic interaction code, and the style change code in a decoder based on the global context vector; obtaining the code generated by the decoder; embedding the static style code, the dynamic interaction code, and the style change code into corresponding tag regions to generate initial web page code; verifying and correcting the initial web page code; and outputting optimized web page code.

[0013] In some embodiments, preprocessing a dynamic image to generate a frame sequence of the dynamic image includes: normalizing multiple frame pages to make the multiple frame pages have the same size; performing contour enhancement processing on the multiple normalized frame pages; and generating the frame sequence from the multiple contour-enhanced frame pages.

[0014] In a second aspect, embodiments of this disclosure provide an electronic device, including: one or more processors; and a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the web page code generation method described in the first aspect of this disclosure.

[0015] Thirdly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the webpage code generation method described in the first aspect of embodiments of this disclosure.

[0016] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the webpage code generation method described in the first aspect of this disclosure.

[0017] In this embodiment, dynamic images are automatically split into frame sequences. Based on these frame sequences, layout structure features, static component features, and dynamic interaction features of the frame page components are extracted, providing a precise dynamic logic basis for webpage code generation. Then, the layout structure features, static component features, and dynamic interaction features are fused spatially and temporally to generate a fused feature sequence recognizable by the code generation model. Finally, based on the fused feature sequence, the code generation model automatically generates complete webpage code containing the layout structure features, static component features, and dynamic interaction features of the dynamic images. This achieves end-to-end conversion from dynamic images to code, significantly improving webpage development efficiency and lowering the technical threshold for webpage development. Furthermore, through precise analysis of dynamic logic, the interactive effects of the generated webpage code better match design expectations, improving the overall quality of webpage development. Attached Figure Description

[0018] Figure 1 This is a flowchart of a webpage code generation method according to an embodiment of this disclosure.

[0019] Figure 2 This is a flowchart of a webpage code generation method according to an embodiment of this disclosure.

[0020] Figure 3 This is a flowchart of the feature fusion based on a three-layer attention mechanism in this embodiment of the disclosure.

[0021] Figure 4 This is a schematic diagram of the composition of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of this disclosure will be described in detail below with reference to the accompanying drawings.

[0023] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0024] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0025] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0027] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0028] The inventors discovered that directly generating webpage code from webpage prototype design images would significantly improve webpage development efficiency and lower the technical barrier, bringing substantial advantages to webpage development. However, current methods for directly generating webpage code are typically based on static images, only converting fixed styles into webpage code. They cannot directly convert dynamic images, which include interactive elements and style changes common in most webpages, into dynamic interactive webpage code.

[0029] In view of this, this disclosure provides a webpage code generation method. It automatically splits dynamic images into frame sequences, extracts layout structure features, static component features, and dynamic interaction features of frame page components based on the frame sequences of the dynamic images, providing accurate dynamic logic for webpage code generation. Based on a three-layer attention mechanism, it performs feature fusion of layout structure features, static component features, and dynamic interaction features from both spatial and temporal perspectives, generating a fused feature sequence recognizable by the webpage code generation module. Finally, it automatically generates complete dynamic interactive webpage code based on the fused feature sequence through a code generation model, greatly improving webpage development efficiency.

[0030] Figure 1 This is a flowchart of a webpage code generation method according to an embodiment of this disclosure. Figure 2 This is a flowchart of a webpage code generation method according to an embodiment of this disclosure.

[0031] Firstly, referring to Figure 1 This disclosure provides a webpage code generation method, including: S1. Preprocess the animated image to generate a frame sequence of the animated image, wherein the frame sequence consists of multiple frame pages; S2. Extract frame page component features based on the frame sequence to generate a dynamic feature sequence, wherein the frame page component features include layout structure features, static component features, and dynamic interaction features, and the dynamic feature sequence characterizes the interactive change information of web page components in the dynamic image; S3. Based on the dynamic feature sequence, the frame page component features are dynamically interacted and fused to generate a fused feature sequence; S4. Input the fused feature sequence into the code generation model to generate web page code, wherein the web page code includes static style code, dynamic interactive code, and style change code.

[0032] In this embodiment, feature extraction can only be performed after the animated image is segmented into a frame sequence. The frame sequence is generated by processing the animated image using image processing tools. For example, `cv2.VideoCapture()` loads a GIF file, `cap.get()` retrieves the total number of frames (60), and `ret, frame = cap.read()` reads each frame sequentially. Each frame is converted from BGR to RGB format and then saved as a single-frame PNG using `cv2.imwrite()`, from "store_frame_00.png" to "store_frame_59.png". Each frame in the resulting animated image frame sequence is a micro-static image. Each frame contains web page components, which include three types of feature information: static component features, layout structure features, and dynamic interaction features. Image processing tools include tools from the OpenCV library, etc., and this disclosure does not specifically limit their use.

[0033] In this embodiment of the disclosure, a micro-static image refers to an image that primarily presents a static image while also containing micro-dynamic elements. Static component characteristics refer to the business-meaning component categories and border sizes within a frame page component, such as search buttons, input boxes, images, and their border sizes. Layout structure characteristics refer to the position and area of ​​each static component, such as navigation bars, content areas, sidebars, and footers. Dynamic interaction characteristics refer to the dynamic changes in the position, shape, and content of the same component, such as the appearance of a pop-up window after clicking a specific button, or the expansion of a menu after clicking a drop-down menu, reflecting the dynamic changes in different states before and after webpage interaction.

[0034] In this embodiment, a frame page component feature extraction module extracts key information such as layout structure features, static component features, and dynamic interaction features from dynamic images based on computer vision and deep learning technologies. (Refer to...) Figure 2Based on the frame sequence obtained by the dynamic image preprocessing module, the business meaning and position layout of the static components of each frame page in the frame sequence are identified, and then the relationship between the components of each frame page and the dynamic interaction features are extracted.

[0035] In this embodiment of the disclosure, reference is made to Figure 2 The webpage code generation method described in this disclosure segments dynamic images into multiple frame sequences through a dynamic image preprocessing module; identifies the webpage components of each frame page and their three feature information: static component features, layout structure features, and dynamic interaction features through a frame page component feature extraction module; fuses the three features of the frame page components based on a three-layer attention mechanism through a component feature fusion module, clarifying the business meaning of each component, the layout structure features of each component in the webpage, and the relationship and dynamic interaction features between the components of each frame page, and generates a feature sequence based on this; finally, the code generation model in the webpage code generation module automatically generates complete webpage code containing the layout structure features, static component features, and dynamic interaction features of the dynamic images. By achieving end-to-end conversion from dynamic images to code through full-process automation, the method significantly improves webpage development efficiency, lowers the technical threshold for webpage development, and, through precise parsing of dynamic logic, makes the interactive effects of the generated webpage code more in line with design expectations, thereby improving the overall quality of webpage development.

[0036] In some embodiments, preprocessing a dynamic image to generate a frame sequence of the dynamic image includes: normalizing multiple frame pages to make the multiple frame pages have the same size; performing contour enhancement processing on the multiple normalized frame pages; and generating the frame sequence from the multiple contour-enhanced frame pages.

[0037] In this embodiment of the disclosure, reference is made to Figure 2 The dynamic image preprocessing module normalizes the size of dynamic images, scaling them to a fixed size, such as 1920×1080 pixels, to ensure the consistency of feature vector dimensions for subsequent frame page components, thereby improving the efficiency of subsequent modules. Gradient operators, such as Sobel and Prewitt, are used to perform edge detection and contour enhancement on the scaled-to-fixed-size dynamic images, strengthening the contour features of components such as search buttons and input boxes, facilitating component recognition by the subsequent frame page component feature extraction module.

[0038] In some embodiments, extracting frame page component features based on the frame sequence to generate a dynamic feature sequence includes: dividing the layout structure of a webpage based on the frame sequence, identifying functional area information in the webpage, and obtaining the layout structure features, wherein the functional area information includes function, position, and range; identifying and locating frame page components in the frame sequence, marking the bounding boxes of webpage components in the frame pages and determining the categories of the webpage components, and obtaining static component features; determining the association relationship between webpage components in different frame pages based on the layout structure features and the static component features, determining the dynamic interaction features of the webpage, and generating the dynamic feature sequence.

[0039] In this embodiment of the disclosure, based on the frame sequence of the dynamic image, the layout structure features, static component features, and dynamic interaction features of the web page components in the frame sequence are extracted. The method also includes: dividing the layout structure of the web page components using an image segmentation model; identifying the segmented regions, which include the navigation bar, content area, sidebar, and footer; and determining the position and range of each segmented region to obtain the layout structure features.

[0040] In this embodiment of the disclosure, the layout structure regions of web page components are identified by an image segmentation model, and the location and extent of each region are determined. For example, the layout regions such as the "navigation bar" (coordinates (0,0,1920,100)) and the "content area" (coordinates (0,100,1920,900)) are identified by the image segmentation model, and layout feature vectors are generated. Layout feature vectors This disclosure includes a layout structure feature representation of web page components. It does not specifically limit the type of image segmentation model; for example, it can be U-Net or other image segmentation models.

[0041] In this embodiment of the disclosure, based on the frame sequence of the dynamic image, the layout structure features, static component features, and dynamic interaction features of the frame page components in the frame sequence are extracted. The method further includes: detecting web page components in the frame sequence using object detection technology; and obtaining the bounding boxes of the web page components. The categories of web page components include search buttons, input boxes, and images. For frame index, For component indexing, and These are the coordinates of the top-left corner and the bottom-right corner of the bounding box, respectively.

[0042] In this embodiment, object detection technology is used to perform object detection frame by frame in each frame sequence to obtain the category and bounding box coordinates of static components in each web page component. For example, the YOLOv8 model is used to perform object detection on each frame sequence to identify the "Add to Cart button" (category: button, bounding box coordinates (800, 800, 850, 850) in frame 0) and the "Product Type List Area" (category: list, bounding box coordinates (200, 200, 400, 800) in frame 0), generating a static feature vector S. The static feature vector S contains the static component feature representation of the web page component. This disclosure does not specifically limit the type of object detection technology; for example, it can be the YOLOv8 model or other object detection technologies.

[0043] In some embodiments, determining the association relationship between frame page components in different frame pages, determining the dynamic interaction change information of the web page components, and generating the dynamic feature sequence includes: matching and tracking web page components in different frame pages to determine the association relationship between web page components in different frame pages, and identifying web page components with association relationships in different frame pages as the same web page component; for the same web page component in different frame pages, performing position change judgment, shape change judgment, and content change judgment to obtain the dynamic interaction change information of the web page component; generating a dynamic feature vector of the web page component based on the association relationship and the change information; and generating the dynamic feature sequence based on the dynamic feature vector.

[0044] In this embodiment, after detecting the bounding box coordinates of each webpage component using object detection technology, the Hungarian algorithm is used to calculate the matching degree of dynamic components between frames. The matching degree is calculated based on the bounding box IOU and category similarity, thereby establishing the correspondence between components across frames. For example, the bounding box of the "Add to Cart" button in frames 10-29 has an IOU > 0.8 with the bounding box of the same button in frames 0-9, and is therefore determined to be the same component; the bounding box of the "Product Type" pop-up in frames 30-45 has a matching cost < 0.2 with the bounding box of the "Product Type" selection text in frames 0-29, and is therefore determined to be an associated component.

[0045] In some embodiments, for the same webpage component in different frame pages, position change judgment, shape change judgment, and content change judgment are performed to obtain dynamic interaction change information of the webpage component, including: calculating the Euclidean distance between the center coordinates of the bounding boxes of the same webpage component in adjacent frame pages, and determining that the position of the webpage component has changed when the Euclidean distance changes; calculating the relative rate of change of the bounding box area of ​​the same webpage component in different frame pages, and determining that the shape of the webpage component has changed when the relative rate of change is not 0; calculating the structural similarity index of the same webpage component in different frame pages, and determining that the content of the webpage component has changed when the structural similarity index is less than a preset threshold.

[0046] In this embodiment, after identifying the category, coordinates, and layout structure features of static components using image segmentation models and object detection techniques, cross-frame matching and tracking, as well as dynamic change analysis, are performed on dynamic components to determine the association relationships and dynamic interaction logic between page components in each frame. The association relationship refers to whether two components are the same component, and the dynamic interaction logic refers to the changes in the position, shape, and content of the dynamic components. For each component in each frame of the frame sequence, a corresponding dynamic feature vector is generated based on the matching and tracking and dynamic change analysis results. Dynamic feature vectors Includes feature representations of dynamic components in a single frame or a single state, where, , This represents the position, shape, and content characteristics of the frame page component in the Kth frame of the frame sequence. The frame page components containing the frame sequence have characteristic information such as position, shape, and content.

[0047] In this embodiment of the disclosure, a time-series model is used based on dynamic feature vectors. The system extracts the position, shape, and content features of the frame page components in each frame of the frame sequence. It also extracts features such as trigger conditions and change patterns of dynamic components. Trigger conditions include actions like clicking and hovering, while change patterns include carousel speed and pop-up appearance methods. Based on the extracted trigger conditions and change patterns, a dynamic feature sequence is output. Dynamic feature sequence It integrates the temporal features of the entire dynamic image change process of a moving image, whereby... , This represents the position, shape, and content characteristics of the frame page components in the first k frames of the frame sequence. It incorporates the feature vector of the current frame. Together with historical frame feature information, it can reflect the evolution of dynamic components over time.

[0048] In this embodiment of the disclosure, the Hungarian algorithm is used to match and track web page components, establishing a correspondence between components across frames, that is, for the first frame... Frame (i.e., frame) Component i and the first in ) Frame (i.e., frame) Component j in ) is assumed to be in frame and frame There is a matching relationship in it. This indicates that component i in frame k and component i in frame k+1 are... These are the same component or related components. For example, the IOU (Intersection over Union) of the bounding box of the "Add to Cart" button in frames 10-29 and the bounding box of the "Add to Cart" button in frames 0-9. If they are the same component, then the matching relationship is determined. Let the bounding box of component i and the bounding box of component j be... For example, if the IOU of the bounding box of the "Product Type" pop-up in frames 30-45 and the bounding box of the "Product Type" selection text in frames 0-29 is less than 0.2, then they are determined to be related components, and the matching relationship is as follows. Let the bounding box of component i and the bounding box of component j be... .

[0049] In this embodiment of the disclosure, when it is determined that component i in frame k and component j in frame k+1 are the same component, the Euclidean distance between the center coordinates of the bounding box of component i is calculated. Euclidean distance from the center coordinates of the bounding box of component j ,like If the position of component i in frame k has changed compared to the position of component j in frame k+1, then the Euclidean distance is calculated using the following formula (1): (1) For frames The center coordinates of the component.

[0050] For example, the Intersection over Union (IOU) of the bounding box of component i in frame 9 and the bounding box of component j in frame 10. The components are identified as the same, for example, the YOLOv8 model detects that the component is the "Add to Cart" button; according to formula (1) and the center coordinates of the "Add to Cart" button in frame 9 detected by the YOLOv8 model. Center coordinates of the "Add to Cart" button in frame 10 Calculate the Euclidean distance between the "Add to Cart" button in frame 9 and the button in frame 10. If the two are equal, it is determined that the position of the "Add to Cart" button in frames 9 and 10 has not changed.

[0051] In this embodiment of the disclosure, when it is determined that component i in frame k and component j in frame k+1 are the same component, the relative rate of change of the bounding box area of ​​component i and component j is calculated. ,like If the shape of component i in frame k changes compared to the shape of component j in frame k+1, then when... When a change occurs, it is determined that the component has undergone a shape change. The relative rate of change of the bounding box area is calculated using the following formula (2): (2) For frames The bounding box area of ​​the middle component.

[0052] For example, the Intersection over Union (IOU) of the bounding box of component i in frame 19 and the bounding box of component j in frame 29. The components were identified as the same component. The YOLOv8 model was used to detect the category and border coordinates of the component, which was identified as the "Add to Cart" button. The area of ​​the "Add to Cart" button in frames 19 and 29 was calculated. For example, the area of ​​the "Add to Cart" button in frame 19 was 5000px² and the area in frame 29 was 8400px². The area was calculated according to formula (2). If the shape of the "Add to Cart" button changes in frames 19 and 29, and the area calculated indicates that the change is an enlargement, then the "Add to Cart" button has changed shape in frames 19 and 29.

[0053] In this embodiment of the disclosure, when component i in frame k and component j in frame k+1 are determined to be the same component, the structural similarity index of the region images of component i and component j is calculated using PyTorch MS-SSIM. ,when ( When the content change threshold (usually set to 0.8) is used, it is determined that the content of component i in frame k has changed compared to the content of component j in frame k+1. For example, the Intersection over Union (IOU) of the bounding box of component i in frame 46 and the bounding box of component j in frame 53. The components are identified as the same; the YOLOv8 model detects that this component is a "product image". Using PyTorchMS – SSIM, the structural similarity index (SSIM) of the "product image" component region in frame 46 is calculated to be 0.95, and the SSIM of the "product image" component region in frame 53 is calculated to be 0.72. The preset threshold T_SSIM is 0.8. Since the SSIM of the "product image" in frame 53 is 0.72, which is less than 0.8, it can be determined that the content has changed. The YOLOv8 model can detect the switch to a "product side view" in frame 53. For example, the IOU (Intersection over Union) between the bounding box of component i in frame 46 and the bounding box of component j in frame 59 is also considered. The components are identified as the same; the YOLOv8 model detects that the component is a "product image". The structural similarity index (SSIM) of the "product image" component region in frame 46 is calculated to be 0.95 using PyTorch MS –SSIM, and the structural similarity index (SSIM) of the "product image" component region in frame 59 is calculated to be 0.68. The preset threshold T_SSIM is 0.8. The SSIM of the "product image" in frame 59 is 0.68, which is less than 0.8, indicating that the content has changed. The YOLOv8 model can detect that the component has switched to "product detail image" in frame 59.

[0054] In some embodiments, dynamic interaction feature fusion is performed on frame page component features based on dynamic feature sequence to generate fused feature sequence, including: standardizing layout structure features, static component features, and dynamic interaction features to unify feature dimensions and formats; and performing feature fusion on layout structure features, static component features, and dynamic interaction features based on a three-layer attention mechanism to generate fused feature sequence.

[0055] In this embodiment, the component feature extraction module extracts three types of feature information: layout structure features, static component features, and dynamic interaction features. These three types of feature information include image type, text type, etc. These three types of feature information involve time and / or spatial dimensions, and each type of feature information has different sources, meanings, and levels of importance. Therefore, a component feature fusion module is needed to perform feature fusion based on a three-layer attention mechanism to obtain a feature sequence that the webpage code generation module can recognize.

[0056] In some embodiments, the layout structure features, static component features, and dynamic interaction features are standardized to unify feature dimensions and formats, including: vectorizing the static component features to generate static feature vectors; vectorizing the layout structure features to generate layout feature vectors; and using the dynamic feature sequence as a dynamic feature vector representing the dynamic interaction features. In this embodiment of the disclosure, text features in the static component features are converted into text feature vectors using word embedding; spatial features (i.e., the bounding box of the static component) are obtained by normalizing the bounding box coordinates of the static component to obtain coordinate feature vectors, and the text feature vectors and coordinate feature vectors of the same static component are concatenated to form a static feature vector. The layout structure features are normalized to obtain coordinate feature vectors. The hierarchical relationships in the layout structure features are transformed into hierarchical relationship vectors using an adjacency matrix. Hierarchical relationships refer to the positional relationships between two layout structures. For example, if the area boxes of the sidebar and navigation bar do not overlap (i.e., the corresponding position in the adjacency matrix is ​​0), they are determined to be at the same level, and a corresponding hierarchical relationship vector is generated. The coordinate feature vectors and hierarchical relationship vectors are concatenated to form the layout feature vector. Dynamic feature sequences It can itself serve as a dynamic feature vector that preserves temporal sequence.

[0057] In some embodiments, feature fusion is performed on the layout structure features, static component features, and dynamic interaction features based on a three-layer attention mechanism to generate the fused feature sequence. This includes: matching the position and category of the static component with the layout structure based on the static feature vector and the layout feature vector to achieve spatial attention fusion and generate a spatial fused feature sequence, where the spatial fused feature sequence represents the layout features of the static component; determining the temporal correlation of the dynamic interaction features of different frame pages based on the dynamic feature sequence, performing temporal attention fusion on the dynamic interaction features of the different frame pages based on the temporal correlation to generate a temporal fused feature sequence, where the temporal fused feature sequence represents the dynamic temporal features; and assigning matching weights between the position and category of the static component and the layout structure based on the dynamic temporal features after temporal attention fusion to achieve dynamic-static attention fusion and generate a multimodal fused feature sequence.

[0058] In the embodiments disclosed herein, such as Figure 3 As shown, the process of generating a multimodal fused feature sequence includes the following three layers of attention fusion: (1) First layer: Spatial attention fusion The first layer of spatial attention fusion is based on the spatial attention fusion mechanism, which integrates static feature vectors. and layout feature vectors The spatial fusion is performed to form a spatial fusion feature sequence. Able to enhance static feature vectors With layout feature vector Spatial relevance. When designing a webpage, the position and layout area of ​​components within the webpage need to be reasonably set according to the business meaning of the component elements. Static feature vectors With layout feature vector While the information and business meaning contained in feature extraction may differ, the spatial dimension remains the same. Static feature vectors can be used... Position, category, and layout feature vector of static components The layout structure is matched accordingly. For example, "the 'Add Product' button must be in the product list area," strengthening the spatial relationship between static components and the layout structure and achieving spatial integration. Here, the "Add Product" button represents the category of the static component, i.e., its business meaning; the product list area represents the layout structure.

[0059] Specifically, using layout feature vectors Calculate the static eigenvector as a "spatial reference". Spatial overlap between each component element and layout features Based on spatial overlap Spatial attention weights are generated using the following formula (3). : (3) in, It is the dimension of the feature vector. It is the first A static component, It is the first Static feature vectors of static components It is the layout feature vector of the layout region.

[0060] Based on spatial attention weights The spatial fusion feature sequence of the first layer is obtained. The first number is obtained through the following formula (4). Spatial fusion characteristics of static components: (4) (2) Second layer: Temporal attention fusion The second layer of temporal attention fusion is based on the temporal attention mechanism, which is used to process dynamic feature sequences. Temporal feature fusion is performed to enhance the temporal correlation between dynamic features. The component feature extraction module extracts dynamic feature sequences. It represents the dynamic change characteristics of components across all frames in a frame sequence. It can show whether components change in different frames (i.e., at different times), but it cannot show the temporal order of changes or business relationships. For different frames (i.e., at different times), the changes of the components of interest are different. We can use temporal attention to capture cross-frame dependencies and mine the information of interest in the current frame.

[0061] By employing the self-attention mechanism of Transformer, dynamic feature sequences are... Each of them Treating it as a "Query", all (j = 1, 2, ..., t) serve as the "key" and "value". The "query" is used to locate the information of interest in the current frame, the "key" is used to match content related to the query, and the "value" is used to fuse feature information.

[0062] Specifically, with Calculate when querying and The degree of association (as a key) is obtained by formula (5) as follows. (As a key) for the current Importance weight, i.e., time attention weight : (5) Based on time attention weight ,right (As values) are weighted and summed to obtain dynamic temporal features that incorporate dynamic information related to themselves from other frames, resulting in the second-layer temporal fusion feature sequence. The dynamic features of the k-th frame with enhanced temporal correlation are obtained through the following formula (6): (6) in, It is the dimension of the feature vector; This is the time decay coefficient; the weight of historical frames that are further away from the current frame is lower. , , It is the parameter matrix used for attention calculation during feature fusion.

[0063] (3) Third layer: Dynamic-static attention fusion The first layer of spatial attention fusion completes the spatial feature fusion of static components and layout structures. The second layer of temporal attention fusion completes the temporal feature fusion of dynamic interaction features. The third layer of dynamic-static attention fusion assigns matching weights to static components and layout structures based on the temporal features of dynamic interaction features, thereby completing multimodal feature fusion and realizing dynamic-static attention fusion.

[0064] Specifically, the first time series features fused together Frame dynamic features For example, As a query, a spatially fused feature sequence with static-layout characteristics As "keys and values", the cross-modal attention weights are obtained through the following formula (7). : (7) Finally, the fusion result of the third layer is obtained, that is, the result of the fusion of the third layer is obtained by the following formula (8). The frame contains the final fused features of dynamic logic and spatial constraints: (8) The final output is a multimodal fused feature sequence. .

[0065] In some embodiments, the method of inputting the fused feature sequence into the code generation model to generate web page code includes: inputting the static component layout features and the dynamic temporal features in the multimodal fused feature sequence into an encoder to generate a global context vector; generating the static style code, the dynamic interaction code, and the style change code in a decoder based on the global context vector; obtaining the code generated by the decoder; embedding the static style code, the dynamic interaction code, and the style change code into corresponding tag regions to generate initial web page code; verifying and correcting the initial web page code; and outputting optimized web page code.

[0066] Secondly, referring to Figure 4 This disclosure provides an electronic device, which includes: One or more processors 401; The memory 402 stores one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the web page code generation method described in the first aspect of the present disclosure. One or more I / O interfaces 403 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0067] Among them, processor 401 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 402 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 403 is connected between processor 401 and memory 402, and can realize information interaction between processor 401 and memory 402, including but not limited to data bus (Bus).

[0068] In some embodiments, the processor 401, memory 402, and I / O interface 403 are interconnected via bus 404, and thus connected to other components of the computing device.

[0069] Thirdly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the webpage code generation method described in the first aspect of embodiments of this disclosure.

[0070] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the webpage code generation method described in the first aspect of this disclosure.

[0071] Fifthly, embodiments of this disclosure provide a webpage code generation apparatus, comprising: The dynamic image preprocessing module is configured to preprocess dynamic images to generate a frame sequence of the dynamic images, wherein the frame sequence consists of multiple frame pages; The frame page component feature extraction module is configured to extract frame page component features based on the frame sequence and generate a dynamic feature sequence, wherein the frame page component features include layout structure features, static component features, and dynamic interaction features, and the dynamic feature sequence characterizes the interactive change information of web page components in the dynamic image; The dynamic interaction feature fusion module is configured to perform dynamic interaction feature fusion on the features of the frame page component based on the dynamic feature sequence to generate a fused feature sequence. The webpage code generation module is configured to input the fused feature sequence into the code generation model to generate webpage code, which includes static style code, dynamic interactive code, and style change code.

[0072] In some embodiments, the dynamic image preprocessing module is further configured to normalize the plurality of frame pages so that the plurality of frame pages have the same size; perform contour enhancement processing on the plurality of frame pages after normalization; and generate the frame sequence from the plurality of frame pages after contour enhancement processing.

[0073] In some embodiments, the frame page component feature extraction module is further configured to: divide the layout structure of the webpage according to the frame sequence; identify functional area information in the webpage to obtain the layout structure features, wherein the functional area information includes function, position, and range; identify and locate the frame page components in the frame sequence; mark the bounding boxes of the webpage components in the frame page and determine the category of the webpage components to obtain static component features; and determine the association relationship of webpage components in different frame pages according to the layout structure features and the static component features, determine the dynamic interaction features of the webpage, and generate the dynamic feature sequence.

[0074] In some embodiments, the frame page component feature extraction module is further configured to match and track web page components in different frame pages, determine the association relationship between web page components in different frame pages, identify the web page components with the same relationship in different frame pages as the same web page component; for the same web page component in different frame pages, perform position change judgment, shape change judgment, and content change judgment to obtain dynamic interaction change information of the web page component; generate a dynamic feature vector of the web page component based on the association relationship and the change information; and generate the dynamic feature sequence based on the dynamic feature vector.

[0075] In some embodiments, the frame page component feature extraction module is further configured to calculate the Euclidean distance between the center coordinates of the bounding boxes of the same web page component in adjacent frame pages, and determine that the position of the web page component has changed when the Euclidean distance changes; calculate the relative rate of change of the bounding box area of ​​the same web page component in different frame pages, and determine that the shape of the web page component has changed when the relative rate of change is not 0; calculate the structural similarity index of the same web page component in different frame pages, and determine that the content of the web page component has changed when the structural similarity index is less than a preset threshold.

[0076] In some embodiments, the dynamic interaction feature fusion module is further configured to standardize the layout structure features, the static component features, and the dynamic interaction features to unify the feature dimensions and formats; and to perform feature fusion on the layout structure features, the static component features, and the dynamic interaction features based on a three-layer attention mechanism to generate the fused feature sequence.

[0077] In some embodiments, the dynamic interaction feature fusion module is further configured to vectorize the static component features to generate a static feature vector; vectorize the layout structure features to generate a layout feature vector; and use the dynamic feature sequence as a dynamic feature vector characterizing the dynamic interaction features.

[0078] In some embodiments, the dynamic interaction feature fusion module is further configured to: match the position and category of the static component with the layout structure based on the static feature vector and the layout feature vector to achieve spatial attention fusion and generate a spatial fusion feature sequence, wherein the spatial fusion feature sequence represents the layout features of the static component; determine the temporal correlation of the dynamic interaction features of different frame pages based on the dynamic feature sequence; perform temporal attention fusion on the dynamic interaction features of the different frame pages based on the temporal correlation to generate a temporal fusion feature sequence, wherein the temporal fusion feature sequence represents the dynamic temporal features; and assign matching weights between the position and category of the static component and the layout structure based on the dynamic component features of the component after temporal attention fusion to achieve dynamic-static attention fusion and generate a multimodal fusion feature sequence.

[0079] In some embodiments, the webpage code generation module is further configured such that the dynamic interaction feature fusion module is further configured to input the static component layout features and the dynamic temporal features in the multimodal fusion feature sequence into the encoder to generate a global context vector; based on the global context vector, generate the static style code, the dynamic interaction code, and the style change code in the decoder respectively; obtain the code generated by the decoder; embed the static style code, the dynamic interaction code, and the style change code into corresponding tag regions respectively to generate initial webpage code; verify and correct the initial webpage code, and output the optimized webpage code.

[0080] To enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of this disclosure, the technical solutions provided by the embodiments of this disclosure will be described in detail below through specific embodiments: Example 1 Reference Figure 2 This disclosure provides a method for generating webpage code, the specific process of which is as follows: 1) Animated Image Preprocessing Module: This module preprocesses animated images to generate a frame sequence of multiple frame pages. Specifically, it includes... a. Perform size normalization and contour enhancement on frame pages: Scale all frame pages to the same size, and use the gradient operator Prewitt to perform contour enhancement on the frame pages scaled to the same size; b. Generate frame sequence: Use tools in the OpenCV library to generate a frame sequence of dynamic images from the frame pages that have undergone contour enhancement processing.

[0081] 2) Frame Page Component Feature Extraction Module: Extracts frame page component features from the frame sequence and generates a dynamic feature sequence. The frame page component features include layout structure features, static component features, and dynamic interaction features. The dynamic feature sequence characterizes the interactive changes of web page components in dynamic images, specifically including... a. Using a trained image segmentation model, the overall layout structure of the webpage is divided according to the frame sequence, and the functional area information such as navigation bar, content area, sidebar, and footer is identified to obtain layout structure features. The functional area information includes function, location, and range. b. Using a trained YOLOv8 model capable of recognizing and locating static component elements, frame page components in the frame sequence are identified and located, the bounding boxes of web page components in the frame page are marked and the categories of the web page components are determined to obtain static component features. c. Match and track web page components in different frame pages, determine the relationship between web page components in different frame pages, and identify related web page components in different frame pages as the same web page component; d. Calculate the Euclidean distance between the center coordinates of the bounding boxes of the same web page component in adjacent frames using formula (1). When the Euclidean distance changes, it is determined that the position of the web page component has changed. e. Use formula (2) to calculate the relative rate of change of the bounding box area of ​​the same web page component in different frame pages. When the relative rate of change is not 0, it is determined that the shape of the web page component has changed. f. Calculate the structural similarity index of the same web page component in different frames using PyTorch MS - SSIM. When the structural similarity index is less than a preset threshold, it is determined that the content of the web page component has changed. g. By judging the position change, shape change, and content change of the same web page component in different frame pages, the dynamic interaction change information of the web page component is obtained. Based on the above-mentioned correlation and change information, the dynamic feature vector of the web page component is generated, and the dynamic feature sequence is generated based on the dynamic feature vector.

[0082] 3) Component Dynamic Interaction Feature Fusion Module: This module fuses the dynamic interaction features of frame page components based on the dynamic feature sequence, generating a fused feature sequence. Specifically, it includes... a. Standardize the layout structure features, static component features, and dynamic interaction features extracted in the above process to obtain the static feature vector. Layout feature vectors and dynamic feature sequences ; b. Based on the static feature vector and the layout feature vector, the position, category and layout structure of the static component are matched to achieve spatial attention fusion and generate a spatial fusion feature sequence, which represents the layout features of the static component; c. Based on the dynamic feature sequence, determine the temporal correlation of the dynamic interaction features of different frame pages, perform temporal attention fusion on the dynamic interaction features of different frame pages based on the temporal correlation, and generate a temporal fusion feature sequence, which represents the dynamic temporal features. d. Based on the dynamic component features of the components after temporal attention fusion, assign matching weights to the position, category, and layout structure of the static components to achieve dynamic-static attention fusion and generate a multimodal fusion feature sequence.

[0083] 4) Webpage code generation module: Input the fused feature sequence into the code generation model to generate webpage code. The webpage code includes static style code, dynamic interaction code, and style change code. Specifically, it includes inputting the "static component layout features" and "dynamic temporal features" from the multimodal fused feature sequence into the code generation model. The encoder section uses a structure combining a convolutional neural network (CNN) and a Transformer: a. Using a convolutional neural network (CNN) submodule, extract the spatial layout features (such as component positions and region boundaries) of each frame in the multimodal fusion feature sequence, and output the spatial encoding features; b. After passing through the Transformer Encoder submodule containing 6 encoder layers, the global context vector is output. .

[0084] The decoder section uses a "multi-task Transformer Decoder": a. HTML Decoder: Based on the spatial layout features in the global context vector C, it generates skeleton tags such as `<html>`, ``, and ``.<h1> <button>, <input> Component tags, embedding component IDs for associating styles and interactions; b. CSS Decoder: Based on static styles and dynamic animation information (such as hover color change, pop-up expansion) in the global context vector C, it generates basic styles and @keyframes animation rules; c. JavaScript Decoder: Generates event binding code and interaction parameters based on dynamic triggering conditions (click / hover) and response logic in the global context vector C.

[0085] Code combination: After reading the output code of the HTML, CSS, and JavaScript decoders, the three types of code are embedded into the corresponding tag areas according to the syntax structure of a standard HTML file to form the initial webpage code.

[0086] Code optimization: The generated initial HTML code is syntax-checked using HTML, CSS, and JavaScript parsers. It automatically corrects detected syntax errors, such as unclosed tags, incorrect attributes, and incorrect function definitions, optimizes the code structure, removes redundant code (such as duplicate CSS styles and unused JavaScript functions), merges identical code blocks, and outputs optimized webpage code.

[0087] Example 2 In this embodiment of the disclosure, it is assumed that there is an existing prototype design of an e-commerce website, which includes three dynamic modules: a "Add to Cart" button that zooms in and changes color when hovered, a "Product Type" selection pop-up that fades in and expands, and a "Product Images" carousel that switches between images.

[0088] Reference Figure 2 The process for generating the webpage code corresponding to the above three dynamic modules is as follows: The prototype functions of the three dynamic modules mentioned above were recorded to form a GIF animation with a duration of 6 seconds and a frame rate of 10fps.

[0089] (a) Dynamic Image Preprocessing Module 1) Normalize the size of GIF animations: Call the OpenCV cv2.resize() function to uniformly scale the GIF animations to 1920×1080 pixels.

[0090] 2) Edge enhancement processing for GIF animations: The Sobel gradient operator is used to perform edge detection on GIF animations to enhance the outline features of components such as the "Add to Cart" button and the "Product Image Display Area".

[0091] 3) Generate the frame sequence of the GIF animation: a. Call cv2.VideoCapture() to load the GIF animated image file; b. Call cap.get() to get the total number of frames in the frame sequence, which is 60 frames; c. Execute `ret, frame = cap.read()` repeatedly to read 60 frames of the frame sequence one by one; d. Convert each of the 60 frames from BGR format to RGB format; e. Call cv2.imwrite() to save 60 frames as single-frame PNGs named from "store_frame_00.png" to "store_frame_59.png". Frames 0-9 are the initial state, frames 10-29 are the button hover process, frames 30-45 are the pop-up animation, and frames 46-59 are the image switching.

[0092] (ii) Frame page component feature extraction module processing 1) Static component feature extraction: Using the YOLOv8 model, target detection was performed frame by frame in a 60-frame sequence. The category of the "Add to Cart" button was "button", and the bounding box coordinates at frame 0 were (800, 800, 850, 850); the category of the "Product Type" selection pop-up was "list", and the bounding box coordinates at frame 0 were (200, 200, 400, 800). Static feature vector S was generated based on the category and bounding box coordinate information of the "Add to Cart" button and the "Product Type" selection pop-up.

[0093] 2) Layout structure feature extraction: The layout regions such as "navigation bar" (coordinates (0,0,1920,100)) and "content area" (coordinates (0,100,1920,900)) are identified by the U-Net image segmentation model. The coordinates of the "navigation bar" region are (0,0,1920,100) and the coordinates of the "content area" region are (0,100,1920,900). The layout feature vector L is generated based on the coordinates of the "navigation bar" and "content area".

[0094] 3) Component Detection and Tracking: The bounding box coordinates of the "Add to Cart" button and the "Product Type" selection pop-up component in each frame are detected using the YOLOv8 model. Based on the bounding box IOU and category similarity, the correspondence between components across frames is established. For example, the bounding box of the "Add to Cart" button in frames 10-29 has an IOU > 0.8 with the bounding box of the "Add to Cart" button in frames 0-9, and is therefore determined to be the same component; the bounding box of the "Product Type" selection pop-up in frames 30-45 has a matching cost < 0.2 with the bounding box of the "Product Type" selection text in frames 0-29, and is therefore determined to be a related component.

[0095] 4) Component Change Judgment: The "Add to Cart" button is detected using the YOLOv8 model. At frame 10, the area is 5000px², and at frame 29, the area is 8400px², indicating the "Add to Cart" button has been enlarged. The structural similarity index (SSIM) of the "Product Image" component area is calculated to be 0.95 at frame 46, indicating the "Product Image" is a "Front View". At frame 53, the SSIM is calculated to be 0.72, indicating the "Product Image" has switched to a "Side View". At frame 59, the SSIM is calculated to be 0.68, indicating the "Product Image" has switched to a "Detail View". Compared to frame 46, the SSIM values ​​for frames 53 and 59 are both <0.8 (preset threshold T_SSIM), indicating a change in content.

[0096] 5) Dynamic feature sequence generation: The LSTM model is used to model the change process of dynamic components. The dynamic feature vectors of each frame component are input, and the temporal dependencies are extracted through 3 layers of LSTM units (hidden layer dimension 256) to output the dynamic feature sequence.

[0097] (III) Processing of Component Dynamic Interaction Feature Fusion Module 1) First layer - Spatial attention fusion: Substitute into the spatial attention weight formula (feature vector dimension d=256) to obtain the spatial attention weight. The spatial fusion features of each frame component are generated, and the final output is a spatial fusion feature sequence. .

[0098] 2) Second layer - Temporal attention fusion: Substitute the temporal attention weight formula (feature vector dimension d=256) and perform weighted summation to generate a temporal fusion feature sequence with enhanced temporal correlation. .

[0099] 3) Third layer - Dynamic-Static Attention Fusion: Substituting the cross-modal attention weight formula (feature vector dimension d=256), the final output is a multimodal fused feature sequence. .

[0100] (iv) Webpage code generation module processing Multimodal fusion feature sequences A code generation model based on an input encoder-decoder architecture.

[0101] 1) The encoder generates a global context vector: a. Extract each frame using a CNN submodule with 3 convolutional layers. Spatial layout features, output spatial coding features (Dimensions 60×256); b. Capture spatial coding features using a Transformer Encoder submodule containing six encoder layers. Inter-frame timing dependencies (such as "button hover") (The order of pop-up window expansion) c. Output the global context vector C (dimension 512).

[0102] 2) Decoder-generated code: a. Based on the spatial layout features in the global context vector C, the HTML decoder generates webpage skeleton tags and component tags, such as top-level tags, and generates layout containers such as product display area and operation button area inside, while embedding component IDs for associating styles and interactions. b. In the CSS decoder, style rules are generated based on the static style and dynamic animation information in the global context vector C; c. Generate interactive code in the JavaScript decoder based on the dynamic triggering conditions and response logic in the global context vector C.

[0103] 3) Code assembly: Following the standard HTML file structure, embed the CSS code entirely within the style tags and the JavaScript code within the javascript tag at the end to form the initial webpage code.

[0104] 4) Code optimization: Detect and automatically complete unclosed tags using HTMLParser (e.g., ...). (Unclosed parentheses) Correct JavaScript function syntax errors (such as missing parentheses) using ESLint, and output e-commerce website code that can be run directly in the browser.

[0105] The webpage code generated through the above process can fully reproduce the dynamic effects of the "Add to Cart" button changing color and enlarging when hovered, the "Product Type" pop-up fading in and expanding, and the "Product Images" carousel switching.

[0106] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0107] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims. < / button>

Claims

1. A webpage code generation method, comprising: preprocessing a dynamic picture to generate a frame sequence of the dynamic picture, the frame sequence consisting of a plurality of frame pages; extracting frame page component features according to the frame sequence to generate a dynamic feature sequence, wherein the frame page component features include layout structure features, static component features, and dynamic interaction features, and the dynamic feature sequence represents interactive change information of webpage components in the dynamic picture; performing dynamic interaction feature fusion on the frame page component features according to the dynamic feature sequence to generate a fused feature sequence; inputting the fused feature sequence into a code generation model to generate webpage code, the webpage code including static style code, dynamic interaction code, and style change code.

2. The web page code generation method of claim 1, wherein, extracting frame page component features according to the frame sequence to generate a dynamic feature sequence, comprising: dividing a layout structure of a webpage according to the frame sequence to identify functional area information in the webpage to obtain the layout structure features, the functional area information including functions, positions, and ranges; identifying and positioning frame page components in the frame sequence to mark a bounding box of a webpage component in the frame page and determine a category of the webpage component to obtain static component features; judging an association relationship of webpage components in different frame pages according to the layout structure features and the static component features to determine dynamic interaction features of the webpage components to generate the dynamic feature sequence.

3. The web page code generation method of claim 2, wherein, judging an association relationship of webpage components in different frame pages to determine dynamic interaction features of the webpage components to generate the dynamic feature sequence, comprising: matching and tracking webpage components in different frame pages to determine an association relationship between webpage components in different frame pages, and identifying webpage components having an association relationship in different frame pages as the same webpage component; judging position change, shape change, and content change of the same webpage component in different frame pages to obtain change information of dynamic interaction of the webpage component; generating a dynamic feature vector of the webpage component according to the association relationship and the change information; generating the dynamic feature sequence according to the dynamic feature vector.

4. The web page code generation method of claim 3, wherein, judging position change, shape change, and content change of the same webpage component in different frame pages to obtain change information of dynamic interaction of the webpage component, comprising: calculating Euclidean distances between center coordinates of bounding boxes of the same webpage component in adjacent frame pages, and determining that the position of the webpage component changes when the Euclidean distances change; calculating a relative change rate of areas of bounding boxes of the same webpage component in different frame pages, and determining that the shape of the webpage component changes when the relative change rate is not 0; calculating a structural similarity index of the same webpage component in different frame pages, and determining that the content of the webpage component changes when the structural similarity index is less than a preset threshold.

5. The web page code generation method according to any one of claims 1 to 4, wherein performing dynamic interaction feature fusion on the frame page component features according to the dynamic feature sequence to generate a fused feature sequence, comprising: standardize the layout structure features, the static component features, and the dynamic interaction features, and unify feature dimensions and formats; perform feature fusion on the layout structure features, the static component features, and the dynamic interaction features based on a three-layer attention mechanism to generate the fusion feature sequence.

6. The web page code generation method of claim 5, wherein, standardize the layout structure features, the static component features, and the dynamic interaction features, and unify feature dimensions and formats, including: perform vectorization processing on the static component features to generate a static feature vector; perform vectorization processing on the layout structure features to generate a layout feature vector; take the dynamic feature sequence as a dynamic feature vector representing the dynamic interaction features.

7. The web page code generation method of claim 6, wherein, perform feature fusion on the layout structure features, the static component features, and the dynamic interaction features based on a three-layer attention mechanism to generate the fusion feature sequence, including: match the positions and categories of the static components with the layout structure according to the static feature vector and the layout feature vector, implement spatial attention fusion, and generate a spatial fusion feature sequence representing static component layout features; determine the time sequence correlation of dynamic interaction features of different frame pages according to the dynamic feature sequence, perform time sequence attention fusion on the dynamic interaction features of the different frame pages according to the time sequence correlation, and generate a time sequence fusion feature sequence representing dynamic time sequence features; distribute matching weights of the positions and categories of the static components with the layout structure according to the dynamic component features of the components after time sequence attention fusion, implement dynamic-static attention fusion, and generate a multi-modal fusion feature sequence.

8. The web page code generation method of claim 7, wherein, input the static component layout features and the dynamic time sequence features in the multi-modal fusion feature sequence into an encoder to generate a global context vector, generate the static style code, the dynamic interaction code, and the style change code in a decoder according to the global context vector; obtain the codes generated by the decoder, embed the static style code, the dynamic interaction code, and the style change code in corresponding label areas respectively, and generate initial web page codes; perform verification and correction on the initial web page codes, and output optimized web page codes. perform preprocessing on a dynamic picture to generate a frame sequence of the dynamic picture, including:

9. The web page code generation method according to any one of claims 1 to 4, wherein, perform normalization processing on a plurality of the frame pages to make the sizes of the plurality of frame pages the same; perform contour enhancement processing on the plurality of frame pages after normalization processing; generate the frame sequence from the plurality of frame pages after contour enhancement processing. 10.An electronic device, comprising: one or more processors; a memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the web page code generation method according to any one of claims 1 to 9. ​ 11. A computer readable medium having stored thereon a computer program which, when executed by a processor, implements the web page code generation method according to any one of claims 1 to 9.

12. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the web page code generation method according to any one of claims 1 to 9.

Images