Webpage interface reconstruction method and system based on parameterized modeling and continuously differentiable optimization

Abstract

The webpage interface reconstruction method based on parameterized modeling and continuous differentiable optimization provided by the application comprises the following steps: parameter modeling of components in a webpage interface to obtain geometric parameters and hierarchical parameters, and differentiable expression; based on the differentiable geometric parameters and hierarchical parameters, a continuous differentiable mapping function is designed to obtain a spatial layout function value; the hierarchical parameters are converted into a differentiable hierarchical weight distribution, and the spatial layout function value is converted into a continuous differentiable variable, and a synthesized webpage interface is obtained after fusion; similarity comparison is performed on the synthesized webpage interface and a target webpage interface, and a staged optimization strategy is adopted, loss functions are designed for the geometric parameters and hierarchical parameters of the components respectively to guide the optimization and update of the parameters, and webpage interface reconstruction is completed. By converting the discrete hierarchical parameters of the components into a continuous differentiable hierarchical weight distribution and decoupling the optimization objectives of different types of parameters in stages, the efficiency and accuracy of component reconstruction are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer graphics, specifically to a method and system for reconstructing web page interfaces based on parametric modeling and continuous differentiable optimization. Background Technology

[0002] In the field of computer vision and graphics, web page interface reconstruction is a critical task. Its goal is to achieve a high-precision reconstruction of the original web page layout using a set of editable interface components and their parameters such as position, size, and hierarchy. This technology has wide and important applications in areas such as web design automation, visual editing, front-end layout optimization, and industrial inspection.

[0003] In related research, differentiable vector graphics rendering methods such as DiffVG model the vector graphics rendering process as a differentiable function, enabling gradient optimization of parameters such as path, color, and width. Differentiable rendering methods based on signed distance fields (SDF) generate smooth, differentiable shape contours through continuous boundary representations and combine this with color mixing to generate two-dimensional images. These methods demonstrate that by transforming discrete structural parameters into a continuously differentiable space, efficient structural optimization and visual reconstruction can be achieved, providing a technical reference for the continuous and differentiable optimization of web interface layouts.

[0004] For example, invention application CN113140033A discloses a method for reconstructing a single cumulus image based on differentiable rendering, including: acquiring a single natural cumulus image; determining the image density through a convolutional neural network with an encoder-decoder structure; determining the loss function of the cumulus structure in three-dimensional space; determining the gradient of each variable in the equation through an inverse model, and iteratively updating the cumulus structure using a gradient optimization algorithm; projecting the cumulus model onto a two-dimensional plane to generate a depth map and a contour map; and generating the corresponding cumulus image using a texture generation network.

[0005] For example, invention application CN113888694A discloses a method and system for reconstructing transparent objects based on SDF field differentiable rendering. This method involves capturing panoramic images and photographs containing transparent objects that need calibration. A rotation matrix is ​​obtained using camera calibration techniques based on the panoramic images and the photographs containing the transparent objects. The initial shape of the transparent object is then obtained using a spatial sculpting algorithm. The initial shape is represented using SDF, and a differentiable ray tracing algorithm is proposed to render images of the 3D model of the transparent object from any viewpoint and in the environment. By minimizing the loss function between the input image and the rendered image, the 3D model of the transparent object is gradually optimized.

[0006] However, existing methods often suffer from problems such as difficulty in optimizing discrete hierarchical parameters of components and insufficient staged optimization when dealing with combinations of multiple types of components. This results in low reconstruction efficiency, low structural accuracy, and difficulty in supporting flexible editing of complex interfaces.

[0007] Therefore, there is an urgent need for an innovative method to transform the discrete hierarchical parameters of components into continuously differentiable parameter representations, so that the component parameters can be efficiently adjusted through gradient optimization methods to achieve high-precision, controllable and flexible interface reconstruction. Summary of the Invention

[0008] The purpose of this invention is to provide a web interface reconstruction method and system based on parametric modeling and continuous differentiable optimization. Addressing the problems of difficulty in optimizing discrete hierarchical parameters and parameter coupling in existing technologies, this invention transforms discrete hierarchical parameters into continuously differentiable parameter representations, decouples the optimization objectives of different types of parameters in stages, and utilizes JAX's efficient automatic differentiation and parallel computing capabilities. This reduces parameter coupling interference while specifically strengthening the core constraints at each stage, significantly improving the efficiency and accuracy of component reconstruction.

[0009] To achieve the above-mentioned objectives, an embodiment provides a web page interface reconstruction method based on parametric modeling and continuous differentiable optimization, comprising the following steps:

[0010] Step 1: Model the parameters of the components in the web page interface to obtain the geometric parameters and hierarchical parameters of the components, and represent the geometric parameters and hierarchical parameters through several differentiable parameters;

[0011] Step 2: Based on the differentiable geometric parameters and hierarchical parameters, design the corresponding continuously differentiable mapping function to obtain the spatial layout function value;

[0012] Step 3: Convert the hierarchical parameters in Step 1 into a differentiable hierarchical weight distribution, and at the same time convert the spatial layout function value in Step 2 into a continuously differentiable variable. Merge the hierarchical weight distribution with the continuously differentiable variable to obtain the synthesized web page interface.

[0013] Step 4: Compare the similarity between the synthesized web page interface and the target web page interface, and adopt a phased optimization strategy. Design loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, and complete the reconstruction of the web page interface.

[0014] This invention addresses the problems of difficulty in optimizing discrete hierarchical parameters and parameter coupling in the process of optimizing parameters of multiple components in web page interfaces. It designs a method to transform the discrete geometric parameters and hierarchical parameters in the web page interface into continuously differentiable parameter representations for optimization, decouples the optimization objectives of different types of parameters in stages, and completes the reconstruction of the web page interface.

[0015] Simultaneously, the efficient parallel computing and automatic differentiation capabilities of JAX significantly improve the reconstruction accuracy and optimization convergence speed of the web page layout while ensuring computational stability.

[0016] In one embodiment, the components in the web page interface are various interface elements extracted from the web page interface, including button components, text box components, and image components.

[0017] In one embodiment, a continuously differentiable mapping function maps any point on a webpage to the value of the correspondence between that point and the component layout. The layout correspondence represents the distance between the point and the component, used to measure the membership relationship between the point and the component. The continuously differentiable mapping function is expressed as follows:

[0018] ,

[0019] In the formula, Point To Component A continuously differentiable mapping function, It is a component The geometric parameters or hierarchical parameters.

[0020] In one embodiment, step 3, converting the hierarchical parameters from step 1 into a differentiable hierarchical weight distribution, includes:

[0021] The discrete layer parameters are transformed into continuous layer weights (depth_weights) using the softmax function. The layer weights of all components in the webpage are then normalized to obtain a differentiable layer weight distribution, represented as:

[0022] ,

[0023] ,

[0024] In the formula, These are discrete hierarchical parameters. For normalization operations, For the summation operation, Indicates the direction along the column.

[0025] In one embodiment, fusing the hierarchical weight distribution with continuously differentiable variables to obtain the synthesized web page interface includes:

[0026] The component coverage is obtained by applying a sigmoid function to the spatial layout function value in step 2:

[0027] ,

[0028] In the formula, This is represented as component coverage. For spatial layout function values, It is a softening parameter used to control the sharpness of component edges;

[0029] Based on the hierarchical weight distribution, the component coverage is weighted and fused to obtain the synthesized web page interface.

[0030] In one embodiment, the phased optimization strategy adopted is a two-stage optimization strategy; the first stage fixes the geometric parameters of the components, optimizes the hierarchical parameters, and designs mean squared error loss and hierarchical loss to guide the hierarchical parameter optimization process; the second stage fixes the hierarchical parameters optimized in the first stage, optimizes the geometric parameters, designs mean squared error loss and edge consistency loss to guide the geometric parameter optimization process, and completes the reconstruction of the web page interface.

[0031] In one embodiment, the mean squared error loss is:

[0032] ,

[0033] In the formula, This indicates the first page of the target webpage. Each sample point value, The first part represents the synthesized web page interface. Each sample point value, This indicates the total number of sampling points.

[0034] In one embodiment, the hierarchical loss is:

[0035] ,

[0036] In the formula, This indicates that the original hierarchical array is sorted in ascending order to obtain an ordered hierarchical array. τ is the hierarchical interval threshold. This is the maximum value function, where m is the hierarchical array variable and M is the total number of hierarchical arrays.

[0037] In one embodiment, the edge consistency loss is:

[0038] ,

[0039] In the formula, and These are the height and width of the synthesized webpage interface, respectively; For the target webpage interface in location The edge strength value is calculated using the Sobel operator. For the synthesized web page interface in the location The edge strength values ​​are calculated using the same Sobel operator. For the mask matrix at position The value of , where, This indicates that the location is included in the loss calculation. This indicates that the position is ignored.

[0040] On the other hand, the present invention also provides a web page interface reconstruction system based on parametric modeling and continuous differentiable optimization, used to complete the aforementioned web page interface reconstruction method based on parametric modeling and continuous differentiable optimization, including:

[0041] Differentiable component modeling unit is used to model the parameters of components in a web page interface to obtain the geometric parameters and hierarchical parameters of the components. The geometric parameters and hierarchical parameters are represented by several differentiable parameters. Based on the differentiable geometric parameters and hierarchical parameters, the corresponding continuously differentiable mapping function is designed to obtain the spatial layout function value.

[0042] The differentiable interface generation unit is used to convert hierarchical parameters into differentiable hierarchical weight distributions, while converting spatial layout function values ​​into continuously differentiable variables. The hierarchical weight distributions are then fused with the continuously differentiable variables to obtain a synthesized web page interface.

[0043] The differentiable parameter optimization unit is used to compare the similarity between the synthesized web page interface and the target web page interface, and adopts a phased optimization strategy. It designs loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, thereby completing the reconstruction of the web page interface.

[0044] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0045] (1) The discrete hierarchical parameters are transformed into a continuously differentiable hierarchical weight distribution. The combination and layout of components are realized through the differentiable fusion mechanism to generate the final layout result of the web page interface.

[0046] (2) A phased optimization strategy is adopted. By dividing the optimization of component parameters into two stages, namely hierarchical and geometric parameters, the optimization objectives of different types of parameters are effectively decoupled, and mutual interference caused by parameter coupling is avoided.

[0047] (3) A specific loss function is proposed for different optimization stages, which, combined with a masking mechanism, can flexibly focus on key areas. Compared with existing technologies using a single loss function, this approach can simultaneously ensure global visual consistency and local detail accuracy, significantly improving the structural integrity and visual naturalness of the reconstructed interface.

[0048] (4) Relying on the automatic differentiation, just-in-time (JIT) compilation and parallel computing capabilities of the JAX framework, end-to-end high-efficiency execution with differentiable parameters can be achieved. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization provided in the embodiments.

[0051] Figure 2 This is a parameter modeling diagram of the button component provided in the embodiment.

[0052] Figure 3 This is a schematic diagram of the webpage interface layout reconstruction result provided in the embodiment.

[0053] Figure 4 This is a schematic diagram of the phased optimization process.

[0054] Figure 5 This is a schematic diagram of the web interface reconstruction system based on parametric modeling and continuous differentiable optimization provided in the embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the accompanying drawings and...

[0056] The embodiments further illustrate the present invention in detail. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.

[0057] To address the challenges of optimizing discrete-level parameters and parameter coupling in existing technologies, this embodiment provides a web interface reconstruction method based on parametric modeling and continuously differentiable optimization, such as... Figure 1 As shown, it includes the following steps:

[0058] S1. Model the parameters of the components in the web page interface to obtain the geometric parameters and hierarchical parameters of the components, and represent the geometric parameters and hierarchical parameters through several differentiable parameters.

[0059] like Figure 2As shown in the embodiment, each component in the AIGC-generated webpage interface is parametrically modeled. The components include various interface elements extracted from the given AIGC-generated webpage interface, such as button components, text box components, and image components. Each component is represented by several differentiable parameters, including geometric parameters (position, size, etc.) and hierarchical parameters. For example, the parameters for a text box are represented as center (center point), width (width), height (height), and depth (hierarchy), providing differentiable input for subsequent optimization.

[0060] S2. Based on the differentiable geometric parameters and hierarchical parameters, design the corresponding continuously differentiable mapping function to obtain the spatial layout function value.

[0061] In this embodiment, based on the differentiable geometric parameters and hierarchical parameters of S1, a corresponding continuously differentiable mapping function is designed for each component (such as buttons, text boxes, etc.) in the web page interface:

[0062] ,

[0063] In the formula, Point To Component A continuously differentiable mapping function, It is a component The geometric or hierarchical parameters are determined. By designing a corresponding continuously differentiable mapping function, i.e., a spatial layout function, any point in the web page interface is mapped to a value representing the correspondence between that point and the component layout. Here, the layout correspondence refers to the distance between the point and the component, used to measure the membership relationship between the point and the component. This mapping relationship enables a differentiable description of the component, providing a stable and accurate spatial metric for subsequent differentiable optimization and gradient backpropagation, realizing the transformation of the component from "discrete parameters" to "continuously differentiable variables".

[0064] S3. Transform the hierarchical parameters in S1 into a differentiable hierarchical weight distribution, and at the same time transform the spatial layout function values ​​in S2 into continuously differentiable variables. Merge the hierarchical weight distribution with the continuously differentiable variables to obtain the synthesized web page interface.

[0065] In this embodiment, the spatial layout function value obtained in S2 is processed by sigmoid to obtain the component coverage value. The correspondence between the spatial midpoint and the component layout is transformed into a continuously differentiable quantitative index between 0 and 1, realizing the smooth transition and continuous differentiability of the component boundary.

[0066] ,

[0067] In the formula, This is represented as component coverage. For spatial layout function values, It is a softening parameter used to control the sharpness of component edges.

[0068] Then, the discrete layer parameters (such as 0, 1, 2...) are transformed into continuous layer weights `depth_weights` using the softmax function. The layer weights of all components are then normalized to obtain a differentiable layer distribution.

[0069] ,

[0070] ,

[0071] In the formula, These are discrete hierarchical parameters. For normalization operations, For the summation operation, Indicates the direction along the column.

[0072] Finally, as Figure 3 As shown, based on the normalized hierarchical weights, the component coverage values ​​and hierarchical weights are weighted and fused to obtain the synthesized interface layout.

[0073] S4. Compare the similarity between the synthesized web page interface and the target web page interface, and adopt a phased optimization strategy. Design loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, and complete the reconstruction of the web page interface.

[0074] In the embodiments, such as Figure 4 As shown, a two-stage optimization strategy is adopted. In the first stage, the geometric parameters of the components are fixed, and the hierarchical parameters are optimized. Mean squared error loss and hierarchical loss constraints are designed to guide the parameter optimization process. In the second stage, the hierarchical parameters optimized in the first stage are fixed, and the focus is on adjusting the spatial position and shape of the components. Mean squared error loss and edge consistency loss are designed to guide the component geometric parameter optimization process. A differentiable optimizer based on the JAX framework is used for parameter backpropagation and iterative updates.

[0075] Mean Squared Error Loss Function Used:

[0076] ,

[0077] In the formula, This indicates the first page of the target webpage. Each sample point value, The first part represents the synthesized web page interface. Each sample point value, This indicates the total number of sampling points.

[0078] The hierarchical loss used is:

[0079] ,

[0080] In the formula, This indicates that the original hierarchical array is sorted in ascending order to obtain an ordered hierarchical array. τ is the hierarchical interval threshold. This is the maximum value function, where m is the hierarchical array variable and M is the total number of hierarchical arrays.

[0081] The edge consistency loss used is:

[0082] ,

[0083] In the formula, and These are the height and width of the synthesized webpage interface, respectively; For the target webpage interface in location The edge strength value is calculated using the Sobel operator. For the synthesized web page interface in the location The edge strength values ​​are calculated using the same Sobel operator. For the mask matrix at position The value of , where, This indicates that the location is included in the loss calculation. This indicates that the position is ignored.

[0084] To test the actual effectiveness of the technical solution provided in this embodiment, we will evaluate it on a web page interface reconstruction dataset. The comparison method selected is the existing differentiable vector graphics optimization framework DiffVG for two-dimensional scenes. By running the optimization process under the same initial conditions, the effectiveness of the interface layout reconstruction is measured using two metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Perceptual Loss.

[0085] Table 1

[0086]

[0087] The results are shown in Table 1. Higher PSNR and SSIM values ​​are better, while lower Perceptual Loss values ​​are better. By comparing with the traditional two-dimensional scene differentiable vector graphics optimization framework DiffVG, the experimental results show that the present invention achieves good results in interface layout reconstruction.

[0088] To clearly demonstrate the web interface reconstruction method based on parametric modeling and continuous differentiable optimization, this embodiment also provides a web interface reconstruction system based on parametric modeling and continuous differentiable optimization to complete the aforementioned web interface reconstruction method based on parametric modeling and continuous differentiable optimization, such as... Figure 5 As shown, the web page interface reconstruction system includes a differentiable component modeling unit, which is used to model the parameters of components in the web page interface to obtain the geometric parameters and hierarchical parameters of the components. The geometric parameters and hierarchical parameters are represented by several differentiable parameters. Based on the differentiable geometric parameters and hierarchical parameters, the corresponding continuously differentiable mapping function is designed to obtain the spatial layout function value.

[0089] The differentiable interface generation unit is used to convert hierarchical parameters into differentiable hierarchical weight distributions, while converting spatial layout function values ​​into continuously differentiable variables. The hierarchical weight distributions are then fused with the continuously differentiable variables to obtain a synthesized web page interface.

[0090] The differentiable parameter optimization unit is used to compare the similarity between the synthesized web page interface and the target web page interface, and adopts a phased optimization strategy. It designs loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, thereby completing the reconstruction of the web page interface.

[0091] By using the above system, the problems of component hierarchical parameters being difficult to optimize and parameter coupling in existing web interface layout technologies are addressed. By transforming the discrete hierarchical parameters of components into a continuously differentiable hierarchical weight distribution, the optimization objectives of different types of parameters are decoupled in stages. By leveraging JAX's efficient automatic differentiation and parallel computing capabilities, the core constraints at each stage are strengthened in a targeted manner while reducing parameter coupling interference, significantly improving the efficiency and accuracy of component reconstruction.

[0092] Furthermore, it should be understood that after reading the above description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims.

Claims

1. A webpage interface reconstruction method based on parametric modeling and continuously differentiable optimization, characterized in that, Includes the following steps: Step 1: Model the parameters of the components in the web page interface to obtain the geometric parameters and hierarchical parameters of the components, and represent the geometric parameters and hierarchical parameters through several differentiable parameters; Step 2: Based on the differentiable geometric parameters and hierarchical parameters, design the corresponding continuously differentiable mapping function to obtain the spatial layout function value; A continuously differentiable mapping function maps any point on a webpage to the value of the correspondence between that point and the component layout. This correspondence represents the distance between the point and the component, and is used to measure the membership relationship between the point and the component. The continuously differentiable mapping function is expressed as follows: ， In the formula, Point To Component A continuously differentiable mapping function, It is a component Geometric parameters or hierarchical parameters; Step 3: Transform the hierarchical parameters from Step 1 into a differentiable hierarchical weight distribution, including: using the softmax function to transform the discrete hierarchical parameters into continuous hierarchical weights. Normalize the hierarchical weights of all components in the webpage interface to obtain a differentiable hierarchical weight distribution, represented as: ， ， In the formula, These are discrete hierarchical parameters. For normalization operations, For the summation operation, This indicates the direction along the column; simultaneously, the spatial layout function value from step 2 is transformed into a continuously differentiable variable, and the hierarchical weight distribution is fused with the continuously differentiable variable to obtain the synthesized web page interface, including: applying a sigmoid function to the spatial layout function value from step 2 to obtain component coverage: ， In the formula, This is represented as component coverage. For spatial layout function values, It is a softening parameter used to control the sharpness of component edges; based on the hierarchical weight distribution, the component coverage is weighted and fused to obtain the synthesized web page interface; Step 4: Compare the similarity between the synthesized web page interface and the target web page interface, and adopt a phased optimization strategy. Design loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, and complete the reconstruction of the web page interface.

2. The webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization according to claim 1, characterized in that, The components in a web page interface are various interface elements extracted from the web page interface, including button components, text box components, and image components.

3. The webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization according to claim 1, characterized in that, The phased optimization strategy adopted is a two-stage optimization strategy; The first stage involves fixing the geometric parameters of the components, optimizing the hierarchical parameters, and designing mean square error loss and hierarchical loss to guide the hierarchical parameter optimization process. The second stage fixes the hierarchical parameters optimized in the first stage, optimizes the geometric parameters, and designs mean square error loss and edge consistency loss to guide the geometric parameter optimization process, thus completing the reconstruction of the web interface.

4. The webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization according to claim 3, characterized in that, The mean square error loss is: ， In the formula, This indicates the first page of the target webpage. Each sample point value, The first part represents the synthesized web page interface. Each sample point value, This indicates the total number of sampling points.

5. The webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization according to claim 3, characterized in that, The hierarchical loss is: ， In the formula, This indicates that the original hierarchical array is sorted in ascending order to obtain an ordered hierarchical array. , It is the hierarchical interval threshold. It is a function with maximum value. It is a hierarchical array variable. This represents the total number of levels in the array.

6. The webpage interface reconstruction method based on parametric modeling and continuous differentiable optimization according to claim 3, characterized in that, The edge consistency loss is: ， In the formula, and These are the height and width of the synthesized webpage interface, respectively; For the target webpage interface in location The edge strength value is calculated using the Sobel operator. For the synthesized web page interface in the location The edge strength values ​​are calculated using the same Sobel operator. For the mask matrix at position The value of , where, This indicates that the location is included in the loss calculation. This indicates that the position is ignored.

7. A webpage interface reconstruction system based on parametric modeling and continuously differentiable optimization, characterized in that, The method for reconstructing a web interface based on parametric modeling and continuous differentiable optimization as described in any one of claims 1 to 6 includes: Differentiable component modeling unit is used to model the parameters of components in a web page interface to obtain the geometric parameters and hierarchical parameters of the components. The geometric parameters and hierarchical parameters are represented by several differentiable parameters. Based on the differentiable geometric parameters and hierarchical parameters, the corresponding continuously differentiable mapping function is designed to obtain the spatial layout function value. The differentiable interface generation unit is used to convert hierarchical parameters into differentiable hierarchical weight distributions, while converting spatial layout function values ​​into continuously differentiable variables. The hierarchical weight distributions are then fused with the continuously differentiable variables to obtain a synthesized web page interface. The differentiable parameter optimization unit is used to compare the similarity between the synthesized web page interface and the target web page interface, and adopts a phased optimization strategy. It designs loss functions for the geometric parameters and hierarchical parameters of the components to guide the optimization and updating of the parameters, thereby completing the reconstruction of the web page interface.

Images