High-reduction-degree and efficient webpage screenshot method and processing terminal

Abstract

The invention discloses a high-reduction-degree and efficient webpage screenshot method and a processing terminal.The method comprises the steps that when a webpage first screen where a webpage to be subjected to screenshot is located starts to be loaded, a sandbox is initialized, and an HTML document where the webpage is located is injected into the sandbox; converting a DOM (Document Object Model) structure of the HTML document in the sandbox into an abstract syntax tree AST; original objects of all media nodes in the abstract syntax tree are converted into Blob objects, and the Blob objects are stored in a resource cache table in a key-value pair mode; in response to a received screenshot instruction which comprises a screenshot area, traversing an AST (abstract syntax tree) according to nodes covered by the screenshot area so as to find matched nodes; and performing forward linked list traversal on the matched nodes, performing DFS traversal on child nodes of the matched nodes to generate SVG nodes, exporting each SVG node as a character string, and generating an SVG picture based on the character string. According to the invention, the screenshot efficiency is improved, the screenshot with high reduction degree is realized, and the screenshot quality is ensured.

Description

Technical Field

[0001] This invention relates to the field of webpage screenshot technology, specifically a high-fidelity and efficient webpage screenshot method and processing terminal. Background Technology

[0002] For webpage screenshots, existing technologies generally involve waiting for the webpage to load, then traversing the process containing the webpage to obtain its DOM nodes. The media data, such as images, contained within these DOM nodes are then converted into a target format (e.g., base64). Finally, the resulting image (e.g., JPG) is redrawn on a canvas. For example, patent application CN117873630A uses Html2Canvas to filter DOM nodes before conversion, then creates an SVG for drawing the graphics to obtain a base image. If the media resources are images, they are converted to Base64 format for drawing.

[0003] Using this existing screenshot technology has the following shortcomings: 1. Traverse the DOM nodes, copy the relevant data obtained, and then transform the copied data. Copying data is time-consuming and consumes a lot of storage resources, making it inefficient. In addition, copying data in the main process can easily cause blocking.

[0004] 2. DOM nodes within the DOM structure are lossless vector elements. Images drawn on a canvas are composed of scalar elements, resulting in a compressed and lossy image that is no longer a vector graphic. Enlarging or reducing the image will cause distortion, leading to a loss of fidelity. Furthermore, Base64 encoding itself is scalar encoding, meaning the resulting screenshot is not a lossless vector image.

[0005] 3. Some existing webpage screenshots are easily affected by external operations, which can affect the normal screenshot process and lead to low screenshot efficiency. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide a high-fidelity and efficient webpage screenshot method and processing terminal, which can solve the problems described in the background art.

[0007] The technical solution to achieve the objective of this invention is: a highly accurate and efficient webpage screenshot method, comprising the following steps: Step 1: When the first screen of the webpage containing the webpage to be screenshotted starts loading, initialize the sandbox and inject the HTML document containing the webpage into the sandbox; Step 2: Convert the DOM structure of the HTML document in the sandbox into an Abstract Syntax Tree (AST); Step 3: Convert the original objects of each media node in the abstract syntax tree into Blob objects, and store the Blob URLs of the Blob objects in the preset resource cache table as key-value pairs; Step 4: In response to the received screenshot command, which includes a screenshot area, based on the nodes covered by the screenshot area, traverse the Abstract Syntax Tree (AST) to find the AST node in the Key-Value Table that corresponds to the covered node, and obtain the matching node. Step 5: Perform a forward linked list traversal on the matching node and a depth-first search (DFS) traversal on the child nodes of the matching node to generate SVG nodes. Determine whether the SVG node is a media element. If so, then determine whether the media element can be matched with the corresponding media element in the key-value table. If yes, then associate the media element of the SVG node with the media element in the KV table; otherwise, insert an image tag into the media element, and use the image tag to reference the address of the Blob object to convert it into a Blob object. Then, cache the media element in the KV table and associate the converted Blob object with the corresponding media element in the KV table. If not, parse the style attributes represented by the style field of the AST node, calculate the style attributes by combining external and inline styles, and merge the calculated key style attributes into a single style string. Export each SVG node as a string, and generate an SVG image based on the string.

[0008] Furthermore, in step 1, injecting the HTML document into the sandbox includes injecting the root node of the HTML document into the sandbox.

[0009] Furthermore, the sandbox adopts the WebAssembly format and is loaded on the browser as a sandbox plugin.

[0010] Furthermore, the Abstract Syntax Tree (AST) is represented using a doubly linked list and a tree structure.

[0011] Furthermore, step 3 also includes: By registering a listener for the root node of the HTML document through MutationObserver, which runs within a sandbox and is connected, changes to each node in the HTML document can be captured. These changes include additions or deletions, modifications to node attributes, and changes in text type.

[0012] Furthermore, when a new node is detected, a new AST node is constructed in the sandbox's Abstract Syntax Tree (AST) and inserted into the parent node's child node list. When a node deletion is detected, the corresponding AST node in the sandbox's Abstract Syntax Tree (AST) is removed from the key-value table, and cached resources are released. When a change in attribute or style is detected, the attribute or style field of the corresponding AST node in the sandbox is directly updated in the KV table.

[0013] Furthermore, step 4 also includes binding the media elements involved in the matching node to the matching node via a Blob URL.

[0014] Furthermore, step 4 also includes: collecting styles in the sandbox at preset time intervals and inlining the styles to generate a unified style snapshot.

[0015] Furthermore, in step 4, by calling the SVG rendering engine, the Abstract Syntax Tree (AST) is traversed using a depth-first search and a forward linked list, and the complete SVG node is exported as a string using the XMLSerializer class.

[0016] A processing terminal, comprising: Memory, used to store program instructions; A processor for running the program instructions to perform the steps of the highly accurate and efficient webpage screenshot method.

[0017] The beneficial effects of this invention are as follows: By constructing a reference-based Abstract Syntax Tree (AST), this invention reduces the data transformation of traditional intermediate layers, thereby improving screenshot efficiency and achieving high-fidelity screenshots while ensuring screenshot quality. Furthermore, by utilizing a sandbox to output binary streams for each element, including media elements, the additional storage overhead caused by traditional Base64 encoding locks is reduced, further improving efficiency. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a preferred embodiment of the method of the present invention; Figure 2 This is a schematic diagram of the processing terminal. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: like Figure 1 As shown, a highly accurate and efficient method for taking screenshots of web pages includes the following steps: Step 1: When the first screen of the webpage containing the webpage to be screenshotted begins loading, initialize the sandbox and inject the HTML document containing the webpage into the sandbox to run the webpage within it. This isolates the original logic of the webpage, meaning the sandbox isolates the process environment of the webpage's process, ensuring that any external operations will not affect the webpage within the sandbox. Injecting the HTML document into the sandbox includes injecting the root node (i.e., the `<head>` element) of the HTML document into the sandbox.

[0020] For example, the sandbox adopts the WebAssembly format. It can be loaded on the browser as a sandbox plugin, thereby enabling the injection and execution of HTML documents into the sandbox.

[0021] Step 2: Convert the DOM structure of the HTML document in the sandbox into an Abstract Syntax Tree (AST). The AST is represented by a doubly linked list and a tree structure, that is, a combination of doubly linked list and tree structure. The fields of each AST node object in the AST include identifier, previous neighbor, parent node, next neighbor, child node, associated styley style, associated style, current DOM node instance reference, and associated media data reference.

[0022] It is understandable that using an Abstract Syntax Tree (AST) to represent elements means that the elements within the AST are references to the addresses of the original element objects. This means that the addresses of the objects in the DOM structure outside the sandbox are consistent, so only the addresses need to be referenced and there is no need to copy the data.

[0023] Step 3: Convert the raw media node objects in the Abstract Syntax Tree (AST) into Blob objects, and store the Blob URLs of the Blob objects as key-value pairs in a pre-defined resource cache table (i.e., the media resource KV table). The Blob URL serves as the key, and the Blob object as the value. Update the associated media data reference field of the AST node object with the Blob URL. Store each AST node in the AST as a key-value pair in the AST node object cache table (i.e., the AST node object KV table).

[0024] Understandably, AST node objects and key-value tables enable fast lookups, incremental updates of AST nodes, and reuse of duplicate AST nodes. For base64 formatted data, it is parsed and converted into Blob objects. For external resource URLs, the original data is fetched and converted into Blob objects. For cross-domain restricted data, the original data is fetched and passed to the request headers of the sandbox's specific configuration to ensure resource accessibility.

[0025] It identifies media elements in HTML documents through AST nodes. Media elements generally include images and videos, and establishes a mapping relationship between media elements and AST nodes.

[0026] By registering a listener for the root node of the HTML document through MutationObserver, which runs within the sandbox and is connected, changes to each node in the HTML document can be captured. Changes to nodes include additions or deletions, modifications to node attributes, and changes in text type. Modifications to node attributes include class, style, src, and href.

[0027] When a new node is detected, a new AST node is constructed in the sandbox's Abstract Syntax Tree (AST), which means a new AST node is created in the Key-Value table and inserted into the parent node's child list. When a node is deleted, the corresponding AST node in the sandbox's AST is removed from the Key-Value table, and cache resources (i.e., the cache originally used to store that AST node) are released. When a change in attribute or style is detected, the attribute or style field of the corresponding AST node in the sandbox's AST is directly updated in the Key-Value table, i.e., the styles or classNameList field is updated.

[0028] Specifically, the style attributes represented by the `classNameList` field of the AST node are parsed. Combining external stylesheets and inline styles, `getComputedStyle` is used to obtain the final calculation result, thus obtaining the style attributes. The calculated key style attributes (such as color, font, background, border, shadow, transformation, etc.) are merged into a single style string, which is then written into the `styles` field of the AST node. A mapping relationship is established between the `classNameList` field and inline styles for each `className` to enable rapid reuse.

[0029] By listening to the node number and performing the corresponding operation in the KV table, the AST node can be quickly located and updated.

[0030] Step 4: In response to the received screenshot command, which includes a screenshot area (the area of ​​the webpage to be captured), the Abstract Syntax Tree (AST) is traversed based on the nodes covered by the screenshot area to find the corresponding AST nodes in the Key-Value Table (KV) to obtain the matching nodes. The media elements (i.e., media resources, including img, canvas, video, etc.) involved in the matching nodes are then bound to the matching nodes via Blob URLs.

[0031] Within the sandbox, styles are collected at preset time intervals and inlined to generate a unified style snapshot. This allows for batch collection and inlining of styles using a time-slicing mechanism. The time-slicing mechanism enables the segmented construction of the Abstract Syntax Tree (AST) to balance memory usage and rendering performance.

[0032] Step 5: Perform a forward linked list traversal on the matching node and a DFS traversal on the children of the matching node to generate the SVG node.

[0033] Determine whether the SVG node is a media element. If so (i.e., the SVG node is a media element), then determine whether the media element can be matched with the corresponding media element in the KV table. If a media element is found in the KV table, the SVG node's media element is associated with that media element in the KV table. If no media element is found, an image tag is inserted into the media element, and the image tag references the address of a Blob object to convert it into a Blob object. The media element is then cached in the KV table, and the converted Blob object is associated with the corresponding media element in the KV table. If not (i.e., the SVG node is not a media element), the style attributes represented by the classNameList field of the AST node are parsed. The style attributes are calculated by combining external and inline styles, and the final calculation result is obtained using getComputedStyle. The calculated key style attributes (such as color, font, background, border, shadow, transformation, etc.) are merged into a single style string, which is then written into the styles field of the AST node. A mapping relationship is established between the classNameList field and inline styles for each className to enable rapid reuse.

[0034] Each SVG node is exported as a string, and an SVG image is generated based on the string, thus obtaining a screenshot in high-fidelity vector (i.e., high-fidelity) format.

[0035] For example, in this step, the Abstract Syntax Tree (AST) can be traversed using depth-first search (DFS) and forward linked list traversal by calling the SVG rendering engine. Furthermore, the complete SVG node can be exported as a string using the XMLSerializer class.

[0036] This invention improves screenshot efficiency and achieves high-fidelity screenshots by constructing a reference-based Abstract Syntax Tree (AST), reducing the data transformation required by traditional intermediate layers and ensuring high screenshot quality. Furthermore, it utilizes a sandbox to stream each element, including media elements, into binary form, reducing the additional storage overhead associated with traditional Base64 encoding locks and further enhancing efficiency.

[0037] Based on the inventor's actual tests and comparisons, this solution is compared with existing technologies (using the invention application patent with publication number CN117873630A as an example, which is based on the Html2Canvas solution) in terms of memory usage, processing time, and comprehensive pressure test (including the combination of the number of nodes and the size of media elements). The Html2Canvas in the table below refers to the Html2Canvas solution from the invention application patent with publication number CN117873630A. The results are shown in Tables 1, 2, and 3 respectively.

[0038] Table 1

[0039] Table 2

[0040] Table 3 Based on the comparative data in the three tables above, this application is significantly superior to existing technologies in terms of memory usage, time consumption, and stress resistance.

[0041] like Figure 2 As shown, the present invention also provides a processing terminal 100, which includes: Memory 101 is used to store program instructions; Processor 102 is configured to run the program instructions to perform the steps of the highly accurate and efficient webpage screenshot method.

[0042] The embodiments disclosed in this specification are merely illustrative of one aspect of the invention, and the scope of protection of the invention is not limited to these embodiments. Any other functionally equivalent embodiments fall within the scope of protection of the invention. Those skilled in the art can make various other corresponding changes and modifications based on the technical solutions and concepts described above, and all such changes and modifications should fall within the scope of protection of the claims of this invention.

Claims

1. A high-fidelity and efficient method for taking screenshots of web pages, characterized in that, Includes the following steps: Step 1: When the first screen of the webpage containing the webpage to be screenshotted starts loading, initialize the sandbox and inject the HTML document containing the webpage into the sandbox; Step 2: Convert the DOM structure of the HTML document in the sandbox into an Abstract Syntax Tree (AST); Step 3: Convert the original objects of each media node in the abstract syntax tree into Blob objects, and store the Blob URL of the Blob object in the preset resource cache table as key-value pairs; Step 4: In response to the received screenshot command, which includes a screenshot area, based on the nodes covered by the screenshot area, traverse the Abstract Syntax Tree (AST) to find the AST node in the Key-Value Table that corresponds to the covered node, and obtain the matching node. Step 5: Perform a forward linked list traversal on the matching node and a depth-first search (DFS) traversal on the child nodes of the matching node to generate SVG nodes. Determine whether the SVG node is a media element. If so, then determine whether the media element can be matched with the corresponding media element in the key-value table. If yes, then associate the media element of the SVG node with the media element in the KV table; otherwise, insert an image tag into the media element, and use the image tag to reference the address of the Blob object to convert it into a Blob object. Then, cache the media element in the KV table and associate the converted Blob object with the corresponding media element in the KV table. If not, parse the style attributes represented by the style field of the AST node, calculate the style attributes by combining external and inline styles, and merge the calculated key style attributes into a single style string. Export each SVG node as a string, and generate an SVG image based on the string.

2. The high-fidelity and efficient webpage screenshot method according to claim 1, characterized in that, In step 1, the HTML document is injected into the sandbox, including injecting the root node of the HTML document into the sandbox.

3. The high-fidelity and efficient webpage screenshot method according to claim 1, characterized in that, The sandbox is a WebAssembly format sandbox and is loaded onto the browser as a sandbox plugin.

4. The high-fidelity and efficient webpage screenshot method according to claim 1, characterized in that, Abstract Syntax Tree (AST) is represented using a doubly linked list and a tree structure.

5. The high-fidelity and efficient webpage screenshot method according to claim 1, characterized in that, Step 3 also includes: By registering a listener for the root node of the HTML document through MutationObserver, which runs within a sandbox and is connected, changes to each node in the HTML document can be captured. These changes include additions or deletions, modifications to node attributes, and changes in text type.

6. The high-fidelity and efficient webpage screenshot method according to claim 5, characterized in that, When a new node is detected, a new AST node is constructed in the sandbox's Abstract Syntax Tree (AST) and inserted into the parent node's child node list. When a node deletion is detected, the corresponding AST node in the sandbox's Abstract Syntax Tree (AST) is removed from the key-value table, and cached resources are released. When a change in attribute or style is detected, the attribute or style field of the corresponding AST node in the sandbox is directly updated in the KV table.

7. The high-fidelity and efficient webpage screenshot method according to claim 6, characterized in that, Step 4 also includes binding the media elements involved in the matching node to the matching node via a Blob URL.

8. The high-fidelity and efficient webpage screenshot method according to claim 7, characterized in that, Step 4 also includes: collecting styles in the sandbox at preset time intervals and inlining the styles to generate a unified style snapshot.

9. The high-fidelity and efficient webpage screenshot method according to claim 8, characterized in that, In step 4, the SVG rendering engine is invoked to perform depth-first search traversal and forward linked list traversal of the Abstract Syntax Tree (AST), and the complete SVG node is exported as a string using the XMLSerializer class.

10. A processing terminal, characterized in that, It includes: Memory, used to store program instructions; A processor for running the program instructions to perform the steps of the high-fidelity and efficient webpage screenshot method as described in any one of claims 1-9.

Images