Page component generation method and apparatus, computer device, and storage medium
Patent Information
- Application Number
- HK42021028287
- Authority / Receiving Office
- HK · HK
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-29
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2040-11-02
AI Technical Summary
In front-end project development, manually extracting page components is inefficient and inaccurate, especially in scenarios where there is a lot of identical content on the webpage, where manual operation is prone to errors.
By acquiring page content from multiple pages, a DOM tree is constructed, node feature information is extracted, and the same structural parts are automatically extracted using hash calculation and hierarchical information to generate page components.
It enables automatic generation of page components, improving generation efficiency and accuracy, and supports the reuse of components across multiple pages.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of front-end technology, and in particular to a method, apparatus, computer device, and storage medium for generating page components. Background Technology
[0002] In front-end project development, component-based development is often used to improve the efficiency of project development.
[0003] In related technologies, developers typically extract the common parts from multiple independent pages based on the similarity of their page structures, and then encapsulate them to generate page components.
[0004] However, in the above solutions, the generation process of page components is limited by manual labor. In scenarios where there are many web pages with the same content and a large number of web pages, manual extraction is inefficient and often carries certain risks and errors, resulting in low accuracy. Summary of the Invention
[0005] This application provides a method, apparatus, computer device, and storage medium for generating page components, which can automatically generate page components and improve the efficiency and accuracy of page component generation. The technical solution is as follows:
[0006] On the one hand, a method for generating page components is provided, the method including:
[0007] Retrieve the content of at least two pages;
[0008] The page content of the at least two pages is parsed, and the Document Object Model (DOM) tree of the at least two pages is constructed;
[0009] Extract the node feature information of each node in the DOM tree of the at least two pages;
[0010] Based on the node feature information of each node in the DOM tree of the at least two pages, extract the common structural parts in the DOM tree of the at least two pages;
[0011] Based on the node feature information of the same structural part, the page component corresponding to the same structural part is generated.
[0012] On the other hand, a page component generation apparatus is provided, the apparatus comprising:
[0013] The page content retrieval module is used to retrieve the page content of at least two pages;
[0014] The page content parsing module is used to parse the page content of the at least two pages and construct the Document Object Model (DOM) tree of the at least two pages;
[0015] The node feature information extraction module is used to extract the node feature information of each node in the DOM tree of the at least two pages;
[0016] The identical structure extraction module is used to extract the identical structure parts in the DOM trees of the at least two pages based on the node feature information of each node in the DOM trees of the at least two pages.
[0017] The page component generation module is used to generate page components corresponding to the same structural parts based on the node feature information of the same structural parts.
[0018] In one possible implementation, the node feature information includes first feature information, which includes the class name and node name of the corresponding node;
[0019] The identical structure extraction module includes:
[0020] The hash attribute acquisition submodule is used to perform hash calculation based on the first feature information of each node in the DOM tree of the at least two pages to obtain the hash attribute of each node.
[0021] The identical structure extraction submodule is used to extract the identical structure parts in the DOM trees of the at least two pages based on the hash attribute of each node in the DOM tree of the at least two pages.
[0022] In one possible implementation, the hash attribute acquisition submodule includes:
[0023] The first concatenation result acquisition unit is used to concatenate the class name and the node name of the target node in response to the target node not having valid child nodes, to obtain the concatenation result of the target node; the target node is any one of the nodes in the DOM tree of the at least two pages;
[0024] The first hash value calculation unit is used to calculate the hash value of the concatenation result of the target node and obtain the hash attribute of the target node.
[0025] In one possible implementation, the hash attribute acquisition submodule includes:
[0026] The second concatenation result acquisition unit is used to concatenate the class name and node name of the valid child node in response to the target node having valid child nodes, to obtain the concatenation result of the valid child node; the target node is any one of the nodes in the DOM tree of the at least two pages;
[0027] The second hash value calculation unit is used to calculate the hash value of the concatenation result of the effective child nodes;
[0028] The first splicing result acquisition unit is used to splice the class name of the target node and the node name of the target node;
[0029] The first hash value calculation unit is used to calculate the hash value of the concatenation result of the target node;
[0030] The hash attribute acquisition unit is used to take the hash value of the concatenation result of the target node and the hash value of the concatenation result of the effective child nodes as the hash attribute of the target node.
[0031] In one possible implementation, the second splicing result acquisition unit is configured to, in response to the fact that the effective child nodes include at least two duplicate nodes, splice the class name and node name of one of the at least two duplicate nodes to obtain the splicing result of the effective child nodes.
[0032] In one possible implementation, the class name consists of at least two element class names, the node name consists of at least two element node names, and the device further includes:
[0033] The first sorting module is used to sort the at least two element class names of each node in a first specified order before the same structure extraction module extracts the same structure part in the DOM tree of the at least two pages based on the node feature information of each node in the DOM tree of the at least two pages, so as to obtain the class name of each node.
[0034] The second sorting module is used to sort the at least two element node names of each node according to a second specified order to obtain the node names of each node.
[0035] In one possible implementation, the node feature information also includes the hierarchy information of the corresponding node, which corresponds to the hierarchy of the node in the DOM tree;
[0036] The identical structure extraction module includes:
[0037] The acquisition submodule is used to acquire the hash attribute of a node contained in the DOM tree of each of the at least two pages, thereby obtaining the hash attributes of at least two nodes; the at least two nodes are nodes with the same hierarchical information.
[0038] A determination submodule is used to determine the at least two nodes as nodes belonging to the same structural part in response to the hash attributes between the at least two nodes satisfying a specified condition.
[0039] In one possible implementation, the specified conditions include:
[0040] The overlap ratio of the hash attributes between at least two nodes reaches a specified threshold.
[0041] In one possible implementation, a submodule is used to obtain the hash attribute of a node contained in each of the DOM trees of the at least two pages in the order from the root node to the child node according to the hierarchical information, so as to obtain the hash attributes of at least two nodes.
[0042] In one possible implementation, the page component generation module is used to construct a DOM tree based on the node feature information of nodes in the same structural part, according to the hierarchical information from child nodes to root nodes, to obtain the page components corresponding to the same structural part.
[0043] In one possible implementation, the node feature information also includes the node attributes and child attributes of the corresponding node;
[0044] The page component generation module includes:
[0045] The component node label generation submodule is used to generate component node labels based on the node names of nodes in the same structural part, in the order from child nodes to root nodes according to the hierarchical information.
[0046] The component node attribute generation submodule is used to generate component node attributes based on the node attributes of nodes in the same structural part.
[0047] The component internal structure construction submodule is used to construct the component internal structure based on the child attributes of the nodes in the same structural part;
[0048] The page component generation submodule is used to generate the page component corresponding to the same structural part based on the component node tag, the component node attribute and the internal structure of the component.
[0049] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the page component generation method provided in the various optional implementations described above.
[0050] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer program, which is loaded and executed by a processor to implement the page component generation method provided in the various alternative implementations described above.
[0051] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the page component generation method provided in the various optional implementations described above.
[0052] The technical solution provided in this application may include the following beneficial effects:
[0053] By parsing the content of multiple pages, the DOM tree structure corresponding to each page is obtained. By comparing the feature information of each node in each DOM tree structure, the common structural parts in each DOM tree are obtained. Based on the node feature information of the common structural parts, reusable page components for multiple pages are generated. This automates the generation of reusable page components across multiple pages, improving the efficiency and accuracy of page component generation.
[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0056] Figure 1 This illustration shows a schematic diagram of the DOM tree corresponding to an HTML webpage provided in an exemplary embodiment of this application;
[0057] Figure 2 A flowchart illustrating a page component generation method provided in an exemplary embodiment of this application is shown;
[0058] Figure 3A flowchart illustrating a page component generation method provided in an exemplary embodiment of this application is shown;
[0059] Figure 4 A schematic diagram of a DOM tree illustrating an exemplary embodiment of this application is shown;
[0060] Figure 5 A schematic diagram illustrating a JSON structure as shown in an exemplary embodiment of this application is provided.
[0061] Figure 6 A schematic diagram illustrating a page component generation method according to an exemplary embodiment of this application is shown;
[0062] Figure 7 A block diagram of a page component generation apparatus provided in an exemplary embodiment of this application is shown;
[0063] Figure 8 This application shows a structural block diagram of a computer device provided in an exemplary embodiment;
[0064] Figure 9 A structural block diagram of a computer device illustrated in an exemplary embodiment of this application is shown. Detailed Implementation
[0065] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0066] It should be understood that "several" in this article refers to one or more, and "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0067] This application provides a method for generating page components, which can automatically generate page components and improve the efficiency and accuracy of page component generation. For ease of understanding, the terms used in this application are explained below.
[0068] 1) DOM (Document Object Model)
[0069] The DOM is a set of Web (World Wide Web) standards used to describe how scripts interact with and access structured documents. Its function is to parse browser-supported documents, including HTML (Hypertext Markup Language), XML (Extensible Markup Language), and XHTML (Extensible Hypertext Markup Language), as objects. In fact, it is a programming API (Application Programming Interface) for manipulating the content contained in a document, allowing developers to read, search, modify, and delete data from the document.
[0070] The DOM model can be used to transform a structured document into a DOM tree. Programs can access and modify nodes in the tree, as well as add and delete nodes, thereby achieving the purpose of reading, searching, modifying, and deleting data from the document.
[0071] 2) DOM tree
[0072] The DOM specification does not impose any restrictions on how documents are represented. Therefore, the DOM tree is a common implementation among various document structures. The basic element of the DOM structure is the "node," and the document structure is composed of hierarchical nodes. In the DOM model, the concept of a node is very broad. The entire document is a single node called a document node. In addition, there are element nodes, attribute nodes, entity nodes, comment nodes, and so on. Figure 1 This illustration shows a schematic diagram of the DOM tree corresponding to an HTML webpage provided in an exemplary embodiment of this application, such as... Figure 1 As shown, an HTML page is a structured text, and each node in its corresponding DOM tree corresponds to a component of the HTML page. For example, node 110 represents the document node HTML, which is the root node of the DOM tree. Its child nodes are nodes 120 and 130. Node 120 represents the Head, and node 130 represents the Body. The child nodes corresponding to node 130 are nodes 140 and 150. Node 140 represents the P element, and node 150 represents the DIV (Division) element. Thus, the hierarchical structure of HTML is represented by the DOM tree.
[0073] Figure 2A flowchart illustrating an exemplary embodiment of this application is shown. This page component generation method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 2 As shown, the method for generating this page component includes the following steps:
[0074] Step 210: Obtain the page content of at least two pages.
[0075] The content of these at least two pages consists of completed independent pages provided to the front-end developers after the refactoring team has written the styles. Different pages with the same styles are also generated based on the same DOM structure and styles. Therefore, the front-end developers need to analyze the similarity of the page structures, extract the common parts in different pages, and encapsulate them into components for use when building at least two pages. By setting different parameters in the components, different page content can be displayed on different pages.
[0076] In one possible implementation, the at least two pages are HTML pages or WXML pages, or pages with an HTML webpage structure or a WXML file structure.
[0077] Step 220: Parse the page content of at least two pages and construct the Document Object Model (DOM) tree for at least two pages.
[0078] In one possible implementation, the computer device constructs the DOM tree corresponding to the page file based on the structural identifiers in the page file. These structural identifiers are pre-marked by the refactoring team when completing each individual page.
[0079] Step 230: Extract the node feature information of each node in the DOM tree of at least two pages.
[0080] Step 240: Based on the node feature information of each node in the DOM tree of at least two pages, extract the common structural parts in the DOM tree of at least two pages.
[0081] Step 250: Generate page components corresponding to the same structural parts based on the node feature information of the same structural parts.
[0082] In summary, the page component generation method provided in this application parses the page content of multiple pages to obtain the DOM tree structure corresponding to each page, compares the feature information of each node in each DOM tree structure to obtain the common structural parts in each DOM tree, and generates reusable page components for multiple pages based on the node feature information of the common structural parts. This achieves the automation of page component generation that can be reused in multiple pages, improving the generation efficiency and accuracy of page components.
[0083] In one possible implementation, the feature information of each node includes the class name and the node name of each node. In the above case, the same structural parts in the DOM trees of at least two pages can be extracted based on the class name and the node name of each node. Figure 3 This application illustrates a flowchart of a page component generation method provided in an exemplary embodiment. This page component generation method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 3 As shown, the method for generating this page component includes the following steps:
[0084] Step 310: Obtain the page content of at least two pages.
[0085] In one possible implementation, the computer device reads the content of the page to be processed by calling the `fs` module in the Node.js environment. The `fs` module is a core module of Node.js used for file operations in the operating system, typically involving reading and writing files. Node.js refers to JavaScript running on the server-side.
[0086] Step 320: Parse the page content of at least two pages and construct the DOM tree of at least two pages.
[0087] In one possible implementation, an HTML Hash class is created, and the jsdom library is used to simulate the browser environment to manipulate the DOM, parse the page content, and generate the DOM tree corresponding to each page. Figure 4 A schematic diagram of a DOM tree illustrating an exemplary embodiment of this application is shown, such as... Figure 4 As shown, each element in a document can be considered a node in the DOM tree. An HTML document can be described as a collection of nodes, including three common types of DOM nodes: element nodes, text nodes, and attribute nodes. Figure 4Nodes 410 (page, html), 420 (body), 430 (view), and 440 (a) are all element nodes, i.e., tags; nodes 450 (text a) and 460 (text b) are the content displayed to the user; and node 470 (attribute href) is an attribute node used to represent the attributes of the corresponding element, such as link attributes, loop attributes, selectable attributes, replaceable attributes, etc.
[0088] Step 330: Extract the node feature information of each node in the DOM tree of at least two pages.
[0089] In one possible implementation, the node characteristic information of each node includes attributes such as nodeType (node class name), nodeName (node name), and attributes (node attributes). Among them, nodeName is a read-only attribute that cannot be written or modified. nodeName can be the tag name of an element node, the attribute name of an attribute node, #text of a text node, and #document of a document node.
[0090] In one possible implementation, when extracting node feature information from each node in the DOM trees of at least two pages, a JSON structure corresponding to each node can be constructed. Figure 5 A schematic diagram illustrating the JSON structure shown in an exemplary embodiment of this application is provided, as follows: Figure 5 As shown, the node feature information of node 510 includes node content, node flag, class name, node name, node attributes, and level information; among them, the level information is the node feature information added to the node according to the number of levels traversed in the DOM tree when extracting the node feature information.
[0091] In one possible implementation, in response to a node having child nodes, i.e., a node being a parent node, the corresponding node feature information also includes child information (the children attribute).
[0092] In one possible implementation, when creating the JSON structure, a createSimpleDom class can be defined to pass in the node name and hierarchy information of the nodes.
[0093] In one possible implementation, text nodes and empty nodes should be excluded when generating the JSON structure, and the format of nodeName should be standardized, such as using uppercase English letters, to prevent non-standard format from affecting the comparison results.
[0094] In one possible implementation, the node attributes of the element node are obtained. Each element node has at least two node attributes. The node attributes are traversed. If any element node attribute contains a "-" symbol, a flag is added to that element node attribute to indicate that the element node attribute is selectable.
[0095] In one possible implementation, the text content of the text node is obtained. Since some content in the text is fixed and some is replaceable, the fixed and replaceable text content are separated using a "-" marker, generating corresponding arrays. If the array length is greater than 1, a variable is marked at the replaceable text content. For example, for the text content "Name: Xiaoming", "Name:" is the fixed text content, while "Xiaoming" is replaceable text; therefore, a variable is marked at the array corresponding to "Xiaoming". Simultaneously, the text type of the text content is obtained, determining whether it is a string or a number, so that different parameters can be input based on the text type when using the generated page component.
[0096] Step 340: Perform hash calculation based on the first feature information of each node in the DOM tree of at least two pages to obtain the hash attribute of each node.
[0097] The first feature information includes the class name and node name of the corresponding node.
[0098] In one possible implementation, the node characteristic information of each node also includes the identifier of the corresponding node, which is used to indicate whether the node is an optional node.
[0099] In one possible implementation, the target node has valid child nodes, where a valid child node is a child node among the target node's child nodes that does not have an optional node identifier.
[0100] In response to the target node not having valid child nodes, the class name and node name of the target node are concatenated to obtain the concatenated target node. The target node is any one of the nodes in the DOM tree of at least two pages.
[0101] Calculate the hash value of the concatenated result of the target node to obtain the hash attribute of the target node.
[0102] For example, if the target node's node class name is "test1test2" and the node name is "NameNodeHttpServer", then the concatenation result of the target node is "test1test2 NameNodeHttpServer", which means concatenating the string representing the node class name and the string identifying the node name into a single string.
[0103] In one possible implementation, the target node has valid child nodes. The hash attribute of each node can be obtained by performing hash calculations based on the first feature information of each node in the DOM tree of at least two pages.
[0104] In response to the target node having valid child nodes, the class name and node name of the valid child nodes are concatenated to obtain the concatenated result of the valid child nodes. The target node is any one of the nodes in the DOM tree of at least two pages.
[0105] Calculate the hash value of the concatenation result of valid child nodes;
[0106] Concatenate the class name and node name of the target node;
[0107] Calculate the hash value of the concatenated result of the target node;
[0108] The hash value of the concatenation result of the target node, and the hash value of the concatenation result of the valid child nodes are used as the hash attribute of the target node.
[0109] In other words, for a target node with valid child nodes, its hash attribute is determined by the first characteristic information of the target node and the first characteristic value of the valid child nodes. The hash attribute of the target node can reflect the structural attributes of its child nodes. If the hash attributes of two target nodes are exactly the same, it means that the target node is the same as the other target node, and the valid child nodes of the target node are also the same as the child nodes of the other target node.
[0110] In one possible implementation, in response to the presence of at least two duplicate nodes among the valid child nodes, when concatenating the class name and node name of the valid node, the class name and node name of one of the at least two duplicate nodes are concatenated to obtain the concatenation result of the valid child node. That is, for duplicate nodes among the child nodes, only one is taken as a valid child node, its class name and node name are concatenated, a hash value is obtained by hash calculation, and then it is used together with the hash value of the target node as the hash attribute of the target node.
[0111] In one possible implementation, the concatenated node result is hashed using the MD5 Message-Digest Algorithm.
[0112] In one possible implementation, the class name of the same target node can contain multiple element class names, and the node name of the same target node consists of at least two element node names. The order of element class names and element node names is not important. However, during hash calculation, the order of different element class names or different element node names will affect the concatenation result of the target node, resulting in different hash values calculated based on the concatenation result, that is, different hash attributes of the target node. Taking node class names as an example, one node has the class name "test1 test2", and another node has the class name "test2 test1". In terms of structural style, the class names of these two nodes are the same, but in terms of hash calculation, due to the different order of element class names, the hash results calculated based on the above two class names are different.
[0113] To avoid the above situation, in one possible implementation, when the class name consists of at least two element class names and the node name consists of at least two element node names, before extracting the common structural parts in the DOM trees of at least two pages based on the node feature information of each node in the DOM trees of at least two pages, the following steps are also included:
[0114] Sort at least two element class names of each node in the first specified order to obtain the class name of each node;
[0115] Sort at least two element node names of each node according to the second specified order to obtain the node name of each node.
[0116] The first specified order and the second specified order can be the same or different. In one possible implementation, the first specified order / second specified order can be implemented as an order of the first letter of the element class name / element node name.
[0117] It should be noted that the above descriptions of the first and second sorting orders are illustrative. Front-end developers can set their own sorting rules according to actual needs, and this application does not impose any restrictions on this.
[0118] In one possible implementation, before sorting the element class names in the class names, the following is also included:
[0119] Preprocess the class names of nodes; including:
[0120] Based on the "-" identifier defined in each element class name in the node class name, a flag is marked on the element class name to indicate that the element class name is optional or variable. When obtaining the class name of the node, the element class name marked with the flag can be combined with other class names to form the class name of the node. Alternatively, the element class name marked with the flag can be treated as an invalid class name and excluded when forming the class name of the node.
[0121] Remove leading and trailing empty strings from the node class names, and sort them into arrays separated by spaces, such as "test1 test2".
[0122] Step 350: Extract the common structural parts from the DOM trees of at least two pages based on the hash attributes of each node in the DOM trees of at least two pages.
[0123] In one possible implementation, the node feature information also includes the corresponding node's hierarchy information, which corresponds to the node's hierarchy in the DOM tree. The above-mentioned extraction of the same structural portion of the DOM trees of at least two pages based on the hash attributes of each node in the DOM trees of at least two pages includes:
[0124] Retrieve the hash attribute of a node contained in the DOM tree of at least two pages, and obtain the hash attributes of at least two nodes; these at least two nodes are nodes with the same hierarchical information;
[0125] In response to the fact that the hash properties of at least two nodes satisfy a specified condition, at least two nodes are identified as nodes belonging to the same structural part.
[0126] Taking three pages as an example, we obtain the hash attribute of the first node in the DOM tree of the first page, the hash attribute of the second node in the DOM tree of the second page that has the same hierarchical information as the first node, and the hash attribute of the third node in the DOM tree of the third page that has the same hierarchical information as the first node. We then compare the hash attributes of the three nodes pairwise to determine whether they meet the specified conditions. If the hash attributes of all three nodes meet the specified conditions, the three nodes are determined to belong to the same structural part, that is, the three nodes are the same. If the hash attributes of all three nodes do not meet the specified conditions, the three nodes are determined to belong to different structural parts, that is, the three nodes are determined to have different nodes.
[0127] In one possible implementation, the specified condition includes: the overlap ratio of hash attributes between at least two nodes reaches a specified threshold.
[0128] Since the hash value of a node and the hash values of its child nodes are used as the hash attributes of a node when obtaining the hash attributes, a specified threshold can be set when comparing hash attributes to determine the similarity between node structures.
[0129] In one possible implementation, the hash attributes of one node from each of the DOM trees in at least two pages are obtained, following the hierarchical information from the root node to the child nodes, thus obtaining the hash attributes of at least two nodes. In other words, when comparing the hash attributes of nodes in the DOM tree, the comparison is performed in order from the root node to the child nodes. This eliminates the need to determine whether the child nodes of the parent node are the same when the hash attribute of the parent node differs from those of the corresponding nodes on other pages, based on the hierarchical structure of the DOM tree. The child nodes corresponding to the parent node are skipped during the next level of traversal, thereby reducing unnecessary waste of computer resources.
[0130] For example, when comparing the hash attributes of the first node and the second node, if only the hash values of the first and second nodes are the same, and all other hash values in the hash attributes of the first and second nodes are different, then further comparisons at the corresponding branches of the first and second nodes in the DOM tree are meaningless. Therefore, in response to the fact that the hash attributes of the first and second nodes do not meet the specified conditions, the matching of the hash attributes of the nodes in the corresponding branches of the first and second nodes in the DOM tree is stopped.
[0131] In one possible implementation, nodes with optional node identifiers in the DOM tree can be obtained as optional nodes of the same structural part, and added or deleted as needed when generating page components.
[0132] Step 360: Based on the node feature information of the same structural part, generate the page component corresponding to the same structural part.
[0133] In one possible implementation, the node feature information of the same structural part can be converted into a JSON file. By traversing the JSON file, DOM nodes can be generated in reverse, and then page components corresponding to the same structural part can be generated.
[0134] In one possible implementation, when generating page components with the same structural parts, a DOM tree is constructed based on the node feature information of the nodes in the same structural parts, following the hierarchical information from child nodes to root nodes, to obtain the page components corresponding to the same structural parts.
[0135] In one possible implementation, node feature information also includes the node attributes of the corresponding node and the child attributes.
[0136] When retrieving page components corresponding to the same structural parts, component node tags are generated based on the node names of the nodes in the same structural parts, following the hierarchical information from child nodes to root nodes.
[0137] Component node attributes are generated based on the node attributes of nodes in the same structural part.
[0138] The internal structure of a component is built based on the child attributes of nodes in the same structural part;
[0139] Based on component node tags, component node attributes, and the internal structure of the component, page components with the same structural parts are generated.
[0140] The method described in this application can be applied to the page component generation environment of mobile data display pages or management backends. Since the page structure in the above environment is relatively simple and the pages are relatively simple, the solution described in this application can greatly improve the generation efficiency and accuracy of page components.
[0141] In summary, the page component generation method provided in this application parses the page content of multiple pages to obtain the DOM tree structure corresponding to each page, compares the feature information of each node in each DOM tree structure to obtain the common structural parts in each DOM tree, and generates reusable page components for multiple pages based on the node feature information of the common structural parts. This achieves the automation of page component generation that can be reused in multiple pages, improving the generation efficiency and accuracy of page components.
[0142] Figure 6 A schematic diagram illustrating a page component generation method according to an exemplary embodiment of this application is shown, such as... Figure 6As shown, this page component generation method can extract page content 610 from multiple pages, construct the DOM tree structure 620 corresponding to each page, extract the node feature information of each node in the DOM tree structure of each page, and form the JSON structure corresponding to each node. The figure takes node 1 in page 611, node 2 in page 612, and node 3 in page 613 as examples to illustrate the method. The corresponding JSON structure 630 is obtained. The JSON structure of each node contains the node feature information of the corresponding node. Based on the class name and node name in the node feature information, the hash attribute of the corresponding node is generated. It is determined whether the hash attributes of nodes 1, 2, and 3 at the same level meet the specified conditions. If they do, it is determined that nodes 1, 2, and 3 belong to the same structural part. The leaf node feature information is converted and added to the JSON file. Then, the subsequent judgment of its child nodes is performed (not shown in the figure) until all nodes are traversed. The page component is generated in reverse according to the JSON file corresponding to the same structural part.
[0143] Figure 7 This illustration shows a block diagram of a page component generation apparatus provided in an exemplary embodiment of this application. This page component generation apparatus can be applied in a computer device, which can be implemented as a server or a terminal, such as... Figure 7 As shown, the page component generation device includes:
[0144] Page content acquisition module 710 is used to acquire the page content of at least two pages;
[0145] Page content parsing module 720 is used to parse the page content of at least two pages and construct the Document Object Model (DOM) tree of at least two pages;
[0146] The node feature information extraction module 730 is used to extract the node feature information of each node in the DOM tree of at least two pages;
[0147] The same structure extraction module 740 is used to extract the same structure parts in the DOM trees of at least two pages based on the node feature information of each node in the DOM trees of at least two pages.
[0148] The page component generation module 750 is used to generate page components corresponding to the same structural parts based on the node feature information of the same structural parts.
[0149] In one possible implementation, the node feature information includes first feature information, which includes the class name and node name of the corresponding node;
[0150] The same structure extraction module 740 includes:
[0151] The hash attribute retrieval submodule is used to perform hash calculations based on the first feature information of each node in the DOM tree of at least two pages to obtain the hash attribute of each node;
[0152] The Same Structure Extraction submodule is used to extract the same structural parts of the DOM trees of at least two pages based on the hash attributes of each node in the DOM trees of at least two pages.
[0153] In one possible implementation, the hash attribute acquisition submodule includes:
[0154] The first concatenation result acquisition unit is used to concatenate the class name and node name of the target node in response to the target node not having valid child nodes, and obtain the concatenation result of the target node; the target node is any one of the nodes in the DOM tree of at least two pages;
[0155] The first hash value calculation unit is used to calculate the hash value of the concatenation result of the target node and obtain the hash attribute of the target node.
[0156] In one possible implementation, the hash attribute acquisition submodule includes:
[0157] The second concatenation result acquisition unit is used to concatenate the class name and node name of the valid child nodes in response to the target node having valid child nodes, and obtain the concatenation result of the valid child nodes; the target node is any one of the nodes in the DOM tree of at least two pages;
[0158] The second hash value calculation unit is used to calculate the hash value of the concatenation result of valid child nodes;
[0159] The first concatenation result acquisition unit is used to concatenate the class name and the node name of the target node.
[0160] The first hash value calculation unit is used to calculate the hash value of the concatenation result of the target node;
[0161] The hash attribute acquisition unit is used to obtain the hash value of the concatenation result of the target node and the hash value of the concatenation result of the effective child nodes as the hash attribute of the target node.
[0162] In one possible implementation, the second splicing result acquisition unit is used to splice the class name and node name of one of the at least two duplicate nodes in response to the presence of at least two duplicate nodes among the valid child nodes, thereby obtaining the splicing result of the valid child nodes.
[0163] In one possible implementation, the class name consists of at least two element class names, the node name consists of at least two element node names, and the device further includes:
[0164] The first sorting module is used to sort at least two element class names of each node in a first specified order before the same structure extraction module 740 extracts the same structure part in the DOM tree of at least two pages based on the node feature information of each node in the DOM tree of at least two pages, so as to obtain the class name of each node.
[0165] The second sorting module is used to sort at least two element node names of each node according to a second specified order to obtain the node names of each node.
[0166] In one possible implementation, the node feature information also includes the hierarchy information of the corresponding node, which is used to indicate the hierarchy of the node in the DOM tree;
[0167] The same structure extraction module 740 includes:
[0168] The `getSubmodule` is used to retrieve the hash attribute of a node contained in the DOM tree of at least two pages, resulting in the hash attributes of at least two nodes; the at least two nodes are nodes with the same hierarchical information.
[0169] The determination submodule is used to determine at least two nodes as belonging to the same structural part in response to the hash properties between at least two nodes satisfying specified conditions.
[0170] In one possible implementation, the specified conditions include:
[0171] At least two nodes must have a hash attribute overlap ratio that reaches a specified threshold.
[0172] In one possible implementation, a submodule is obtained to retrieve the hash attribute of a node contained in the DOM tree of at least two pages, in order from the root node to the child node according to the hierarchical information, so as to obtain the hash attributes of at least two nodes.
[0173] In one possible implementation, the page component generation module 750 is used to construct a DOM tree based on the node feature information of nodes in the same structural part, according to the hierarchical information from child nodes to root nodes, to obtain the page components corresponding to the same structural part.
[0174] In one possible implementation, the node feature information also includes the node attributes and child attributes of the corresponding node;
[0175] This page component generation module 750 includes:
[0176] The component node label generation submodule is used to generate component node labels based on the node names of nodes in the same structural part, in the order from child nodes to root nodes according to the hierarchical information.
[0177] The component node property generation submodule is used to generate component node properties based on the node properties of nodes in the same structural part;
[0178] The component internal structure construction submodule is used to build the component internal structure based on the child attributes of nodes in the same structural part;
[0179] The page component generation submodule is used to generate page components with the same structural parts based on the component node tag, the component node attributes, and the component's internal structure.
[0180] In summary, the page component generation apparatus provided in this application embodiment, applied in a computer device, parses the page content of multiple pages to obtain the DOM tree structure corresponding to each page, compares the feature information of each node in each DOM tree structure to obtain the common structural parts in each DOM tree, and generates reusable page components for multiple pages based on the node feature information of the obtained common structural parts. This automates the generation of page components that can be reused in multiple pages, improving the generation efficiency and accuracy of page components.
[0181] Figure 8 This illustration shows a structural block diagram of a computer device 800 provided in an exemplary embodiment of this application. The computer device 800 may be a portable mobile terminal, such as a smartphone, tablet computer, laptop computer, or desktop computer. The computer device 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.
[0182] Typically, computer device 800 includes a processor 801 and a memory 802.
[0183] Processor 801 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0184] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 802 are used to store at least one instruction, which is executed by the processor 801 to implement the page component generation method provided in the method embodiments of this application.
[0185] In some embodiments, the computer device 800 may also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 707, a positioning assembly 808, and a power supply 809.
[0186] Peripheral device interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 801 and memory 802. In some embodiments, processor 801, memory 802 and peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 801, memory 802 and peripheral device interface 803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.
[0187] In some embodiments, the computer device 800 further includes one or more sensors 810. The one or more sensors 810 include, but are not limited to: an accelerometer 811, a gyroscope 812, a pressure sensor 813, a fingerprint sensor 814, an optical sensor 815, and a proximity sensor 816.
[0188] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0189] Figure 9 A structural block diagram of a computer device 900 illustrated in an exemplary embodiment of this application is shown. This computer device can be implemented as the protective blocking device described above in this application. The computer device 900 includes a Central Processing Unit (CPU) 901, a system memory 904 including Random Access Memory (RAM) 902 and Read-Only Memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. The computer device 900 also includes a basic input / output system (I / O system) 906 that facilitates information transfer between various devices within the computer, and a mass storage device 907 for storing the operating system 913, application programs 914, and other program modules 915.
[0190] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 910 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, printer, or other types of output devices.
[0191] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include computer-readable media (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.
[0192] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.
[0193] According to various embodiments of this disclosure, the computer device 900 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 900 can be connected to a network 912 via a network interface unit 911 connected to the system bus 905, or it can use the network interface unit 911 to connect to other types of networks or remote computer systems (not shown).
[0194] The memory further includes at least one instruction, at least one program, code set, or instruction set, which are stored in the memory. The central processing unit 901 executes the at least one instruction, at least one program, code set, or instruction set to implement all or part of the steps in the page component generation method shown in the above embodiments.
[0195] In one exemplary embodiment, a computer-readable storage medium is also provided, wherein at least one computer program is stored therein, the computer program being loaded and executed by a processor to implement the above. Figure 2 or Figure 3 All or part of the steps of the method shown in any embodiment. For example, the non-transitory computer-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0196] In one exemplary embodiment, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned actions. Figure 2 or Figure 3 All or part of the steps of the method shown in any embodiment.
[0197] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0198] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method of generating a page component, characterized by, The method includes: Retrieve the content of at least two pages; The page content of the at least two pages is parsed, and the Document Object Model (DOM) tree of the at least two pages is constructed; Extract node feature information of each node in the DOM tree of the at least two pages; the node feature information includes first feature information and the corresponding node's hierarchy information, the first feature information includes the class name and node name of the corresponding node, and the hierarchy information is used to indicate the hierarchy of the corresponding node in the DOM tree; Hash calculation is performed on the first feature information of each node in the DOM tree of the at least two pages to obtain the hash attribute of each node; the hash attribute of one node contained in the DOM tree of each of the at least two pages is obtained to obtain the hash attribute of at least two nodes; the at least two nodes are nodes with the same hierarchical information; in response to the hash attribute between the at least two nodes satisfying the specified condition, the at least two nodes are determined to be nodes belonging to the same structural part. Based on the node feature information of the same structural part, the page component corresponding to the same structural part is generated.
2. The method of claim 1, wherein, The step of performing hash calculations based on the first feature information of each node in the DOM tree of the at least two pages to obtain the hash attribute of each node includes: In response to the fact that the target node does not have valid child nodes, the class name and the node name of the target node are concatenated to obtain the concatenated result of the target node; the target node is any one of the nodes in the DOM tree of the at least two pages; Calculate the hash value of the concatenation result of the target node to obtain the hash attribute of the target node.
3. The method of claim 1, wherein, The step of performing hash calculations based on the first feature information of each node in the DOM tree of the at least two pages to obtain the hash attribute of each node includes: In response to the target node having valid child nodes, the class name and node name of the valid child nodes are concatenated to obtain the concatenation result of the valid child nodes; the target node is any one of the nodes in the DOM tree of the at least two pages; Calculate the hash value of the concatenation result of the valid child nodes; The class name and the node name of the target node are concatenated; Calculate the hash value of the concatenation result of the target node; The hash value of the concatenation result of the target node and the hash value of the concatenation result of the effective child nodes are used as the hash attribute of the target node.
4. The method according to claim 3, characterized in that, The concatenation of the class name and node name of the valid child nodes to obtain the concatenation result of the valid child nodes includes: In response to the fact that the valid child nodes include at least two duplicate nodes, the class name and node name of one of the at least two duplicate nodes are concatenated to obtain the concatenation result of the valid child nodes.
5. The method according to any one of claims 1 to 4, characterized in that, The class name consists of at least two element class names, the node name consists of at least two element node names, and before extracting the common structural parts in the DOM trees of the at least two pages based on the node feature information of each node in the DOM trees of the at least two pages, the method further includes: Sort the at least two element class names of each node in a first specified order to obtain the class name of each node; The node names of each node are obtained by sorting the at least two element node names of each node in a second specified order.
6. The method according to claim 1, characterized in that, The specified conditions include: The overlap ratio of the hash attributes between at least two nodes reaches a specified threshold.
7. The method according to claim 1, characterized in that, The step of obtaining the hash attribute of a node contained in the DOM trees of the at least two pages, and obtaining the hash attribute of the at least two nodes, includes: According to the hierarchical information from the root node to the child node, obtain the hash attribute of one node contained in the DOM tree of each of the at least two pages, and obtain the hash attributes of at least two nodes.
8. The method according to claim 1, characterized in that, The step of generating page components corresponding to the same structural parts based on the node feature information of the same structural parts includes: Following the hierarchical information from child nodes to root nodes, a DOM tree is constructed based on the node feature information of nodes in the same structural part, thereby obtaining the page components corresponding to the same structural part.
9. The method according to claim 8, characterized in that, The node feature information also includes the node attributes and child attributes of the corresponding node; The step of constructing a DOM tree based on the node feature information of nodes in the same structural part, according to the hierarchical information from child nodes to root nodes, to obtain the page components corresponding to the same structural part, includes: Component node labels are generated based on the node names of nodes in the same structural part, in order from child nodes to root nodes according to the hierarchical information. Component node attributes are generated based on the node attributes of the nodes in the same structural part. The internal structure of the component is constructed based on the child attributes of the nodes in the same structural part; Based on the component node tag, the component node attribute, and the internal structure of the component, the page component corresponding to the same structural part is generated.
10. A page component generation device, characterized in that, The device includes: The page content retrieval module is used to retrieve the page content of at least two pages; The page content parsing module is used to parse the page content of the at least two pages and construct the Document Object Model (DOM) tree of the at least two pages; The node feature information extraction module is used to extract the node feature information of each node in the DOM tree of the at least two pages; the node feature information includes first feature information and the corresponding node's hierarchy information, the first feature information includes the class name and node name of the corresponding node, and the hierarchy information is used to indicate the hierarchy of the corresponding node in the DOM tree. The same structure extraction module is used to perform hash calculation based on the first feature information of each node in the DOM tree of the at least two pages to obtain the hash attribute of each node; obtain the hash attribute of a node contained in the DOM tree of each of the at least two pages to obtain the hash attributes of at least two nodes; the at least two nodes are nodes with the same hierarchical information; in response to the hash attributes between the at least two nodes satisfying a specified condition, the at least two nodes are determined to be nodes belonging to the same structural part; The page component generation module is used to generate page components corresponding to the same structural parts based on the node feature information of the same structural parts.
11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set; the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the page component generation method as described in any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the page component generation method as described in any one of claims 1 to 9.
13. A computer program product, characterized in that, The computer program product includes computer instructions that are executed by a processor of a computer device to implement the page component generation method as described in any one of claims 1 to 9.