Application service state modeling method, device and equipment based on close page recognition

Abstract

The invention discloses an application service state modeling method, device and equipment based on close page recognition, and belongs to the field of data processing. Comprising the following steps: traversing different pages of an application by using a reinforcement learning method to obtain all to-be-identified pages of a target application; defining four types of close pages, and combining a plurality of nodes contained in the DOM trees of the two to-be-identified pages to obtain a simplified DOM tree by taking elimination of interference of the first type, the second type and the third type of close pages as a target; coding the simplified DOM tree to obtain a simplified page coding sequence by aiming at eliminating the interference of the fourth type of close pages; inputting the obtained simplified page coding sequence into a pre-trained contrast learning model, generating vector embedding, and performing close page identification based on the vector embedding; under the condition that the two to-be-recognized pages are close pages, modeling the two to-be-recognized pages to be in the same service state; and under the condition that the two to-be-identified pages are not the close pages, modeling the two to-be-identified pages into different business states.

Description

Technical Field

[0001] This application belongs to the field of data processing, and specifically relates to an application business state modeling method, apparatus and equipment based on close relative page recognition. Background Technology

[0002] In recent years, data backfeeding has involved collecting user data scattered across multiple applications through various methods. The data backfeeding process mainly comprises two stages: data location and data extraction. The first stage primarily involves finding the required data from various modules of social media apps, such as extracting "conversation data" from the "chat" function of an app. The second stage involves extracting the found data from the app, such as extracting "conversation data" from a "chat with someone" page into corresponding text, images, voice, and video. Considering that user data backfeeding is not a one-time task, and that different users, different social media applications, different time periods, and different AI models all have different backfeeding needs, both data location and data extraction face significant challenges.

[0003] To address the issue of data redundancy caused by accessing the same data through different paths within a single application, and considering user convenience in app design, the same data can often be found through different access paths. For example, when using WeChat, to retrieve the conversation content with "User A," one can access the conversation from the contacts list or search for the user in the search box. If accessed during the backflow process, the obtained data may be redundant, leading to uneven distribution and potentially causing model training errors. Furthermore, because data frequently changes—for example, conversation data increases with each conversation round, the data accessed in the current session differs from the data accessed in the next session—detecting data redundancy solely through data comparison becomes extremely difficult. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, and device for application business state modeling based on close-related page identification, which can solve the problem of data redundancy in the business state modeling of target applications caused by the difficulty in identifying close-related pages.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an application business state modeling method based on close relative page identification, the method comprising: Use reinforcement learning methods to traverse different pages of the application and obtain all pages of the target application to be identified. Four types of close relatives are defined. With the goal of eliminating interference from the first, second and third types of close relatives, multiple nodes contained in the DOM trees of the two pages to be identified are merged to obtain two simplified DOM trees. Taking two simplified DOM trees as input, and aiming to eliminate interference from close relatives of the fourth type, the two simplified DOM trees are encoded to obtain two simplified page encoding sequences; The two simplified page encoding sequences are input into a pre-trained contrastive learning model to generate vector embeddings, and close relative page identification is performed based on the vector embeddings. If two pages to be identified are closely related, they will be modeled as having the same business state; if two pages to be identified are not closely related, they will be modeled as having different business states. The first type is dynamic data loading on the page, the second type is dynamic attribute generation on the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement on the page.

[0006] Optionally, multiple nodes contained in the DOM trees of the two pages to be identified are merged to obtain two simplified DOM trees, including: Assign hash values ​​to the tags corresponding to each node in the DOM tree of the two pages to be identified, with tags of the same type having the same hash value; Based on the hash values ​​of multiple tags, an encoding is generated for each node in the DOM tree in the order from leaf node to root node, resulting in an encoded value for each node. The encoded value of each node contains the complete path from the root node to that node. For each DOM tree, nodes with the same encoding value are merged according to the encoding values ​​of multiple nodes contained in the DOM tree to obtain a simplified DOM tree.

[0007] Optionally, multiple nodes contained in the DOM trees of the two pages to be identified are merged to obtain two simplified DOM trees, including: Determine whether the common ancestor node of the multiple subtrees contained in the DOM tree is greater than a preset number of nodes; if the common ancestor node of the multiple subtrees is greater than the preset number of nodes, merge the multiple subtrees; And / or, determine whether the DOM tree contains a preset type tag; if the DOM tree contains a preset type tag, delete the child node of the node corresponding to the preset type tag.

[0008] Optionally, for each DOM tree, nodes with the same encoded value in the DOM tree are merged according to the encoded values ​​of multiple nodes contained in the DOM tree, including: Determine whether there are multiple subtrees with the same encoded value in the DOM tree; In the case of multiple subtrees with the same encoded value, determine the node containing each of the multiple subtrees with the same encoded value; Multiple subtrees with the same encoded value belonging to the same node are merged into one subtree.

[0009] Optionally, the method further includes: Retain multiple subtrees with the same encoded value belonging to different nodes; Determine whether the DOM structure contains clickable or invisible nodes; If the DOM structure contains clickable nodes and / or invisible nodes, the clickable nodes and invisible nodes are retained; Preserve all leaf nodes.

[0010] Optionally, the training process of the contrastive learning model includes the following steps: Obtain a real page dataset, which includes a positive sample set and a negative sample set. The positive sample set contains multiple positive samples, and the negative sample set contains multiple negative sample pairs. Each positive sample pair includes a query sample and a positive sample, which correspond to closely related pages with the same business status. Each negative sample pair includes a query sample and a negative sample, which correspond to pages with different business statuses. During the training of the contrastive learning model, the vector embedding of the query sample in the positive sample pair is input into the standard encoder to obtain the query embedding of the query sample, and the positive and negative samples are input into the momentum encoder to obtain the key embedding; the parameters of the momentum encoder are fixed during the training process and are gradually adjusted through the momentum update strategy. Based on the query embedding and key embedding of the positive sample pair, the positive sample cosine similarity of the positive sample pair is calculated, and based on the query embedding and key embedding of the negative sample pair, the negative sample cosine similarity of the negative sample pair is calculated. A similarity matrix is ​​constructed based on the cosine similarity of the positive samples and the cosine similarity of the negative samples; The parameters of the standard encoder are updated based on the loss function value calculated from the similarity matrix, and the trained contrastive learning model is obtained by minimizing the loss function value through multiple iterations. During training, the key embeddings of the negative samples are multiple historical embeddings generated by the momentum encoder and stored in a queue of preset capacity. The queue is updated using a first-in-first-out mechanism. During training, new key embeddings are added to the tail of the queue, and old embeddings at the head of the queue are removed to keep the capacity of the queue constant.

[0011] Optionally, the process of constructing the positive samples includes the following steps: Obtain the node sequence from the real page dataset; For the node sequence in the real page dataset, the nodes in the node sequence are deleted or retained using the dropout method to obtain multiple positive samples.

[0012] Optionally, the two simplified DOM trees are encoded to obtain two simplified page encoding sequences that include at least one of the following methods; The frequency of each tag in the DOM tree is counted; the frequency of each tag is encoded to obtain a simplified page encoding sequence of the page to be identified. Alternatively, multiple tags contained in the DOM tree can be encoded sequentially to obtain a simplified page encoding sequence of the page to be identified.

[0013] Secondly, embodiments of this application provide a business state machine modeling apparatus based on close relative page recognition, the apparatus comprising: The page acquisition module is used to traverse different pages of the application using reinforcement learning methods to acquire all pages of the target application to be identified. The node merging module defines four types of close relatives pages. With the goal of eliminating interference from the first, second, and third types of close relatives pages, it merges multiple nodes contained in the DOM trees of two pages to be identified to obtain two simplified DOM trees. The contrastive learning module takes two simplified DOM trees as input and aims to eliminate interference from close relative pages of type IV. It encodes the two simplified DOM trees to obtain two simplified page encoding sequences. The close relative page recognition module is used to input the two simplified page encoding sequences into a pre-trained contrastive learning model to generate vector embeddings, and to perform close relative page recognition based on the vector embeddings. The state modeling module is used to model two pages to be identified as having the same business state when they are closely related pages, and to model two pages to be identified as having different business states when they are not closely related pages. The first type is dynamic data loading on the page, the second type is dynamic attribute generation on the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement on the page.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] In this embodiment, to address dynamic data loading (first type of interference), dynamic attribute generation (second type of interference), and the presence of expandable elements (third type of interference), two simplified DOM trees are obtained by merging multiple nodes in the DOM trees of the two pages to be identified. This reduces noise from dynamic content and variable attributes while preserving the core structural features of the page. Next, to eliminate dynamic element rearrangement (fourth type of interference), the two simplified DOM trees are encoded to generate two simplified page encoding sequences. This encoding method minimizes the impact of dynamic element rearrangement on the identification process. Finally, the two simplified page encoding sequences are input into a pre-trained contrastive learning model to generate vector embeddings, and close-related page identification is performed based on these embeddings. The contrastive learning model effectively distinguishes between semantically similar and semantically dissimilar pages by learning the similarity between samples. When two pages to be identified are close-related, they are modeled as having the same business state; otherwise, they are modeled as having different business states. This method not only improves the accuracy of close-related page identification but also provides a more accurate and reliable basis for modeling application business states. By eliminating various types of interference and combining a contrastive learning model, accurate identification of related pages is achieved, improving the accuracy and efficiency of target application business state modeling. Business state modeling through this solution can play a crucial role in subsequent expansion of model training datasets, returning data control to users, and enhancing user-driven personalized services and innovative applications. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an application business state modeling method based on close relative page identification, provided in one embodiment of this application. Figure 2 This is a complete flowchart of an application business state modeling method based on close relative page identification provided in one embodiment of this application; Figure 3 This is a flowchart of generating a simplified DOM tree provided in one embodiment of this application; Figure 4 This is a schematic diagram of node merging provided in an embodiment of this application; Figure 5 This is a schematic diagram of the framework of an application business state modeling device based on close relative page recognition according to an embodiment of this application; Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0019] The following description, in conjunction with the accompanying drawings, details an application business state modeling method based on close relative page identification provided in this application, through specific embodiments and application scenarios.

[0020] This application provides a method for modeling application business states based on close-related page identification. By employing this method within an Automated Web GUI Testing Approaches (AWGTs), it automatically models the business states of a given target application (such as a social media application). Close-related pages (pages that appear different but are actually the same) are merged together through state abstraction. In other words, by merging redundant structures and combining contrastive learning, pages are effectively abstracted and classified. The classification results guide the testing software in modeling, avoiding misclassification of the same business state due to subtle differences when reached through different paths. This ensures that the data for the corresponding business state is unique, reducing the potential risk of data redundancy.

[0021] Figure 1 This is a flowchart illustrating an application business state modeling method based on close relative page identification, provided in one embodiment of this application. Figure 2 This is a complete flowchart of an application business state modeling method based on close relative page identification provided in one embodiment of this application.

[0022] refer to Figure 1 One embodiment of this application provides an application service state modeling method based on close relative page identification, the method including steps S11 to S15: Step S11: Use reinforcement learning methods to traverse different pages of the application and obtain all pages of the target application to be identified.

[0023] refer to Figure 2 In this embodiment, during the automated testing process, the agent selects the next action to be executed through a reinforcement learning algorithm to obtain the current page of the target application according to different access paths and add the current page to the page to be identified. Through a series of actions, multiple pages to be identified are obtained. Any two pages to be identified are used for subsequent close relative page identification.

[0024] Step S12: Define four types of close relatives pages. With the goal of eliminating interference from the first, second, and third types of close relatives pages, merge the multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees.

[0025] Specifically, from a visual perspective, nearly identical pages can be divided into three types: those with changing appearances, those with dynamic data, and those with repetitive pages.

[0026] (1) Appearance changes: The changes between pages are mainly reflected in aesthetics (such as changes in background images), while the functions remain unchanged.

[0027] (2) Dynamic data: The two pages are initialized from the same template, but are populated with different data.

[0028] (3) Duplicate pages: There are additional web page elements on the page, but their functions and semantics are completely reflected in another page.

[0029] To better understand close-knit pages (pairs of nearly identical pages) from the perspective of the DOM, this application uses the DOM as input for the page classification method and further optimizes it based on the three classification methods mentioned above, dividing close-knit pages into four types, each of which will receive specialized processing.

[0030] Specifically, in this embodiment, the four types of near-repetition mentioned above can be accurately detected to improve the accuracy of page classification and the effectiveness of state abstraction. The first type is dynamic data loading of the page, the second type is dynamic attribute generation of the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement of the page.

[0031] Type 1: Dynamic Data Loading. The data content of page modules (such as tweets or images in the travel module) will be dynamically refreshed, but because of the consistent theme and structure, these pages should be identified as related pages.

[0032] The second type: dynamically generated attributes. HTML tag attributes (such as IDs) on a page change dynamically across different access paths. These pages should be identified as close relatives.

[0033] The third type: Expandable elements. Pages contain expandable elements (such as lists or grids) that change dynamically based on user behavior. For example, expandable elements appear when a user posts new content (such as a tweet); if there is no content, the elements are not displayed. These types of pages should be identified as close relatives.

[0034] The fourth type: Dynamic element rearrangement. The order of page elements may change due to loading, addition, or deletion, but these changes in position do not affect the functionality of the page. This type of page should be identified as a close relative page.

[0035] The above four categories are the four closely related page types summarized in this application. This application can effectively classify these four types of similar duplicate page pairs, ensuring that HTML pages with the same business logic are merged into a unified business state, thereby improving the accuracy of page classification and the effect of state abstraction.

[0036] Specifically, in this embodiment, HTML pages with the same business logic should be merged into a unified business state. The goal of the DOM structure merging stage is to simplify the HTML DOM tree of the page and reduce the interference of dynamic data loading, dynamic attribute generation, and extensible elements on page classification.

[0037] Therefore, the goal is to eliminate interference from closely related pages of the first, second, and third types. This is achieved by filtering attributes to retain only the tags corresponding to the nodes, merging redundant structures in the DOM tree, and eliminating the interference of redundant structures on page classification. This simplifies the process of multiple nodes contained in the DOM trees of the two pages to be identified, ultimately resulting in two simplified DOM trees corresponding to the two pages to be identified.

[0038] The simplified DOM tree output by this step is a simplified sequence of DOM nodes containing only HTML tags. By merging subtrees with the same structure and removing unnecessary DOM text and attributes, i.e. merging nodes with the same structure and filtering out some meaningless nodes, the length of the original DOM tree is significantly reduced, thereby effectively eliminating the impact of the three types of dynamic content changes.

[0039] Step S13: Taking two simplified DOM trees as input, and aiming to eliminate interference from the fourth type of close relative pages, encode the two simplified DOM trees to obtain two simplified page encoding sequences; In this embodiment, after generating two simplified DOM trees, the problem of classifying three types of close relatives pages is solved by eliminating redundant structures in the DOM trees. Therefore, only the fourth type of close relative page problem remains unsolved, namely dynamic element rearrangement.

[0040] To address the issue of closely related pages in the fourth type of dynamic element rearrangement, this embodiment eliminates the sequential relationships between multiple nodes in the simplified DOM tree by encoding two simplified DOM trees, thereby obtaining two simplified page encoding sequences.

[0041] Step S14: Input the two simplified page encoding sequences into the pre-trained contrastive learning model to generate vector embeddings, and perform close relative page identification based on the vector embeddings.

[0042] In this embodiment, after training the contrastive learning model, the model encodes two simplified page encoding sequences into vector embeddings. Next, to achieve better classification performance, a Support Vector Machine (SVM) is used as the classifier. Compared to threshold-based classification methods, the classification results obtained using the SVM classifier have stronger generalization ability. In terms of adaptability, the SVM classifier can flexibly learn decision boundaries and handle more complex classification tasks. Therefore, SVM can provide stronger classification capabilities for this task, especially when facing diverse and complex page data, and is more effective than traditional threshold-based classification methods.

[0043] After input, the input embedding vector is normalized and the vector dimension is preserved by element-wise multiplication. At the same time, cosine similarity is calculated and InfoNCE loss is used to use cosine similarity as the similarity measure.

[0044] The following formula illustrates the process of cosine similarity calculation, where A and B represent the input vector embeddings, C represents the result vector obtained through similarity calculation, and the result vector is then fed into a support vector machine (SVM) for classification.

[0045] .

[0046] Step S15: If the two pages to be identified are closely related pages, model the two pages to be identified as having the same business state; if the two pages to be identified are not closely related pages, model the two pages to be identified as having different business states.

[0047] In this embodiment, if the support vector machine's close-family page identification result indicates that the two pages to be identified are close-family pages, it means that the two pages have the same business logic but may have some differences in content or structure. Therefore, these two pages need to be modeled as the same business state. Conversely, if the support vector machine's close-family page identification result indicates that the two pages to be identified are not close-family pages, it means that the two pages have different business logic. Therefore, these two pages need to be modeled as different business states.

[0048] This embodiment addresses the interference from dynamic data loading (first type of interference), dynamic attribute generation (second type of interference), and the presence of extensible elements (third type of interference) on a page. By merging multiple nodes in the DOM trees of the two pages to be identified, two simplified DOM trees are obtained. This reduces noise from dynamic content and variable attributes while preserving the core structural features of the page. Next, to eliminate dynamic element rearrangement (fourth type of interference), the two simplified DOM trees are encoded, generating two simplified page encoding sequences. This encoding method minimizes the impact of dynamic element rearrangement on the identification process. Finally, the two simplified page encoding sequences are input into a pre-trained contrastive learning model to generate vector embeddings, which are then used for close-related page identification. The contrastive learning model effectively distinguishes between semantically similar and semantically dissimilar pages by learning the similarity between samples. When two pages are identified as close-related, they are modeled as having the same business state; otherwise, they are modeled as having different business states. This method not only improves the accuracy of close-related page identification but also provides a more precise and reliable basis for modeling application business states. By eliminating various types of interference and combining a contrastive learning model, accurate identification of related pages is achieved, improving the accuracy and efficiency of target application business state modeling. Business state modeling through this solution can play a crucial role in subsequent expansion of model training datasets, returning data control to users, and enhancing user-driven personalized services and innovative applications.

[0049] Figure 3 This is a flowchart of generating a simplified DOM tree according to an embodiment of this application, see reference. Figure 3 In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, step S12, "merging multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees," specifically includes steps S12-1 to S12-3: Step S12-1: Assign hash values ​​to the tags corresponding to each node in the DOM trees of the two pages to be identified. Tags of the same type have the same hash value.

[0050] In this embodiment, for the DOM trees of the two pages to be identified, each page first contains multiple tags, and a hash value is assigned to each tag on the page. The hash value assignment for the tags starts from "001" and is carried out step by step until the hash values ​​of all tags have been assigned.

[0051] It is important to pay special attention to tags of the same type (e.g.)and<divclass="Chat"> Although their attributes (such as class) are different, they are considered to be the same type of tag when hashing, and therefore have the same hash value.

[0052] Step S12-2: Based on the hash values ​​of multiple tags, generate an encoding for each node in the DOM tree in the order from leaf node to root node, and obtain the encoding value of each node. The encoding value of each node contains the complete path from the root node to that node.

[0053] In this embodiment, each tag corresponds to a node in the DOM tree, and the encoding process of each node in the DOM tree follows these rules: Encoding begins from the bottom up, starting with the leaf nodes and recursively working towards the parent nodes. Each node's encoded value contains the complete path from the root node to that node. When a node has multiple child nodes, they are sorted according to the hash values ​​of their corresponding tags, with nodes having smaller hash values ​​appearing first. This sorting uses a prefix sorting method, where each parent node's encoded value consists of its own hash value and the encoded values ​​of all its child nodes, thus ensuring the consistency and integrity of the entire DOM tree structure.

[0054] The following example illustrates the node encoding process. Suppose we have a page with the following DOM tree: <section> < / section> ; Includes the following tags: The label was assigned the hash value "009". The label was assigned the hash value "003". <section>The label was assigned the hash value "020". The tag was assigned the hash value "017", and the tag was assigned the hash value "002". The label was assigned the hash value "016".

[0055] The following is the encoding for each node of the DOM tree in this example: ： It is a leaf node with a hash value of "017", so the encoded value of this node is 017.

[0056] This node is also a leaf node, and its hash value is "016". Therefore, the encoded value of this node is 016.

[0057] <section>:for <sapn>and The parent node has a hash value of "020". When sorting sibling nodes, the node with the smaller hash value is ranked first. (Hash value "017") is ranked in Before the hash value "016", according to the prefix sorting rule, the final encoded value is 020 / 016 / 017.

[0058] :for <section>The parent node has a hash value of "003", so the encoded value of this node is 003 / 020 / 016 / 017.

[0059] :for The parent node has a hash value of "002", so the encoded value of this node is 002 / 003 / 020 / 016 / 017.

[0060] : is the parent node of, and its hash value is "009", so the encoded value of this node is 009 / 002 / 003 / 020 / 016 / 017.

[0061] Step S12-3: For each DOM tree, merge nodes with the same encoding value according to the encoding values ​​of multiple nodes contained in the DOM tree to obtain a simplified DOM tree.

[0062] In this embodiment, nodes are merged after encoding. The node merging process is to merge sibling nodes with the same encoding value. Considering that these duplicate nodes have a certain degree of redundancy, which will affect the accuracy of subsequent close relative page recognition, the nodes need to be merged according to the redundancy during the merging process.

[0063] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, step S12-3, "merging multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees," specifically includes steps S12-3-1 and / or step S12-3-2: Step S12-3-1: Determine whether the common ancestor node of the multiple subtrees contained in the DOM tree is greater than the preset number of nodes; if the common ancestor node of the multiple subtrees is greater than the preset number of nodes, merge the multiple subtrees.

[0064] In this embodiment, the redundancy in the DOM tree can be merged by determining whether the number of common ancestor nodes among any number of subtrees in the DOM tree is greater than a preset number of nodes. If the number of common ancestor nodes among multiple subtrees is greater than the preset number of nodes, the multiple subtrees under these common ancestor nodes are merged to solve the close relative page problem of the first and third types.

[0065] In the DOM tree, a common ancestor node refers to the ancestor node shared by two or more subtrees. By determining whether the number of these common ancestor nodes exceeds a preset number (e.g., 3), a decision is made on whether to merge these subtrees, thereby resolving the issues of dynamic data loading and expandable elements in close-knit pages.

[0066] Specifically, there are issues related to dynamic data loading and scalable elements. Dynamic data loading refers to situations where certain parts of a page (such as list items, table rows, etc.) change based on dynamically loaded data. For example, a news page might dynamically load different news items based on backend data. By merging these dynamically loaded parts (i.e., subtrees with a common ancestor node exceeding a preset number of nodes), the complexity of the DOM tree can be reduced, thereby improving the accuracy of identifying subsequent close-kin pages.

[0067] Extensible elements refer to elements on a page whose quantity may change dynamically. For example, the number of items in the shopping cart page of a shopping app may change dynamically depending on the number of items added by the user. However, they are essentially part of the same business logic. By merging these extensible elements, the complexity of the DOM tree can be reduced, thereby improving the accuracy of identifying subsequent related pages.

[0068] By determining whether the number of common ancestor nodes is greater than a preset value, it can be decided whether to merge these subtrees.

[0069] The specific steps are as follows: 1. Traverse the DOM tree: Starting from the root node, traverse each subtree of the DOM tree level by level.

[0070] 2. Find the common ancestor node: For multiple subtrees, find their common ancestor node.

[0071] 3. Determine the number of common ancestor nodes: If the number of common ancestor nodes is greater than the preset value, it means that these subtrees have similar structures and can be merged.

[0072] 4. Merge subtrees: Merge multiple subtrees under these common ancestor nodes into one subtree.

[0073] And / or, in step S12-3-2, determine whether the DOM tree contains a preset type tag; if the DOM tree contains a preset type tag, delete the child node of the node corresponding to the preset type tag.

[0074] In this embodiment, the second type of problem of closely related pages generated by dynamic attributes refers to the existence of dynamically generated attributes (such as class, id, style, etc.) for certain tags on a page. Changes in these attributes may lead to the page being misclassified. For example, two pages may be structurally identical, but if the class attribute values ​​of certain tags are different, they may be incorrectly identified as pages with different business logic.

[0075] Therefore, to address the issue of closely related pages generated by dynamic attributes, we can first determine whether the DOM tree contains predefined type tags: predefined type tags refer to those tags that contain dynamically generated attributes. For example, 、 , Then, traverse the DOM tree, checking if these preset type tags exist. If they are found, delete their child nodes. This eliminates interference from dynamic attribute generation on subsequent close-knit page identification.

[0076] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, step S12-3, "for each DOM tree, according to the encoding values ​​of multiple nodes contained in the DOM tree, merge nodes with the same encoding value in the DOM tree", specifically includes steps S12-3-3 to S12-3-5: Step S12-3-3: Determine whether there are multiple subtrees with the same encoded value in the DOM tree; Step S12-3-4: In the case of multiple subtrees with the same encoding value, determine the node where each of the multiple subtrees with the same encoding value is located; Step S12-3-5: Merge multiple subtrees with the same encoded value belonging to the same node into one subtree.

[0077] Specifically, in this embodiment, subtrees with the same encoding under the same node will be merged. Since some dynamically generated content can cause redundant structures on the page, for example, list items or table rows may generate duplicate subtrees due to data-driven generation. Merging such redundant structures can eliminate noise caused by changes in data volume and preserve the templated features of the DOM tree. However, the merging scope is limited to the same parent node to avoid merging across parent nodes and disrupting the page hierarchy logic of the page to be identified.

[0078] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification, which further includes steps S21 to S24: Step S21: Retain multiple subtrees with the same encoded value belonging to different nodes; Step S22: Determine whether the DOM structure contains clickable or invisible nodes; Step S23: If the DOM structure contains clickable nodes and / or invisible nodes, retain the clickable nodes and invisible nodes. Step S24: Retain all leaf nodes.

[0079] Specifically, in this embodiment, subtrees with the same structure under different nodes will not be merged, because subtrees with the same structure under different parent nodes usually carry independent functional modules (such as navigation bars and sidebars both containing list structures, but belonging to subtrees under different parent nodes). Preserving their independence can maintain the modularity of the page and prevent semantic confusion caused by accidental merging.

[0080] Clickable nodes (such as buttons) will not be merged. The functionality of interactive elements depends on their independent event bindings and context. Merging may disrupt click event positioning; for example, if two "Submit" buttons are merged, it will be impossible to distinguish their respective form areas.

[0081] Invisible nodes (such as those with the style attribute "display: none;") will not be merged. Although visually invisible, they may serve as backend functions such as dynamic loading placeholders and script control. For example, different modules in some applications are rendered using the invisible attribute, but their structure in the DOM tree is completely consistent.

[0082] All leaf nodes will not be merged. Leaf nodes are typically content endpoints (such as text nodes, images, and links), directly carrying valid information or function entry points. Merging will result in the loss of content features; for example, two... The nodes contain the text for "price" and "inventory" respectively, and their semantic differences cannot be distinguished after merging.

[0083] Figure 4 This is a schematic diagram of node merging provided in an embodiment of this application, see reference. Figure 4 .

[0084] After the above merging operation, the DOM tree structure is further simplified. The number of nodes in the simplified DOM tree is no greater than the number of nodes in the original DOM tree. The merged nodes will then have their tags re-extracted to generate the final tag sequence. In this way, it is ensured that the DOM tree structure of the page to be identified is not only concise but also accurately reflects the core content of the page.

[0085] For example, the above steps can be referenced in the following page merging example: <section class="main-content"> < / section> <button class="btn primary">< / button> <button class="btn primary">< / button> The code above is an example of page merging, and the corresponding DOM tree structure is as follows: Figure 4 As shown, in this example, the node and nodes Nodes with the same encoding should be merged, while nodes... and nodes<style="display: none;"> Although they have the same encoding, they are actually invisible nodes and therefore will be skipped during merging. <section class="content">and nodes They have the same encoding, but because the two nodes belong to different subtrees, they will not be merged. The next node Since it is a leaf node, it will not be merged. under the node <button class="btnprimary"> Node is< / button> <button>They will not merge.

[0086] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, the training process of the contrastive learning model includes the following steps S31 to S35: Step S31: Obtain the real page dataset, which includes a positive sample set and a negative sample set. The positive sample set contains multiple positive samples, and the negative sample set contains multiple negative sample pairs. The positive sample pair includes a query sample and a positive sample, which correspond to closely related pages with the same business status. The negative sample pair includes a query sample and a negative sample, which correspond to pages with different business statuses.

[0087] In this embodiment, a real-world page dataset needs to be obtained before training the contrastive learning model. This dataset includes a positive sample set and a negative sample set. The positive sample set contains multiple positive sample pairs, each pair consisting of a query sample and a positive sample. These pairs correspond to closely related pages in the same business state; that is, the two pages in a positive sample pair are identical in business logic, but may differ in content or structure. The negative sample set contains multiple negative sample pairs, each pair consisting of a query sample and a negative sample. These pairs correspond to pages in different business states. The two pages in a negative sample pair are different in business logic. This provides the model with sufficient data to learn the similarities and differences between pages.

[0088] In step S32, during the training of the contrastive learning model, the vector embedding of the query sample in the positive sample pair is input into the standard encoder to obtain the query embedding of the query sample, and the positive and negative samples are input into the momentum encoder to obtain the key embedding; the parameters of the momentum encoder are fixed during the training process and are gradually adjusted through the momentum update strategy.

[0089] In this embodiment, during the training of the contrastive learning model, the vector embedding of the query sample in the positive sample pair is input into the standard encoder to obtain the query embedding of the query sample. Simultaneously, the positive and negative samples are input into the momentum encoder to obtain the key embedding.

[0090] The standard encoder and momentum encoder have the same structure, but they update their parameters differently. The parameters of the standard encoder are updated during training using gradient descent, while the parameters of the momentum encoder are gradually adjusted using a momentum update strategy to maintain relative stability.

[0091] Step S33: Calculate the positive sample cosine similarity of the positive sample pair based on the query embedding and key embedding of the positive sample pair; and calculate the negative sample cosine similarity of the negative sample pair based on the query embedding and key embedding of the negative sample pair.

[0092] In this embodiment, after obtaining the query embedding of the query sample and the key embeddings of the positive and negative samples, the positive sample cosine similarity of the positive sample pair is calculated based on the query embedding and key embedding of the positive sample pair. Similarly, the negative sample cosine similarity of the negative sample pair is calculated based on the query embedding and key embedding of the negative sample pair. Cosine similarity is a commonly used similarity measurement method, which measures the similarity between two vectors by calculating the cosine value of the angle between them. The value of cosine similarity ranges from -1 to 1; the closer the value is to 1, the more similar the two vectors are, and the closer the value is to -1, the less similar the two vectors are. The similarity between positive and negative sample pairs is quantified by calculating cosine similarity.

[0093] Step S34: Construct a similarity matrix based on the cosine similarity of the positive samples and the cosine similarity of the negative samples.

[0094] In this embodiment, a similarity matrix is ​​constructed based on the calculated cosine similarity of positive and negative samples. The similarity matrix is ​​a two-dimensional array where each element represents the similarity between a query sample and either a positive or negative sample. In this matrix, positive sample pairs typically have higher similarity values, while negative sample pairs have lower similarity values. The similarity matrix is ​​constructed to integrate the similarity information of all sample pairs for calculating the loss function in subsequent steps. The similarity matrix provides a visual representation of the similarity distribution between different sample pairs.

[0095] Step S35: Calculate the loss function value based on the similarity matrix to update the parameters of the standard encoder, and minimize the loss function value through multiple iterations to obtain the trained contrastive learning model; During training, the key embeddings of the negative samples are multiple historical embeddings generated by the momentum encoder and stored in a queue of preset capacity. The queue is updated using a first-in-first-out mechanism. During training, new key embeddings are added to the tail of the queue, and old embeddings at the head of the queue are removed to keep the capacity of the queue constant.

[0096] In this embodiment, during training, a loss function value is calculated based on the constructed similarity matrix, and this loss function value is used to update the parameters of the standard encoder. The loss function used is the InfoNCE loss function, which encourages high similarity between positive sample pairs and low similarity between negative sample pairs. By minimizing the loss function value, the model parameters can be optimized, enabling the model to better learn the similarities and differences between pages. Furthermore, during training, a larger batch size of negative samples results in better training performance; therefore, to enhance the model's learning of negative samples, large-scale training with negative samples is necessary.

[0097] However, when processing very long input data, the batch size of negative samples cannot be set too large due to limited training resources, making it difficult to apply a large number of negative samples, which may affect the final training effect. To address this issue, this embodiment uses a queue to cache the key embeddings generated by the momentum encoder in each training batch, and stores this queue in CPU memory.

[0098] Specifically, during training, the key embeddings of negative samples are obtained from a queue of pre-defined capacity, which stores multiple historical embeddings generated by the momentum encoder. After each training batch, the key embeddings generated in the current batch are pushed to the tail of the queue, while the oldest key embedding is removed from the head of the queue. The embeddings stored in the queue then participate as negative samples in the training of subsequent batches to maintain a constant queue capacity. This mechanism ensures the diversity of negative samples, thereby improving the training effect and generalization ability of the model. Through multiple iterations of training, the loss function value is gradually minimized, ultimately resulting in a well-trained contrastive learning model.

[0099] In this way, CPU memory can be used to expand the size of the negative sample queue, thereby providing rich and stable negative samples for contrastive learning and further improving the model's embedding representation capabilities. The advantage of this method is that the negative samples in the queue are gradually accumulated through multiple training batches, providing a stable and abundant negative sample pool, which can effectively improve the model's learning performance and promote feature representation learning in close-page classification tasks.

[0100] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, the process of constructing the positive sample includes the following steps S41 and S42: Step S41: Obtain the node sequence from the real page dataset; Step S42: For the node sequence in the real page dataset, the nodes in the node sequence are deleted or retained in the dropout method to obtain multiple positive samples.

[0101] Specifically, contrastive learning is an unsupervised learning method whose core idea is to construct positive and negative samples so that the model can learn effective feature representations of the input data. By maximizing the similarity between positive samples while minimizing the similarity between positive and negative samples, the model can learn discriminative features. However, if there are significant differences between positive samples and query samples, the model may have difficulty distinguishing them from negative samples, resulting in poor learning performance; conversely, if positive samples are highly similar to or identical to query samples, the model may easily select them, leading to overfitting or model collapse.

[0102] Therefore, constructing positive and negative samples is a crucial step in contrastive learning, directly impacting the model's learning performance. Negative samples represent different pages; since their construction is relatively simple and can be obtained through sampling, the focus of sample construction is concentrated on building positive samples.

[0103] The construction of positive samples typically relies on data augmentation techniques. In the image domain, common methods include image cropping, rotation, and color dithering, which do not alter the core features of the image. In natural language processing, positive samples can be generated using synonym replacement, back-translation, and word deletion. However, these data augmentation methods are not suitable for HTML page classification tasks because the node sequences of HTML pages lack semantic features, and node deletion may disrupt the page structure. Therefore, conventional data augmentation methods cannot be directly applied.

[0104] Based on this, this embodiment proposes an innovative method for constructing positive samples. Specifically, positive samples can be generated by dropping out the node sequences of multiple HTML pages contained in a real page dataset, thus effectively solving the problem of positive sample construction. Using this method, the model can generate sufficiently effective positive samples without disrupting the page structure, providing more stable and effective training data for page classification tasks.

[0105] In conjunction with the technical solutions of the above embodiments, another embodiment of this application also provides an application business state modeling method based on close relative page identification. In this method, the step S13 of "encoding two simplified DOM trees to obtain two simplified page encoding sequences" includes at least one of the following methods; This embodiment provides two encoding methods: encoding the quantity of each type of HTML tag and encoding only the total number of tags, as detailed below: Step S13-1: Calculate the frequency of occurrence of each tag among the multiple tags contained in the DOM tree; encode the frequency of occurrence of each tag to obtain a simplified page encoding sequence of the page to be identified.

[0106] In this embodiment, encoding the quantity of each HTML tag refers to treating each DOM tree as a tag set by counting the number of times each HTML tag appears on the page, and encoding the frequency of these tags contained in the DOM tree. In this way, the model can classify based on the frequency of tag occurrence, regardless of the order of elements.

[0107] This process uses the 120 standard HTML elements defined by the W3C specification and sorts these elements. It's important to note that the encoding starts from the second position because the first position represents the total number of nodes. For example, if a tag is position 51 in the HTML document and appears 3 times on the page, then the 52nd position in the encoded vector corresponds to the number 3. When counting tag occurrences, the number of occurrences is stored in a fixed-length vector of length 121, where the first position stores the total number of tags, and the remaining positions store the frequency of each HTML tag.

[0108] The following example illustrates the coding process. First, the number of occurrences of the tags contained in the DOM tree is counted: total (16), html (1), body (1), div (5), span (3), section (1), p (3), button (2).

[0109] Assuming the nodes are arranged in order from one to seven, the encoding result is: [16, 1, 1, 5, 3, 1,3, 2, 0, 0, 0, ..., 0].

[0110] Since the BERT model requires a fixed-length input, the input sequence needs further processing to meet the format requirements of the Transformer model: a start symbol ([CLS], number 101) is added to the beginning of the input sequence, and an end symbol ([SEP], number 102) is added to the end of the input sequence; the input sequence is padded to a length of 512 using a padding symbol ([PAD], number 0). Therefore, the final input sequence to BERT is:

[101] 16 1 1 5 3 1 3 2 0 0 0 ... 0

[102] 0 ... 0. By converting the frequency information of the occurrence of HTML tags into a fixed-length vector through this encoding method and encoding it with BERT, the structural information of the page can be effectively processed and classification tasks can be performed.

[0111] Alternatively, in step S13-2, the multiple tags contained in the DOM tree are encoded sequentially to obtain a simplified page encoding sequence of the page to be identified.

[0112] In this embodiment, encoding only the total number of tags is another encoding method. This method does not require counting the number of tags. All tags are treated as the same type of tag and encoded starting from 1. Each tag is encoded sequentially until the last element. For example, if the total number of tags in the DOM tree is 16, its encoding would be: [1, 2, 3,..., 16].

[0113] This approach is useful in practical applications when behavioral differences between two pages are often difficult to discern when the number of tags is exactly the same, especially in modern web pages where the number of tags often reaches tens of thousands. However, directly encoding a single value as a high-dimensional vector is not suitable for the Transformer model. Therefore, instead of directly using the tag count value itself, a series of consecutive positive integers, the same number as the total number of tags, is chosen as the input.

[0114] Meanwhile, in order to effectively handle such long inputs, Longformer can be chosen as the encoder. Longformer is a Transformer model that uses an attention mechanism. By linearizing the extension of the sequence length, it can handle long input sequences. For this purpose, the input window size is increased to 4,096 tokens. Since the input DOM has been processed by the above structure merging algorithm before processing, according to our evaluation, this algorithm can shorten the length of the DOM tree by more than 90%. Therefore, a window of 4,096 tokens is sufficient to cope with the complexity of modern web pages. The Longformer starts with a symbol ("0") and ends with a symbol ("2"), and then fills with padding symbols "1" until the maximum input size (4,096) is reached. The final sequence is: [0] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [2][1] [1]...[1] (filled to 4,096 tokens). This encoding method simplifies tag representation by assigning each tag a consecutive integer from 1 to 16. At the same time, the Longformer encoder effectively handles lengthy inputs, thus meeting the challenges of complex modern web page structures.

[0115] It should be noted that the application service state modeling method based on close-related page identification provided in this application embodiment can be executed by an application service state modeling device based on close-related page identification, or by a control module in the application service state modeling device for executing the method of loading application service state modeling based on close-related page identification. This application embodiment uses the execution of the method of loading application service state modeling based on close-related page identification by the application service state modeling device as an example to illustrate the application service state modeling method based on close-related page identification provided in this application embodiment.

[0116] Figure 5 This is a schematic diagram of the framework of an application business state modeling device based on close relative page recognition according to an embodiment of this application. (Refer to...) Figure 5 .

[0117] One embodiment of this application provides a business state machine modeling apparatus based on close-related page recognition, the apparatus comprising: Page acquisition module 11 is used to traverse different pages of the application using reinforcement learning methods to acquire all pages to be identified in the target application. The node merging module 12 is used to define four types of close relatives pages. With the goal of eliminating interference from the first, second and third types of close relatives pages, it merges multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees. The contrastive learning module 13 is used to encode two simplified DOM trees as input, with the goal of eliminating interference from close relatives of the fourth type, to obtain two simplified page encoding sequences; The close relative page recognition module 14 is used to input the two simplified page encoding sequences into a pre-trained contrastive learning model to generate vector embeddings and perform close relative page recognition based on the vector embeddings. The state modeling module 15 is used to model two pages to be identified as having the same business state when the two pages to be identified are closely related pages; and to model two pages to be identified as having different business states when the two pages to be identified are not closely related pages. The first type is dynamic data loading on the page, the second type is dynamic attribute generation on the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement on the page.

[0118] Optionally, the node merging module 12 includes: The hash value allocation unit is used to assign hash values ​​to the tags corresponding to each node in the DOM tree of the two pages to be identified. Tags of the same type have the same hash value. The node encoding unit is used to generate an encoding for each node in the DOM tree based on the hash values ​​of multiple tags, in the order from leaf node to root node, to obtain the encoding value of each node. The encoding value of each node contains the complete path from the root node to that node. The node merging unit is used to merge nodes with the same encoding value in each DOM tree according to the encoding values ​​of multiple nodes contained in the DOM tree, so as to obtain a simplified DOM tree.

[0119] Optionally, the node merging unit includes: The first merging unit is used to determine whether the common ancestor node of the multiple subtrees contained in the DOM tree is greater than a preset number of nodes; if the common ancestor node of the multiple subtrees is greater than the preset number of nodes, the multiple subtrees are merged. And / or, the second merging unit is used to determine whether the DOM tree contains a preset type tag; if the DOM tree contains a preset type tag, the child nodes of the node corresponding to the preset type tag are deleted.

[0120] Optionally, the node merging unit further includes: The judgment unit is used to determine whether there are multiple subtrees with the same encoded value in the DOM tree; The determining unit is used to determine the node of each of the multiple subtrees with the same encoded value when there are multiple subtrees with the same encoded value. The subtree merging unit is used to merge multiple subtrees with the same encoded value belonging to the same node into one subtree.

[0121] Optionally, the device further includes: The first retention unit is used to retain multiple subtrees with the same encoded value belonging to different nodes; A special node determination unit is used to determine whether the DOM structure contains clickable or invisible nodes; The second retention unit is used to retain the clickable nodes and / or invisible nodes when the DOM structure contains clickable nodes and / or invisible nodes; The third reserved unit is used to retain all leaf nodes.

[0122] Optionally, the device further includes: The real page dataset acquisition module is used to acquire a real page dataset, which includes a positive sample set and a negative sample set. The positive sample set contains multiple positive samples, and the negative sample set contains multiple negative sample pairs. The positive sample pair includes a query sample and a positive sample, and the query sample and the positive sample correspond to closely related pages with the same business status. The negative sample pair includes a query sample and a negative sample, and the query sample and the negative sample correspond to pages with different business statuses. The embedding acquisition module is used to input the vector embedding of the query sample in the positive sample pair into the standard encoder to obtain the query embedding of the query sample during the training process of the contrastive learning model, and to input the positive and negative samples into the momentum encoder to obtain the key embedding; the parameters of the momentum encoder are fixed during the training process and are gradually adjusted through the momentum update strategy. The cosine similarity calculation module is used to calculate the positive sample cosine similarity of the positive sample pair based on the query embedding and key embedding of the positive sample pair, and to calculate the negative sample cosine similarity of the negative sample pair based on the query embedding and key embedding of the negative sample pair. The similarity matrix calculation module is used to construct a similarity matrix based on the cosine similarity of the positive samples and the cosine similarity of the negative samples. The iterative training module is used to update the parameters of the standard encoder by calculating the loss function value based on the similarity matrix, and to obtain the trained contrastive learning model by minimizing the loss function value through multiple iterations. During training, the key embeddings of the negative samples are multiple historical embeddings generated by the momentum encoder and stored in a queue of preset capacity. The queue is updated using a first-in-first-out mechanism. During training, new key embeddings are added to the tail of the queue, and old embeddings at the head of the queue are removed to keep the capacity of the queue constant.

[0123] Optionally, the device further includes: The node sequence acquisition module is used to obtain the node sequence from the real page dataset; The positive sample construction module is used to delete or retain nodes in the node sequence of the real page dataset using dropout to obtain multiple positive samples.

[0124] Optionally, the contrastive learning module 13 includes: The first encoding module is used to count the frequency of each tag among the multiple tags contained in the DOM tree; and to encode the frequency of each tag to obtain a simplified page encoding sequence of the page to be identified. Alternatively, a second encoding module is used to encode multiple tags contained in the DOM tree in sequence to obtain a simplified page encoding sequence of the page to be identified.

[0125] The business state machine modeling device based on close-relationship page recognition in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the specific implementation.

[0126] The business state machine modeling device based on close-related page recognition in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0127] The business state machine modeling device based on close relative page recognition provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented by the business state machine modeling device based on close relative page identification in the method embodiment will not be described again here to avoid repetition.

[0128] Optionally, Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. This application also provides an electronic device; it should be noted that the electronic device in this application includes the mobile electronic device and non-mobile electronic device described above.

[0129] The electronic device includes, but is not limited to, components such as: radio frequency unit, network module, audio output unit, input unit, sensor, display unit, user input unit, interface unit, memory, and processor.

[0130] Those skilled in the art will understand that electronic devices may also include a power supply (such as a battery) that supplies power to various components. The power supply can be connected to the processor logic through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0131] As an example, such as Figure 6 As shown, the electronic device 600 includes a memory 610 and a processor 620. The memory 610 and the processor 620 are connected via a bus for communication. The memory 610 stores a computer program that can run on the processor 620 to implement the steps in the business state machine modeling method based on close relative page recognition disclosed in the above embodiments of this application.

[0132] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.

[0133] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0134] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.

[0135] Furthermore, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described business state machine modeling method embodiment based on close relative page identification and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0136] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0137] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described business state machine modeling method embodiment based on close relative page identification, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0138] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0141] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.< / button> < / section> < / section> < / sapn> < / section> < / section>

Claims

1. A method for modeling application business status based on close relative page recognition, characterized in that, The method includes: Use reinforcement learning methods to traverse different pages of the application and obtain all pages of the target application to be identified. Four types of close relatives are defined. With the goal of eliminating interference from the first, second and third types of close relatives, multiple nodes contained in the DOM trees of the two pages to be identified are merged to obtain two simplified DOM trees. Taking two simplified DOM trees as input, and aiming to eliminate interference from close relatives of the fourth type, the two simplified DOM trees are encoded to obtain two simplified page encoding sequences; The two simplified page encoding sequences are input into a pre-trained contrastive learning model to generate vector embeddings, and close relative page identification is performed based on the vector embeddings. If two pages to be identified are closely related, they will be modeled as having the same business state; if two pages to be identified are not closely related, they will be modeled as having different business states. The first type is dynamic data loading on the page, the second type is dynamic attribute generation on the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement on the page.

2. The application business state modeling method based on close relative page identification according to claim 1, characterized in that, Merge multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees, including: Assign hash values ​​to the tags corresponding to each node in the DOM tree of the two pages to be identified, with tags of the same type having the same hash value; Based on the hash values ​​of multiple tags, an encoding is generated for each node in the DOM tree in the order from leaf node to root node, resulting in an encoded value for each node. The encoded value of each node contains the complete path from the root node to that node. For each DOM tree, nodes with the same encoding value are merged according to the encoding values ​​of multiple nodes contained in the DOM tree to obtain a simplified DOM tree.

3. The application business state modeling method based on close relative page identification according to claim 2, characterized in that, Merge multiple nodes contained in the DOM trees of the two pages to be identified to obtain two simplified DOM trees, including: Determine whether the common ancestor node of the multiple subtrees contained in the DOM tree is greater than a preset number of nodes; if the common ancestor node of the multiple subtrees is greater than the preset number of nodes, merge the multiple subtrees; And / or, determine whether the DOM tree contains a preset type tag; if the DOM tree contains a preset type tag, delete the child node of the node corresponding to the preset type tag.

4. The application business state modeling method based on close relative page identification according to claim 2, characterized in that, For each DOM tree, nodes with the same encoded value are merged according to the encoded values ​​of multiple nodes contained in that DOM tree, including: Determine whether there are multiple subtrees with the same encoded value in the DOM tree; In the case of multiple subtrees with the same encoded value, determine the node containing each of the multiple subtrees with the same encoded value; Multiple subtrees with the same encoded value belonging to the same node are merged into one subtree.

5. The application business state modeling method based on close relative page identification according to claim 4, characterized in that, The method further includes: Retain multiple subtrees with the same encoded value belonging to different nodes; Determine whether the DOM structure contains clickable or invisible nodes; If the DOM structure contains clickable nodes and / or invisible nodes, the clickable nodes and invisible nodes are retained; Preserve all leaf nodes.

6. The application business state modeling method based on close relative page identification according to claim 1, characterized in that, The training process of the contrastive learning model includes the following steps: Obtain a real page dataset, which includes a positive sample set and a negative sample set. The positive sample set contains multiple positive samples, and the negative sample set contains multiple negative sample pairs. Each positive sample pair includes a query sample and a positive sample, which correspond to closely related pages with the same business status. Each negative sample pair includes a query sample and a negative sample, which correspond to pages with different business statuses. During the training of the contrastive learning model, the vector embedding of the query sample in the positive sample pair is input into the standard encoder to obtain the query embedding of the query sample, and the positive and negative samples are input into the momentum encoder to obtain the key embedding; the parameters of the momentum encoder are fixed during the training process and are gradually adjusted through the momentum update strategy. Based on the query embedding and key embedding of the positive sample pair, the positive sample cosine similarity of the positive sample pair is calculated, and based on the query embedding and key embedding of the negative sample pair, the negative sample cosine similarity of the negative sample pair is calculated. A similarity matrix is ​​constructed based on the cosine similarity of the positive samples and the cosine similarity of the negative samples; The parameters of the standard encoder are updated based on the loss function value calculated based on the similarity matrix, and the loss function value is minimized through multiple iterations to obtain the trained contrastive learning model. During training, the key embeddings of the negative samples are multiple historical embeddings generated by the momentum encoder and stored in a queue of preset capacity. The queue is updated using a first-in-first-out mechanism. During training, new key embeddings are added to the tail of the queue, and old embeddings at the head of the queue are removed to keep the capacity of the queue constant.

7. The application business state modeling method based on close relative page identification according to claim 6, characterized in that, The process of constructing the positive samples includes the following steps: Obtain the node sequence from the real page dataset; For the node sequence in the real page dataset, the nodes in the node sequence are deleted or retained using the dropout method to obtain multiple positive samples.

8. The application business state modeling method based on close relative page identification according to claim 1, characterized in that, Encoding two simplified DOM trees results in two simplified page encoding sequences that include at least one of the following methods; The frequency of each tag in the DOM tree is counted; the frequency of each tag is encoded to obtain a simplified page encoding sequence of the page to be identified. Alternatively, multiple tags contained in the DOM tree can be encoded sequentially to obtain a simplified page encoding sequence of the page to be identified.

9. A business state machine modeling device based on close relative page recognition, characterized in that, The device includes: The page acquisition module is used to traverse different pages of the application using reinforcement learning methods to acquire all pages of the target application to be identified. The node merging module is used to define four types of close relatives. With the goal of eliminating interference from the first, second, and third types of close relatives, it merges multiple nodes contained in the DOM trees of two pages to be identified to obtain two simplified DOM trees. The contrastive learning module takes two simplified DOM trees as input and aims to eliminate interference from close relative pages of type IV. It encodes the two simplified DOM trees to obtain two simplified page encoding sequences. The close relative page recognition module is used to input two simplified page encoding sequences into a pre-trained contrastive learning model to generate vector embeddings and perform close relative page recognition based on the vector embeddings. The state modeling module is used to model two pages to be identified as having the same business state when they are closely related pages, and to model two pages to be identified as having different business states when they are not closely related pages. The first type is dynamic data loading on the page, the second type is dynamic attribute generation on the page, the third type is the presence of expandable elements on the page, and the fourth type is dynamic element rearrangement on the page.

10. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the application service state modeling method based on close-related page identification as described in claims 1-8.

Images