Page content processing method, system, electronic device, and storage medium

Abstract

This disclosure provides a method, system, electronic device, and storage medium for processing page content. The method includes the following steps: in response to receiving a page processing instruction, obtaining target page association information of the target page; employing a multi-level verification strategy combined with page semantic information and execution operation description information to filter out content nodes in the target page that meet preset verification conditions as target nodes; wherein, the multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are conditions that characterize the validity of the execution operation; using the execution operation description information, performing a rendering operation on the content located at the target node in the initial explanation content to obtain the target explanation content. This disclosure can significantly reduce element positioning errors and the proportion of non-executable instructions, effectively improve guidance accuracy, and provide personalized and targeted explanations and guidance, thereby improving interactivity and learning efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, system, electronic device, and storage medium for processing page content. Background Technology

[0002] With the explosive growth of online information, users often face the pain points of information overload and operational confusion when reading long documents (such as technical manuals and papers), operating complex business systems (such as SaaS backends, data dashboards, etc.) or learning online courses.

[0003] Traditional assistance methods typically rely on users searching for keywords, consulting separate user manuals, or requesting human customer service, which is not only inefficient but also often provides guidance that is disconnected from the current page status.

[0004] Based on this, several page content processing solutions have emerged in the existing technology. Among them, the current online learning and training widely adopts the solution of using pre-recorded videos as the main carrier. However, such solutions usually require high course production and maintenance costs (recording, editing, updating, etc.) and are mainly one-way playback. In addition, most content assistants based on LLM (Large Language Model) only stay at the level of text summarization or general question and answer, and have not yet deeply solved the problem of interaction loop on the client side. They often produce illusions, such as suggesting to click non-existent buttons or quoting text not displayed on the page. There are also problems such as poor follow-up questioning effect, loss of control of reading pace of long content, weak teaching interaction, high cost, and difficulty in engineering implementation. Summary of the Invention

[0005] The technical problem to be solved by this disclosure is to overcome the problems of existing page content processing solutions, such as frequent hallucinations, poor follow-up questioning effect, loss of control over the reading pace of long content, weak teaching interaction, high cost, and difficulty in engineering implementation, as well as significant problems in visual guidance and automated reading of long processes, which cannot meet the actual use needs. The purpose is to provide a page content processing method, system, electronic device and storage medium.

[0006] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0007] In a first aspect, this disclosure provides a method for processing page content, the method comprising:

[0008] In response to receiving a page processing instruction, obtain the target page association information of the target page;

[0009] The target association information includes the page semantic information of the target page, the execution operation description information in the target page, and the corresponding initial explanation content;

[0010] A multi-level verification strategy is adopted, combining the page semantic information and the execution operation description information, to filter out content nodes in the target page that meet the preset verification conditions as target nodes;

[0011] The multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are conditions that characterize the validity of the operation.

[0012] Using the execution operation description information, a rendering operation is performed on the content located at the target node in the initial explanation content to obtain the target explanation content.

[0013] Optionally, the multi-level verification strategy includes a semantic matching verification mechanism, a structural position verification mechanism, and an interaction state verification mechanism executed sequentially.

[0014] The semantic matching verification mechanism is used to verify whether the content corresponding to the operation to be executed actually exists.

[0015] The structural position verification mechanism is used to verify whether the content corresponding to the operation to be performed is within the visual area.

[0016] The interaction state verification mechanism is used to verify whether the content corresponding to the operation to be performed is in an interactive state.

[0017] And / or,

[0018] The step of obtaining the page semantic information of the target page includes:

[0019] Based on the structure tree and layout information corresponding to the target page, the page semantic information of the target page is obtained.

[0020] Optionally, the step of using a multi-level verification strategy to combine the page semantic information and the execution operation description information to filter out content nodes in the target page that meet preset verification conditions as target nodes includes:

[0021] Based on the page semantic information and the execution operation description information, obtain the initial candidate node set corresponding to the target page;

[0022] Using the semantic matching verification mechanism, content nodes with a semantic similarity higher than a first preset value to the execution operation description information are selected from the initial candidate node set to form a first candidate node set;

[0023] The structural position verification mechanism is used to verify whether the content nodes in the first candidate node set meet the preset structural position conditions; wherein, the preset structural position conditions include at least one of the following: within the visual area, not occluded, and matching the sequence number in the DOM (Document Object Model) tree;

[0024] In response to the content nodes in the first candidate node set satisfying the preset structural position condition, the operation type of the operation to be executed is determined based on the execution operation description information, and the content nodes in the first candidate node set are verified using the interaction state verification mechanism and the operation type.

[0025] In response to the fact that the operation to be performed at the content node in the first candidate node set is in an interactive state, the corresponding content node is taken as the target node.

[0026] Optionally, the processing method further includes:

[0027] In response to the content nodes in the first candidate node set being selected through a one- or two-level verification mechanism, the content node with the highest semantic similarity to the execution operation description information is selected as the target node; and / or, the content node with the highest semantic similarity to the execution operation description information is selected as the target node, and a label with moderate confidence is added to the target node.

[0028] In response to a content node in the first candidate node set failing any level of verification mechanism, the corresponding page processing operation is performed using the local default heuristic search method; or an interactive prompt is generated to perform the corresponding page processing operation based on external interactive operations.

[0029] Optionally, the processing method is implemented based on a preset model;

[0030] The processing method further includes:

[0031] In response to a content node in the first candidate node set failing any level of verification mechanism, feedback information is generated;

[0032] Perform a rollback operation; or, adjust the preset model based on the feedback information and perform corresponding processing operations using the adjusted preset model;

[0033] And / or,

[0034] The preset model includes models of different sizes, and the processing method further includes:

[0035] The complexity of the processing task corresponding to the page processing instruction is obtained through analysis;

[0036] The matching target model is invoked based on the complexity; wherein the complexity is positively correlated with the size of the target model;

[0037] And / or,

[0038] The processing method further includes:

[0039] In response to the fact that the output of the preset model does not meet the preset conditions, a preset backup model or a local heuristic strategy is invoked to perform the corresponding page processing operation.

[0040] Optionally, the processing method further includes:

[0041] The current page is scrolled using a preset step size.

[0042] Get the currently scrolling content and trigger the generation of corresponding segmented explanation content.

[0043] Optionally, after obtaining the currently scrolling content and triggering the generation of corresponding segmented explanation content, the method further includes:

[0044] Obtain the first interactive operation and execute the corresponding operation on the current page;

[0045] The first interactive operation includes clicking the current page, and the corresponding operation includes stopping scrolling;

[0046] The first interactive operation includes scrolling the current page, and the corresponding operation includes updating the content of the context of the current position based on the new position after scrolling;

[0047] And / or,

[0048] The processing method further includes:

[0049] Obtain the character playback rate of the target explanation content;

[0050] Based on the character playback rate, control the scrolling rate of the reading cursor on the current page to synchronize the playback speed with the page scrolling speed;

[0051] And / or,

[0052] The processing method further includes:

[0053] In response to the broadcast of the explanation of the target content, an external voice interaction operation is received;

[0054] The text content corresponding to the external voice interaction operation is identified;

[0055] The text content is inserted into the broadcast queue so that it can be broadcast when explaining the page content at the corresponding location;

[0056] And / or,

[0057] The processing method further includes:

[0058] In response to broadcasting the content of the explanation of the target, upon receiving the second interactive operation, the broadcasting stops and the next round of interactive state begins.

[0059] Optionally, the processing method further includes:

[0060] Monitor the current page and adjacent pages, and use a multi-level hashing strategy to analyze the content of the current page and adjacent pages in order to perform incremental DOM fingerprint updates.

[0061] Optionally, the multi-layer hashing strategy includes content hashing, structure hashing, and interactive hashing;

[0062] The step of analyzing the content of the current page and the adjacent pages using a multi-level hashing strategy to perform incremental DOM fingerprint updates includes:

[0063] In response to a change in content hash while the structure hash remains unchanged, an incremental update is triggered; wherein, the incremental update represents a local update performed on the content node whose content has changed by performing differential fingerprint calculation;

[0064] A full reconstruction is triggered in response to a change in the structure hash that exceeds the second preset value.

[0065] A second aspect of this disclosure provides a page content processing system, the processing system comprising:

[0066] The target association information acquisition module is used to acquire the target page association information of the target page in response to receiving a page processing instruction;

[0067] The target association information includes the page semantic information of the target page, the execution operation description information in the target page, and the corresponding initial explanation content;

[0068] The verification processing module is used to use a multi-level verification strategy to combine the page semantic information and the execution operation description information to filter out content nodes in the target page that meet the preset verification conditions as target nodes.

[0069] The multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are conditions that characterize the validity of the operation.

[0070] The page processing module is used to perform a rendering operation on the content located at the target node in the initial explanation content using the execution operation description information, so as to obtain the target explanation content.

[0071] Optionally, the multi-level verification strategy includes a semantic matching verification mechanism, a structural position verification mechanism, and an interaction state verification mechanism executed sequentially.

[0072] The semantic matching verification mechanism is used to verify whether the content corresponding to the operation to be executed actually exists.

[0073] The structural position verification mechanism is used to verify whether the content corresponding to the operation to be performed is within the visual area.

[0074] The interaction state verification mechanism is used to verify whether the content corresponding to the operation to be performed is in an interactive state.

[0075] And / or,

[0076] The target association information acquisition module is also used to obtain the page semantic information of the target page based on the structure tree and layout information corresponding to the target page.

[0077] Optionally, the verification processing module includes:

[0078] The initial node set acquisition unit is used to acquire the corresponding initial candidate node set in the target page based on the page semantic information and the execution operation description information;

[0079] The first node set acquisition unit is used to use the semantic matching verification mechanism to filter out content nodes from the initial candidate node set whose semantic similarity to the execution operation description information is higher than a first preset value, so as to form a first candidate node set.

[0080] The verification processing unit is used to verify whether the content nodes in the first candidate node set meet the preset structural position conditions using the structural position verification mechanism; wherein, the preset structural position conditions include at least one of being within the visual area, not being occluded, and matching the sequence number in the DOM tree;

[0081] In response to the content nodes in the first candidate node set satisfying the preset structural position condition, the operation type of the operation to be executed is determined based on the execution operation description information, and the content nodes in the first candidate node set are verified using the interaction state verification mechanism and the operation type.

[0082] In response to the fact that the operation to be performed at the content node in the first candidate node set is in an interactive state, the corresponding content node is taken as the target node.

[0083] Optionally, the processing system further includes:

[0084] The first auxiliary processing module is configured to, in response to the content nodes in the first candidate node set being verified through a one-level or two-level verification mechanism, select the content node with the highest semantic similarity to the execution operation description information as the target node; and / or, select the content node with the highest semantic similarity to the execution operation description information as the target node, and add label information representing a medium confidence level to the target node;

[0085] In response to a content node in the first candidate node set failing any level of verification mechanism, the corresponding page processing operation is performed using the local default heuristic search method; or an interactive prompt is generated to perform the corresponding page processing operation based on external interactive operations.

[0086] Optionally, the processing system is implemented based on a preset model;

[0087] The processing system also includes:

[0088] The feedback information generation module is used to generate feedback information in response to the fact that a content node in the first candidate node set fails to pass any level of verification mechanism.

[0089] The second auxiliary processing module is used to perform a rollback operation; or, based on the feedback information, adjust the preset model and use the adjusted preset model to perform corresponding processing operations.

[0090] And / or,

[0091] The preset models include models of different sizes, and the processing system further includes:

[0092] The complexity analysis module is used to analyze the complexity of the processing task corresponding to the page processing instruction;

[0093] A model matching module is used to call a matching target model based on the complexity; wherein the complexity is positively correlated with the size of the target model;

[0094] And / or,

[0095] The processing system includes:

[0096] The backup solution invocation module is used to invoke a preset backup model or a local heuristic strategy to perform corresponding page processing operations in response to the fact that the output of the preset model does not meet the preset conditions.

[0097] Optionally, the processing system further includes:

[0098] The display control module is used to control the scrolling display of the current page using a preset step size;

[0099] The content triggering module is used to obtain the currently scrolling content and trigger the generation of corresponding segmented explanation content.

[0100] Optionally, the processing system includes:

[0101] The first interactive operation response module is used to obtain the first interactive operation and perform the corresponding operation on the current page.

[0102] The first interactive operation includes clicking the current page, and the corresponding operation includes stopping scrolling;

[0103] The first interactive operation includes scrolling the current page, and the corresponding operation includes updating the content of the context of the current position based on the new position after scrolling;

[0104] And / or,

[0105] The processing system also includes:

[0106] The character broadcast rate acquisition module is used to acquire the character broadcast rate of the broadcast content of the target explanatory content;

[0107] The page synchronization control module is used to control the scrolling rate of the reading cursor on the current page based on the character playback rate, so as to synchronize the playback speed with the page scrolling speed.

[0108] And / or,

[0109] The processing system also includes:

[0110] The voice interaction operation acquisition module is used to receive external voice interaction operations in response to the broadcast of the target explanation content;

[0111] The text content recognition module is used to recognize the text content corresponding to the external voice interaction operation;

[0112] The content broadcasting processing module is used to insert the text content into the broadcasting queue so that it can be broadcast when explaining the page content at the corresponding position;

[0113] And / or,

[0114] The processing system also includes:

[0115] The second interactive operation response module is used to respond to the broadcast of the target explanation content, receive the second interactive operation, stop broadcasting and enter the next round of interactive state.

[0116] Optionally, the processing system further includes:

[0117] The content analysis module is used to monitor the current page and adjacent pages, and uses a multi-level hashing strategy to analyze the content of the current page and adjacent pages in order to perform incremental DOM fingerprint updates.

[0118] Optionally, the multi-layer hashing strategy includes content hashing, structure hashing, and interactive hashing;

[0119] The content analysis module is also used to trigger an incremental update in response to a change in the content hash while the structure hash remains unchanged; wherein, the incremental update represents a local update performed on the content node whose content has changed by performing differential fingerprint calculation.

[0120] The content analysis module is also used to trigger a full reconstruction in response to a change in the structure hash value that exceeds a second preset value.

[0121] A third aspect of this disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the page content processing method provided in the first aspect.

[0122] In a fourth aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the page content processing method provided in the first aspect.

[0123] In a fifth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the page content processing method provided in the first aspect.

[0124] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0125] The positive and progressive effects of this disclosure are as follows:

[0126] This disclosure introduces a multi-level verification strategy to effectively verify the relevant content of the target page, thereby achieving accurate and reliable positioning of page elements to avoid unexecutable situations. It ensures that every generated annotation, click suggestion, etc., is real and visible on the interface, which can significantly reduce element positioning errors and the proportion of unexecutable instructions, effectively improve the accuracy of guidance, and provide personalized and targeted explanations and guidance, thereby improving interactivity and learning efficiency.

[0127] In addition, compared with existing solutions that mainly rely on pre-recorded videos, this embodiment can generate explanations and interactive Q&A in real time based on existing text / pages / teaching materials, without the need for large-scale recording, editing and repeated updates. This transforms content production from a pre-set resource mode to an on-demand generation mode, thereby significantly reducing production and maintenance costs.

[0128] Furthermore, compared to the unidirectional pixel-stream propagation of video, this embodiment can achieve a two-way closed loop based on structured instructions and client-side models. This allows learners to interrupt, ask questions, follow up, and correct errors at any time. It also combines the current viewport / reading cursor to output dynamic, personalized, and targeted explanations and guidance, thereby improving interactivity and learning efficiency. Attached Figure Description

[0129] Figure 1 This is a flowchart of a page content processing method according to Embodiment 1 of this disclosure.

[0130] Figure 2 This is a first flowchart of the page content processing method of Embodiment 2 of this disclosure.

[0131] Figure 3 This is a second flowchart of the page content processing method of Embodiment 2 of this disclosure.

[0132] Figure 4 This is a schematic diagram of the page content processing flow of Embodiment 2 of this disclosure.

[0133] Figure 5 This is a schematic diagram of the page content processing system of Embodiment 3 of this disclosure.

[0134] Figure 6 This is a schematic diagram of the page content processing system of Embodiment 4 of this disclosure.

[0135] Figure 7 This is a schematic diagram of the structure of the electronic device according to Embodiment 5 of this disclosure. Detailed Implementation

[0136] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0137] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0138] Example 1

[0139] like Figure 1 As shown, the page content processing method in this embodiment includes:

[0140] S101. In response to receiving a page processing instruction, obtain the target page association information of the target page;

[0141] The target association information includes the target page's semantic information, the description of the operations performed on the target page, and the corresponding initial explanation content.

[0142] Target pages include, but are not limited to, web pages, display interfaces in document readers, reading interfaces of e-books / textbooks, and interfaces of business systems.

[0143] S102. A multi-level verification strategy is adopted, combining page semantic information and execution operation description information, to filter out content nodes in the target page that meet the preset verification conditions as target nodes.

[0144] Among them, the multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are the conditions that characterize the validity of the operation.

[0145] S103. Using the execution operation description information, perform a rendering operation on the content located at the target node in the initial explanation content to obtain the target explanation content.

[0146] In real-world scenarios, the target explanation content is used to automatically and specifically explain and interact with relevant content on the target page.

[0147] In this embodiment, the page content processing method is mainly applied to the user-side terminal (i.e., the client); the client implementation includes, but is not limited to, browser extensions, desktop applications, mobile applications, and reader plugins.

[0148] In this embodiment, by introducing a multi-level verification strategy, the relevant content of the target page is effectively verified, and the page elements are accurately and reliably located to avoid unexecutable situations. This ensures that every generated annotation, click suggestion, etc., is real and visible on the interface, which can significantly reduce element positioning errors and the proportion of unexecutable instructions, effectively improve the accuracy of guidance, and provide personalized and targeted explanations and guidance, thereby improving interactivity and learning efficiency.

[0149] In addition, compared with existing solutions that mainly rely on pre-recorded videos, this embodiment can generate explanations and interactive Q&A in real time based on existing text / pages / teaching materials, without the need for large-scale recording, editing and repeated updates. This transforms content production from a pre-set resource mode to an on-demand generation mode, thereby significantly reducing production and maintenance costs.

[0150] Example 2

[0151] The page content processing method in this embodiment is a further improvement on embodiment 1, specifically:

[0152] In one feasible solution, the step of obtaining the page semantic information of the target page in S101 includes:

[0153] Based on the structure tree and layout information of the target page, the page semantic information of the target page is obtained.

[0154] In this solution, a Page Semantic Model (PSM) is used as an intermediate layer between the LLM and the page on the client to obtain the corresponding content from the target page. The content collected will vary depending on the carrier object, including the webpage DOM, document rendering tree / layout objects, and application UI (user interface) control tree. Furthermore, the model can collect not only text but also element coordinates, interaction attribute information, and fingerprint information.

[0155] Among them, the Page Semantic Model (PSM) is an intermediate data structure that can abstract online content or visual interface structures (such as web page HTML (Hypertext Markup Language) / DOM, document structure and layout objects, application control tree, etc.), remove redundant information that is irrelevant to understanding, and retain the hierarchy, text, coordinates (bounding rectangle or equivalent layout coordinates) and interaction type of content nodes.

[0156] Specifically, the system collects the tree structure and layout information corresponding to the current content on the target page in real time on the client side to construct a lightweight page semantic graph. Further, in the browser, it collects the DOM tree of the target page, calculates CSS (Cascading Style Sheets) styles (visibility, position, etc.), and extracts text nodes; in the document / reader, it collects the object tree, paragraph / heading levels, page numbers, and coordinates of the target page.

[0157] In a feasible solution, the multi-level verification strategy includes a semantic matching verification mechanism, a structural position verification mechanism, and an interaction state verification mechanism executed sequentially.

[0158] Among them, the semantic matching verification mechanism is used to verify whether the content corresponding to the operation to be executed actually exists;

[0159] The structural location verification mechanism is used to verify whether the content corresponding to the operation to be performed is within the visual area.

[0160] The interaction state verification mechanism is used to verify whether the content corresponding to the operation to be performed is in an interactive state.

[0161] Among them, the semantic matching verification mechanism is mainly used to match the semantics of the content corresponding to the operation to be performed; the structural position verification mechanism is mainly used to verify whether the candidate node is within the viewport, the occlusion relationship, the sequence number / hierarchy consistency, etc.; the interaction state verification mechanism is mainly used to verify whether the candidate node is clickable, editable, disabled, etc.

[0162] This solution cleverly employs a three-tiered verification mechanism: semantic matching, structural location, and interaction state. Specifically, by sequentially executing semantic matching, structural location, and interaction state verification mechanisms, the relevant content of the target page is effectively validated, ensuring accurate and reliable positioning of page elements. This avoids unexecutable situations and ensures that every generated annotation and click suggestion is genuinely present and visible on the interface. This significantly reduces element positioning errors and the proportion of unexecutable instructions, effectively improving guidance accuracy and consequently enhancing interactivity and learning efficiency. For example, the page content processing solution implemented through this three-tiered verification mechanism reduces the element positioning error rate from a high level (e.g., approximately 20%~40%) to a low level (e.g., below 5%), thus effectively reducing element positioning errors and improving guidance accuracy.

[0163] In a feasible solution, such as Figure 2 As shown, step S102 includes:

[0164] S1021. Based on page semantic information and execution operation description information, obtain the corresponding initial candidate node set in the target page;

[0165] S1022. Using a semantic matching verification mechanism, content nodes with a semantic similarity higher than the first preset value to the description information of the operation to be selected from the initial candidate node set are used to form the first candidate node set.

[0166] S1023. Use a structural position verification mechanism to verify whether the content nodes in the first candidate node set meet the preset structural position conditions.

[0167] Among them, the preset structural location conditions include at least one of the following: within the visual area, not obscured, and matching the sequence number in the DOM tree;

[0168] S1024. In response to the content nodes in the first candidate node set satisfying the preset structural position conditions, determine the operation type of the operation to be executed based on the operation description information, and verify the content nodes in the first candidate node set using an interactive state verification mechanism and the operation type.

[0169] If the operation to be performed at the content node in the first candidate node set is in an interactive state, the corresponding content node will be selected as the target node.

[0170] In this solution, a semantic matching verification mechanism is used to first select the first set of candidate nodes that meet the requirements. Then, a structural position verification mechanism is used to further filter these content nodes to obtain content nodes that are within the visual area and are not occluded. Finally, an interaction state verification mechanism is used to verify these content nodes that meet the preset structural position conditions to obtain nodes that belong to the interactive state, thereby obtaining the target node. This avoids the occurrence of non-executable situations and ensures that every generated annotation, click suggestion, etc., is real and visible on the interface. It can significantly reduce element positioning errors and the proportion of non-executable instructions, effectively improve guidance accuracy, and thus improve interactivity and learning efficiency.

[0171] In a feasible solution, such as Figure 3 As shown, the processing method also includes:

[0172] S1025. In response to the content nodes in the first candidate node set, the content node with the highest semantic similarity to the operation description information is selected as the target node through a one-level or two-level verification mechanism; and / or, the content node with the highest semantic similarity to the operation description information is selected as the target node, and a label with moderate confidence is added to the target node.

[0173] S1026. In response to the fact that the content nodes in the first candidate node set fail to pass any level of verification mechanism, the corresponding page processing operation is performed using the local default heuristic search method; or an interactive prompt is generated to perform the corresponding page processing operation based on the external interactive operation.

[0174] There is no restriction on the specific execution order between S1025 and S1026; they are executed according to the actual situation.

[0175] In this solution, for situations where multi-level verification strategies cannot be met, corresponding auxiliary processing schemes are pre-set to carry out targeted processing in a timely and efficient manner, thereby effectively ensuring the feasibility and reliability of page content processing.

[0176] In one feasible solution, the processing method is implemented based on a pre-defined model;

[0177] like Figure 3 As shown, the processing method also includes:

[0178] S1027. In response to the fact that the content node in the first candidate node set fails the verification mechanism at any level, feedback information is generated; among them, there is no restriction on the specific execution order between S1025 and S1027, and the execution is carried out according to the actual situation.

[0179] S1028. Perform a rollback operation; or, adjust the preset model based on the feedback information and perform corresponding processing operations using the adjusted preset model.

[0180] In this solution, if a content node in the first candidate node set fails any level of verification mechanism, the solution directly generates corresponding feedback information indicating failure and rolls back; or it automatically optimizes and adjusts the model based on the feedback information so that the adjusted model can process page content with higher quality.

[0181] In a feasible solution, the pre-defined model includes models of different sizes, i.e., based on multi-model collaborative processing. Specifically, the processing method also includes:

[0182] The complexity of the processing tasks corresponding to the page processing instructions is analyzed.

[0183] The target model is matched based on complexity; where complexity is positively correlated with the size of the target model.

[0184] In this solution, the processing complexity of each task is automatically analyzed in advance, and the appropriate model is automatically matched for task processing, which effectively ensures the efficiency of page content processing and avoids unnecessary waste of computing resources.

[0185] In one feasible solution, the processing method further includes:

[0186] If the output of a preset model does not meet the preset conditions, a preset backup model or a local heuristic strategy is invoked to perform the corresponding page processing operation.

[0187] Among them, through the task-aware routing and fallback mechanism, when the preferred model is unavailable or the output does not meet the preset conditions, the backup model or local heuristic strategy can be automatically switched to improve the overall availability.

[0188] In this solution, if the output of the preset model fails to meet the conditions, the model will no longer be used, a fallback mechanism will be enabled, and a backup model or local heuristic strategy will be automatically called to assist in follow-up processing. This ensures the feasibility and efficiency of page content processing and avoids situations where page content processing cannot be executed due to anomalies, or where actual processing needs cannot be met.

[0189] In one feasible solution, the processing method further includes:

[0190] The current page is scrolled using a preset step size.

[0191] Get the currently scrolling content and trigger the generation of corresponding segmented explanation content.

[0192] The page scrolling distance can be estimated based on the voice broadcast progress or reading speed to determine the preset step size; of course, the preset step size can be reset or adjusted according to the actual needs of the scenario.

[0193] This solution achieves automatic and smooth scrolling and page turning of the content on the current page by setting a preset step size, realizing adaptive scrolling control of the page. It can sense changes in the viewport content and achieve an adaptive reading experience. The viewport is the visible rectangular area on the current page, which changes with the user's scrolling or pagination operations.

[0194] In a feasible solution, after obtaining the currently scrolling content and triggering the generation of corresponding segmented explanatory content, the following steps are also included:

[0195] Obtain the first interaction operation and execute the corresponding operation on the current page;

[0196] The first interactive operation includes clicking the current page, and the corresponding operation includes stopping scrolling;

[0197] The first interactive operation includes scrolling the current page, and the corresponding operation includes updating the content of the context of the current position based on the new position after scrolling.

[0198] In this solution, the display method or content of the page can be matched and responded to according to the user's interactive operation on the current interface, such as clicking or scrolling. That is, automatic scrolling can be paused or adjusted when the user interrupts or when the page fingerprint changes significantly, so as to further enhance the user's interactive and personalized needs for page content processing.

[0199] In one feasible solution, the processing method further includes:

[0200] Get the character playback rate of the target explanation content;

[0201] Based on the character playback rate, control the scrolling rate of the reading cursor on the current page to synchronize the playback speed with the page scrolling speed.

[0202] Specifically, by using view area perception and reading cursor control, the voice broadcast and page scrolling / turning are kept synchronized, reducing interference from inconsistencies between the narration and the view area. For example, the synchronization error between voice and page scrolling is controlled within the range of hundreds of milliseconds (e.g., <300ms) to effectively ensure consistency between reading and narration.

[0203] In this solution, the character playback rate of the target content to be explained is obtained in real time, and the scrolling rate of the reading cursor on the current page is automatically controlled accordingly. This enables dynamic adjustment of the scrolling rhythm of the page content according to the progress of the explanation, thereby achieving adaptive reading companionship that integrates "eye-voice-scrolling" and greatly enhances intelligence, interactivity and fun.

[0204] Furthermore, compared to the unidirectional pixel-stream propagation of video, this embodiment can achieve a two-way closed loop based on structured instructions and client-side models. This allows learners to interrupt, ask questions, follow up, and correct errors at any time. It also combines the current viewport / reading cursor to output dynamic, personalized, and targeted explanations and guidance, thereby improving interactivity and learning efficiency.

[0205] In one feasible solution, the processing method further includes:

[0206] In response to the broadcast of the explanation of the target content, it receives external voice interaction operations;

[0207] The text content corresponding to external voice interaction operations is recognized;

[0208] Insert the text content into the broadcast queue so that it can be broadcast when explaining the page content at the corresponding location.

[0209] This solution can acquire the user's voice interaction input in real time, recognize it as corresponding text, and then automatically insert it into the corresponding position on the page. This enables the automatic playback of the inserted content when explaining the page content at the corresponding position, thereby further improving the personalization, automation, and convenience of page content processing.

[0210] In one feasible solution, the processing method further includes:

[0211] In response to the broadcast of the content to be explained to the target, upon receiving the second interactive operation, the broadcast stops and enters the next round of interactive state.

[0212] In this solution, once a specific interactive operation is received from the user, it indicates that the user needs to stop the current content playback, control the cessation of playback and enter the next round of interaction. In other words, the user can actively intervene to directly stop the playback in the current round without any waiting, further improving the convenience of the user in processing page content.

[0213] In one feasible solution, the processing method further includes:

[0214] Monitor the current page and adjacent pages, and use a multi-level hashing strategy to analyze the content of the current page and adjacent pages in order to perform incremental DOM fingerprint updates.

[0215] In this solution, DOM fingerprinting and incremental updates can maintain the stability of annotations in scenarios where the page is constantly changing, reduce the probability of annotations pointing to invalid elements, and thus achieve adaptability to dynamic pages.

[0216] In a feasible scheme, a multi-layered hashing strategy includes content hashing, structure hashing, and interactive hashing;

[0217] The steps for incremental DOM fingerprint updates, which involve analyzing the content of the current page and adjacent pages using a multi-level hashing strategy, include:

[0218] In response to a change in content hash while the structure hash remains unchanged, an incremental update is triggered; wherein, the incremental update represents a local update performed on the content node whose content has changed by performing differential fingerprint calculation.

[0219] A full reconstruction is triggered in response to a change in the structure hash that exceeds the second preset value.

[0220] The page fingerprint employs a multi-layered hashing strategy. This includes: content hashing (SHA256) on visible text to generate a 32-byte digest; structure hashing (breadth-first traversal sequence of the DOM tree (node ​​type + depth), calculating the hash of the first 1000 nodes); and interaction hashing (sorting the selector set of all button / input / a tags and then hashing it). Based on this, the fingerprint is calculated as: final_fingerprint = hash(content_hash + structure_hash + interaction_hash). Specifically, when the fingerprint similarity between two pages is > 0.95, it is considered a minor change to the same page (no need to rebuild the PSM); when the structure hash changes by > 30%, a full PSM reconstruction is triggered; when the content hash changes but the structure hash remains unchanged, an incremental update is performed (only refreshing the subtree of the changed node).

[0221] In this solution, Mutation Observer (a JavaScript API provided by the browser for asynchronously listening to changes in the DOM tree) is used to listen for DOM change events within the viewport; a debouncing threshold (e.g., 300ms) is set to avoid frequent triggering; and only the viewport and the area one screen above and below it are listened to to reduce computation.

[0222] Furthermore, differential fingerprint calculation is performed only on affected nodes; incremental PSM updates are triggered when the change exceeds a threshold, otherwise the cache is reused. Compared to full reconstruction, incremental PSM updates can reduce CPU (central processing unit) overhead by approximately 80%; the proportion of tags pointing to failed elements can be reduced from a high level to a low level.

[0223] In one feasible solution, the processing method further includes:

[0224] By filtering the visible areas on the page, the amount of data uploaded can be reduced (e.g., by more than 70%), thereby reducing token consumption and end-to-end latency.

[0225] In addition, combining de-identification strategies with domain / access control can reduce the risk of sensitive information being leaked.

[0226] This solution achieves efficient multi-model collaboration while protecting privacy. It reduces reliance on large cloud models and the risk of data exposure by using front-end structured extraction and local anonymization, combined with on-demand routing strategies.

[0227] In one feasible solution, the method for processing the page content includes:

[0228] When the target element being labeled is detected to be invalid, search for similar content in the adjacent area (e.g., ±200px) (e.g., text similarity threshold T2=0.8, configurable); if a replacement element is found, the label is smoothly transitioned to the new position; if not found, the label fades out and the user is notified that the content has been updated, so as to realize the invalidation detection and automatic recovery of contextual evidence.

[0229] Among them, grounding evidence is anchor information used to connect LLM output with page facts, and is usually composed of the triple {Selector, Keyword_Hash, Coordinates}.

[0230] The following is combined with Figure 4 The implementation principle of the page content processing scheme in this embodiment will be explained in general:

[0231] (1) Overall Architecture

[0232] The system can take the form of a browser extension, desktop application, mobile application, or reader plugin, and includes client-side rendering components and backend service collaboration; the following example uses a browser extension:

[0233] Content Script: Responsible for DOM scraping, annotation rendering, scrolling / clicking, and other page-related actions;

[0234] Background / Service Worker: Responsible for model requests, session management, configuration management, and cross-domain request proxying (within the scope of permissions);

[0235] Popup / Panel UI: Responsible for user interaction entry points (explanation / question / voice switch / model selection);

[0236] Model Adapter Layer (LLM Provider Layer): Connects to different model services, providing unified capabilities such as chat(), analyze_page(), and locate_elements().

[0237] Verifier: Verifies the result of element location and the result of action execution, and triggers retry or degradation strategies when necessary.

[0238] Among them, the adaptive closed loop of validator and PSM collaboration: the validator feeds back to the PSM construction and sampling strategy according to the failure type to trigger local resampling, threshold adjustment or degradation.

[0239] Trajectory recording and playback method: Record page fingerprint snapshots, annotation commands, verification results and user feedback for playback review and strategy optimization.

[0240] (2) Page capture and structured modeling

[0241] Listen for user triggers (such as clicking "explain the current page") or page events (URL change, scroll pause);

[0242] Collect visible text and key DOM element features (headings, paragraphs, lists, buttons, input boxes, tables, etc.);

[0243] Generate page_model and de-identify sensitive fields (e.g., email / phone number / order number, etc.);

[0244] Use page_model as model input or as a retrieval index for subsequent dialogue and element location.

[0245] (3) Evidence constraint mechanism for the explanation and Q&A of page content

[0246] Level 1: Semantic Matching Verification Mechanism

[0247] Perform NLP semantic parsing on the target descriptions output by LLM (such as "paragraph 3" or "login button");

[0248] Fuzzy search is performed in PSM to generate a set of candidate nodes (such as Top-K, where K is usually 3-5).

[0249] Calculate the semantic similarity score: sim_score = cosine(embed(llm_desc), embed(node.content));

[0250] Filter out candidate nodes with sim_score > T1; where the threshold T1 is configurable, for example, 0.7; embed() can be implemented by a sentence vector model, including but not limited to BERT-like encoders, contrastive learning encoders or other semantic vectorization models.

[0251] Level 2: Structural Location Verification Mechanism

[0252] Check if the candidate node's bounding box is within the current viewport (allowing a tolerance of ±50px).

[0253] Check the z-index (a special property for webpage display) level to exclude obscured elements; this is achieved through the document.elements FromPoint API.

[0254] For the description of the Nth item, verify whether the sequence number in the DOM tree matches.

[0255] Level 3: Interaction State Verification Mechanism

[0256] For click-type operations, verify that the element's pointer-events is not none;

[0257] For input-related operations, the validation element is not in a disabled state;

[0258] Perform a tentative focus attempt (focus()), and if it fails, the user is deemed unable to interact.

[0259] Comprehensive decision-making:

[0260] Only candidate nodes that pass all three levels of verification will be selected as the final target.

[0261] If no node passes through, then:

[0262] Scenario A (partially passed): Select the node with the highest score, but add a "Medium confidence level" prompt to the UI;

[0263] Case B (all failures): Trigger the fallback strategy, or use local ARIA role heuristic search, request the user to manually select, etc.

[0264] (4) Automatic page scrolling / turning

[0265] The automatic browsing module generates steps based on browse_plan to control the page to scroll automatically at a constant speed;

[0266] Update the viewport and the collection of visible text blocks after each scroll step;

[0267] The triggering agent explains newly presented content in segments.

[0268] If a user interrupts or scrolls manually, automatic browsing is paused and the session context is updated.

[0269] (5) Voice interaction

[0270] When a user initiates voice input, the voice module captures the audio and performs recognition.

[0271] Use the recognized text as dialogue input;

[0272] After obtaining the model output, it enters the broadcast queue. The TTS (text-to-speech) broadcast can be synchronized with the annotation animation.

[0273] If a user interrupts during the broadcast (using hot words / buttons / keyboard), the broadcast will stop and a new round of interaction will begin.

[0274] The following examples further illustrate this point in real-world scenarios:

[0275] Scenario 1: Segmenting and automatically scrolling through long web pages.

[0276] The user opens the long tutorial page and clicks "Explain the current page";

[0277] The system collects the page_model of the first screen, segments the visible text blocks (title-body text-code block / table) and generates an explanation outline;

[0278] The visual annotation module highlights titles and key paragraphs, and labels key points with numbers;

[0279] The voice module will read out the first paragraph of the explanation; after the reading is completed, it will automatically scroll one step to move on to the next paragraph.

[0280] When a user asks, "What is the core conclusion of this paragraph?", the system locates the corresponding paragraph in explained_ranges and answers the question, while also highlighting the evidence for that paragraph (highlighted range).

[0281] Scenario 2: Page operation instructions (buttons / forms) and arrow annotations

[0282] A user asked on a business backend page, "How do I export reports?"

[0283] The system extracts candidate button and menu item information from elements[], and the model outputs the "click path" and element evidence for each step;

[0284] The content script verifies each piece of evidence, drawing arrows and bubble descriptions in sequence, for example: 1. Click [More], 2. Select [Export];

[0285] If an element is not visible at a certain step, the "expand menu" action will be automatically triggered or the user will be prompted to scroll to the corresponding area until the evidence is approved.

[0286] When the task is "element localization / annotation", the system prioritizes models with stronger visual or structured output capabilities; when the task is "summarization / rewriting", it selects models with lower cost and lower latency.

[0287] If the preferred model times out or returns output without evidence fields, the routing module triggers a fallback: switching to an alternative model or generating candidate evidence using local heuristics (e.g., element retrieval based on ARIA role and text matching).

[0288] The validator verifies the candidate evidence and selects the set with the highest pass rate for annotation.

[0289] Scenario 3: Real-time detection of dynamic page changes and incremental PSM updates

[0290] The user is reading a news page that is updated in real time (such as market data, live broadcasts, or data dashboards), and the page content is automatically refreshed in sections every second.

[0291] However, using existing traditional methods presents several problems: rebuilding the entire PSM every time the DOM changes, leading to significant performance overhead and page lag; and failing to detect or validate changes, resulting in annotations pointing to disappeared elements and creating non-executable guidelines. The solution presented in this embodiment eliminates these problems, effectively improving the efficiency of page content processing.

[0292] Example 3

[0293] like Figure 5 As shown, the page content processing system in this embodiment includes:

[0294] The target association information acquisition module 1 is used to acquire the target page association information of the target page in response to receiving a page processing instruction;

[0295] Among them, the target association information includes the page semantic information of the target page, the description information of the execution operation in the target page, and the corresponding initial explanation content;

[0296] Verification processing module 2 is used to use a multi-level verification strategy to combine page semantic information and execution operation description information to filter out content nodes in the target page that meet the preset verification conditions as target nodes.

[0297] Among them, the multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are the conditions that characterize the validity of the operation.

[0298] Page processing module 3 is used to perform rendering operations on the content located at the target node in the initial explanation content using the execution operation description information, so as to obtain the target explanation content.

[0299] In this embodiment, by introducing a multi-level verification strategy, the relevant content of the target page is effectively verified, and the page elements are accurately and reliably located to avoid unexecutable situations. This ensures that every generated annotation, click suggestion, etc., is real and visible on the interface, which can significantly reduce element positioning errors and the proportion of unexecutable instructions, effectively improve the accuracy of guidance, and provide personalized and targeted explanations and guidance, thereby improving interactivity and learning efficiency.

[0300] In addition, compared with existing solutions that mainly rely on pre-recorded videos, this embodiment can generate explanations and interactive Q&A in real time based on existing text / pages / teaching materials, without the need for large-scale recording, editing and repeated updates. This transforms content production from a pre-set resource mode to an on-demand generation mode, thereby significantly reducing production and maintenance costs.

[0301] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0302] Example 4

[0303] like Figure 6 As shown, the page content processing system in this embodiment is a further improvement on embodiment 3, specifically:

[0304] In a feasible solution, the multi-level verification strategy includes a semantic matching verification mechanism, a structural position verification mechanism, and an interaction state verification mechanism executed sequentially.

[0305] Among them, the semantic matching verification mechanism is used to verify whether the content corresponding to the operation to be executed actually exists;

[0306] The structural location verification mechanism is used to verify whether the content corresponding to the operation to be performed is within the visual area.

[0307] The interaction state verification mechanism is used to verify whether the content corresponding to the operation to be performed is in an interactive state;

[0308] And / or,

[0309] The target association information acquisition module 1 is also used to obtain the page semantic information of the target page based on the structure tree and layout information corresponding to the target page.

[0310] In one feasible solution, the verification processing module 2 includes:

[0311] The initial node set acquisition unit 4 is used to acquire the corresponding initial candidate node set in the target page based on page semantic information and execution operation description information;

[0312] The first node set acquisition unit 5 is used to use a semantic matching verification mechanism to filter out content nodes from the initial candidate node set whose semantic similarity to the operation description information is higher than a first preset value, so as to form a first candidate node set;

[0313] The verification processing unit 6 is used to verify whether the content nodes in the first candidate node set meet the preset structural position conditions using a structural position verification mechanism; wherein, the preset structural position conditions include at least one of the following: within the visual area, not occluded, and matching the sequence number in the DOM tree;

[0314] In response to the content nodes in the first candidate node set satisfying the preset structural position conditions, the operation type of the operation to be executed is determined based on the operation description information, and the content nodes in the first candidate node set are verified using an interactive state verification mechanism and the operation type.

[0315] If the operation to be performed at the content node in the first candidate node set is in an interactive state, the corresponding content node will be selected as the target node.

[0316] In one feasible embodiment, the processing system further includes:

[0317] The first auxiliary processing module 7 is used to respond to the content nodes in the first candidate node set through a one-level or two-level verification mechanism, select the content node with the highest semantic similarity to the execution operation description information as the target node; and / or, select the content node with the highest semantic similarity to the execution operation description information as the target node, and add label information with moderate confidence to the target node;

[0318] If a content node in the first candidate node set fails any level of verification mechanism, the corresponding page processing operation is performed using the local default heuristic search method; or an interactive prompt is generated to perform the corresponding page processing operation based on the external interactive operation.

[0319] In one feasible solution, the processing system is implemented based on a pre-defined model;

[0320] The processing system also includes:

[0321] Feedback information generation module 8 is used to generate feedback information in response to the fact that a content node in the first candidate node set fails any level of verification mechanism;

[0322] The second auxiliary processing module 9 is used to perform rollback operations; or, adjust the preset model based on feedback information and use the adjusted preset model to perform corresponding processing operations.

[0323] And / or,

[0324] The preset models include models of different sizes, and the processing system also includes:

[0325] Complexity analysis module 10 is used to analyze the complexity of the processing tasks corresponding to the page processing instructions.

[0326] The model matching module 11 is used to call the target model for matching based on complexity; where the complexity is positively correlated with the size of the target model.

[0327] And / or,

[0328] The processing system includes:

[0329] The backup solution call module 12 is used to call the preset backup model or local heuristic strategy in response to the fact that the output of the preset model does not meet the preset conditions, so as to perform the corresponding page processing operation.

[0330] In one feasible solution, the processing system also includes:

[0331] The display control module 13 is used to control the scrolling display of the current page using a preset step size;

[0332] Content triggering module 14 is used to obtain the currently scrolling content and trigger the generation of corresponding segmented explanation content.

[0333] In one feasible solution, the processing system includes:

[0334] The first interactive operation response module 15 is used to obtain the first interactive operation and perform the corresponding operation on the current page;

[0335] The first interactive operation includes clicking the current page, and the corresponding operation includes stopping scrolling;

[0336] The first interactive operation includes scrolling the current page, and the corresponding operation includes updating the content of the context of the current position based on the new position after scrolling.

[0337] And / or,

[0338] The processing system also includes:

[0339] The character broadcast rate acquisition module 16 is used to acquire the character broadcast rate of the broadcast content of the target explanation content;

[0340] The page synchronization control module 17 is used to control the scrolling rate of the reading cursor on the current page based on the character broadcasting rate, so as to synchronize the broadcasting speed with the page scrolling speed.

[0341] And / or,

[0342] The processing system also includes:

[0343] The voice interaction operation acquisition module 18 is used to receive external voice interaction operations in response to the broadcast of the target explanation content;

[0344] Text content recognition module 19 is used to recognize the text content corresponding to external voice interaction operations;

[0345] The content broadcasting processing module 20 is used to insert text content into the broadcasting queue so that it can be broadcast when explaining the page content at the corresponding position;

[0346] And / or,

[0347] The processing system also includes:

[0348] The second interactive operation response module 21 is used to respond to the broadcast of the target explanation content, receive the second interactive operation, stop the broadcast and enter the next round of interactive state.

[0349] In one feasible solution, the processing system also includes:

[0350] Content analysis module 22 is used to monitor the current page and adjacent pages, and uses a multi-level hashing strategy to analyze the content of the current page and adjacent pages in order to perform incremental DOM fingerprint updates.

[0351] In a feasible scheme, a multi-layered hashing strategy includes content hashing, structure hashing, and interactive hashing;

[0352] The content analysis module is also used to trigger incremental updates in response to changes in content hash while the structure hash remains unchanged; wherein, incremental update represents a local update performed on the content node whose content has changed by performing differential fingerprint calculation.

[0353] The content analysis module is also used to trigger a full reconstruction in response to a change in the structure hash that exceeds a second preset value.

[0354] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0355] Example 5

[0356] Figure 7 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the method described in any of the above embodiments. Figure 7 The electronic device 90 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0357] like Figure 7 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0358] Bus 93 includes a data bus, an address bus, and a control bus.

[0359] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0360] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, such program module 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0361] The processor 91 executes various functional applications and data processing, such as the methods provided in any of the above embodiments, by running computer programs stored in the memory 92.

[0362] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0363] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0364] Example 6

[0365] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in any of the above embodiments.

[0366] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0367] Example 7

[0368] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the above embodiments.

[0369] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0370] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A method of processing page content, characterized by, The processing method includes: In response to receiving a page processing instruction, obtain the target page association information of the target page; The target association information includes the page semantic information of the target page, the execution operation description information in the target page, and the corresponding initial explanation content; A multi-level verification strategy is adopted, combining the page semantic information and the execution operation description information, to filter out content nodes in the target page that meet the preset verification conditions as target nodes; The multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are conditions that characterize the validity of the operation. Using the execution operation description information, a rendering operation is performed on the content located at the target node in the initial explanation content to obtain the target explanation content.

2. The method of claim 1, wherein, The multi-level verification strategy includes a semantic matching verification mechanism, a structural position verification mechanism, and an interaction state verification mechanism that are executed sequentially. The semantic matching verification mechanism is used to verify whether the content corresponding to the operation to be executed actually exists. The structural position verification mechanism is used to verify whether the content corresponding to the operation to be performed is within the visual area. The interaction state verification mechanism is used to verify whether the content corresponding to the operation to be performed is in an interactive state. And / or, The step of obtaining the page semantic information of the target page includes: Based on the structure tree and layout information corresponding to the target page, the page semantic information of the target page is obtained.

3. The method of claim 2, wherein, The step of using a multi-level verification strategy, combining the page semantic information and the execution operation description information, to filter out content nodes in the target page that meet preset verification conditions as target nodes includes: Based on the page semantic information and the execution operation description information, obtain the initial candidate node set corresponding to the target page; Using the semantic matching verification mechanism, content nodes with a semantic similarity higher than a first preset value to the execution operation description information are selected from the initial candidate node set to form a first candidate node set; The structural position verification mechanism is used to verify whether the content nodes in the first candidate node set meet the preset structural position conditions; wherein, the preset structural position conditions include at least one of the following: within the visual area, not obscured, and matching the sequence number in the DOM tree; In response to the content nodes in the first candidate node set satisfying the preset structural position condition, the operation type of the operation to be executed is determined based on the execution operation description information, and the content nodes in the first candidate node set are verified using the interaction state verification mechanism and the operation type. In response to the fact that the operation to be performed at the content node in the first candidate node set is in an interactive state, the corresponding content node is taken as the target node.

4. The method of claim 3, wherein, The processing method further includes: In response to the content nodes in the first candidate node set being selected through a one- or two-level verification mechanism, the content node with the highest semantic similarity to the execution operation description information is selected as the target node; and / or, the content node with the highest semantic similarity to the execution operation description information is selected as the target node, and a label with moderate confidence is added to the target node. In response to a content node in the first candidate node set failing any level of verification mechanism, the corresponding page processing operation is performed using the local default heuristic search method; or an interactive prompt is generated to perform the corresponding page processing operation based on external interactive operations.

5. The method of claim 4, wherein, The processing method is implemented based on a preset model; The processing method further includes: In response to a content node in the first candidate node set failing any level of verification mechanism, feedback information is generated; Perform a rollback operation; or, adjust the preset model based on the feedback information and perform corresponding processing operations using the adjusted preset model; And / or, The preset model includes models of different sizes, and the processing method further includes: The complexity of the processing task corresponding to the page processing instruction is obtained through analysis; The matching target model is invoked based on the complexity; wherein the complexity is positively correlated with the size of the target model; And / or, The processing method further includes: In response to the fact that the output of the preset model does not meet the preset conditions, a preset backup model or a local heuristic strategy is invoked to perform the corresponding page processing operation.

6. The method of processing page content according to any one of claims 1 to 5, wherein, The processing method further includes: The current page is scrolled using a preset step size. Get the currently scrolling content and trigger the generation of corresponding segmented explanation content.

7. The method for processing page content as described in claim 6, characterized in that, After obtaining the currently scrolling content and triggering the generation of corresponding segmented explanation content, the method further includes: Obtain the first interactive operation and execute the corresponding operation on the current page; The first interactive operation includes clicking the current page, and the corresponding operation includes stopping scrolling; The first interactive operation includes scrolling the current page, and the corresponding operation includes updating the content of the context of the current position based on the new position after scrolling; And / or, The processing method further includes: Obtain the character playback rate of the target explanation content; Based on the character playback rate, control the scrolling rate of the reading cursor on the current page to synchronize the playback speed with the page scrolling speed; And / or, The processing method further includes: In response to the broadcast of the explanation of the target content, an external voice interaction operation is received; The text content corresponding to the external voice interaction operation is identified; The text content is inserted into the broadcast queue so that it can be broadcast when explaining the page content at the corresponding location; And / or, The processing method further includes: In response to broadcasting the content of the explanation of the target, upon receiving the second interactive operation, the broadcasting stops and the next round of interactive state begins.

8. The method for processing page content as described in claim 7, characterized in that, The processing method further includes: Monitor the current page and adjacent pages, and use a multi-level hashing strategy to analyze the content of the current page and adjacent pages in order to perform incremental DOM fingerprint updates.

9. The method for processing page content as described in claim 8, characterized in that, The multi-layered hashing strategy includes content hashing, structure hashing, and interactive hashing; The step of analyzing the content of the current page and the adjacent pages using a multi-level hashing strategy to perform incremental DOM fingerprint updates includes: In response to a change in content hash while the structure hash remains unchanged, an incremental update is triggered; wherein, the incremental update represents a local update performed on the content node whose content has changed by performing differential fingerprint calculation; A full reconstruction is triggered in response to a change in the structure hash that exceeds the second preset value.

10. A page content processing system, characterized in that, The processing system includes: The target association information acquisition module is used to acquire the target page association information of the target page in response to receiving a page processing instruction; The target association information includes the page semantic information of the target page, the execution operation description information in the target page, and the corresponding initial explanation content; The verification processing module is used to use a multi-level verification strategy to combine the page semantic information and the execution operation description information to filter out content nodes in the target page that meet the preset verification conditions as target nodes. The multi-level verification strategy is determined based on at least two preset verification mechanisms, and the preset verification conditions are conditions that characterize the validity of the operation. The page processing module is used to perform a rendering operation on the content located at the target node in the initial explanation content using the execution operation description information, so as to obtain the target explanation content.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the method for processing page content according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for processing page content according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for processing page content as described in any one of claims 1 to 9.

Images