A web page text adaptive positioning system based on DOM tree structure comparison

Abstract

The application belongs to the field of data analysis and processing, and discloses a web text self-adaptive positioning system based on DOM tree structure comparison. The system extracts web text, title and date based on forest simulation succession type DOM self-adaptive architecture comparison, and performs XPath positioning, including: using a forest state evolution mechanism to determine a heat value to mark a potential text area; dynamically updating energy according to real-time operation types; forming a stable network, when a new node is inserted into a DOM tree, dynamically updating according to ecological niche factors, competition intensity and cooperation factors; identifying a DOM path meeting growth conditions as a text trunk, and selecting a path with an inner product higher than an expectation as a final text path; pre-deploying a forest window event and information spore propagation; forming a structure-semantic mapping, generating a semantic transmission network, and activating semantic related nodes according to a semantic energy package, so as to realize web text self-adaptive positioning based on DOM tree structure comparison.

Description

Technical Field

[0001] This invention relates to the field of data analysis and processing, and more specifically, to a webpage text adaptive positioning system based on DOM tree structure comparison. Background Technology

[0002] With the rapid development of the internet, web page content extraction technology has significant application value in areas such as search engine optimization, data crawling, and accessibility. However, existing information extraction technologies face the following problems when dealing with complex web page structures, dynamic content loading, and cross-site semantic differences: First, in terms of quantifying structural similarity, traditional methods rely on DOM path matching or rule matching, which struggles to overcome tag differences (such as...). and

[0003]

[0004]

[0005]

[0006]

[0007]

[0008]

[0009]

[0010]

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020] Figure 1 Figure 2

[0021]

[0022] Figure 1 Figure 2 、 (etc.) or text content. The parser automatically converts the nested relationships of HTML code into a tree structure. The generated DOM tree structure after parsing is as follows: html ├── head │ └── title │ └── "Example" (text node) └── body └── div (id="content") └── p └── "Hello World" (text node) To ensure the integrity of the DOM tree, parsing operations need to be performed after the webpage has finished loading to avoid missing dynamically loaded content. This can be achieved by listening to the DOMContentLoaded event or using the page load completion callback in a headless browser.

[0023] "Basic information" refers to the key features describing each node in the DOM tree, used for subsequent structural analysis and text location. Specifically: A tag is the HTML tag name of a node, for example... 、 、 etc. For example, for nodes<divid="content"> The tag is "div".

[0024] Depth refers to the hierarchical depth of a node in the DOM tree. The root node has a depth of 0, and the depths of its child nodes increase sequentially. For example, consider the nodes in the DOM tree... The depth is 1. The depth is 2. The depth is 3.

[0025] Attributes refer to the HTML properties of a node, including key-value pairs such as id, class, and style. For example, for a node... The attribute is {id:"content",class:"main"}.

[0026] The number of child nodes refers to the total number of direct child nodes of a node, including element nodes and text nodes. For example, for a node... Text More The number of child nodes is 2 ( and ).

[0027] The browser rendering engine generates an initial visual snapshot of the webpage (represented as a data structure that records the rendering position, size, and content characteristics of each DOM node), and records the visual coverage area and text density of each node. Specifically, the visual coverage area refers to the rectangular area occupied by a DOM node after the webpage is rendered. It describes the position and size of the node on the page. In a browser environment, it can be obtained using the getBoundingClientRect() method and is usually represented as a four-tuple containing the following information: top: The distance (in pixels) of the top edge of the node's rectangular region relative to the top of the viewport.

[0028] left: The distance (in pixels) of the left edge of the node's rectangular region relative to the left side of the viewport.

[0029] width: The width of the node's rectangular area (in pixels).

[0030] height: The height of the node's rectangular region (in pixels).

[0031] Text density refers to the proportion of text content within a DOM node's visual coverage area, used to measure the richness of a node's content. For example, consider a... The node contains the text "Hello World" (11 characters), with a width of 300 and a height of 200. Its area is 300 × 200 = 60,000 pixels². Therefore, the text density is approximately 11 / 60,000 ≈ 0.000183 (which can be adjusted to the range of 0-1 through normalization). This can be used to distinguish between main text areas (which typically have higher text density) and non-main text areas (such as advertising images with lower text density).

[0032] Initialize energy quota allocation to simulate the (biological) energy management mechanism of a forest-like successional DOM adaptive positioning architecture. Specifically, the initial energy quota (e.g., E=100) is a virtual resource pool allocated at startup to dynamically control the consumption and allocation of computing resources. The unit of energy quota is abstract and its value and consumption rules can be adjusted according to the actual scenario. Static or dynamic allocation can be used. Static allocation involves directly setting a fixed initial energy value of E=100 for the system in the program's global variables or configuration objects. Dynamic allocation involves dynamically allocating initial energy based on the device's computing power (e.g., number of CPU cores, memory size) or the complexity of the webpage (e.g., number of DOM nodes). Energy is dynamically updated (consumed or restored) based on the real-time operation type (e.g., DOM traversal, evaluation calculation); for example, DOM traversal may consume ΔE=-0.1×number of nodes, while capturing effective spores may restore ΔE=+0.5×effective spores.

[0033] For each DOM node, an evaluation function Ψ(node) = α × Tag + β × Depth + γ × VisualDensity + δ × RSI is generated, and the evaluation function value of the node is stored in the basic information data structure of the node. In this system, α, β, γ, and δ are initial coefficients (adjustable through adaptive learning), and Tag is the node's label, which needs to be converted to a numerical representation [e.g., using the hash value of the label or a predefined label mapping table (e.g., div=1, p=2, span=3)]. Depth is the label value for node depth, VisualDensity is the label value for node text density, and RSI is the label value for the root stability index, initially set to 0. The label values ​​are not directly derived from the node's label, depth, text density, and root stability index; instead, they are obtained by fitting a curve using historical data. Real-time data is input into the fitted curve to obtain the corresponding label values. These values ​​are dynamically updated based on real-time data. All possible text paths enter the evaluation state, awaiting subsequent observation and collapse.

[0034] The evaluation function is a mathematical model used to describe the uncertainty state of DOM node features in structural similarity quantification. Example: for a node... Assuming the tag values ​​are {Tag:1,Depth:2,VisualDensity:0.001,RSI:0} and the initial coefficients are {α:0.25,β:0.25,γ:0.25,δ:0.25}, then: Ψ(node)=0.25×1+0.25×2+0.25×0.001+0.25×0=0.25+0.5+0.00025+0=0.75025.

[0035] Converting HTML code into a complete DOM tree data structure is the foundation for "retrieving the HTML code of at least two detail pages from the same list page and constructing the DOM tree." Without the DOM tree, XPath positioning cannot be performed.

[0036] Basic information is used directly when "getting the main HTML and a list of direct child tag names of the current tag in the comparison HTML" and "by comparing with list page title information". For example, tag names are used to compare the list of child tag names, and attributes are used to match titles and dates.

[0037] The initial visual snapshot provides data support for subsequent steps (such as heatmap generation), indirectly optimizing the accuracy of XPath localization.

[0038] The evaluation function provides a quantized representation of node features for subsequent steps (such as semantic mapping), indirectly supporting semantic enhancement of XPath localization.

[0039] Based on the overall rendering size of the webpage [through the browser viewport size or the scroll range of the entire document], initialize and construct a blank two-dimensional matrix [assuming the webpage size is W×H pixels, the heatmap can be represented as a W×H two-dimensional matrix, or to reduce computational complexity, it can be discretized into a smaller grid (such as W / 10×H / 10)], and discretize it into grid cells; Traverse all DOM nodes and map the visual coverage area of ​​each node (including {top, left, width, height}) to the corresponding grid area in a blank two-dimensional matrix. For each grid cell in the matrix (corresponding to the pixel position of the webpage), if its coordinates fall within the visual coverage area of ​​any node, the heat value of that grid cell is determined by the characteristics of that node. If the visual coverage areas of multiple nodes overlap, the heat value is determined using the forest evolution mechanism, thus forming a heatmap. The webpage is then represented in the form of a heatmap.

[0040] The thermal value is a scalar that indicates the importance of the content in the corresponding region, and is usually between 0 and 1 (normalizable).

[0041] Noea is a normalized value of node area. Node area is the area of ​​the visual coverage region (width × height). Rendering priority is an indicator that infers the importance of a node based on its visual characteristics (such as CSS styles and hierarchy). Text density, Noea, and rendering priority also need to be converted into tag values ​​and then weighted and summed or calculated using a non-linear function to obtain heat values.

[0042] When generating a heatmap, do not include the label name (e.g., ...). 、 The presence of visual deception (such as ads mimicking the style of the main text) makes it difficult to accurately identify structurally similar areas when extracting webpage text, titles, and dates, especially in cross-site scenarios. Secondly, in terms of dynamic content management, for dynamically loaded content in Single-Page Applications (SPAs), existing technologies struggle to accurately capture effective updates and filter noise (such as ads and comment sections), leading to interference with text extraction and decreased stability of XPath localization in dynamic webpages. Especially when faced with frequent variations in webpage structure or visual deception, existing technologies are prone to misjudging text boundaries, resulting in unstable XPath localization paths, particularly when ads mimic the style of the main text or when the webpage structure changes. At a deeper level, regarding semantic interconnectivity, the significant semantic differences between cross-site webpages mean that traditional methods result in poor universality of XPath localization across different sites. Furthermore, XPath localization consumes significant computational resources, making efficient operation difficult in low-computation scenarios such as mobile devices, leading to performance degradation in resource-constrained environments. Therefore, adaptive localization of webpage text based on DOM tree structure comparison requires design and innovation to meet practical needs. To overcome the aforementioned deficiencies of the prior art and achieve the above objectives, the present invention provides the following technical solution: a webpage text adaptive localization system based on DOM tree structure comparison, which extracts webpage text, title, and date based on a forest-like succession DOM adaptive architecture comparison and performs XPath localization, including: after loading the webpage, converting the webpage's HTML code into a complete DOM tree data structure and recording node data; using a forest evolution mechanism to determine heat values, forming a heatmap, and marking potential text areas; initializing energy quota allocation and dynamically updating energy according to real-time operation types; evaluating the root stability index of each DOM node to form a stability network, and dynamically updating it according to niche factors, competition intensity, and cooperation factors when a new node is inserted into the DOM tree; starting from the root node, traversing downwards along the path with the highest text density, identifying DOM paths that meet growth conditions as the main text backbone, calculating the inner product of the evaluation function of all candidate paths, and selecting the path with an inner product higher than expected as the final text path; pre-deploying forest window events and information spore propagation; forming a structure-semantic mapping, generating a semantic transmission network, and activating semantically related nodes according to semantic energy packets. Preferably, the method for converting the HTML code of a webpage into a complete DOM tree data structure includes: each HTML tag, attribute, and text content is represented as a node in the tree, and the basic information of each node is recorded; the visual coverage area and text density of each node are recorded through an initial visual snapshot of the webpage; energy quota allocation is initialized, and energy is dynamically updated according to the real-time operation type; an evaluation function is generated for each DOM node, the evaluation function value of the node is generated, and it is stored in the basic information data structure of the node.Preferably, the method for obtaining the potential text region includes: initializing and constructing a blank two-dimensional matrix; traversing all DOM nodes and mapping the visual coverage area of ​​each node to the blank two-dimensional matrix; for each grid cell in the matrix, if its coordinates fall within the visual coverage area of ​​any node, the heat value of the grid cell is determined by the features of that node; if the visual coverage areas of multiple nodes overlap, the heat value is determined using a forest evolution mechanism to form a heat map; grid cells with heat values ​​below a threshold are marked as 0, and those above the threshold are marked as 1, identifying connected regions; for each continuous text region, if its boundary is surrounded by a continuous blank region, it is marked as a potential text region and retained; for the heat map distribution of different site pages, if the similarity is higher than expected, it is determined to be structurally equivalent, and the potential text region is recorded and marked; the energy consumed in heat map generation and canopy area detection is recorded. Preferably, the method for determining the heat value using the forest evolution mechanism is as follows: Define an initial heat value for each node; for overlapping areas, evaluate the competition intensity of all nodes, select the node with the highest competition intensity as the dominant canopy, and use its heat value as the base value; the heat values ​​of other nodes are suppressed according to the difference in competition intensity, and the base value and the contribution value of the suppressed nodes are combined to obtain the final heat value; define a cooperation factor between any two nodes to measure the cooperation intensity between nodes; if the cooperation factor is higher than a threshold, the heat value of the weaker node is enhanced, and the enhanced heat value is fused with the base value to obtain the final heat value; define a hierarchy factor, sort the nodes according to the hierarchy factor, the node with the highest hierarchy factor is the dominant node, and use its heat value as the base value; the heat values ​​of other nodes are suppressed according to the difference in hierarchy factor, and the base value and the contribution value of the suppressed nodes are fused to obtain the final heat value. Preferably, the method for forming a stable network includes: evaluating the root stability index of each DOM node; neighboring nodes forming a stable network through the decay propagation of the root stability index; when a new node is inserted into the DOM tree, classifying the new node as a candidate for main text, marginal content, or dynamic noise based on niche factors; updating the competition intensity of the new node and the cooperation factor with neighboring main text nodes; if the probability of main text attribution is higher than expected, the new node is initially determined to be a main text expansion area; otherwise, it is considered as dynamic noise pending determination; and recording the energy consumed in the formation and expansion of the stable network.Preferably, the method for obtaining the final text path includes: initializing a candidate path set, starting from the root node, traversing downwards along the path with the highest text density, and recursively traversing all child node paths; identifying DOM paths that meet growth conditions as the main text backbone, the growth conditions including continuously increasing path depth, node text density gradient changes lower than expected, and being surrounded by at least 3 layers of branch and leaf nodes; during the traversal, prioritizing paths containing potential text regions; if the average root stability index is lower than expected, then removing the path; forming a tree ring structure for the nodes on the core path, with the inner tree ring representing the core text, the middle tree ring representing the supplementary content, and the outer tree ring representing the peripheral content; calculating the inner product of the evaluation function of all candidate paths, and selecting the path with an inner product higher than expected as the final text path; and recording the energy during the backbone path extraction process. Preferably, the method for pre-deploying forest window events and information spore propagation includes: defining and listening to forest window events to capture dynamic changes in the DOM tree, including tree-falling events for unloading the main content container, seed events for new DOM nodes containing more than the expected number of text child nodes, and lighting events for the increased rendering area within the viewport exceeding expectations; defining information spores to propagate dynamic changes, including structural spores triggered by DOM subtree changes; state spores triggered by user clicks or scrolling; and semantic spores triggered by semantic delivery network updates; performing heatmap differential analysis on the newly added areas triggered by forest window events, merging the newly added areas into the main text if they are continuous text areas, and marking the newly added areas as dynamic noise if they are isolated fragments; and recording the energy consumed and recovered during the forest window events and information spore propagation process. Preferably, the method for updating the semantic transitive network includes: defining the initial root node of the semantic transitive network; identifying high heat value regions in the heatmap, using a stable network to evaluate the root stability index of the nodes, and prioritizing those higher than expected as semantic mapping candidates to form a structure-semantic mapping; adjusting weights using user confirmation feedback for semantic transitive network updates; traversing nodes on the core path, identifying inner and middle rings, and for each inner ring node, calculating its co-occurrence frequency and path distance with middle ring nodes to generate a semantic energy packet; if the energy value of the semantic energy packet exceeds expectations, activating and marking it as a semantically relevant node; otherwise, filtering and marking it as an isolated label; and recording the energy consumed in constructing and updating the semantic transitive network. Preferably, the method for dynamically updating energy according to the real-time operation type includes: after each operation is completed, checking the current updated energy and selecting the operating mode according to the energy range; the operating modes include full-function mode, cache mode, and sleep mode; in full-function mode, all operations are run; in cache mode, semantic mappings in the semantic transitive network are used preferentially; and in sleep mode, only key events are monitored; and energy is restored by monitoring idle time of operations through a timer.Preferably, the step of extracting webpage content, title, and date based on a forest-simulated succession DOM adaptive architecture and performing XPath localization includes: obtaining the HTML code of at least two detail pages from the same list page; converting the HTML code of the webpage into a complete DOM tree data structure after loading the webpage and removing invisible tags; selecting the main page and the comparison page; based on the basic information of each node, comparing the DOM structure to set the main_same_tags list to store tags with the same direct child tag name and number but different text, and the main_unsame_tags list to store tags with inconsistent direct child tag number or name; based on text density, extracting the tag with the most text in main_unsame_tags as the content and recording its XPath path; combining structure-semantic mapping, extracting the title and date tags in main_same_tags and recording their XPath paths; counting the XPath paths of all detail pages and selecting the path with the most occurrences as the final content, title, and date XPath. This invention presents the technical effects and advantages of a webpage text adaptive localization system based on DOM tree structure comparison: Through forest evolution mechanism, stability network, forest window events, and semantic transfer network, it accurately locates webpage text, titles, and dates, maintaining high accuracy even in scenarios with complex webpage structures, dynamic content loading, and cross-site semantic differences. Forest window events and information spores are used to capture dynamically loaded content in real time and filter noise, ensuring the stability of XPath localization paths in dynamic webpages. Stability network, growth conditions, and evaluation collapse resist frequent variations in webpage structure and visual deception, ensuring the accuracy of text boundaries and the reliability of XPath paths. Semantic transfer network and structure-semantic mapping achieve cross-site semantic interconnection, significantly improving the versatility of XPath localization. Energy adaptive decision-making enables efficient operation in low-computing-power scenarios, optimizing computational resource allocation and improving XPath localization performance. The accompanying drawings illustrate the structure of the webpage text adaptive localization system based on DOM tree structure comparison and the processing flow of XPath localization based on the forest-simulated succession DOM adaptive architecture of this invention. Detailed Description of Embodiments The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.Please refer to the examples shown in [examples omitted]. This example describes a webpage text adaptive positioning system based on DOM tree structure comparison, comprising: The core technical solution of this invention is based on a forest-like succession DOM adaptive architecture, including the following steps: After loading the webpage, the HTML code of the webpage is converted into a complete DOM tree data structure, wherein each HTML tag, attribute, and text content is represented as a node in the tree, and the basic information of each node is recorded, including tag, depth, attributes, and number of child nodes; For example: The HTML code is converted into a tree data structure by an HTML parser to generate a complete DOM tree. In actual implementation, the browser's built-in DOMParser or third-party libraries such as Cheerio can be used to ensure the capture of static and dynamically generated DOM structures. The webpage is loaded using a browser or a headless browser (such as Puppeteer, Selenium), ensuring that the HTML code of the webpage is completely obtained, including the DOM structure dynamically generated by JavaScript, including: using an HTML parser (such as DOMParser, BeautifulSoup, Cheerio, etc.) to parse the HTML code into a DOM tree. In a browser environment, the built-in DOMParser object can be used; in a non-browser environment (such as Node.js), a third-party library such as Cheerio can be used; the DOM tree is a tree structure with the root node as , and child nodes including and , and each node may contain child nodes (e.g., ).

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Tags may be linked to strong text nodes (such as...) Instead of using tag names as direct input for heatmap values, features unrelated to tags (such as text density, area, and rendering priority) are used. HTML tag differences are ignored by not using tag names as features, standardizing node representations, and extracting semantically irrelevant features. The heatmap itself reflects the content distribution in visual space, and continuity can be measured by the spatial connectivity of heatmap values. Focusing solely on the continuity of visual representation is achieved through spatial distribution analysis, smoothing, and connectivity measurement of the heatmap, ensuring the system transcends tag differences and focuses on the spatial continuity of content. The heatmap is converted into a binary image, with grid cells with heatmap values ​​below a threshold (e.g., 0.1) marked as 0 (blank) and those above the threshold marked as 1 (non-blank). Connected component labeling algorithms (such as Depth-First Search (DFS) or Breadth-First Search (BFS)) are used to identify connected regions in the binary image; a connected region is a region formed by all grid cells with values ​​of 0 connected in the top, bottom, left, right (4-connected) or eight directions (8-connected). For each connected region, its area (i.e., the number of grid cells it contains) is calculated. If the area is greater than a certain threshold (e.g., 5% of the total webpage area), it is considered a continuous blank area, possibly corresponding to an advertisement, navigation bar, or footer. Assume the heatmap is as follows (simplified representation, 0 represents blank, 1 represents non-blank): 1 1 1 1 11 0 0 0 11 0 0 0 11 1 1 1 1. After binarization, the heatmap remains unchanged. Using a connected component labeling algorithm, a continuous blank area (a region with a value of 0) is identified, containing 6 grid cells. If the total webpage area is 20 grid cells, the blank area accounts for 6 / 20 = 30%, far exceeding the 5% threshold, and is therefore marked as a continuous blank area, possibly corresponding to an advertisement. Identifying continuous blank areas in the heatmap is achieved through binarization of the heatmap, connected component labeling, and area filtering. Connected component labeling algorithms (such as DFS) are used to find all connected regions with a value of 0, and the area threshold is used to confirm whether it is an advertisement, navigation bar, or other non-text area. For each continuous text region, if its boundary is surrounded by continuous blank areas, it is considered a canopy region, marked as a potential text region, and preserved. A canopy region refers to a connected area in the heatmap where the text block has a high heat value (e.g., >0.5), similar to the dense distribution of tree canopies in a forest, and is surrounded by blank areas with low heat values ​​(canopy gaps). The marking method involves assigning a unique identifier to each preserved continuous text region and recording its position (bounding box) and features (such as average heat value) in the heatmap. For the heatmap distribution of different site pages, similarity is calculated (based on heatmap inner product or cosine similarity). If the similarity is higher than expected (>0.8), it is determined to be structurally equivalent, and the potential text region is recorded and marked. The energy consumed in heatmap generation and canopy region detection is recorded. Specifically, the initial energy quota (E=100) is a virtual resource pool allocated by the system at startup, used for energy management in all subsequent steps.The energy consumption for heat map generation and gap detection (ΔE=-0.1×number of nodes) is directly deducted from the initial energy quota, affecting the current energy state of the system. If the current energy E is insufficient to support the operation (for example, E<|ΔE|), the system may enter a low-power mode (such as a cache mode or a sleep mode), which is the mechanism defined in the simulated photosynthesis-respiration balance and energy management. For illustration: assume that the initial energy E of the system at startup is 100, and the DOM tree has 500 nodes. The energy consumption for heat map generation and gap detection is: ΔE=-0.1×500=-50; after the operation, the current energy of the system becomes: E=100+ΔE=100-50=50; if subsequent steps continue to consume energy, E may further decrease, triggering an energy adaptive decision (for example, E>50 full-function mode, 20<E<50 cache mode, E<20 sleep mode). The energy consumption mechanism and the initial energy quota allocation together form the energy efficiency optimization framework of the system, which simulates the photosynthesis-respiration balance of a forest system to ensure the sustainable operation of the system in low-computing-power scenarios (such as mobile terminals). In conventional methods, if the coordinates of a grid cell fall within the visual coverage area of any node, the heat value of the grid cell is determined by the feature of that node. If the visual coverage areas of multiple nodes overlap, the maximum value or weighted average of the heat values is taken. However, such heat value calculation is too average and simple, does not consider the dynamic competition between text and non-text, it is difficult to enhance the recognition ability of the text area, and the overly average calculation makes it more difficult to filter the interference of non-text areas (such as advertisements and navigation bars). In a web page, the text area usually occupies a large visual space and has a higher "competitive advantage", while non-text areas such as advertisements may be "occluded" or "suppressed"; similarly, in a forest, tall trees compete for sunlight through their canopies, and low plants may be occluded. In a web page, weak text nodes (such as a single. <article>The collaboration of tags enhances their popularity; similarly, trees in a forest share nutrients through root networks, and weaker trees may receive support from stronger trees. Furthermore, in a webpage, the main text area is typically located in the "canopy layer" of visual space, while non-text areas (such as advertisements) may be located in the "shrub layer" or "herb layer," and their popularity should be suppressed by hierarchy; similarly, plants at different levels in a forest have different ecological niches, with the canopy layer receiving priority sunlight, while the shrub and herb layers rely on the gaps in the canopy layer.

[0055] Based on the above description, the simulation of the competition (sunlight competition), cooperation (resource sharing), and stratification (canopy, shrubs, herbs) characteristics of the forest canopy better aligns with the heat value fusion rules required for the main text positioning. It avoids simply taking the maximum value or weighted average, introducing the dynamic competition and cooperation mechanisms of the forest ecosystem, making the heat value calculation more ecologically intelligent.

[0056] The method for determining thermal values ​​using forest evolution mechanisms is as follows: Define an initial thermal value for each node. The initial thermal value can be the same as the thermal value determined by the characteristics of that node. For overlapping regions, the competition intensity of all nodes is calculated based on heat values ​​and layer depth. For example, competition intensity = initial heat value × e^(-k × Depth(node)), where k is a decay coefficient (e.g., 0.1), and Depth(node) is the depth of the node in the DOM tree. Nodes with greater depth have lower competition intensity (simulating the canopy layer). The node with the highest competition intensity is selected as the dominant canopy, and its heat value is used as the base value. The heat values ​​of other nodes are suppressed according to their competition intensity differences. The suppression formula can be set as: contribution value of the suppressed node = initial heat value × e^(-α × highest node competition intensity - suppressed node competition intensity), where α is the suppression coefficient (e.g., 0.5). The base value and the contribution value of the suppressed node are combined to obtain the final heat value; final heat value = base value + Σ(suppressed node contribution value) × β, where β is the cooperation coefficient (e.g., 0.1), which controls the degree of contribution of the suppressed node.

[0057] Based on the similarity of DOM tree paths, the semantic roles of tags, and the distance of visual coverage areas, a collaboration factor is defined between any two nodes to measure the strength of collaboration between nodes. For example, the collaboration factor (node1, node2) = w1 × Str(node1, node2) + w2 × Sem(node1, node2) + w3 × Vis(node1, node2), where w1, w2, and w3 are weights (e.g., 0.4, 0.3, 0.3); Str represents the similarity based on the DOM tree path, such as whether two nodes share the same parent or ancestor node; Sem represents the semantic role based on the tag, such as... <article>and The semantic similarity is higher than and Vis represents the distance based on the visual coverage area, such as the percentage of overlapping area between two nodes.

[0058] If the cooperation factor is higher than a threshold (e.g., 0.7), the thermal value of weak nodes is enhanced. The enhanced thermal value is then merged with the base value to obtain the final thermal value. For example, the thermal value of a weak node = initial thermal value (weak node) + γ × cooperation factor (weak node, strong node) × initial thermal value (strong node), where γ is the enhancement coefficient (e.g., 0.2), strong nodes are nodes with higher thermal values, and weak nodes are nodes with lower thermal values. The final thermal value = base value + Σ(enhanced thermal value of the node) × β, where β is the cooperation coefficient (e.g., 0.1).

[0059] A hierarchy factor is defined based on the semantic weight and depth of a node to measure its hierarchical importance. For example, the hierarchy factor is calculated as e^(-λ × depth) × semantic weight, where λ is a decay coefficient (e.g., 0.1). The semantic weight of a node is as follows: <article> =1, =0.5, =0.2).

[0060] For overlapping regions, nodes are sorted by hierarchical factor. The node with the highest hierarchical factor is the dominant node, and its thermal value is used as the base value. The thermal values ​​of other nodes are suppressed according to the differences in hierarchical factors. The contribution value of the suppressed node = base value × e^(-μ × (highest node hierarchical factor - suppressed node hierarchical factor)), where μ is the suppression coefficient (e.g., 0.5). The base value and the contribution value of the suppressed node are merged to obtain the final thermal value. For example, the final thermal value = base value + Σ(suppressed node contribution value) × β, where β is the cooperation coefficient (e.g., 0.1).

[0061] The specific implementation process can be as follows: first, determine the dominant node (canopy layer priority) through hierarchical factors; then, weaken the contribution of non-text nodes through a competitive suppression mechanism; and finally, increase the heat value of text nodes through a collaborative enhancement mechanism.

[0062] The root stability index (RSI) of each DOM node is evaluated using a linear representation based on historical frequency of occurrence, resource consumption ratio, and text density. Historical frequency of occurrence, such as the path of an ad node, varies with each load, resulting in a lower RSI. Resource consumption ratio represents the time spent loading images and scripts or rendering text, and is typically higher for ad nodes.

[0063] Neighboring nodes form a stable network through root stability index decay propagation. That is, nodes with high root stability index (main text) transmit stability signals to nodes along their path, enhancing the overall stability of the path. Nodes with low root stability index (advertisements) are isolated as dead zones and marked as non-main text. The propagation formula can be regarded as: high root stability index of a new node = original root stability index + λ × ∑(root stability index of neighboring nodes), where λ is the decay coefficient.

[0064] When a new node is inserted into the DOM tree, it is classified into three categories based on its niche factor: candidate content (canopy layer), marginal content (shrub layer), or dynamic noise (herbaceous layer). If the niche factor is greater than 0.7, it is marked as "canopy layer" and given priority for inclusion in the main text. If the niche factor is between 0.3 and 0.7, it is marked as "shrub layer" and considered as marginal content pending further investigation. If the niche factor is less than or equal to 0.3, it is marked as "herbaceous layer" and initially identified as dynamic noise.

[0065] The competition intensity of new nodes is updated by combining RSI and heat value. For example, the updated competition intensity = RSI × the original competition intensity. The text assignment probability of the new node is suppressed according to the difference in competition intensity; that is, the text assignment probability = e^(-α × (the highest competition intensity of the neighboring nodes - the competition intensity of the new node)), where α is the suppression coefficient (e.g., 0.5).

[0066] Calculate the collaboration factor between the new node and its neighboring text nodes. If the collaboration factor is higher than a threshold (e.g., 0.7), the probability of the new node being assigned to the text is enhanced. For nodes that have undergone enhanced competition suppression and cooperation, if the probability of them belonging to the main text is higher than expected (>0.7), the new node is initially determined to be a main text expansion area; otherwise, it is considered as dynamic noise to be determined.

[0067] Record the energy consumed in the formation and expansion of the stable network, such as ΔE = -0.2 × number of nodes.

[0068] In the specific implementation, the above first classifies the hierarchy of new nodes (canopy, shrub, herb) through niche factors, then weakens the probability of dynamic noise attribution through a competition suppression mechanism, and finally enhances the probability of text nodes attribution through a cooperation enhancement mechanism.

[0069] The dynamic node classification mechanism better aligns with the dynamic adaptability of forest ecosystems, surpassing conventional RSI mean threshold judgment. The competition suppression mechanism effectively weakens the probability of dynamic noise nodes (such as advertisements) belonging to the main text, enhancing the identification ability of the main text expansion region. The cooperation enhancement mechanism improves the synergistic effect among main text nodes, avoiding the omission of weak main text nodes. The niche assessment mechanism ensures the hierarchical advantage of the main text region, filtering out interference from deep non-main text nodes.

[0070] By combining heatmap values, the classification process not only relies on RSI but also considers visual heatmap values ​​and ecological competition and cooperation characteristics, making the classification results closer to the visual appearance of web pages. The comprehensive mechanism dynamically adjusts the text assignment probability based on the structural, semantic, visual, and stability characteristics of new nodes, adapting to different web page structures and dynamic changes. Dominant nodes are quickly identified through niche factors and competition intensity, reducing unnecessary computational overhead.

[0071] Suppose a new node is inserted into the DOM tree of a webpage, and there is an existing node A: <article id="content">RSI=0.8, initial thermal value=0.8, Depth=2, semantic weight=1.

[0072] Existing node B: RSI=0.3, initial thermal value=0.4, Depth=2, semantic weight=0.5.

[0073] Newly inserted node C: RSI=0.4, initial thermal value=0.6, Depth=2, semantic weight=0.8.

[0074] The niche factor of node C (assuming λ=0.1) is calculated as e^(-0.1×2)×0.8×0.6≈0.8187×0.8×0.6≈0.393; Results: The niche factor of node C is 0.393, belonging to the "shrub layer", and its status as edge content is pending.

[0075] Canopy competition: Find the neighboring nodes of node C (e.g., child nodes A and B of the parent node), and calculate the competition strength (assuming k=0.1): Competition intensity (A) = 0.8 × 0.8 × e^(-0.1 × 2) ≈ 0.524; Competition intensity (B) = 0.3 × 0.4 × e^(-0.1 × 2) ≈ 0.098; Competition intensity (C) = 0.4 × 0.6 × e^(-0.1 × 2) ≈ 0.196; Node A is the dominant tree canopy.

[0076] The probability of text attribution for node C (assuming α=0.5) is calculated as e^(-0.5×(0.524-0.196))≈e^(-0.5×0.328)≈e^(-0.164)≈0.849; Result: The probability of node C belonging to the main text is 0.849 > 0.7, and it is preliminarily determined to be the main text extension area.

[0077] Root network collaboration: Calculate the collaboration factor between node C and node A (assuming Str = 0.8 (sibling node), Sem = 0.9, Vis = 0.7): The collaboration factor (C,A) = 0.4 × 0.8 + 0.3 × 0.9 + 0.3 × 0.7 = 0.32 + 0.27 + 0.21 = 0.8; Increase the probability of text attribution for node C (assuming γ=0.2): The probability of attribution to the main text (C) = 0.849 + 0.2 × 0.8 × 0.524 ≈ 0.849 + 0.0838 ≈ 0.933; Result: The enhanced text attribution probability of node C is 0.933 > 0.7, and it is determined to be a text extension region.

[0078] Newly inserted node C (<pid="new"> ) is marked as a text extension region because its niche factor, competition intensity, and cooperation factor all indicate that it is related to the existing text node A ( <article>It is highly correlated with ad node B, and with ad node B. Competition from other countries was suppressed.

[0079] Initialize the candidate path set, starting from the root node, and traverse downwards along the path with the highest text density, recursively traversing all child node paths. Based on the basic information of each node (including tags, depth, attributes, and number of child nodes) and text density, identify DOM paths that meet the growth conditions as the main text trunk. The growth conditions include: the path depth continuously increases [simulating the vertical growth of the tree trunk, that is, the depth of each node on the path strictly increases]; the change in the node text density gradient is lower than expected [e.g., ≤10%, to ensure content continuity; if the difference in text density between any two adjacent nodes is >10%, the path is removed]; and it is surrounded by at least 3 layers of branch and leaf nodes [that is, the parent node, grandparent node, and great-grandparent node of the path's end node all have at least one child node (branch and leaf node), enhancing anti-interference ability]. In a DOM tree structure, nodes with child nodes are called branch nodes (or internal nodes). Branch nodes are the branches of the tree, responsible for connecting the root node and other nodes. Nodes without child nodes are called leaf nodes (or terminal nodes). Leaf nodes are the ends of the tree and typically represent the final storage location of the data.

[0080] During the traversal, paths containing potential text regions are selected first; For each candidate path, calculate the average root stability index of the nodes on the path (based on the stable network). If it is lower than expected (e.g., <0.7), then the path is removed. Based on node depth and text density, nodes on the core path are marked to form a tree ring structure. The inner tree ring is used to represent the core text, the middle tree ring is used to represent the supplementary content, and the outer tree ring is used to represent the peripheral content. By combining heatmaps and stability networks, nodes along the core path are traversed, and the comprehensive importance factor of each node is calculated. The comprehensive importance factor is linearly affected by text density, heat value, and root stability index. Based on the comprehensive importance factor, nodes along the path are stratified and marked as tree rings. If the comprehensive importance factor > 0.8, it is marked as an inner tree ring (high text density area, such as the main body of the article). If 0.5 < comprehensive importance factor ≤ 0.8, it is marked as a middle tree ring (title, author, date, etc.). If the comprehensive importance factor ≤ 0.5, it is marked as an outer tree ring (pagination, citation, related recommendations, etc.).

[0081] Based on the evaluation function generated for each DOM node, user scrolling or interaction triggers an "observation event", which causes the evaluation function of the main text path to collapse. The inner product of the evaluation functions of all candidate paths is calculated, and the path with the inner product higher than expected is selected as the final main text path. For each candidate path, calculate the mean value of the evaluation function values of the nodes on the path as the comprehensive evaluation function value of the path. For example, it is predefined that the evaluation function value of the path at this time is weighted by the heat value and the root system stability index.

[0082] Calculate the state function inner product of all candidate paths: for two paths, calculate the inner product of their evaluation function values, where inner product (path 1, path 2) = comprehensive evaluation function value (path 1) × comprehensive evaluation function value (path 2); select paths whose inner product with the core path (backbone path) is greater than 0.8 as the final body path. A user's scrolling or interactive behavior (such as clicking "expand more") triggers an "observation event", which causes the evaluation function to collapse into a single path.

[0083] Record the energy ΔE in the backbone path extraction process as ΔE = -2 × number of superimposed paths. The number of superimposed paths refers to the size of the candidate path set. For example, if there are 3 candidate paths in the above example, the number of superimposed paths is 3.

[0084] Based on the initial visual snapshot of the web page, three types of gap events are defined and monitored to capture dynamic changes of the DOM tree, including a tree-felling event for unloading the main content container, a seed event for a new DOM node containing more text child nodes than expected, and a light event for the newly added rendering area in the viewport exceeding the expected value; Specifically, for the tree-felling event, the unload event of the main content container is monitored, for example, MutationObserver is used to detect the removal of nodes on the core path (extracted based on the backbone path), and if removal is detected, a snapshot save is triggered to record the current DOM tree state.

[0085] For the seed event, the insertion event of new DOM nodes (based on dynamic filtering) is monitored. If the number of child nodes of the new node is greater than 5 and all are text nodes, monitoring is started, and the text density thereof is evaluated in combination with a heat map.

[0086] For the light event, rendering changes in the viewport are monitored, for example, IntersectionObserver is used to detect the visual coverage area of newly added nodes in the viewport. If the newly added rendering area is greater than 15%, it is determined as a valid update.

[0087] Example: For the following dynamic changes of the DOM tree: <article id="content"> Main content text... < / article> <!-- Dynamic loading --> Additional content text... More text... If <article id="content">If the tree is unloaded, a tree-falling event is triggered, and a snapshot is saved.

[0088] like If there are 6 text child nodes, a seed event is triggered, and monitoring is started.

[0089] like If the viewport is entered and the newly rendered area is greater than 15%, a lighting event is triggered, and the update is considered valid.

[0090] Based on the basic information of each node, three types of information spores are defined to efficiently propagate dynamic changes. These include: when a structural spore is received, it carries the hash of the difference between the old and new DOM subtree structures, and the trigger condition is a DOM subtree change; when a state spore is received, it records the sequence of interaction events and the viewport state, and the trigger condition is user click / scroll; when a semantic spore is received, it propagates new semantic patterns. When a structural spore is received, the structural hash value of the newly inserted DOM subtree is calculated. Combined with the heatmap, it is evaluated whether the new subtree passes through the potential text region (canopy region). If the new structural hash matches the text library (based on the core path) by more than 90%, the forest window succession engine is activated and the new node is merged into the text.

[0091] When a state spore is received, the user behavior path is recorded, such as the viewport position where the user clicks or scrolls. Combined with evaluation collapse, the inner product of the evaluation function of the candidate path is re-evaluated.

[0092] When a semantic spore is received, the semantic similarity between the new node and the neighboring text nodes is calculated based on the cooperation factor, in conjunction with the stability network. If the similarity is higher than expected (e.g., >0.7), the new semantic pattern is propagated and the mycorrhizal network weights are updated, for example, by adjusting the root stability index of the new node through semantic similarity.

[0093] Example: New Insertion Trigger the structural spore, calculate its structural hash, combine it with the heatmap to confirm that it passes through the potential text area (canopy area), and if the matching degree is >90%, activate the forest window succession engine and merge it into the text.

[0094] Users scroll to Trigger the state spore, record the viewport state, and assist in the assessment of collapse in step 4.

[0095] like and <article id="content">If the semantic similarity is greater than 0.7, a semantic spore is triggered, and the mycorrhizal network weights are updated.

[0096] Based on the heatmap, pixel-level difference operations are performed between the newly added region triggered by the forest window event and the existing heatmap to conduct heatmap difference analysis. A connected component identification algorithm is used to determine whether the difference region is a continuous text area or an isolated fragment. If the newly added region is a continuous text area, it is merged into the main text; if the newly added region is an isolated fragment, it is marked as dynamic noise. Example, newly inserted Generate a heatmap, calculate the difference between it and the existing heatmap, and merge the heatmap regions with a heat value > 0.5 and continuous text regions into the main text.

[0097] If newly inserted If the fragments are isolated (thermal value < 0.3, not surrounded by continuous blank areas, such as the comment section) and the mean RSI is < 0.7, they are marked as dynamic noise.

[0098] Energy consumed and restored during the recording of forest window events and information spores.

[0099] For example, the energy recovery from spore generation and capture is ΔE = +0.5 × effective spores (capturing effective spores restores energy). The energy consumption for forest window succession analysis is ΔE = -0.3 × number of new nodes. Effective spores refer to spores that trigger text merging or path updates; for example, structural spores cause new nodes to be merged into the text. The energy consumption of forest window succession analysis is proportional to the number of new nodes. Extract semantic templates from open semantic libraries, such as news templates. <article>Tags usually correspond to the main text. <h1>The tag corresponds to the title. <time>The label corresponds to the date and serves as the initial root node of the semantic transitive network; Identify high heat value regions (potential text regions, canopy regions) in the heat map and extract their corresponding DOM node labels and attributes. Use a stability network to evaluate the root stability index of the nodes and prioritize nodes with RSI higher than expected (e.g., >0.7) as candidates for semantic mapping to form a structure-semantic mapping and establish links. Example: If node If the thermal value is >0.5 and the RSI is >0.7, then a mapping is established. <article> ↔ .

[0100] The semantic delivery network updates use user confirmation feedback to adjust weights. If the user confirms that it is the main text, the confirmation feedback is 1; otherwise, the confirmation feedback is 0. The construction of the semantic delivery network enables cross-site semantic delivery, bridging the semantic gap and achieving zero-sample adaptation.

[0101] Example: Weight update formula: New weight = e^(-Δt / τ) × Initial weight + (1-e^(-Δt / τ)) × Confirmation feedback, where Δt is the time interval and τ is the decay constant. For mapping... <article> ↔ ↔ is the mapping symbol, initial weight = 0.5, time interval Δt = 1, decay constant τ = 2, user confirmation = 1, then update the weight, new weight = e^(-1 / 2)×0.5+(1-e^(-1 / 2))×1≈0.6065×0.5+(1-0.6065)×1≈0.3033+0.3935≈0.6968, the mapping weight is updated to 0.6968.

[0102] Traverse the nodes on the core path, identify inner and middle growth rings, and for each inner growth ring node, calculate its co-occurrence frequency and path distance with the middle growth ring nodes to generate a semantic energy packet; if the energy value of the semantic energy packet exceeds the expectation, activate and mark it as a semantically relevant node; otherwise, filter and mark it as an isolated label. Co-occurrence frequency refers to the number of times a text node and its dependent nodes appear together in the core path or potential text area (canopy area), normalized to between 0 and 1. Path distance is the shortest path length between a text node and its dependent nodes in the DOM tree; for example, the distance between a parent and child node is 1, and the distance between sibling nodes is 2.

[0103] The text node sends semantic energy packets to nodes such as the title and date to activate semantically related nodes: Energy value (node ​​1, node 2) = co-occurrence frequency (node ​​1, node 2) × path distance (node ​​1, node 2) - ¹. Node 1 is the main text node, and node 2 is the auxiliary node. If the energy value is >0.5, node 2 is activated and marked as a semantically relevant node; otherwise, node 2 is filtered and marked as an isolated tag (e.g., irrelevant date).

[0104] Record the energy consumed in the construction and updating of the mycorrhizal network as ΔE = -0.4 × number of links.

[0105] Photosynthesis-respiration balance and energy management: Introducing an energy budget model to dynamically manage computing resources. Optimizing computing resource allocation and improving system sustainability in low-computing-power scenarios.

[0106] During the process of dynamically updating energy based on the real-time operation type, after each operation is completed, the updated energy is checked, and the operating mode is selected according to the energy range. The operating modes include full-featured mode, cache mode and hibernation mode. In full-featured mode, all operations are run. In cache mode, semantic mappings in the mycorrhizal network are used first. In hibernation mode, only critical events are monitored. Monitor idle time using a timer (such as setInterval). If there is no user interaction or DOM change, restore energy. For example, restore energy ΔE = +0.1 per second. For example: After the operation of each step is completed, check the current energy, and select the operation mode according to the energy range.

[0107] E>50: Start the full-function mode, including evaluation function generation, heat map generation, stability network construction, evaluation collapse, forest gap succession and mycorrhizal network update; 20<E<50: Enable the cache mode, preferentially use the semantic mapping cached by the mycorrhizal network, reduce DOM traversal and heat map generation, and avoid repeated calculation of heat maps or stability networks.

[0108] E<20: Enter the sleep mode, and only monitor key events, such as user scrolling (triggering a state spore) or information spore events (triggering a structural spore or a semantic spore).

[0109] Assume that the current energy E=-20.5, the operation is idle for 10 seconds, the recovered energy=+0.1×10=+1, 3 valid spores are captured, the recovered energy=+0.5×3=+1.5, the total energy is updated to -20.5+1+1.5=-18, the current energy E=-18, which is still in the sleep mode, but the energy is gradually recovering.

[0110] Through the XPath positioning method based on DOM tree structure comparison, the text, title and date information of multiple detail pages are extracted in batches from the news list page.

[0111] Extract the web page text, title and date based on DOM tree structure comparison, perform XPath positioning, which is used to extract the text, title and date information of multiple detail pages from the same list page, including the following steps: Obtain the HTML code of at least 2 detail pages from the same list page, convert the HTML code of the web page into a complete DOM tree data structure after loading the web page, construct the DOM tree and remove invisible tags; Select the main page and the control page, compare the DOM structure based on the basic information of each node, set the main_same_tags list to store tags with the same name and number of direct child tags but different text, and set the main_unsame_tags list to store tags with inconsistent number or name of direct child tags; Based on the text density, optimize through the canopy area information, extract the tag with the most text in main_unsame_tags as the text, for example, preferentially select the nodes in the canopy area, optimize the trunk path extraction, and record its XPath path; optimize through RSI evaluation and the stability network, for example, preferentially select nodes with RSI>0.7.

[0112] By combining structure-semantic mapping, title and date tags are extracted from main_same_tags, and optimization is performed using tree ring tags, such as prioritizing mid-level tree ring nodes. Their XPath paths are recorded.

[0113] Analyze the XPath paths of all detail pages and select the most frequently occurring path as the final XPath for the main text, title, and date. Verify the stability of the XPath paths using cross-site similarity metrics. Also verify the stability using dynamic filtering to ensure that dynamically inserted noisy nodes do not affect the stability of the XPath paths.

[0114] A comparison of the DOM tree structure in XPath localization includes: Based on parsing the complete DOM tree after loading the webpage, a depth-first search preorder traversal is adopted. Using basic information such as the recorded node tags and the number of child nodes, all child tags of the body tag in the main HTML are traversed, only tag tags are traversed, and plain text tags are skipped. Get the list of direct child tag names of the current tag in the main HTML and the reference HTML, and check if the list length and the child tag names in the same position are equal; if the child tag name list is the same but the text information is different, add the tag to main_same_tags; if the child tag name list is different, add the tag to main_unsame_tags and record the number of descendant tags. Title and date extraction in XPath location methods includes: Based on the basic information of each node, by comparing it with the list page title information, tags containing title text are matched in main_same_tags, and their XPath paths are recorded; Combining structure-semantic mapping, regular expressions are used to extract date strings, formatted as 'yyyy-mm-dd', and tags containing date information are matched in main_same_tags. If the text length is less than 30, it is marked as a date tag, and its XPath path is recorded.

[0115] `main_same_tags`, `main_unsame_tags`, `body`, `tag`, and `yyyy-mm-dd` are common technical terms used in DOM and XPath localization. Specifically, `main_same_tags` is a list (or data structure) that stores tags with the same direct child tag name and number but different text, used to extract titles and dates. `main_unsame_tags` is a list (or data structure) that stores tags with inconsistent direct child tag numbers or names, used to extract the body text. XPath is a query language used to locate nodes in XML or HTML documents. `body` is the `<body>` tag in an HTML document, the starting point for DOM structure comparison. `yyyy-mm-dd` is a standardized date format representing "year-month-day", used for date extraction and matching. Example

[0116] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described operation mode of the webpage text adaptive positioning system based on DOM tree structure comparison.

[0117] Since the electronic device described in this embodiment is the electronic device used to implement the webpage text adaptive positioning system based on DOM tree structure comparison in this application embodiment, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the webpage text adaptive positioning system based on DOM tree structure comparison described in this application embodiment. Therefore, how the electronic device implements the method in this application embodiment will not be described in detail here. As long as those skilled in the art implement the electronic device used in the webpage text adaptive positioning system based on DOM tree structure comparison in this application embodiment, it falls within the scope of protection of this application.

[0118] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0119] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention. < / article> < / article> < / time> < / h1> < / article> < / article> < / article> < / article> < / article> < / article> < / article> < / article>

Claims

1. A webpage text adaptive positioning system based on DOM tree structure comparison, characterized in that, Based on a forest-like succession DOM adaptive architecture, the webpage body, title, and date are extracted and XPath localization is performed, including: After the webpage is loaded, the HTML code of the webpage is converted into a complete DOM tree data structure and the node data is recorded; the forest evolution mechanism is used to determine the heat value, form a heat map, and mark potential text areas; Initialize energy quota allocation and dynamically update energy based on real-time operation type; The root stability index of each DOM node is evaluated to form a stable network. When a new node is inserted into the DOM tree, it is dynamically updated based on niche factors, competition intensity, and cooperation factors. Starting from the root node, traverse downwards along the path with the highest text density, identify DOM paths that meet the growth conditions as the main text backbone, calculate the inner product of the evaluation function of all candidate paths, and select the path with an inner product higher than expected as the final main text path. Pre-deploy forest window events and information spore propagation; form a structure-semantic mapping, generate a semantic transmission network, and activate semantically related nodes based on semantic energy packets.

2. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 1, characterized in that, The method for converting the HTML code of a webpage into a complete DOM tree data structure includes: Each HTML tag, attribute, and text content is represented as a node in a tree, recording basic information about each node; Record the visual coverage area and text density of each node through the initial visual snapshot of the webpage; Initialize energy quota allocation and dynamically update energy based on real-time operation type; An evaluation function is generated for each DOM node, and the evaluation function value of the node is stored in the node's basic information data structure.

3. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 2, characterized in that, The method for obtaining the potential text region includes: Initialize and construct a blank two-dimensional matrix; Traverse all DOM nodes and map the visual coverage area of ​​each node to a blank two-dimensional matrix. For each grid cell in the matrix, if its coordinates fall within the visual coverage area of ​​any node, the heat value of the grid cell is determined by the characteristics of that node. If the visual coverage areas of multiple nodes overlap, the heat value is determined by the forest evolution mechanism, thus forming a heat map. Grid cells with heat values ​​below the threshold are marked as 0, and those above the threshold are marked as 1, thus identifying connected regions; for each continuous text region, if its boundary is surrounded by a continuous blank region, it is marked as a potential text region and retained. If the similarity of heatmap distributions across different website pages is higher than expected, the site is considered structurally equivalent, and potential text regions are recorded and marked. Record the energy consumed in generating the heat map and detecting the canopy area.

4. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 3, characterized in that, The method for determining thermal values ​​using forest evolution mechanisms is as follows: Define the initial thermal value for each node; For overlapping regions, the competition intensity of all nodes is evaluated, and the node with the highest competition intensity is selected as the dominant tree crown, with its heat value as the base value. The heat values ​​of other nodes are suppressed according to the difference in competition intensity, and the base value and the contribution value of the suppressed nodes are combined to obtain the final heat value. Define a collaboration factor between any two nodes to measure the strength of collaboration between them; If the cooperation factor is higher than the threshold, the heat value of the weak node will be enhanced. The enhanced heat value is then merged with the base value to obtain the final heat value. Define a hierarchy factor, sort the nodes according to the hierarchy factor, and the node with the highest hierarchy factor is the dominant node, whose thermal value is used as the base value. The thermal values ​​of other nodes are suppressed according to the differences in hierarchy factors. The base value and the contribution value of the suppressed nodes are merged to obtain the final thermal value.

5. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 4, characterized in that, The method for forming a stable network includes: Evaluate the root stability index of each DOM node; Adjacent nodes form a stable network through root stability index decay propagation; When a new node is inserted into the DOM tree, it is classified as a candidate for main content, edge content, or dynamic noise based on the niche factor. Update the competition intensity of the new node and the cooperation factor with neighboring text nodes; if the probability of text ownership is higher than expected, the new node is initially determined to be a text expansion area; otherwise, it is treated as dynamic noise pending determination. Record the energy consumed in the formation and expansion of stable networks.

6. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 5, characterized in that, The method for obtaining the final text path includes: Initialize the candidate path set, starting from the root node, and traverse downwards along the path with the highest text density, recursively traversing all child node paths. Identify DOM paths that meet the growth conditions as the main text backbone. The growth conditions include a continuous increase in path depth, a node text density gradient change that is lower than expected, and being surrounded by at least 3 layers of branch and leaf nodes. During the traversal, paths containing potential text regions are selected first; If the mean root stability index is lower than expected, the path is removed. A tree-ring structure is formed for the nodes on the core path, with the inner tree rings representing the core text, the middle tree rings representing the supplementary content, and the outer tree rings representing the peripheral content. Calculate the inner product of the evaluation functions of all candidate paths, and select the path with the inner product higher than expected as the final text path; Record the energy consumed during the extraction of the main path.

7. A webpage text adaptive positioning system based on DOM tree structure comparison according to claim 6, characterized in that, The methods for pre-deploying forest window events and information spore propagation include: Define and listen for forest window events to capture dynamic changes in the DOM tree, including tree fall events for unloading the main content container, seed events for new DOM nodes containing more text child nodes than expected, and lighting events for additional rendered area in the viewport exceeding expectations. Define information spores to propagate dynamic changes, including structural spores triggered by DOM subtree changes; state spores triggered by user clicks or scrolling; and semantic spores triggered by semantic delivery network updates. Perform heatmap differential analysis on the newly added regions triggered by the forest window event. If the newly added region is a continuous text area, it is merged into the main text; if the newly added region is an isolated fragment, it is marked as dynamic noise. Energy consumed and restored during the recording of forest window events and information spores.

8. The webpage text adaptive positioning system based on DOM tree structure comparison according to claim 7, characterized in that, The method for updating the semantic delivery network includes: Define the initial root node of the semantic transitive network; Identify regions with high thermal values ​​in the heat map, use a stable network to evaluate the root stability index of nodes, and prioritize those with higher-than-expected values ​​as semantic mapping candidates to form a structure-semantic mapping. Semantic delivery network updates adjust weights using user confirmation feedback; Traverse the nodes on the core path, identify inner and middle growth rings, and for each inner growth ring node, calculate its co-occurrence frequency and path distance with the middle growth ring nodes to generate a semantic energy packet; if the energy value of the semantic energy packet exceeds the expectation, activate and mark it as a semantically relevant node; otherwise, filter and mark it as an isolated label. Record the energy consumed in building and updating the semantic transitive network.

9. A webpage text adaptive positioning system based on DOM tree structure comparison according to claim 8, characterized in that, The method for dynamically updating energy based on real-time operation type includes: After each operation is completed, check the current updated energy level and select the operating mode based on the energy range; The operating modes include full-featured mode, cache mode, and hibernation mode. In full-featured mode, all operations are run. In cache mode, semantic mappings in the semantic delivery network are used first. In hibernation mode, only critical events are monitored. Energy recovery is performed by monitoring idle time using a timer.

10. A webpage text adaptive positioning system based on DOM tree structure comparison according to claim 9, characterized in that, The method of extracting webpage content, title, and date based on a forest-simulated succession DOM adaptive architecture and performing XPath localization includes: Obtain the HTML code of at least two detail pages from the same list page, and convert the HTML code of the webpage into a complete DOM tree data structure after loading the webpage, and remove invisible tags; Select the main page and the comparison page. Based on the basic information of each node, compare the DOM structure and set the main_same_tags list to store tags with the same direct child tag name and number but different text, and the main_unsame_tags list to store tags with different direct child tag number or name. Based on text density, the tag containing the most text in main_unsame_tags is extracted as the main text, and its XPath path is recorded; combined with structure-semantic mapping, the title and date tags are extracted in main_same_tags, and their XPath paths are recorded. Count the XPath paths of all detail pages and select the path that appears most frequently as the final XPath for the main text, title, and date.

Images