Method, device and storage medium for processing browser page

Abstract

The present application relates to the field of data processing, and provides a browser page processing method and device and a storage medium. The browser page processing method comprises: obtaining a target document object model tree corresponding to a browser page; dividing the target document object model tree into multiple groups of target document object model nodes to serve as multiple target document object model nodes corresponding to each target business object respectively; for each target business object, determining an object check value corresponding to the target business object based on the multiple target document object model nodes corresponding to the target business object; determining a change type corresponding to the target business object based on the object check value corresponding to the target business object; and in the case where the change type is used to indicate that the target business object has changed, determining a set of evidence data corresponding to the target business object. The above scheme can realize efficient, accurate and structured collection and visualized evidence storage of social platform information.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically to a method for processing browser pages, a device for processing browser pages, a storage medium, and a computer program product. Background Technology

[0002] With the booming development of social media, platforms such as Weibo, WeChat official accounts, Douyin, Kuaishou, Xiaohongshu, and Zhihu have become core carriers for public information dissemination, public opinion formation, and brand communication. The dynamic content on these platforms, including posts, multi-level comments, interactive data (likes, favorites, reposts), and author information, contains extremely high social and commercial value, serving as a key data source for business scenarios such as public opinion analysis, brand reputation management, market trend analysis, and public information governance.

[0003] Existing web scraping methods typically involve traversal crawling and page-level snapshot comparison. These methods are coarse-grained and prone to data redundancy and content mixing. They also have extremely poor adaptability to the high-frequency, micro-updating scenarios of social media platforms. Those skilled in the art urgently need a lightweight, highly compatible, and low-maintenance-cost social media platform information extraction technology to overcome the inherent shortcomings of existing scraping solutions and achieve efficient, accurate, structured collection and visual evidence storage of publicly available information on social media platforms. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. The present invention provides a method for processing browser pages, a device for processing browser pages, a storage medium, and a computer program product.

[0005] According to one aspect of the present invention, a method for processing a browser page is provided. The method is applied to a browser plugin and includes: obtaining a target document object model tree (DOM) corresponding to the browser page; dividing the DOM into multiple groups of DOM nodes, each serving as a set of DOM nodes corresponding to a specific target business object; wherein the DOM nodes in each group are related; for each target business object, determining an object verification value based on the DOM nodes corresponding to that target business object; determining a change type corresponding to the target business object based on the object verification value; wherein the change type indicates a change in the target business object; and determining a storage dataset corresponding to the target business object when the change type indicates a change in the target business object.

[0006] For example, the target document object model tree is divided into multiple groups of target document object model nodes, each serving as a separate target document object model node for each target business object, including:

[0007] The target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object; wherein, the trained object segmentation model is trained based on multiple trained document object model trees and multiple labeled document object model nodes corresponding to each of the multiple trained document object model trees.

[0008] For example, the target document object model tree is divided into multiple groups of target document object model nodes, each serving as a separate target document object model node for each target business object, including:

[0009] Based on the publishing platform to which the browser page belongs, determine the preset matching template corresponding to the publishing platform; wherein, the preset matching template includes the node identifiers corresponding to each of the multiple preset document object model nodes included in the target business object;

[0010] Based on a preset matching template, multiple target document object model nodes corresponding to each target business object in the browser page are determined.

[0011] For example, based on a preset matching template, multiple target document object model nodes corresponding to each target business object in the browser page are determined, including:

[0012] If the preset document object model nodes included in the preset matching template are not matched successfully, the target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object; wherein, the trained object segmentation model is trained based on multiple trained document object model trees and multiple labeled document object model nodes corresponding to each of the multiple trained document object model trees.

[0013] For example, based on multiple target document object model nodes corresponding to the target business object, the object validation value corresponding to the target business object is determined, including:

[0014] Based on the node content of at least some of the target document object model nodes among the multiple target document object model nodes corresponding to the target business object, determine the field validation value corresponding to the target business object;

[0015] Based on the target document object model branch corresponding to the target business object in the target document object model tree, determine the structural verification value corresponding to the target business object;

[0016] Based on the appearance information of the target business object in the browser page, determine the appearance information verification value corresponding to the target business object;

[0017] The field validation value, structure validation value, and appearance information validation value corresponding to the target business object are concatenated to obtain the object validation value corresponding to the target business object.

[0018] For example, the change type is used to represent a newly added target business object. Based on the object validation value corresponding to the target business object, the change type corresponding to the target business object is determined, including:

[0019] If the field validation value, structure validation value, and appearance information validation value of the target business object are all different from all historical business objects on the browser page, the change type of the target business object is determined to represent the newly added target business object; among them, the historical business object is the target business object obtained before the target business object at the time of acquisition.

[0020] For example, the change type is used to indicate that the node content of the target document object model node has changed. Based on the object validation value corresponding to the target business object, the change type corresponding to the target business object is determined, including:

[0021] If the field validation value corresponding to the target business object is different from that of the historical business object corresponding to the target business object, and the structure validation value corresponding to the target business object is the same as that of the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the node content of the target document object model node has changed; wherein, the historical business object corresponding to the target business object is the historical business object with the highest similarity to the target business object among all historical business objects corresponding to the browser page.

[0022] For example, the change type is used to indicate a change in the structure of the target business object. Based on the object validation value corresponding to the target business object, the change type corresponding to the target business object is determined, including:

[0023] If the matching degree between the field verification value corresponding to the target business object and the field verification value corresponding to the historical business object is higher than the matching degree threshold, and the structure verification value corresponding to the target business object is different from the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the structure of the target business object has changed.

[0024] For example, the processing method further includes:

[0025] For each target business object, the public opinion level corresponding to the target business object is determined based on the change type corresponding to the target business object and the node content of the multiple target document object model nodes corresponding to the target business object; wherein, the public opinion level is used to indicate the urgency of reviewing the target business object.

[0026] For example, the evidence dataset includes at least one of the following: content structured data generated based on the node content of each of the multiple target document object model nodes corresponding to the target business object, an image of the object region corresponding to the target business object in the browser page, a Uniform Resource Locator corresponding to the browser page, the path of the target document object model branch corresponding to the target business object in the target document object model tree, the time of acquisition of the target document object model tree, the object verification value corresponding to the target business object, and the visual feature verification value of the image of the object region corresponding to the target business object in the browser page.

[0027] According to another aspect of the present invention, a browser page processing apparatus is also provided. The browser page processing apparatus includes a display and a processor. The processor is configured to run a browser plugin to perform the following steps: obtaining a target document object model tree corresponding to the browser page displayed on the display; dividing the target document object model tree into multiple groups of target document object model nodes, each serving as a plurality of target document object model nodes corresponding to each target business object; wherein the plurality of target document object model nodes in each group are related; for each target business object, determining an object verification value corresponding to the target business object based on the plurality of target document object model nodes corresponding to the target business object; determining a change type corresponding to the target business object based on the object verification value corresponding to the target business object; wherein the change type is used to indicate a change in the target business object; and determining a storage dataset corresponding to the target business object when the change type indicates a change in the target business object.

[0028] According to another aspect of the present invention, a storage medium is also provided, which stores computer program instructions that, when executed, are used to perform the above-described browser page processing method.

[0029] According to another aspect of the present invention, a computer program product is also provided, the computer program product comprising computer program instructions, which, when executed by a processor, are used to perform the above-described browser page processing method.

[0030] According to the above-described solution of the present invention, it can be applied to browser plugins. It can obtain the target document object model tree corresponding to the browser page. Then, based on the target document object model tree, it determines multiple target document object model nodes corresponding to each target business object in the browser page. For each target business object, based on the multiple target document object model nodes corresponding to that target business object, it can determine the object verification value corresponding to that target business object. Based on the object verification value corresponding to that target business object, it determines the change type corresponding to that target business object. When the change type indicates that the target business object has changed, it determines the evidence dataset corresponding to that target business object. The above solution continuously monitors the full-dimensional change behavior of the target document object model tree and reconstructs the target document object model tree corresponding to the browser page in real time, accurately distinguishing between valid business content and invalid redundant page elements. It can accurately capture refined dynamic changes such as single comments, single interaction data, and partial content modifications. It can achieve efficient, accurate, structured collection and visual evidence storage of publicly available information on social platforms. Through browser plugins, it is possible to extract information from social platforms in a lightweight, highly compatible, and low-maintenance manner. Attached Figure Description

[0031] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0032] Figure 1 A schematic flowchart illustrating a method for processing browser pages according to an embodiment of the present invention is shown;

[0033] Figure 2 A schematic block diagram of a processing apparatus according to an embodiment of the present invention is shown. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0035] To at least partially address the aforementioned problems, embodiments of the present invention provide a method for processing browser pages. This processing method can be applied to browser plugins. The browser plugin can be a lightweight data collection plugin developed based on the Browser Extensions Application Programming Interface (API). In one example, the browser plugin may include a Content Script injection module, a Document Object Model (DOM) scraping module, a DOM monitoring module, and a data processing module. Specifically, the browser plugin can inject collection scripts and monitoring scripts into the browser page through the Content Script injection module, scrape the DOM tree of the browser page through the DOM scraping module, monitor node changes to determine changes in the DOM tree through the DOM monitoring module, and process the acquired data through the data processing module.

[0036] Figure 1 A schematic flowchart illustrating a browser page processing method according to an embodiment of the present invention is shown. Figure 1 As shown, the processing method may include steps S110 to S150.

[0037] In step S110, the target document object model tree corresponding to the browser page is obtained.

[0038] If a user has a browser plugin installed and opens a social media platform's browser page, the plugin (with user authorization) can automatically inject the data collection and listening scripts. After the browser page loads, it can retrieve the target document object model (DOM) tree corresponding to the current browser page. Specifically, DOM change listeners (such as MutationObserver) can be used to detect asynchronous loading, node changes, and dynamic browser page rendering. The initial data collection can occur before the DOM tree is retrieved and after the browser page's dynamic rendering has finished. If asynchronous loading and / or node changes occur and the browser page's dynamic rendering has finished, the changed content can be added to the DOM tree to update it. In short, when the browser page changes, the corresponding DOM tree can change accordingly, and this change can be synchronized after the browser page's rendering is complete.

[0039] In step S120, the target document object model tree is divided into multiple groups of target document object model nodes, which serve as multiple target document object model nodes corresponding to each target business object.

[0040] A target business object can correspond to multiple related target data in a browser page. In one example, the multiple related target data corresponding to target business object A may include a question, multiple replies to that question, the author information of the question, the publication time of the question, the number of likes for the question, etc. In another example, the multiple related target data corresponding to target business object B may include multiple pieces of information from the content author's homepage, specifically, the target data may include the content author's profile picture, number of followers, number of followings, homepage background, bio, etc. The multiple target document object model nodes in each group are related. For example, in the hierarchical structure of a browser page, all data under a preset level can be used as the above target data.

[0041] In one example, a target document object model node can be used to represent a document object model node in the target document object model tree that has valid text (such as the specific content of the title, the specific content of the body, etc.). In another example, the target document object model tree may include a first node with valid text, a second node for representing the type of the valid text (such as title type, body type, etc.), and a target document object model node may include at least one first node and a corresponding second node for each first node.

[0042] The target document object model (DEM) nodes can be categorized and divided based on their respective node content (representing the aforementioned valid text), their corresponding object area in the browser page, and their hierarchical relationship within the DEM tree, resulting in multiple groups of DEM nodes. Specifically, for target business object A, a group of DEM nodes corresponding to target business object A might include DEM nodes whose content is related to the aforementioned question information, DEM nodes whose corresponding object area in the browser page is the object area related to the aforementioned question information, and DEM nodes belonging to the branch of the DEM tree corresponding to the post to which the aforementioned question information belongs. A DEM branch represents a portion of the DEM tree from a preset node to a leaf node. This preset node is the second node of a preset type. The preset type is a user-preset node type. It can be understood that the starting point of the DEM branches belonging to the same group of DEM nodes is the same preset node.

[0043] In one example, the plain text in the target document object model node can be obtained, and the business characters can be extracted using regular expressions. These business characters can then be identified as the node content corresponding to the target document object model node.

[0044] In step S130, for each target business object, the object verification value corresponding to the target business object is determined based on the multiple target document object model nodes corresponding to the target business object.

[0045] The object verification value corresponding to the target business object can include hash values, parity check codes, CRC values, etc., corresponding to various data of the target business object. The following explanation uses hash values ​​as an example. It is understood that developers can also use parity check codes, CRC values, etc., as verification values ​​according to actual needs. This embodiment of the invention does not impose limitations on this, and will not be elaborated further below.

[0046] In some embodiments, hash values ​​can be calculated for the node content, node type, and corresponding position in the browser page of the multiple target document object model nodes, and then combined as the object verification value corresponding to the target business object.

[0047] In some other embodiments, a hash value can be calculated based on the hierarchical and nesting relationships between multiple target document object model nodes corresponding to the target business object, in order to determine the object verification value corresponding to the target business object.

[0048] In step S140, for each target business object, the change type corresponding to the target business object is determined based on the object verification value corresponding to the target business object.

[0049] It is understandable that the target business object can change after the target document object model tree is updated.

[0050] The change type indicates the changes made to the target business object. Changes to the target business object can include the creation of a new target business object, changes to the content of the corresponding target document object model node, or changes to the structure of the target business object. Changes to the structure of the target business object can include the addition or deletion of target data corresponding to the target business object.

[0051] Specifically, when a content creator publishes a new video, the multiple target data corresponding to that video can correspond to a newly added target business object C. For the aforementioned target business object A, an increase in the target data corresponding to the target business object could be due to the addition of new reply information, while a decrease in the target data corresponding to the target business object could be due to the deletion of reply information. For the aforementioned target business object B, a change in the node content of the target document object model node corresponding to the target business object could be due to an increase in the content creator's follower count.

[0052] It is understandable that, given a defined target document object model tree, step S130 can be used to determine the object verification value corresponding to each target business object. These object verification values ​​can be stored in the database. After the target document object model tree is updated, step S130 can be used to re-determine the object verification value corresponding to each target business object. For each re-determined target business object, its corresponding object verification value can be compared with the object verification value corresponding to the target business object in the database. Target business objects with completely identical corresponding object verification values ​​can be identified as the same target business object. If the object verification value only partially matches the target business object's corresponding object in the database, it is determined that the target business object has changed (see the example below where the object verification value is obtained through concatenation processing).

[0053] When a target business object changes, the change can be determined by comparing the difference between the newly determined object validation value and the corresponding object validation value in the database. For example, if the object validation value in the database does not match the target business object at all, the change type of the target business object can be used to indicate that a new target business object has been added.

[0054] In step S150, for each target business object, if the change type indicates that the target business object has changed, the evidence storage dataset corresponding to the target business object is determined.

[0055] When the change type indicates a change in the target business object, the node content of all or part of the target document object model nodes corresponding to the target business object can be stored in the evidence storage dataset corresponding to the target business object. Specifically, when the change type indicates a change in the node content of the target document object model nodes corresponding to the target business object, this part of the target document object model nodes can be the target document object model nodes whose node content has changed.

[0056] The image of the object region corresponding to the target business object in the browser page can be stored in the evidence dataset corresponding to the target business object. Traditional data collection only outputs pure structured text data, without establishing the association and binding relationship between content, structure, and image, and lacks the ability to solidify visual evidence. In actual business scenarios such as social information forensics, content tracing, and risk control evidence storage, structured text data alone cannot restore the original display state of the page, resulting in problems such as data without visual corroboration, weak evidentiary value, inability to verify tampering, and insufficient basis for backtracking. However, the embodiment of this invention can simultaneously save screenshot results, which helps to improve the representativeness of the evidence dataset.

[0057] The Uniform Resource Locator (URL) corresponding to the browser page can be stored in the evidence storage dataset corresponding to the target business object.

[0058] The path of the target document object model branch corresponding to the target business object in the target document object model tree can be stored in the evidence dataset corresponding to the target business object. The sequence of unique identifiers of all nodes from the preset node of the target document object model branch to the leaf node can be used as the path of the target document object model branch.

[0059] The acquisition time of the target document object model tree can be stored in the evidence storage dataset corresponding to the target business object.

[0060] The object verification value corresponding to the target business object can be stored in the evidence storage dataset corresponding to the target business object.

[0061] The appearance information verification value of the object area image corresponding to the target business object in the browser page can be stored in the evidence storage dataset corresponding to the target business object.

[0062] The evidence dataset corresponding to the target business object allows users to verify the relevant network data of the target business object.

[0063] The above-described solution according to embodiments of the present invention can be applied to browser plugins. It can obtain the target document object model tree corresponding to the browser page. Then, based on the target document object model tree, it determines multiple target document object model nodes corresponding to each target business object in the browser page. For each target business object, based on the multiple target document object model nodes corresponding to that target business object, it can determine the object verification value corresponding to that target business object. Based on the object verification value corresponding to that target business object, it determines the change type corresponding to that target business object. When the change type indicates that the target business object has changed, it determines the evidence dataset corresponding to that target business object. The above solution continuously monitors the full-dimensional change behavior of the target document object model tree and reconstructs the target document object model tree corresponding to the browser page in real time, accurately distinguishing between valid business content and invalid redundant page elements. It can accurately capture refined dynamic changes such as single comments, single interaction data, and partial content modifications. It can achieve efficient, accurate, structured collection and visual evidence storage of publicly available information on social platforms. Through browser plugins, it is possible to extract information from social platforms in a lightweight, highly compatible, and low-maintenance manner. For solutions applied to servers for automatic crawling, this solution can only parse static browser page content. The above-described solution of the present invention can determine the change type in real time through a browser plugin on the user's end, thereby determining the evidence storage dataset. This greatly improves the timeliness of the evidence storage dataset (which is especially important in the scenario of public opinion reporting) and also avoids storing redundant information.

[0064] For example, step S120 involves dividing the target document object model tree into multiple groups of target document object model nodes, each serving as a corresponding target document object model node for each target business object, including step S121a.

[0065] In step S121a, the target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object.

[0066] The trained object segmentation model is obtained based on multiple training document object model trees and multiple labeled document object model nodes corresponding to each of the multiple training document object model trees. The training document object model trees can be determined based on the browser page during the training phase, referring to the relevant steps in step S110.

[0067] The trained document object model tree includes multiple trained document object model nodes, and the labeled document object model nodes can be used to represent the preset segmentation results for the trained document object model tree. The trained document object model tree can be input into an incompletely trained object segmentation model to obtain predicted document object model nodes. A first loss value can be determined based on the degree of difference between the predicted document object model nodes and the labeled document object model nodes to adjust the model parameters of the incompletely trained object segmentation model. The magnitude of the first loss value can be positively correlated with the degree of difference. It is understood that developers can add or delete the selected loss function according to actual needs, and this embodiment of the invention does not impose any restrictions.

[0068] Multiple training document object model trees can be determined for different social platforms to obtain a more adaptive trained object segmentation model.

[0069] According to the above-described scheme of the present invention, the target document object model tree can be input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object. The above scheme, through the trained object segmentation model, can determine the multiple target document object model nodes corresponding to each target business object without writing separate adaptation code for each social platform. The above scheme can maintain the accuracy of the target document object model nodes corresponding to the target business object in various business scenarios, thereby improving the accuracy of the evidence storage dataset.

[0070] For example, step S120 divides the target document object model tree into multiple groups of target document object model nodes, which serve as multiple target document object model nodes corresponding to each target business object, including steps S121b and S122b.

[0071] In step S121b, a preset matching template corresponding to the publishing platform is determined based on the publishing platform to which the browser page belongs.

[0072] The preset matching template includes the node identifiers corresponding to each of the multiple preset document object model nodes included in the target business object.

[0073] It's understandable that content creators on different social media platforms may have different writing habits, and the page designs of these platforms may also vary. Multiple preset matching templates can be determined for various preset platforms. It's also understandable that the preset matching templates for different pages, such as the homepage of a preset platform and the content creator's profile, can differ. Pages on different social media platforms can have similar structures; for example, a content creator's profile can include a profile picture, number of followers, number of following, profile background, and bio. The same preset matching template can be used for different preset platforms.

[0074] The browser page's publishing platform can be matched with these multiple preset platforms to determine the preset matching template corresponding to the publishing platform.

[0075] In step S122b, based on a preset matching template, multiple target document object model nodes corresponding to each target business object in the browser page are determined.

[0076] Based on the node identifiers corresponding to the preset document object model nodes in the preset matching template, multiple target document object model nodes that match the preset matching template can be determined (the target document object model nodes may also have corresponding node identifiers, which are used to match the node identifiers corresponding to the preset document object model nodes). These multiple target document object model nodes are then determined as the multiple target document object model nodes corresponding to each target business object in the browser page.

[0077] According to the above-described solution of the present invention, a preset matching template corresponding to the publishing platform to which the browser page belongs can be determined. Then, based on the preset matching template, multiple target document object model nodes corresponding to each target business object in the browser page are determined. The above solution, through the preset matching template, can simply and directly determine multiple target document object model nodes corresponding to each target business object, and when the publishing platform to which the browser page belongs is a preset platform, it has higher accuracy and efficiency.

[0078] For example, step S122b involves determining multiple target document object model nodes corresponding to each target business object in the browser page based on a preset matching template, including step S122b1.

[0079] In step S122b1, if it is determined that the preset document object model nodes included in the preset matching template have not been matched successfully, the target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object.

[0080] The trained object segmentation model is obtained based on multiple training document object model trees and multiple labeled document object model nodes corresponding to each of the multiple training document object model trees.

[0081] The preset matching template can be determined with reference to step S121b. The training steps for the trained object segmentation model can be referred to step S121a. The embodiments of the present invention will not be described in detail here.

[0082] In some embodiments, the determination of multiple target document object model nodes corresponding to each target business object can be based on a target matching threshold. Specifically, if a target document object model node successfully matches a preset document object model node in a preset matching template, and the degree of matching is higher than the target matching threshold, multiple target document object model nodes corresponding to each target business object can be determined using the preset matching template. Conversely, the target document object model tree can be input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object.

[0083] According to the above-described scheme of the present invention, when it is determined that the preset document object model nodes included in the preset matching template have not been matched successfully, the target document object model tree can be input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object. The above scheme combines the preset matching template and the trained object segmentation model to determine the multiple target document object model nodes corresponding to each target business object, which can balance accuracy and efficiency.

[0084] For example, in step S130, the object verification value corresponding to the target business object is determined based on multiple target document object model nodes corresponding to the target business object, including steps S131 to S134.

[0085] In step S131, based on the node content of at least some of the target document object model nodes among the multiple target document object model nodes corresponding to the target business object, the field verification value corresponding to the target business object is determined.

[0086] The content of each node in the target document object model can include the post body text, author name, posting time, number of likes, etc. Hash values ​​can be calculated for each field and used as the field verification values ​​for the target business object.

[0087] In step S132, based on the target document object model branch corresponding to the target business object in the target document object model tree, the structure verification value corresponding to the target business object is determined.

[0088] It is understandable that, for multiple target document object model nodes within a target document object model branch, changes in the hierarchical relationship, node type, and nesting depth of these nodes will alter the structural validation value corresponding to the target business object. Specifically, when the response information to a question increases or decreases, the hierarchical relationship, node type, and nesting depth of the multiple target document object model nodes within the target document object model branch corresponding to target business object A in the target document object model tree can change, thereby changing the structural validation value corresponding to target business object A.

[0089] In step S133, based on the appearance information of the target business object in the browser page, the appearance information verification value corresponding to the target business object is determined.

[0090] It can obtain the Hyper Text Markup Language (HTML) code of the browser page. Based on the HTML code, it determines the appearance information of the target business object in the browser page, and then determines the appearance information verification value corresponding to the target business object. Specifically, the appearance information of the target business object in the browser page can include information such as the position, size, and style of the target business object node in the browser page.

[0091] In step S134, the field verification value, the structure verification value, and the appearance information verification value corresponding to the target business object are concatenated to obtain the object verification value corresponding to the target business object.

[0092] The above field verification values, structure verification values, and appearance information verification values ​​can be concatenated in simple order, with staggered positions, or with string concatenation using length identifiers / delimiters to obtain the object verification value corresponding to the target business object.

[0093] According to the above-described scheme of the present invention, the field verification value corresponding to the target business object can be determined based on the node content corresponding to at least some of the target document object model nodes among multiple target document object model nodes corresponding to the target business object. Then, based on the target document object model branch corresponding to the target business object in the target document object model tree, the structure verification value corresponding to the target business object is determined. Next, based on the object area corresponding to the target business object in the browser page, the appearance information verification value corresponding to the target business object is determined. Finally, the field verification value, the structure verification value, and the appearance information verification value corresponding to the target business object are concatenated to obtain the object verification value corresponding to the target business object. The above scheme determines the object verification value through field verification value, structure verification value, and appearance information verification value, enabling triple verification, which is beneficial for accurately identifying the change type of the target business object, thereby improving the accuracy of the evidence dataset.

[0094] For example, the change type is used to represent a newly added target business object. In step S140, the change type corresponding to the target business object is determined based on the object verification value corresponding to the target business object, including step S141a.

[0095] In step S141a, if the field verification value, structure verification value, and appearance information verification value corresponding to the target business object are all different from all historical business objects corresponding to the browser page, the change type corresponding to the target business object is determined to represent the newly added target business object.

[0096] Historical business objects are target business objects retrieved prior to the current target business object.

[0097] It is understandable that if the field validation value, structure validation value, and appearance information validation value of the target business object are all different from all historical business objects on the browser page, then the target business object is different from all historical business objects and can be identified as a new target business object. The change type corresponding to the target business object is then determined to represent the newly added target business object.

[0098] For example, the change type is used to indicate that the node content of the target document object model node has changed. In step S140, the change type corresponding to the target business object is determined based on the object verification value corresponding to the target business object, including step S141b.

[0099] In step S141b, if the field verification value corresponding to the target business object is different from that of the historical business object corresponding to the target business object, and the structure verification value corresponding to the target business object is the same as that of the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the node content of the target document object model node has changed.

[0100] The historical business object corresponding to the target business object is the historical business object whose object verification value is most similar to the target business object among all historical business objects corresponding to the browser page.

[0101] It can determine the similarity between the object verification value of the target business object and the historical business objects in the database, and identify the historical business object with the highest similarity as the historical business object corresponding to the target business object.

[0102] It's understandable that if the structure validation value corresponding to the target business object is the same as that of the corresponding historical business object, then the target business object and its corresponding historical business object can be identified as the same target business object. Conversely, if the field validation value corresponding to the target business object differs from that of its corresponding historical business object, then the change type of the target business object can be used to indicate a change in the node content of the target document object model node. For example, a content creator might modify the post's body or summary content.

[0103] For example, the change type is used to indicate that the structure of the target business object has changed. In step S140, the change type corresponding to the target business object is determined based on the object verification value corresponding to the target business object, including step S141c.

[0104] In step S141c, if the matching degree between the field verification value corresponding to the target business object and the field verification value corresponding to the historical business object is higher than the matching degree threshold, and the structure verification value corresponding to the target business object is different from the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the structure of the target business object has changed.

[0105] If the matching degree between the field validation value corresponding to the target business object and the field validation value corresponding to the historical business object is higher than the matching degree threshold, the target business object and its historical counterpart can be identified as the same target business object. If the structure validation value corresponding to the target business object differs from that of its historical counterpart, the change type of the target business object can be determined to indicate a change in its structure. Specific examples include content creators deleting comments or adding new replies to posts.

[0106] For example, the processing method further includes step S210.

[0107] In step S210, for each target business object, the public opinion level corresponding to the target business object is determined based on the change type corresponding to the target business object and the node content of the multiple target document object model nodes corresponding to the target business object.

[0108] Public opinion level is used to indicate the urgency of reviewing a target business entity. This embodiment of the invention does not limit the number or categories of these levels, which can be determined by the developers based on actual needs. In one example, the public opinion level may include levels one to five (corresponding to progressively increasing urgency).

[0109] When the change type corresponding to the target business object is used to represent a newly added target business object, the public opinion level corresponding to the target business object can be determined based on the node content of multiple target document object model nodes corresponding to that target business object. Specifically, the public opinion level corresponding to the target business object can be determined based on the amount of text involved in the content of these multiple nodes or based on the sensitive words involved in the content of these multiple nodes.

[0110] When the change type corresponding to the target business object indicates a change in the node content of a target document object model node, the changed node content of the target document object model nodes among multiple target document object model nodes corresponding to the target business object can be identified, thereby determining the public opinion level corresponding to the target business object. Specifically, the public opinion level corresponding to the target business object can be determined based on the amount of text involved in the node content of the changed target document object model node, or based on the sensitive words involved in the node content.

[0111] When the change type corresponding to the target business object indicates a change in the structure of the target business object, the increase or decrease of target data corresponding to the target business object can be determined based on the node content of the target document object model nodes. For example, if the target data corresponding to the target business object decreases, the public opinion level corresponding to the target business object can be determined to be lower, and vice versa. It can be understood that a decrease in target data may also mean that sensitive content has been deleted, which can also increase the public opinion level, depending on the needs of the developers.

[0112] According to the above-described scheme of the embodiments of the present invention, the change situation may include at least one of the following: adding a new target business object, changing the node content of the target document object model node, or changing the structure of the target business object. When the field verification value, structure verification value, and appearance information verification value corresponding to the target business object are all different from all historical business objects corresponding to the browser page, the change type corresponding to the target business object can be determined to indicate a newly added target business object. When the field verification value corresponding to the target business object is different from the historical business object corresponding to the target business object, but the structure verification value corresponding to the target business object is the same as the historical business object corresponding to the target business object, the change type corresponding to the target business object can be determined to indicate a change in the node content of the target document object model node. When the matching degree between the field verification value corresponding to the target business object and the field verification value corresponding to the historical business object corresponding to the target business object is higher than a matching degree threshold, and the structure verification value corresponding to the target business object is different from the historical business object corresponding to the target business object, the change type corresponding to the target business object can be determined to indicate a change in the structure of the target business object. The public opinion level corresponding to the target business object can be determined based on the change type corresponding to the target business object and the node content of the target document object model node. The above solution determines the public opinion level corresponding to the target business object by using the target document object model node that has changed, which makes it easier for users to review the target data corresponding to the target business object, thus improving the targeting of the review.

[0113] For example, the evidence dataset includes at least one of the following: content structured data generated based on the node content of each of the multiple target document object model nodes corresponding to the target business object, an image of the object region corresponding to the target business object in the browser page, a Uniform Resource Locator corresponding to the browser page, the path of the target document object model branch corresponding to the target business object in the target document object model tree, the time of acquisition of the target document object model tree, the object verification value corresponding to the target business object, and the visual feature verification value of the image of the object region corresponding to the target business object in the browser page.

[0114] The system can perform various cleanup processes on the node content of multiple target document object model nodes corresponding to the target business object, such as cleaning up spaces and line breaks, filtering invalid HTML tags, deduplicating duplicate content, standardizing time formats, and normalizing numeric fields. By standardizing the pattern of social content business objects, the cleaned data is categorized and aligned, mapping the post body, author information, posting time, comment list, and interaction data to their respective fields to obtain assembled structured JSON data as the content structured data of the target business object.

[0115] The Uniform Resource Locator (URL) corresponding to a browser page can include query parameters and anchor information. Combining these parameters and anchor information allows for automatic scrolling to the corresponding page position after loading a specified pagination, facilitating browser page location. Referring to step S133, the location information of the target business object within the browser page is determined based on the HTML code, and a screenshot is taken of the object area corresponding to the target business object on the browser page to determine the image of the object area corresponding to the target business object within the browser page. Specifically, the screenshot result can undergo preprocessing such as invalid element removal, resolution unification, and compression optimization to obtain the image of the object area corresponding to the target business object.

[0116] Visual feature verification values ​​can be determined for the image of the object region. For example, feature extraction operations can be performed on the image of the object region (e.g., through visual feature extraction algorithms or machine learning models used for feature extraction) to obtain visual feature verification values.

[0117] When reviewers examine object region images in the evidence dataset, the authenticity of the image to be tested can be determined by comparing the visual feature verification values ​​of the image to be tested obtained by the reviewers with those of the object region image. Specifically, the image to be tested can be an object region image obtained by the reviewers based on the time of acquisition of the target document object model tree and the Uniform Resource Locator (URL) corresponding to the browser page (which can be saved via snapshot).

[0118] In any embodiment of this application, multiple target document object model nodes corresponding to the target document object can be determined as follows: For a browser page, determine whether it is a page that the user does not have permission to access. If the browser page requires login authentication or has access restrictions, re-determine other browser pages. If the browser page does not require login authentication and has no access restrictions, determine the target document object model tree. For each second node in the target document object model tree, determine the collection permission identifier corresponding to the second node. If the corresponding collection permission identifier indicates that collection is not allowed, delete the node in the target document object model tree to obtain an updated target document object model tree. Based on the updated target document object model tree, determine multiple target document object model nodes corresponding to the target document object.

[0119] In any embodiment of this application, the evidence storage dataset corresponding to the target business object can be determined as follows: For the target business object, the collection permission identifiers corresponding to multiple target document object model nodes corresponding to the target business object are re-determined, and the access permission range of the target data corresponding to the target business object is determined. When all the above collection permission identifiers are used to indicate that collection is permissible and the access permission range is unlimited, the evidence storage dataset corresponding to the target business object is determined. Traditional general collection technologies lack a clear content permission filtering mechanism, do not distinguish between publicly visible content and private restricted content, lack compliant collection constraint logic, and have unclear collection boundaries. Most solutions rely on circumventing platform anti-scraping and breaking access restrictions as core technologies, lacking compliant collection boundary design and unable to adapt to formal and compliant business scenarios such as government affairs, state-owned enterprises, and public information. However, the above-mentioned solution in this embodiment of the invention incorporates access permissions and can comply with the permission regulations of various platforms.

[0120] For each target document object, the corresponding evidence dataset can be automatically named and archived chronologically.

[0121] According to the above-described scheme of the present invention, the evidence storage dataset may include at least one of the following: structured content data generated based on the node content of multiple target document object model nodes corresponding to the target business object; an image of the object region corresponding to the target business object in the browser page; a Uniform Resource Locator (URL) corresponding to the browser page; the path of the target document object model branch corresponding to the target business object in the target document object model tree; the acquisition time of the target document object model tree; the object verification value corresponding to the target business object; and the visual feature verification value of the image of the object region corresponding to the target business object in the browser page. The evidence storage system established by the above scheme through the evidence storage dataset can deeply bind each target business object with multi-dimensional traceability evidence. If any dimension is tampered with, it can be discovered through verification in other dimensions, ensuring accurate verification of the evidence storage dataset.

[0122] It should be noted that the data obtained in this application is accessed, collected, stored, and used for subsequent analysis and processing after the user or relevant data owner has been clearly informed of the content of the data collection, the purpose of the data, the processing method, etc., and with the consent and authorization of the user or relevant data owner. Furthermore, the application can provide the user or relevant data owner with the means to access, correct, or delete the data, as well as the method to revoke consent or authorization.

[0123] This invention also provides a browser page processing device. Figure 2 A schematic block diagram of a processing apparatus 300 according to an embodiment of the present invention is shown. (In conjunction with...) Figure 2 As shown, the processing device 300 may include a display 310 and a processor 320.

[0124] Processor 320 is configured to run a browser plugin to perform the following steps: Obtain the target document object model (DEM) tree corresponding to the browser page displayed on monitor 310. Based on the DEM tree, determine multiple DEM nodes corresponding to each target business object in the browser page. All DEM nodes corresponding to the same target business object are related. For each target business object, determine the object verification value corresponding to that target business object based on the multiple DEM nodes. Based on the object verification value corresponding to that target business object, determine the change type corresponding to that target business object. The change type indicates the change status of the target business object. If the change type indicates that the target business object has changed, determine the evidence storage dataset corresponding to that target business object.

[0125] Furthermore, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the browser page processing method described above in the embodiments of the present invention, and is used to implement corresponding modules in the browser page processing apparatus described above in the embodiments of the present invention. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. A computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0126] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when executed by a computer or processor, cause the computer or processor to perform corresponding steps of the browser page processing method.

[0127] Those skilled in the art can understand the specific implementation scheme of the above storage medium by reading the relevant description of the browser page processing method. For the sake of brevity, it will not be described in detail here.

[0128] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0129] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0130] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0131] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0132] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0133] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or elements of any method or apparatus so disclosed may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0134] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0135] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the browser page processing apparatus according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0136] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0137] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for processing browser pages, characterized in that, Applied to browser plugins, the method includes: Obtain the target document object model tree corresponding to the browser page; The target document object model tree is divided into multiple groups of target document object model nodes, which serve as multiple target document object model nodes corresponding to each target business object; wherein, the multiple target document object model nodes in each group are related. For each target business object, Based on the multiple target document object model nodes corresponding to the target business object, determine the object verification value corresponding to the target business object; Based on the object verification value corresponding to the target business object, the change type corresponding to the target business object is determined; wherein, the change type is used to indicate the change status of the target business object; When the change type indicates that the target business object has changed, the evidence storage dataset corresponding to the target business object is determined.

2. The method as described in claim 1, characterized in that, The step of dividing the target document object model tree into multiple groups of target document object model nodes, each serving as a separate set of target document object model nodes for each target business object, includes: The target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object; wherein, the trained object segmentation model is trained based on multiple trained document object model trees and multiple labeled document object model nodes corresponding to each of the multiple trained document object model trees.

3. The method as described in claim 1, characterized in that, The step of dividing the target document object model tree into multiple groups of target document object model nodes, each serving as a separate set of target document object model nodes for each target business object, includes: Based on the publishing platform to which the browser page belongs, a preset matching template corresponding to the publishing platform is determined; wherein, the preset matching template includes the node identifiers corresponding to each of the multiple preset document object model nodes included in the target business object; Based on the preset matching template, multiple target document object model nodes corresponding to each target business object in the browser page are determined.

4. The method as described in claim 3, characterized in that, The step of determining multiple target document object model nodes corresponding to each target business object in the browser page based on the preset matching template includes: If it is determined that the preset document object model node included in the preset matching template has not been matched successfully, the target document object model tree is input into the trained object segmentation model to obtain multiple target document object model nodes corresponding to each target business object; wherein, the trained object segmentation model is trained based on multiple trained document object model trees and multiple labeled document object model nodes corresponding to each of the multiple trained document object model trees.

5. The method as described in claim 1, characterized in that, The step of determining the object verification value corresponding to the target business object based on multiple target document object model nodes corresponding to the target business object includes: Based on the node content of at least some of the target document object model nodes among the multiple target document object model nodes corresponding to the target business object, determine the field validation value corresponding to the target business object; Based on the target document object model branch corresponding to the target business object in the target document object model tree, determine the structural verification value corresponding to the target business object; Based on the appearance information of the target business object in the browser page, determine the appearance information verification value corresponding to the target business object; The field validation value, structure validation value, and appearance information validation value corresponding to the target business object are concatenated to obtain the object validation value corresponding to the target business object.

6. The method as described in claim 5, characterized in that, The change type is used to indicate any of the following: the addition of a new target business object, a change in the node content of a target document object model node, or a change in the structure of the target business object; Determining the change type corresponding to the target business object based on the object verification value corresponding to the target business object includes any one of the following: If the field verification value, structure verification value, and appearance information verification value corresponding to the target business object are all different from all historical business objects corresponding to the browser page, the change type corresponding to the target business object is determined to indicate the addition of a new target business object; wherein, the historical business object is the target business object obtained before the target business object at the time of acquisition; If the field verification value corresponding to the target business object is different from that of the historical business object corresponding to the target business object, and the structure verification value corresponding to the target business object is the same as that of the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the node content of the target document object model node has changed; wherein, the historical business object corresponding to the target business object is the historical business object with the highest similarity to the target business object among all historical business objects corresponding to the browser page; If the matching degree between the field verification value corresponding to the target business object and the field verification value corresponding to the historical business object is higher than the matching degree threshold, and the structure verification value corresponding to the target business object is different from the historical business object corresponding to the target business object, the change type corresponding to the target business object is determined to indicate that the structure of the target business object has changed; The method further includes: For each target business object, the public opinion level corresponding to the target business object is determined based on the change type corresponding to the target business object and the node content of the multiple target document object model nodes corresponding to the target business object; wherein, the public opinion level is used to indicate the urgency of reviewing the target business object.

7. The method as described in claim 1, characterized in that, The evidence dataset includes at least one of the following: structured content data generated based on the node content of each of the multiple target document object model nodes corresponding to the target business object; an image of the object region corresponding to the target business object in the browser page; a Uniform Resource Locator (URL) corresponding to the browser page; the path of the target document object model branch corresponding to the target business object in the target document object model tree; the acquisition time of the target document object model tree; the object verification value corresponding to the target business object; and the visual feature verification value of the image of the object region corresponding to the target business object in the browser page.

8. A browser page processing apparatus, characterized in that, The device includes a display and a processor, the processor being configured to run a browser plugin to perform the following steps: Obtain the target document object model tree corresponding to the browser page displayed on the monitor; The target document object model tree is divided into multiple groups of target document object model nodes, which serve as multiple target document object model nodes corresponding to each target business object; wherein, the multiple target document object model nodes in each group are related. For each target business object, Based on the multiple target document object model nodes corresponding to the target business object, determine the object verification value corresponding to the target business object; Based on the object verification value corresponding to the target business object, the change type corresponding to the target business object is determined; wherein, the change type is used to indicate the change status of the target business object; When the change type indicates that the target business object has changed, the corresponding evidence storage dataset target business object is determined.

9. A storage medium storing computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the browser page processing method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed by a processor, are used to perform the browser page processing method as described in any one of claims 1-7.

Images