Webpage data collection method and device, storage medium and computer equipment

Abstract

The webpage data acquisition method provided by the invention comprises the following steps: acquiring a target webpage data file, and determining a file type corresponding to the target webpage data file; extracting webpage structure information and interaction logic information from the target webpage data file according to the file type, and constructing a webpage interaction diagram of the target webpage data file according to the webpage structure information and the interaction logic information; generating a webpage operation sequence of the target webpage data file on the basis of the webpage interaction diagram in combination with a large model and a reinforcement learning strategy; according to the webpage operation sequence, collecting multi-modal webpage data from the target webpage data file, fusing the multi-modal webpage data, and generating a webpage data report according to the fused multi-modal webpage data. In this way, the adaptability of webpage data collection in the face of novel webpages with frequent content changes and complex interaction is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a web page data collection method, device, storage medium and computer equipment. Background Art

[0002] With the continuous growth of information on the internet, web data collection plays a fundamental role in various data applications. Currently, common data collection methods include fixed rule-based extraction methods, such as extracting static page data through regular expressions or XPath rules, which require custom rules for different websites; template matching-based extraction methods, which analyze the structural features or visual layout of web pages and extract data according to preset templates; and manual tagging of target content using browser plug-ins, with user intervention for data classification and upload. However, existing methods are not adaptable enough to deal with new web pages with frequently changing content and complex interactions. Rules need to be frequently adjusted, resulting in high maintenance costs and reduced collection efficiency. Summary of the Invention

[0003] The purpose of this application is to solve at least one of the above technical deficiencies, especially the technical deficiencies in the prior art of insufficient adaptability when dealing with new web pages with frequently changing content and complex interactions.

[0004] In a first aspect, the present application provides a webpage data collection method, the method comprising:

[0005] Obtaining a target webpage data file and determining the file type corresponding to the target webpage data file;

[0006] Extracting webpage structure information and interaction logic information from the target webpage data file according to the file type, and constructing a webpage interaction graph of the target webpage data file according to the webpage structure information and interaction logic information;

[0007] Based on the web page interaction graph, combined with the large model and reinforcement learning strategy, the web page operation sequence of the target web page data file is generated;

[0008] According to the web page operation sequence, multimodal web page data is collected from the target web page data file and then fused, and a web page data report is generated based on the fused multimodal web page data.

[0009] In one embodiment, the step of extracting webpage structure information and interactive logic information from the target webpage data file according to the file type includes:

[0010] When the file type is a structured file, the tag structure in the target web page data file is parsed to extract web page elements, element attributes and their relationships to determine web page structure information, and the script logic embedded in the target web page data file is analyzed to determine interaction logic information;

[0011] When the file type is unstructured, the target web page data file is processed through layout analysis, image recognition, text recognition and feature modeling to build the logical hierarchical relationship of web page elements to extract web page structure information and infer interactive logic information;

[0012] When the file type is dynamically generated content, the web page structure information and interaction logic information are extracted based on the final page presentation result after the target web page data file is rendered.

[0013] In one embodiment, the step of constructing a webpage interaction graph of a target webpage data file based on the webpage structure information and the interaction logic information includes:

[0014] By analyzing the web page structure information, multiple interactive elements are identified, and based on the interactive logic information, the triggering relationship between each interactive element is determined;

[0015] According to each interactive element and its triggering relationship, a webpage interaction graph of the target webpage data file is obtained with each interactive element as a node and each triggering relationship as an edge.

[0016] In one embodiment, the steps of generating a web page operation sequence for a target web page data file based on a web page interaction graph, combined with a large model and a reinforcement learning strategy, include:

[0017] Analyze the operable states and trigger relationships of interactive elements in web page interaction diagrams through a large model to generate a dynamic behavior model of the web page;

[0018] Construct state space and action space based on the dynamic behavior model of the web page, and train and optimize the reinforcement learning strategy based on the state space and the preset reward function feedback;

[0019] Based on the optimized reinforcement learning strategy, the target action is selected in the action space to generate a web page operation sequence.

[0020] In one embodiment, the multimodal web page data includes text data, image data, and video data; and the step of collecting the multimodal web page data from the target web page data file and fusing the data according to the web page operation sequence includes:

[0021] Collecting text data, image data, and video data from a target web page data file based on a web page operation sequence, and performing layout analysis on the target web page data file to identify text block locations, table area locations, and image area locations;

[0022] According to the text block position, table area position and image area position, the various text data, image data and video data are aligned to obtain the fused multimodal web page data.

[0023] In one embodiment, the step of generating a web page data report based on the fused multimodal web page data includes:

[0024] After identifying the information type of the fused multimodal web page data, a preset annotation model is used to annotate the data according to the information type to obtain the data annotation results;

[0025] Verify the data annotation results based on the pre-acquired user behavior data;

[0026] If there is a deviation between the data annotation results and the user behavior data, the verification fails, triggering the next round of data annotation and verification. The annotation model is optimized when the number of verifications reaches the preset threshold and the verification fails.

[0027] If there is no deviation between the data annotation results and the user behavior data, the verification is passed and a web page data report is generated.

[0028] In one embodiment, the method further comprises:

[0029] Obtain user behavior data and perform data labeling under the federated learning framework.

[0030] In a second aspect, the present application provides a webpage data collection device, comprising:

[0031] A target web page data file acquisition module is used to acquire the target web page data file and determine the file type corresponding to the target web page data file;

[0032] A webpage interaction graph construction module is used to extract webpage structure information and interaction logic information from a target webpage data file according to the file type, and to construct a webpage interaction graph of the target webpage data file according to the webpage structure information and interaction logic information;

[0033] A web page operation sequence generation module is used to generate a web page operation sequence for a target web page data file based on the web page interaction graph, combined with a large model and reinforcement learning strategy;

[0034] The web page data report generation module is used to collect multimodal web page data from the target web page data file according to the web page operation sequence, fuse the data, and generate a web page data report based on the fused multimodal web page data.

[0035] In a third aspect, the present application provides a storage medium: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of any web page data collection method in the above embodiments.

[0036] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0037] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the steps of any of the webpage data collection methods in the above embodiments are performed.

[0038] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0039] The web page data collection method provided by the present application first obtains the target web page data file and determines the corresponding file type, so as to be able to select an adaptive parsing strategy according to different types of web page files, thereby ensuring the accuracy and pertinence of subsequent processing; then, according to the file type, the web page structure information and interaction logic information are extracted from the web page data file, and a web page interaction graph is constructed, which can comprehensively characterize the dynamic structure and interactive behavior characteristics of the web page; based on the web page interaction graph, combined with the understanding ability of the large model and the reinforcement learning strategy, a web page operation sequence is generated, which can autonomously adapt to the content changes and interaction changes of different web pages without the need for frequent adjustment of fixed rules or manual intervention, and significantly improves the adaptability when facing new web pages with frequent content changes and complex interactions; through the execution of operation sequences based on the web page interaction graph and the fusion of multimodal data, the integrity and accuracy of the collected data are further ensured, the maintenance cost is reduced, and the degree of automation and efficiency of data collection are improved, thereby effectively overcoming the problem of insufficient adaptability of the collection method in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart of a web page data collection method provided in an embodiment of the present application;

[0042] Figure 2 This is an example diagram of the architecture of the web page data collection system provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of the structure of a web page data collection device provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] This application provides a web page data collection method. The following embodiments are described using the method applied to a computer device as an example. It is understood that the computer device can be any device with data processing capabilities, including but not limited to a single server, a server cluster, a personal laptop computer, a desktop computer, etc. Figure 1 As shown, the method may include the following steps:

[0047] S101: Acquire a target webpage data file and determine the file type corresponding to the target webpage data file.

[0048] The target webpage data file refers to the electronic file containing webpage content generated by a server-side response or browser-side rendering. Specifically, it may include, but is not limited to, HTML files, JSON files, DOM structured data generated by dynamic scripts, or page rendering result files. The file type refers to the data format category to which the target webpage data file belongs. These are distinguished based on different encoding rules, structural characteristics, or generation methods, such as static HTML pages, asynchronously loaded JSON data, or page content dynamically generated based on a front-end framework.

[0049] In this step, the computer device first sends an access request to the target webpage's server through the network communication module and obtains the response data returned by the webpage. The response data can be a static file directly from the webpage or a dynamic data file required for webpage rendering. To ensure the adaptability and accuracy of data collection, after receiving the response data, the computer device saves the data as a local temporary file or cache object, namely the target webpage data file.

[0050] Subsequently, the computer device can perform a file type determination operation on the target web page data file based on a preset file identification module. Specifically, the computer device can automatically identify the file type of the web page data file by parsing the Content-Type header information of the file or analyzing the structural features of the file content, such as whether it contains standard HTML tags, JSON format features, JavaScript rendering tags, etc.

[0051] Furthermore, to improve the accuracy of file type identification, the computer device can also perform auxiliary detection processes based on the initial identification. For example, it can simulate the page loading process to observe whether there are asynchronous data requests, or analyze the changes in the DOM tree during the loading process, so as to distinguish between truly static pages and pages that appear static but are actually dynamically rendered. In this way, the computer device can more accurately determine the actual type of web page data files, avoiding data extraction errors caused by misjudgment of file types during subsequent processing.

[0052] It can be understood that by performing the operations of acquiring the target web page data file and determining its file type during the initial phase of web page data collection, the computer device can select a matching parsing and processing strategy based on the actual characteristics of different web page content, thereby significantly improving the adaptability and accuracy of web page data collection. Specifically, by accurately identifying the type of web page data file, data extraction failures or omissions caused by mismatched parsing methods can be avoided, reducing the frequency of manual intervention and adjustment, and laying the foundation for subsequent web page structure analysis and interaction logic extraction.

[0053] S102: extracting web page structure information and interaction logic information from the target web page data file according to the file type, and constructing a web page interaction graph of the target web page data file according to the web page structure information and interaction logic information.

[0054] Web page structure information refers to the hierarchical relationships, layout order, and attribute descriptions of each page element on a webpage. Interaction logic information refers to the behavioral associations and page state change rules triggered by user interactions between page elements on a webpage. The web page interaction graph is a graph-structured data constructed based on the extracted web page structure information and interaction logic information.

[0055] In this step, after determining the webpage data file type, the computer device selects an appropriate parsing strategy based on the corresponding file type and activates the webpage data file parsing module. For static HTML files, the computer device can directly parse the HTML tag structure, extracting information such as the tag type, location hierarchy, and ID or Class identifiers for each element in the webpage. For dynamically rendered pages, the computer device simulates the page's runtime environment, such as by executing webpage scripts through a headless browser, dynamically constructing a complete DOM tree, and then extracting structural information.

[0056] While extracting webpage structure information, the computer device also conducts in-depth analysis of the interactive logic within the webpage. Specifically, the computer device can monitor the interaction triggering relationships between elements by listening to DOM events such as click, mouseover, and submit. It also records changes in the webpage state under user simulated operations, such as new window popups, partial content refreshes, or page jumps. Furthermore, the computer device can analyze network requests during webpage loading and extract the asynchronous data loading logic triggered by interactions, thereby fully restoring the dynamic interactive characteristics of the webpage.

[0057] Based on the extracted web page structure and interaction logic information, the computer device further constructs a web page interaction graph. During this process, the computer device uses each element or page state on the web page as a node, and each possible user-triggered interaction as an edge, connecting the interactions between nodes. Node attributes include the element's type, hierarchy, and importance score, while edge attributes include the interaction event type, triggering conditions, and resulting state change descriptions. By organizing the web page interaction graph in a standardized and structured manner, the computer device can clearly represent the overall structure and dynamic behavior flow of the web page.

[0058] In order to improve the expression accuracy and processing efficiency of web page interaction graphs, computer equipment can also introduce graph simplification strategies during the construction process, such as merging repeated interaction paths, ignoring non-critical micro-interactions, or screening high-value interaction paths based on page weight information, so that the generated web page interaction graph can not only fully reflect the essential structure and behavior of the web page, but also control the complexity, facilitating subsequent data collection and automatic operation task execution.

[0059] It can be understood that by extracting web page structure information and interaction logic information based on file type, and further constructing a web page interaction graph, computer equipment can fully understand the static layout and dynamic behavior patterns of web page elements. Compared with the traditional method of relying on simple static structure or rule extraction, by combining the extraction of structural information and interaction information, it can cope with problems such as frequent changes in content and complex interaction logic in web pages, avoid data collection failures caused by web page structure adjustments or changes in interaction methods, and improve the adaptability and robustness to new web pages. At the same time, through the construction of a web page interaction graph, computer equipment can efficiently manage and infer the interaction relationships in web pages in the form of a graph structure, which helps to improve the accuracy of web page data collection and the intelligence of operation execution, thereby effectively reducing maintenance costs and improving the stability and efficiency of web page data collection.

[0060] S103: Based on the web page interaction graph, combined with the large model and reinforcement learning strategy, a web page operation sequence of the target web page data file is generated.

[0061] A large model refers to a deep learning model trained on large amounts of data. It possesses strong feature extraction and reasoning capabilities, enabling it to understand webpage content and interaction intent. A reinforcement learning strategy is a machine learning method that uses a trial-and-error mechanism to learn optimal decision paths by maximizing cumulative rewards. A webpage action sequence refers to a series of action instructions, arranged in a specific order, used to perform specific interactions on a webpage.

[0062] In this step, after extracting the web page interaction graph, the computer device can first perform a structural analysis on the web page interaction graph. Specifically, the computer device can identify the attribute information of each node in the web page interaction graph, such as the type of element, hierarchical position, visibility status, interaction frequency, etc., and extract the attribute information of the edge, such as the interaction event type, triggering conditions and corresponding state change description. Then, based on the web page interaction graph, the computer device can pass the feature data of the nodes and edges as input to the big model for processing. In one example, the big model can be a deep learning model that has been pre-trained for the web page structure understanding task and has the ability to comprehensively understand the semantics and layout patterns of complex web page elements. By calling the big model inference service, the computer device enables the big model to combine the web page element features, interaction relationships, contextual information, etc. to infer the reasonable operation process that the user may perform when visiting the web page, and form a preliminary candidate sequence of web page actions.

[0063] To further improve the accuracy and environmental adaptability of action sequences, computers can incorporate reinforcement learning strategies to train and optimize the initially generated action sequences. During this process, the computer can construct a webpage interactive simulation environment to simulate the webpage's response to different actions. Using the reward and punishment mechanisms of reinforcement learning, the computer can iteratively adjust the action strategy based on feedback after the action is executed, prioritizing action paths with high rewards and good execution results while eliminating inefficient or failure-prone paths.

[0064] In practical applications, such as for news and information websites with frequently updated page layouts, computing devices can dynamically adapt to changes in the webpage structure. Even if a new ad module is added or the navigation bar layout changes, the computing device can reparse the webpage interaction graph and regenerate an optimized webpage operation sequence, flexibly locating target elements such as article titles, body content, and comment section entrances. This allows for continuous and stable collection of target data, avoiding issues with operational process failures caused by minor adjustments to the webpage.

[0065] In another example, for e-commerce platform pages with complex interactive logic and multiple layers of dynamic pop-ups, computer equipment can identify all possible pop-up window entrances and closing paths based on the web page interaction graph, and combine large model reasoning and reinforcement learning training to automatically learn the pop-up window processing priority and the shortest path interactive operation plan, ensuring that complex interactive obstacles can be traversed quickly and accurately when collecting product details, thereby greatly improving the success rate and efficiency of the overall collection task.

[0066] It can be understood that by generating web page operation sequences based on web page interaction graphs combined with large models and reinforcement learning strategies, computer equipment can automatically infer highly adaptable and robust operation paths when faced with complex and frequently changing web page environments, thereby reducing the risk of acquisition rule failure due to web page updates. Using large models to understand web page content and layout can accurately locate key elements and operation processes, avoiding the problem that traditional static rules cannot adapt to web page changes; combining reinforcement learning to optimize operation sequences can continuously correct operation decisions through interactive feedback, improving the success rate and stability of web page operations. Therefore, it can effectively improve the adaptability and maintenance efficiency of web page data collection systems, and significantly improve the problem that existing technologies are prone to failure in dynamic web page environments.

[0067] S104: According to the web page operation sequence, multimodal web page data is collected from the target web page data file and then fused, and a web page data report is generated based on the fused multimodal web page data.

[0068] Multimodal web data refers to the different types of data modalities involved in web pages, including but not limited to text, image, video, audio, structured table data, and script-based dynamic data. Web data reports refer to analytical documents or visual data results generated based on the fused multimodal web data, describing the content, structure, and interactive features of web pages.

[0069] In this step, after generating a web page operation sequence, the computer device first simulates or actually performs operations according to the operation sequence step by step, accessing each target element in the target web page. With each operation, the computer device captures the status data of the corresponding page in real time and collects the data modalities that are visible or interactive on the current interface. For example, when clicking on a link to a product details page, the computer device can simultaneously capture the product image, product description text, price information, and introductory short video after the page is loaded.

[0070] Furthermore, to improve the comprehensiveness and accuracy of data collection, the computer device can be equipped with a modality awareness module during the data collection process. This module automatically determines the modal content to be collected based on the element type. For example, for image elements, it performs image downloading; for video elements, it extracts playback links or downloads video clips; for text paragraphs, it performs text extraction and segmentation; for table structures, it performs structured parsing to extract the correspondence between table headers and table bodies. This on-demand, modality-based data collection ensures fine-grained and diverse data collection.

[0071] After completing the separate collection of each modal data, the computer equipment further performs fusion processing. During the fusion process, data belonging to the same business object can be unified and merged based on clues such as element association, interaction logic, and position relationship. For example, the product name text, price information, product main image, promotional video, etc. are associated with the same product entity record and organized into a standardized structure according to the preset data format. For data with dynamic changes, such as paginated comments and scrolling loading content, the interaction context information can also be combined to automatically complete the fusion field to ensure data consistency and integrity.

[0072] Finally, the computer generates a web data report based on the fused multimodal web data. Specifically, an appropriate report template can be selected based on the target web page type, such as a news element extraction report for information web pages or a product information summary report for e-commerce web pages. The report includes not only a text summary but also relevant images, tables, and interactive process descriptions, supporting subsequent content review, data analysis, or intelligent application invocation, further enhancing the depth and breadth of data utilization.

[0073] It can be understood that by gradually executing web page operations based on a web page operation sequence, the computer device can accurately locate the target element and extract the corresponding multimodal data content. Through sub-modal collection and fusion processing, the heterogeneity and inconsistency between different modal data can be eliminated, ensuring the integrity and availability of the collection results. Furthermore, by generating web page data reports, the utilization of web page data is made more standardized and systematic. Therefore, this method not only significantly improves the accuracy and coverage of collection, but also effectively reduces the risk of data loss and erroneous collection caused by the complexity and variability of web page content, and improves adaptability.

[0074] In the above embodiment, first, by obtaining the target web page data file and determining the corresponding file type, an adaptive parsing strategy can be selected according to different types of web page files to ensure the accuracy and pertinence of subsequent processing; then, according to the file type, the web page structure information and interaction logic information are extracted from the web page data file to construct a web page interaction graph, which can comprehensively characterize the dynamic structure and interactive behavior characteristics of the web page; based on the web page interaction graph, combined with the understanding ability of the large model and the reinforcement learning strategy, a web page operation sequence is generated, which can autonomously adapt to the content changes and interaction changes of different web pages without the need for frequent adjustment of fixed rules or manual intervention, and significantly improves the adaptability when facing new types of web pages with frequent content changes and complex interactions; through the execution of operation sequences based on the web page interaction graph and multimodal data fusion, the integrity and accuracy of the collected data are further ensured, the maintenance cost is reduced, and the degree of automation and efficiency of data collection are improved, thereby effectively overcoming the problem of insufficient adaptability of the collection method in the existing technology.

[0075] In one embodiment, the step of extracting webpage structure information and interaction logic information from the target webpage data file according to the file type includes:

[0076] When the file type is a structured file, the tag structure in the target web page data file is parsed to extract web page elements, element attributes and their relationships to determine web page structure information, and the script logic embedded in the target web page data file is analyzed to determine interaction logic information;

[0077] When the file type is unstructured, the target web page data file is processed through layout analysis, image recognition, text recognition and feature modeling to build the logical hierarchical relationship of web page elements to extract web page structure information and infer interactive logic information;

[0078] When the file type is dynamically generated content, the web page structure information and interaction logic information are extracted based on the final page presentation result after the target web page data file is rendered.

[0079] Structured files refer to data files with a clear markup structure, such as those in formats like HTML and XML, where web page elements and their attributes are described using explicit tags. Unstructured files are data files that lack a distinct markup structure and whose page content primarily consists of images, natural text, or layouts, such as scanned web page snapshots or single pages containing a mix of text and images. Dynamically generated content refers to web page content that is dynamically generated and displayed on the final page, relying on script execution, asynchronous data requests, or client-side rendering mechanisms. Layout analysis refers to the process of parsing the visual layout structure of a web page to identify different sections, regions, and hierarchical relationships. Image recognition refers to the classification, detection, or content recognition of image elements within a web page. Text recognition refers to the process of extracting textual content from web page images or layout areas, such as OCR (optical character recognition). Feature modeling refers to the process of building element classification, hierarchical inference, or interaction relationship reasoning models based on collected element feature information.

[0080] After the computer device receives the target web page data file, it first performs preprocessing operations based on the file type. When the file type is a structured file such as an HTML file or an MHTML archive file, the computer device directly extracts the original web page code content, including the DOM tree structure, CSS style sheet, and JavaScript script. Specifically, it parses the markup elements such as,, in the HTML or MHTML file, extracts each web page element, such as buttons, input boxes, hyperlinks, text paragraphs, and their attribute information, such as id, class, href, style, and establishes a parent-child hierarchical relationship or link relationship between the elements, thereby determining the basic structural layout of the web page. At the same time, it further extracts <script> 标签内嵌的 JavaScript 脚本，分析事件绑定逻辑例如 onclick、addEventListener 等注册的交互行为，以识别用户与网页元素的交互方式，构建交互逻辑信息。对于外部引用的 CSS或 JS 文件，亦支持根据链接路径进行内容抓取与解析，确保结构和交互信息提取的完整性。

[0081] 当文件类型为非结构化文件如 PDF 文件、PNG / JPG 图片时，计算机设备无法直接获得标准化网页结构，因此需通过版面分析、图像识别、文本识别及特征建模等手段对文件内容进行处理。以 PDF 文件为例，应用版面分析模型如 LayoutLMv3 对页面中的文本块、表格、图像进行区域检测，结合光学字符识别技术（OCR）识别文本内容，并基于文本块的空间分布推断逻辑层级关系如标题、正文、页脚等节点的层次结构。通过字体大小、颜色、排列方式等视觉特征，进一步推断每个元素可能对应的网页元素类型及其属性信息，并在此基础上模拟网页 DOM 树结构。对于图片文件如网页截图，采用实例分割模型如 YOLACT 分割出图像中的不同功能区域如按钮、输入框、图片区域，并通过光学字符识别技术（OCR）识别文本内容，通过 CLIP 模型提取图像语义特征，结合区域位置和视觉布局推断网页元素的层级关系和潜在交互逻辑。

[0082] 当文件类型为动态生成内容如通过 URL 实时访问的单页应用（SPA）页面时，由于网页内容在初次加载时未完全包含在 HTML 文件中，计算机设备需通过模拟浏览器环境如使用 Playwright、Puppeteer 等自动化渲染工具，执行完整的页面加载过程以获取最终呈现的 DOM 树结构。具体地，拦截页面加载过程中的网络请求，实时提取 CSS 样式表与JavaScript 代码，并在页面渲染完成后，抓取动态生成的 DOM 结构、元素属性和交互绑定逻辑例如通过 JavaScript 动态添加的事件监听器或异步数据加载行为。该方式能够有效应对 JavaScript 动态渲染内容带来的结构变化，确保提取出的网页结构信息和交互逻辑信息反映最终页面状态。

[0083] 为了支持批量处理，若输入为压缩包如 ZIP、RAR，计算机设备首先执行自动解压操作，识别其中包含的各类文件。针对解压得到的每个文件，依据其具体类型分别适配提取流程，确保在批量文件场景下亦能准确、高效地提取网页结构信息和交互逻辑信息。此外，针对云存储链接如 S3、OSS 中的文件，可以通过认证访问接口下载目标文件，统一纳入文件类型判别与处理流程之中，以保证来源一致性和处理完整性。

[0084] 本实施例中，通过根据文件类型差异分别采取标记结构解析、版面分析与多模态识别、渲染结果提取等处理方式，计算机设备能够针对不同结构特点的网页数据文件，采用最匹配的技术路径进行网页结构信息与交互逻辑信息的提取，从而提升提取过程的准确性与完整性。如此显著降低因文件结构复杂性或动态行为多样性导致的提取遗漏或提取错误的风险，有效保证网页交互图构建与操作序列推理的质量和可靠性，进而增强整个网页理解与操作自动化的稳定性与适应性。

[0085] 在一个实施例中，根据网页结构信息和交互逻辑信息，构建目标网页数据文件的网页交互图的步骤，包括：

[0086] 通过分析网页结构信息，识别多个交互元素，并基于交互逻辑信息，确定各个交互元素之间的触发关系；

[0087] 根据每个交互元素及其触发关系，以每个交互元素为节点且以每个触发关系为边，得到目标网页数据文件的网页交互图。

[0088] 其中，交互元素是指在网页中可与用户互动的组件，例如按钮、输入框、下拉菜单等。这些元素通常附带有事件监听器，用于响应用户的操作，如点击、输入或鼠标悬停等。

[0089] 具体而言，首先解析网页的结构信息，识别页面中所有的交互元素，包括解析网页中的 DOM 树结构、CSS 样式以及 JavaScript 脚本，识别出所有可以与用户交互的元素。例如，按钮、链接、表单字段和下拉菜单等。

[0090] 接下来，计算机设备分析这些交互元素的相互关系，即它们之间的触发关系。可以通过分析嵌入在网页中的 JavaScript 代码来实现的。查找与交互元素相关的事件绑定代码，识别用户交互与页面变化之间的逻辑映射关系。例如，当用户点击某个按钮时，可能会触发一个弹出窗口，或者更新页面上的某个数据区域。通过这种方式，计算机设备能够确定哪些交互元素是相互关联的，哪些操作是由用户触发的。

[0091] 在得到交互元素及其触发关系后，构建一个网页交互图，其中每个交互元素作为一个节点，而元素之间的触发关系则构成了节点之间的有向边。例如，若用户点击按钮 A会触发按钮 B 的变化，则在交互图中会有一个从 A 到 B 的有向边。通过这种方式，网页交互图能够直观地表示网页中各个交互元素的触发顺序和依赖关系。例如，在电商网站中，当用户选择商品并点击加入购物车按钮时，可以通过解析交互图，确定该操作是否会影响购物车的显示、商品数量的更新或页面底部的结算按钮状态。

[0092] 可以理解，通过网页特征提取和交互图的构建，计算机设备能够全面理解网页中的交互元素和其触发关系，从而自动识别和执行网页上的动态行为。

[0093] 在一个实施例中，基于网页交互图，结合大模型和强化学习策略，生成目标网页数据文件的网页操作序列的步骤，包括：

[0094] 通过大模型分析网页交互图中交互元素的可操作状态及其触发关系，生成网页动态行为模型；

[0095] 基于网页动态行为模型构建状态空间和动作空间，并根据状态空间及预设奖励函数反馈，训练优化强化学习策略；

[0096] 基于优化后的强化学习策略，在动作空间中选择目标动作,生成网页操作序列。

[0097] 其中，网页动态行为模型基于交互图，模拟网页中交互元素的状态和行为，以理解和预测网页的动态交互。状态空间是指网页中各种交互状态的集合，动作空间是指与这些状态相关的所有可能的操作。强化学习策略则是基于状态空间和动作空间，通过预设的奖励函数进行优化学习的策略，旨在从中选择最优的操作序列来实现预期目标。

[0098] 具体而言，首先利用大模型分析 DOM 树，提取网页中交互元素的属性，如id、class、onclick 等，并识别事件监听代码，如 JavaScript 中的 addEventListener，获取网页元素的基本信息和其可能的交互行为。在此基础上，通过语义分析，推断元素之间的触发关系。例如，识别出当用户点击某个按钮时，页面会加载新的商品列表，从而推断出按钮与加载商品列表的行为之间的关联。

[0099] 接下来，构建状态空间和动作空间。状态空间包括当前页面用于标识页面结构的DOM 哈希值、已执行的操作序列，如 click(button1)、scroll(200px) 以及交互图中的可用节点，如未点击的按钮，为强化学习算法提供了对当前网页状态的全面了解。动作空间则包括可能的操作，例如点击按钮、滚动页面、输入文本等，具体操作对应交互图中的节点。基于当前的状态空间，选择最优的动作，以期得到期望的网页响应。

[0100] 为了优化操作策略，使用强化学习的 PPO 算法。通过模拟在交互图中选择特定的动作，例如点击某个按钮，观察页面的响应并根据预设的奖励函数对行为进行评分。例如，当点击按钮后页面成功加载了新内容，根据页面哈希值变化或新元素出现给予正向奖励，如 +10；若页面没有变化，则给予负向奖励，如 -5。通过不断的训练，学会根据奖励优化策略，从而生成最优的操作序列。例如，在实际操作中，可能选择执行 click(load-more) →scroll(500px)，通过反复调整和优化，生成最适合当前页面结构和任务目标的操作序列。

[0101] 最后，操作序列可以通过无头浏览器，如 Playwright 进行自动化执行，以无需人工干预地完成自动化的网页操作过程，提高操作效率，减少人为错误，并提升网页交互的响应速度。

[0102] 本实施例中，通过分析网页交互图中的交互元素的可操作状态及其触发关系，能够理解网页中元素的动态行为，并预测它们之间的交互，从而更准确地识别和模拟网页操作，还能生成精确的网页动态行为模型，从而更好地应对用户交互。通过强化学习训练，能基于状态空间和动作空间优化决策策略，以在不同情境下做出最佳决策，自动化生成最有效的网页操作序列，避免了人工规则设定的局限性，能够处理复杂的动态交互，增强了自适应能力，提高了网页交互操作的自动化程度。

[0103] 在一个实施例中，多模态网页数据包括文本数据、图像数据和视频数据；根据网页操作序列，从目标网页数据文件中采集多模态网页数据后融合的步骤，包括：

[0104] 基于网页操作序列采集目标网页数据文件中的文本数据、图像数据和视频数据，并对目标网页数据文件进行版面分析，识别文本块位置、表格区域位置和图像区域位置；

[0105] 根据文本块位置、表格区域位置和图像区域位置，对各个文本数据、图像数据和视频数据进行对齐，得到融合后的多模态网页数据。

[0106] 其中，文本数据、图像数据和视频数据是网页内容的不同形式，分别代表网页中的文字信息、图像信息和视频信息。文本块位置、表格区域位置和图像区域位置则指的是网页中各类内容区域的具体位置。

[0107] 具体而言，首先需要从目标网页中采集文本数据、图像数据和视频数据。计算机设备通过网页操作序列执行一系列操作以加载网页内容，采集到网页中的不同数据类型。接下来，进行版面分析，识别出网页中的文本块位置、表格区域位置和图像区域位置。可以通过分析网页的 DOM 结构和视觉信息，确定每一类数据在页面中的具体布局。例如，通过图像识别技术来识别图像区域，使用文本分析技术识别文本块，并通过网页解析技术定位表格区域。识别完成后，根据这些位置对文本、图像和视频数据进行对齐，使得不同类型的数据能够按照网页版面的实际布局进行合理匹配，确保它们在多模态数据融合时保持一致性。

[0108] 在一个示例中，通过提取网页 HTML 代码，结合样式表和脚本，自动解析页面的视觉布局。此时，不仅需要关注每个数据块的内容，还需要精确确定其在网页中的空间位置。例如，文本块可能会出现在页面的左侧，而图像可能位于右侧的特定区域。对这些元素的位置进行精确的坐标计算后，利用图像处理和文本处理算法，将这些数据类型融合到一个统一的多模态数据结构中。最终，生成的多模态网页数据不仅包含了文本、图像和视频数据，还包括了它们在网页中的空间布局信息。

[0109] 在另一个示例中，在处理包含商品信息的网页时，文本采集模块首先使用BeautifulSoup解析HTML并清除噪声，如广告弹窗，提取商品描述。对于PDF或图片中的文本，使用OCR进行多语言识别。图像处理模块通过YOLACT模型进行图像分割，识别商品图，并利用CLIP模型生成图像特征向量。对于视频内容，FFmpeg提取关键帧，Whisper模型进行语音识别，并通过SAM模型分割视频中的主体。音视频处理模块采用AWS Transcribe或Vosk模型进行实时语音识别，并结合MFCC和情感分析提取音频特征。在多模态数据融合过程中，LayoutLMv3模型检测PDF或图片中的版面信息，合并相邻文本块，识别标题、正文和页脚等逻辑区域。文本与图像通过CLIP模型进行对齐，确保商品描述与图像之间的匹配，而视频中的语音内容与关键帧时间戳进行对齐，生成同步字幕。

[0110] 本实施例中，通过执行网页操作序列采集目标网页数据、进行版面分析，并对数据进行对齐，可以有效实现对网页内容的全面提取与准确重构，并且深入理解网页的结构与内容，确保不同类型的数据在融合过程中保持位置上的一致性。首先，采集不同类型的网页数据能全面获取网页内容，提高数据采集的完整性；其次，通过版面分析，能够精确识别网页的布局结构，确保在对齐过程中，文本、图像和视频等内容能够正确地对应到页面的实际位置，避免数据错位或丢失；最终，融合后的多模态网页数据提供了精准、结构化的数据支持，从而提升网页数据采集的准确性与处理效率。

[0111] 在一个实施例中，根据融合后的多模态网页数据，生成网页数据报告的步骤，包括：

[0112] 识别融合后的多模态网页数据的信息类型后，根据信息类型，采用预设的标注模型进行数据标注，得到数据标注结果；

[0113] 根据预先获取的用户行为数据，对数据标注结果进行验证；

[0114] 若数据标注结果与用户行为数据存在偏差，则验证不通过，触发下一轮数据标注及验证，直至验证次数达到预设阈值且验证未通过时，优化标注模型；

[0115] 若数据标注结果与用户行为数据不存在偏差，则验证通过，生成网页数据报告。

[0116] 其中，用户行为数据指的是通过分析用户在网页上的互动历史获得的数据，用作验证标注结果是否符合实际用户需求的依据。

[0117] 具体而言，首先通过分析融合后的多模态网页数据中的信息类型，识别出各个数据元素的性质和功能。例如，识别网页中的文本块、图片、表格、视频等不同类型的元素，并根据信息类型选择预设的标注模型进行数据标注。可以使用基于深度学习的自动化标注模型，如自然语言处理模型标注文本内容，计算机视觉模型标注图像和视频内容。通过标注模型对原始数据进行自动化处理，给每一类数据分配合适的标签或类别。

[0118] 随后，借助预先收集的用户行为数据，如用户的点击行为、停留时间等，对标注结果进行验证。验证过程将标注结果与用户行为数据进行比对，判断标注结果是否符合用户的实际行为。如果发现标注结果与用户行为数据存在偏差，即标注结果未能准确反映用户的行为，则会触发下一轮数据标注和验证。在此过程中，对标注模型进行不断地优化，直到验证通过为止。如果验证次数达到预设阈值，但验证依然未通过，对标注模型进行进一步优化，调整其算法和策略，以提高准确性。

[0119] 当标注结果与用户行为数据完全一致时，验证通过，生成最终的网页数据报告。该报告不仅包含了准确标注的网页内容信息，还结合了用户的行为数据，提供深度的分析洞察。为了进一步增强报告的准确性和深度，可以检索外部知识库，如历史价格数据库或行业报告，补充上下文信息。例如，在分析产品价格时，可能会生成一条报告："本周iPhone 15Pro价格上涨2.3%，主要因供应商成本增加”，以便为用户提供更为全面的市场动态。这种信息的补充，不仅提升了报告的内容丰富性，也增强了其对用户需求的精准回应。

[0120] 在一个示例中，假设正在处理一篇包含多个数据点的文章，其中有一个文本"iPhone 15 Pro”。最初，LLM根据上下文语义将其错误地标注为"date”，而实际上应标注为"product_model”。与基准库中的历史数据对比后，标注的相似度低于0.85，触发了第一次冲突，进入二次验证阶段。行为数据采集显示，用户频繁点击与"价格”相关的区域，但在"iPhone 15 Pro”这个文本上停留时间较长，并且点击行为更多，表明用户的关注点倾向于产品型号而非日期字段。根据验证规则，数据点的行为特征与典型标签行为模式的偏差超过30%，触发了二次验证。进入人工仲裁池后，人工标注员确认该数据点应标注为"product_model”，并将修正后的数据回馈到模型训练中进行微调。然而，在后续的标注处理中，模型仍错误标注为"date”，未能吸取人工标注员的修正。由于这是第三次冲突，触发了深度优化，利用人工标注数据重新训练模型，优化LLM的标注策略。通过这一过程，能够不断调整和优化标注模型，提高数据标注的准确性，减少人工干预的频率。

[0121] 本实施例中，通过结合用户行为数据对数据标注结果进行验证，能够有效确保标注模型的精度。与用户行为数据进行比对，能够揭示标注模型可能存在的误差或偏差，进而通过迭代优化，不断提高标注准确度。

[0122] 在一个实施例中，方法还包括：

[0123] 在联邦学习框架下获取用户行为数据，以及进行数据标注。

[0124] 其中，联邦学习是一种分布式机器学习方法，允许在多个计算设备上进行模型训练，而无需将数据集中到一个中央服务器。各设备在本地执行数据处理和模型更新，随后通过加密方式将更新后的模型参数上传至服务器进行聚合，从而共享全局模型的优化结果，确保数据隐私不被泄露。

[0125] 具体而言，首先，通过本地采集已授权的用户行为数据，可以来自用户的点击记录、停留时间、滚动轨迹等交互信息，根据与用户的交互记录生成个性化的行为数据。例如，智能手机可能会记录用户点击某个产品信息区域的次数，并分析用户在该区域的停留时长。接下来，在本地应用初步的标注规则对数据进行标注。例如，基于预定义的标签模型对网页中的文本进行自动标注，如将"iPhone 15 Pro”标注为"产品型号”，将日期"2025-03-24”标注为"日期”。当完成本地标注后，标注数据将与其他用户设备进行同步更新，并通过联邦学习框架上传至中央服务器进行汇总。由于本地进行了数据采集和处理，因此不会泄露用户的私有数据，所有数据的隐私性得到保障。

[0126] 在验证阶段，如果本地标注与用户行为数据一致且符合预定标准，验证通过，最终生成准确的报告。

[0127] 为便于理解本申请的方案，下面提供具体的示例进行说明。

[0128] 如图2所示，系统架构可分为输入层、处理层和输出层。输入层支持多文件类型，包括 HTML、PDF、图片、MHTML 归档文件及压缩包，还支持直接输入 URL 和云存储链接，并能批量处理。处理层包含动态解析、多模态采集、智能标注、隐私保护等模块，其中动态解析模块可处理不同类型文件提取相关元素，也能对 URL 输入进行实时网页解析等；多模态采集模块负责文本采集、图像 / 视频处理、音视频处理；智能标注模块有生成式标注和验证机制；隐私保护模块采用联邦学习框架和敏感信息过滤。输出层可输出结构化数据、可视化数据和数据报告多种格式。

[0129] 下面对本申请实施例提供的网页数据采集装置进行描述，下文描述的网页数据采集装置与上文描述的网页数据采集方法可相互对应参照。如图3所示，本申请提供了一种网页数据采集装置，装置包括：

[0130] 目标网页数据文件获取模块201，用于获取目标网页数据文件，并确定目标网页数据文件对应的文件类型；

[0131] 网页交互图构建模块202，用于根据文件类型，从目标网页数据文件中提取网页结构信息和交互逻辑信息，并根据网页结构信息和交互逻辑信息，构建目标网页数据文件的网页交互图；

[0132] 网页操作序列生成模块203，用于基于网页交互图，结合大模型和强化学习策略，生成目标网页数据文件的网页操作序列；

[0133] 网页数据报告生成模块204，用于根据网页操作序列，从目标网页数据文件中采集多模态网页数据后融合，并根据融合后的多模态网页数据，生成网页数据报告。

[0134] 在一个实施例中，网页交互图构建模块202包括：

[0135] 第一目标网页数据文件解析单元，用于当文件类型为结构化文件时，通过解析目标网页数据文件中的标记结构，提取网页元素、元素属性及其相互关系，以确定网页结构信息，并分析目标网页数据文件中嵌入的脚本逻辑，以确定交互逻辑信息；

[0136] 第二目标网页数据文件解析单元，用于当文件类型为非结构化文件时，通过版面分析、图像识别、文本识别及特征建模，对目标网页数据文件进行处理，构建网页元素的逻辑层级关系，以提取网页结构信息，并推断交互逻辑信息；

[0137] 第三目标网页数据文件解析单元，用于当文件类型为动态生成内容时，基于目标网页数据文件渲染完成的最终页面呈现结果，提取网页结构信息和交互逻辑信息。

[0138] 在一个实施例中，网页交互图构建模块202包括：

[0139] 触发关系确定单元，用于通过分析网页结构信息，识别多个交互元素，并基于交互逻辑信息，确定各个交互元素之间的触发关系；

[0140] 网页交互图构建单元，用于根据每个交互元素及其触发关系，以每个交互元素为节点且以每个触发关系为边，得到目标网页数据文件的网页交互图。

[0141] 在一个实施例中，网页操作序列生成模块203包括：

[0142] 网页动态行为模型生成单元，用于通过大模型分析网页交互图中交互元素的可操作状态及其触发关系，生成网页动态行为模型；

[0143] 强化学习策略优化单元，用于基于网页动态行为模型构建状态空间和动作空间，并根据状态空间及预设奖励函数反馈，训练优化强化学习策略；

[0144] 网页操作序列生成单元，用于基于优化后的强化学习策略，在动作空间中选择目标动作,生成网页操作序列。

[0145] 在一个实施例中，多模态网页数据包括文本数据、图像数据和视频数据；网页数据报告生成模块204包括：

[0146] 多模态网页数据采集单元，用于基于网页操作序列采集目标网页数据文件中的文本数据、图像数据和视频数据，并对目标网页数据文件进行版面分析，识别文本块位置、表格区域位置和图像区域位置；

[0147] 多模态网页数据融合单元，用于根据文本块位置、表格区域位置和图像区域位置，对各个文本数据、图像数据和视频数据进行对齐，得到融合后的多模态网页数据。

[0148] 在一个实施例中，网页数据报告生成模块204包括：

[0149] 数据标注结果获取单元，用于识别融合后的多模态网页数据的信息类型后，根据信息类型，采用预设的标注模型进行数据标注，得到数据标注结果；

[0150] 数据标注结果验证单元，用于根据预先获取的用户行为数据，对数据标注结果进行验证；

[0151] 标注模型优化单元，用于若数据标注结果与用户行为数据存在偏差，则验证不通过，触发下一轮数据标注及验证，直至验证次数达到预设阈值且验证未通过时，优化标注模型；

[0152] 网页数据报告生成单元，用于若数据标注结果与用户行为数据不存在偏差，则验证通过，生成网页数据报告。

[0153] 在一个实施例中，装置还包括：

[0154] 隐私保护模块，用于在联邦学习框架下获取用户行为数据，以及进行数据标注。

[0155] 在一个实施例中，本申请还提供了一种存储介质，所述存储介质中存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如上述实施例中任一项所述网页数据采集方法的步骤。

[0156] 在一个实施例中，本申请还提供了一种计算机设备，所述计算机设备中存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如上述实施例中任一项所述网页数据采集方法的步骤。

[0157] 示意性地，如图4所示，图4为本申请实施例提供的一种计算机设备的内部结构示意图，该计算机设备300可以被提供为一服务器。参照图4，计算机设备300包括处理组件302，其进一步包括一个或多个处理器，以及由存储器301所代表的存储器资源，用于存储可由处理组件302的执行的指令，例如应用程序。存储器301中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件302被配置为执行指令，以执行上述任意实施例的网页数据采集方法。

[0158] 计算机设备300还可以包括一个电源组件303被配置为执行计算机设备300的电源管理，一个有线或无线网络接口304被配置为将计算机设备300连接到网络，和一个输入输出（I / O）接口305。计算机设备300可以操作基于存储在存储器301的操作系统，例如WindowsServer TM、Mac OS XTM、Unix TM、Linux TM、Free BSDTM或类似。

[0159] 本领域技术人员可以理解，图4中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。

[0160] 最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语"包括”、"包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句"包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。本文中，"一”、"一个”、"所述”、"该”和"其”也可以包括复数形式，除非上下文清楚指出另外的方式。多个是指至少两个的情况，如2个、3个、5个或8个等。"和 / 或”包括相关所列项目的任何及所有组合。

[0161] 本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间可以根据需要进行组合，且相同相似部分互相参见即可。

[0162] 对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。< / script>

Claims

1. A web page data collection method, characterized in that: The method comprises: Obtaining a target webpage data file and determining a file type corresponding to the target webpage data file; extracting webpage structure information and interaction logic information from the target webpage data file according to the file type, and constructing a webpage interaction graph of the target webpage data file according to the webpage structure information and the interaction logic information; Based on the webpage interaction graph, combined with a large model and a reinforcement learning strategy, a webpage operation sequence of the target webpage data file is generated; According to the web page operation sequence, multimodal web page data is collected from the target web page data file and then fused, and a web page data report is generated based on the fused multimodal web page data.

2. The webpage data collection method according to claim 1, characterized in that: The step of extracting webpage structure information and interactive logic information from the target webpage data file according to the file type includes: When the file type is a structured file, the web page structure information is determined by parsing the tag structure in the target web page data file to extract web page elements, element attributes, and their relationships, and the script logic embedded in the target web page data file is analyzed to determine the interactive logic information; When the file type is an unstructured file, the target web page data file is processed through layout analysis, image recognition, text recognition, and feature modeling to construct a logical hierarchical relationship between web page elements, thereby extracting the web page structure information and inferring the interactive logic information; When the file type is dynamically generated content, the web page structure information and the interactive logic information are extracted based on the final page presentation result after the target web page data file is rendered.

3. The webpage data collection method according to claim 1, characterized in that: The step of constructing a webpage interaction graph of the target webpage data file based on the webpage structure information and the interaction logic information includes: Identifying a plurality of interactive elements by analyzing the webpage structure information, and determining a trigger relationship between each of the interactive elements based on the interactive logic information; According to each of the interactive elements and their triggering relationships, a webpage interaction graph of the target webpage data file is obtained with each of the interactive elements as a node and each of the triggering relationships as an edge.

4. The webpage data collection method according to claim 1, characterized in that: The step of generating a webpage operation sequence for the target webpage data file based on the webpage interaction graph in combination with a large model and a reinforcement learning strategy includes: Analyze the operable states and triggering relationships of the interactive elements in the webpage interaction diagram through the large model to generate a webpage dynamic behavior model; Constructing a state space and an action space based on the webpage dynamic behavior model, and training and optimizing the reinforcement learning strategy according to the state space and the preset reward function feedback; Based on the optimized reinforcement learning strategy, a target action is selected in the action space to generate the web page operation sequence.

5. The webpage data collection method according to claim 1, characterized in that: The multimodal web page data includes text data, image data and video data; The step of collecting and fusing multimodal web page data from the target web page data file according to the web page operation sequence includes: collecting text data, image data, and video data from the target web page data file based on the web page operation sequence, and performing layout analysis on the target web page data file to identify text block locations, table area locations, and image area locations; According to the position of the text block, the position of the table area and the position of the image area, the respective text data, the image data and the video data are aligned to obtain fused multimodal web page data.

6. The webpage data collection method according to claim 1, characterized in that: The step of generating a web page data report based on the fused multimodal web page data includes: After identifying the information type of the fused multimodal webpage data, a preset annotation model is used to annotate the data according to the information type to obtain a data annotation result; Verify the data annotation results based on the pre-acquired user behavior data; If there is a deviation between the data annotation result and the user behavior data, the verification fails, triggering the next round of data annotation and verification. When the number of verifications reaches a preset threshold and the verification fails, the annotation model is optimized. If there is no deviation between the data annotation result and the user behavior data, the verification is passed and the web page data report is generated.

7. The webpage data collection method according to claim 6, characterized in that: The method further comprises: The user behavior data is obtained and the data is labeled under the federated learning framework.

8. A web page data collection device, characterized in that: The device comprises: A target web page data file acquisition module is used to acquire a target web page data file and determine the file type corresponding to the target web page data file; a webpage interaction graph construction module, configured to extract webpage structure information and interaction logic information from the target webpage data file according to the file type, and to construct a webpage interaction graph of the target webpage data file according to the webpage structure information and the interaction logic information; A web page operation sequence generation module is used to generate a web page operation sequence for the target web page data file based on the web page interaction graph, in combination with a large model and a reinforcement learning strategy; The web page data report generating module is used to collect multimodal web page data from the target web page data file according to the web page operation sequence, fuse the data, and generate a web page data report based on the fused multimodal web page data.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the web page data collection method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the web page data collection method according to any one of claims 1 to 7 are performed.

Images