Multi-source literature batch acquisition and management method based on browser automatic control

Abstract

The invention discloses a multi-source literature batch acquisition and management method based on browser automatic control, and relates to the technical field of browser management.The method comprises the steps that a retrieval request input by a user on a graphical user interface is responded, and the retrieval request at least comprises a retrieval keyword and one or more target literature database identifiers; based on the retrieval request, initializing a browser automatic control environment, and navigating to a retrieval entry of the target literature database; automatically executing a retrieval operation in a retrieval entry of the target literature database to obtain a literature list page; and carrying out structural analysis and feature extraction on the literature list page, and dynamically identifying a literature entry container area in the page. According to the method, a page structure dynamic perception and self-adaptive analysis mechanism is introduced, so that page layout changes of different databases can be intelligently identified and adapted without manual intervention, and stable and accurate extraction of multi-source literature information is realized.

Description

Technical Field

[0001] This invention belongs to the field of browser management technology, and in particular relates to a method for batch acquisition and management of multi-source documents based on browser automatic control. Background Technology

[0002] In current practices of bulk document retrieval, there is a persistent lack of effective technologies to handle dynamic changes in page layouts, especially given the diverse structures and frequent front-end updates of academic databases and professional search platforms. Existing automated solutions often fail to fundamentally address the inherent relationship between parsing logic and specific page structures. When the target website undergoes interface redesign, element reorganization, or the introduction of new interaction modes, the existing automated processes become ineffective, requiring manual re-analysis of the page and code adjustments. This deficiency leads to widespread difficulties in cross-platform compatibility, long-term stability, and maintenance costs for technical solutions. Therefore, the following solutions are proposed to address these issues. Summary of the Invention

[0003] The purpose of this invention is to provide a method for batch acquisition and management of multi-source documents based on browser-based automatic control. By introducing a dynamic perception and adaptive parsing mechanism for page structure, it can intelligently identify and adapt to changes in the page layout of different databases without manual intervention, thereby achieving stable and accurate extraction of multi-source document information and solving the problem of automation process failure caused by changes in the front-end structure of the target website in existing methods.

[0004] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0005] This invention relates to a method for batch acquisition and management of multi-source documents based on browser-automated control, comprising the following steps:

[0006] In response to a user's search request entered in a graphical user interface, the search request includes at least search keywords and one or more target literature database identifiers;

[0007] Based on the search request, initialize the browser's automated control environment and navigate to the search entry point of the target document database;

[0008] The search operation is automatically performed in the search portal of the target literature database to obtain a literature list page;

[0009] Structural analysis and feature extraction are performed on the document list page to dynamically identify the document entry container area on the page;

[0010] Based on the content complexity assessment of the document entry container area, the corresponding parsing strategy is adaptively selected.

[0011] Based on the selected parsing strategy, extract the document metadata from the document entry container area and obtain the corresponding document details access link;

[0012] Based on the document details access link, control the browser to navigate to the document details page and automatically trigger the full-text download operation;

[0013] During the full-text download and metadata extraction process, task status monitoring and exception handling are performed to achieve task status persistence and breakpoint resumption.

[0014] The downloaded full-text documents and extracted metadata are linked, organized, and stored in a structured manner, and then output to a specified location.

[0015] Furthermore, the step of initializing the browser automation control environment includes:

[0016] Start a browser instance and inject a script into the browser instance to enhance the document object model access and structure monitoring capabilities;

[0017] Load the adapter template corresponding to the target document database identifier. The adapter template includes at least the database access address and session maintenance logic.

[0018] Furthermore, the step of performing structural analysis and feature extraction on the document list page to dynamically identify the document entry container area includes:

[0019] Obtain the document object model tree of the document list page;

[0020] Calculate the structural feature vector of the element in the document object model tree, wherein the structural feature vector includes element depth, number of child elements, attribute information and visual presentation information;

[0021] Cluster analysis is performed on page elements based on the structural feature vectors to obtain element clusters that reflect different layout patterns;

[0022] Candidate clusters that meet preset conditions are selected from the element clusters and used as the document entry container area.

[0023] Furthermore, the step of adaptively selecting the corresponding parsing strategy based on the content complexity assessment of the document entry container region includes:

[0024] The content complexity evaluation value of the document entry container area is calculated. The content complexity evaluation value is calculated based on the text length distribution characteristics, the proportion of link elements, and the proportion of table elements within the area.

[0025] The content complexity evaluation value is compared with a preset threshold range;

[0026] Based on the comparison results, one of the predefined parsing strategies is selected. The parsing strategy includes at least a rule-based position index parsing strategy, a parsing strategy based on visual block segmentation and text density, and a parsing strategy based on semantic segmentation model.

[0027] Furthermore, the step of extracting document metadata from the document entry container area according to the selected parsing strategy includes:

[0028] For each metadata field to be extracted, multiple candidate text contents are obtained from the document entry container area;

[0029] Based on predefined field feature rules and domain knowledge dictionary, the multiple candidate text contents are cleaned, verified and confidence evaluated;

[0030] The candidate text content with the highest confidence assessment result is selected as the final extraction result of the metadata field.

[0031] Furthermore, the steps of monitoring and handling the execution task status, and implementing task status persistence and breakpoint resumption, include:

[0032] Real-time monitoring of task execution process; capture abnormal events that occur during operation.

[0033] The abnormal events are classified to distinguish between temporary errors and structural errors;

[0034] For temporary errors, an automatic retry mechanism is implemented;

[0035] For structural errors, log the error context information and skip the current failure point to continue executing the task;

[0036] Periodically persist the task execution progress, successfully processed intermediate results, and error logs to the local database;

[0037] When a task is restarted after being interrupted, it locates the last interruption point and continues to execute the unfinished part based on the information stored in the persistent memory.

[0038] Furthermore, the step of associating and structurally storing the successfully downloaded full-text document file with the extracted document metadata includes:

[0039] The metadata extracted from multiple target literature databases is merged and deduplicated, and the deduplication operation is based on the similarity calculation of literature identifiers or metadata.

[0040] The deduplicated document metadata is generated into a structured data file;

[0041] The structured data file is stored together with the corresponding full-text document file, and a report file containing task statistics is generated.

[0042] The present invention has the following beneficial effects:

[0043] 1. This invention effectively improves the system's compatibility with changes in the page layout of different academic databases by dynamically sensing page structure and using an adaptive parsing mechanism. Traditional methods based on fixed path parsing are costly to maintain due to strong coupling with specific website structures and are prone to functional failure due to page updates. This invention, by analyzing the topological features and visual attribute distribution of the DOM tree in real time, can identify document entry areas without relying on pre-configured parsing rules and adaptively select appropriate parsing strategies based on the complexity of the page content. This method reduces the dependence on the stability of the target website structure, enabling the system to adapt to more professional databases and their page redesigns, extending the effective lifespan of the tool, and reducing the maintenance workload caused by changes in website structure.

[0044] 2. This invention enhances the robustness and accuracy of the document information extraction process. By employing a multi-verification mechanism and a confidence fusion strategy, the system can reliably extract structured metadata from complex and inconsistent page elements. Traditional methods often extract based on a single attribute or location, which can easily lead to extraction failures or errors due to the diverse presentation of data. This invention improves the success rate of identifying key fields such as titles, authors, and abstracts by combining regular expressions, domain dictionaries, and evaluation of candidate texts from multiple locations. This design reduces information omissions or mis-extractions during the extraction process, providing a more complete and accurate data foundation for subsequent document management and analysis.

[0045] 3. This invention enables interruptible recovery and stable execution of batch acquisition tasks. Through built-in task queue management, state persistence, and hierarchical error handling mechanisms, the system can cope with unstable factors such as network fluctuations and page loading anomalies. Traditional scripting solutions often experience overall interruption and are difficult to recover when an anomaly occurs. This invention decomposes the task process into monitorable sub-steps and periodically saves progress snapshots, allowing the task to continue execution from the breakpoint, avoiding repetitive work and resource waste. This design improves the reliability of completing long-term, large-scale document acquisition tasks and reduces the risk of overall task failure due to unstable external environments.

[0046] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the multi-source document batch acquisition and management method based on browser automatic control according to the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Please see Figure 1 As shown, this invention is a method for batch acquisition and management of multi-source documents based on browser-automatic control, including the following steps:

[0051] In response to a user's search request entered in the graphical user interface, the search request includes at least search keywords and one or more target literature database identifiers;

[0052] Based on the search request, initialize the browser's automated control environment and navigate to the search entry point of the target literature database;

[0053] The search operation is automatically performed in the search portal of the target literature database to obtain a list of literature pages;

[0054] Perform structural analysis and feature extraction on the literature list page to dynamically identify the container areas of literature entries on the page;

[0055] Based on the content complexity assessment of the document entry container area, the corresponding parsing strategy is adaptively selected;

[0056] Based on the selected parsing strategy, extract the document metadata from the document entry container area and obtain the corresponding document details access link;

[0057] Based on the document details access link, control the browser to navigate to the document details page and automatically trigger the full-text download operation;

[0058] During the full-text download and metadata extraction process, task status monitoring and exception handling are performed to achieve task status persistence and breakpoint resumption.

[0059] The downloaded full-text documents and extracted metadata are linked, organized, and stored in a structured manner, and then output to a specified location.

[0060] The steps for initializing the browser automation control environment include:

[0061] Start a browser instance and inject a script into the browser instance to enhance document object model access and structure monitoring capabilities;

[0062] Load the adapter template corresponding to the target literature database identifier. The adapter template must contain at least the database access address and session maintenance logic.

[0063] The steps for performing structural analysis and feature extraction on the document list page, and dynamically identifying the container area of ​​document entries, include:

[0064] Retrieve the document object model tree from the literature list page;

[0065] Calculate the structural feature vector of an element in the Document Object Model tree. The structural feature vector contains element depth, number of child elements, attribute information, and visual presentation information.

[0066] Cluster analysis of page elements based on structural feature vectors yields element clusters that reflect different layout patterns;

[0067] Candidate clusters that meet preset conditions are selected from the element clusters and used as the container area for document entries.

[0068] The steps for adaptively selecting the corresponding parsing strategy based on the content complexity assessment of the document entry container region include:

[0069] The content complexity evaluation value of the document entry container area is calculated. The content complexity evaluation value is based on the text length distribution characteristics, the proportion of link elements, and the proportion of table elements within the area.

[0070] The content complexity evaluation value is compared with a preset threshold range;

[0071] Based on the comparison results, one of the predefined parsing strategies is selected. The parsing strategies include at least a rule-based position index parsing strategy, a parsing strategy based on visual block segmentation and text density, and a parsing strategy based on semantic segmentation models.

[0072] The steps for extracting document metadata from the document entry container area, based on the selected parsing strategy, include:

[0073] For each metadata field to be extracted, multiple candidate text contents are obtained from the document entry container area;

[0074] Based on predefined field feature rules and domain knowledge dictionaries, multiple candidate text contents are cleaned, verified, and their confidence is evaluated.

[0075] The candidate text content with the highest confidence assessment result is selected as the final extraction result of the metadata field.

[0076] The steps for performing task status monitoring and exception handling, and implementing task status persistence and breakpoint resumption include:

[0077] Real-time monitoring of task execution process; capture abnormal events that occur during operation.

[0078] Classify abnormal events, distinguishing between temporary errors and structural errors;

[0079] For temporary errors, an automatic retry mechanism is implemented;

[0080] For structural errors, log the error context information and skip the current failure point to continue executing the task;

[0081] Periodically persist the task execution progress, successfully processed intermediate results, and error logs to the local database;

[0082] When a task is restarted after being interrupted, it uses the information stored in persistent storage to locate the point of interruption and continue executing the unfinished part.

[0083] The steps for associating and structurally storing the successfully downloaded full-text document files with the extracted metadata include:

[0084] The metadata extracted from multiple target document databases is merged and deduplicated. The deduplication operation is based on document identifiers or metadata similarity calculation.

[0085] The deduplicated document metadata is generated into a structured data file;

[0086] The structured data files are stored together with the corresponding full-text documents, and a report file containing task statistics is generated.

[0087] One specific application of this embodiment is:

[0088] Step S1: System Startup and Task Configuration

[0089] Step S11: The user starts the system through the graphical user interface; in the interface, the user enters search keywords (e.g., "maize drought stress meteorological model") and selects the professional databases in the field of agricultural meteorology to be searched (e.g., AGRIS, CABAbstracts, Chinese Agricultural Meteorology Academic Journal Database).

[0090] Step S12: The user sets the storage path to save the downloaded document PDF files and the extracted metadata table;

[0091] Step S13: After configuration, the user clicks the "Start Execution" button. The system receives the configuration parameters and generates a unique task identifier (TaskID), which then enters the task queue to wait for scheduling.

[0092] Step S2: Initialize the browser automation environment

[0093] Step S21: The system calls the browser automation control module to launch a headless or visible Microsoft Edge browser instance based on the Selenium framework;

[0094] Step S22: After the browser instance is loaded, the system injects a custom JavaScript script, which is used to enhance the page's DOM access capabilities and real-time structure monitoring.

[0095] At the same time, the system loads the initial adapter template for each selected database. This template includes database login information (if required), the base URL, and session persistence logic, but does not include a fixed document list page parsing path.

[0096] Step S3: Navigate to the target database and perform a search.

[0097] Step S31: The browser instance accesses each selected database website in sequence according to the task configuration; for databases that require authentication, the system automatically fills in the pre-stored credentials to complete the login;

[0098] Step S32: The system enters the search keywords submitted by the user in the database search box and simulates clicking the "Search" button;

[0099] Step S33: The system waits for the page to finish loading and confirms that the search results page is ready by detecting the stable state of the main content area of ​​the page (the interval between two consecutive DOMContentLoaded events is greater than 500ms and the network is idle).

[0100] Step S4: Dynamic Perception and Feature Extraction of Page Structure

[0101] The system performs structure-aware analysis on the loaded search results page;

[0102] Step S41: Obtain the DOM tree of the entire page and calculate its topological features; define any element in the page. Structural feature vectors for:

[0103]

[0104]

[0105] In the formula, Indicates the depth of the element in the DOM tree. The number of direct child elements. and These represent the lengths of the class and id attribute values, respectively. Encoding value for label name, This represents the proportion of the visible area of ​​the element within the viewport.

[0106] Step S42: The system then performs a breadth-first traversal of the DOM tree, selects all leaf nodes and intermediate nodes with specific tags (such as div, li, tr), calculates their structural feature vectors, and divides the page elements into several structural clusters based on a density clustering algorithm (such as DBSCAN).

[0107] Each cluster represents a recurring layout pattern on the page; the system selects the cluster with the most elements and the largest visible area. Clusters (usually) () serves as a container area for candidate document entries.

[0108] Step S5: Adaptive parsing strategy selection and document entry location

[0109] For each candidate document entry container area, the system further analyzes the structure and content pattern of its internal sub-elements;

[0110] Step S51: Define the region Content complexity for:

[0111]

[0112] In the formula, Let V be the variance of the lengths of all text nodes within the region. Maximum text length , , These are the link elements, table elements, and the total number of elements. The weighting coefficients are preset based on the characteristics of the agricultural meteorology page (e.g., ...). );

[0113] Step S52: The system according to Value selection parsing strategy:

[0114] like The layout is determined to be a simple list layout, and rule-based position index parsing is used (e.g., extracting one document for every li element).

[0115] like The layout was determined to be of medium complexity (common in entries containing abstracts and keywords). A parsing method based on visual block segmentation and text density was used to further divide the region into title blocks, author blocks, abstract blocks, etc.

[0116] like If the data is identified as having a high complexity layout (such as data-intensive entries with embedded tables or charts), a deep learning-assisted parsing mode is activated. A pre-trained lightweight CNN model (already trained offline on the agricultural meteorology page dataset) is called to perform semantic segmentation of the region and identify the boundaries of the document metadata fields.

[0117] Step S6: Extraction and Structured Storage of Document Information

[0118] Step S61: Based on the selected parsing strategy, the system extracts the following metadata fields from each identified document entry container: title, author, publication year, journal / conference name, abstract, keywords, DOI / link;

[0119] Step S62: The extraction process employs a multi-validation mechanism: For each field, the system attempts to obtain candidate texts from different attributes (such as innerText, data-field, aria-label) and different sub-positions, and then cleans and merges them based on predefined regular expressions and domain dictionaries (such as agricultural meteorological terminology database), selecting the result with the highest confidence.

[0120] Step S63: The extracted metadata is temporarily stored in a memory structure and linked to the document details page.

[0121] Step S7: Batch download full text of documents

[0122] Step S71: The system iterates through the extracted document detail page links and controls the browser to navigate to each detail page in sequence; on each detail page, the system detects the PDF download button or link;

[0123] Step S72: The detection method combines multiple features: the element text contains "download", "PDF", or "FullText", and the href attribute ends with ".pdf", or the onclick event contains download-related keywords;

[0124] Step S73: After finding the download element, the system simulates a click and monitors the browser's download behavior;

[0125] Step S74: The downloaded PDF file is renamed in the format "First Author_Year_First 5 Words of Title.pdf" and saved to the user-specified directory; at the same time, the system records the download status (success / failure) and timestamp.

[0126] Step S8: Task Status Management and Error Recovery

[0127] Step S81: Throughout the process, the task management module monitors the execution status of each subtask in real time;

[0128] Step S82: For any exceptions that occur in any step (such as element location timeout, network disconnection, download failure), the system captures and classifies the exceptions;

[0129] If it is a temporary error (such as network timeout), the system will automatically retry up to 3 times, with a random delay (1~5 seconds) between each retry.

[0130] If there is a structural error (such as a significant change in page layout causing the parsing strategy to fail), the system marks the database as "requiring structural update" and records the current page screenshot and DOM snapshot. The task skips the database and continues to execute the rest of the task.

[0131] Step S83: Every 10 document entries, the system persists the current progress (including successfully extracted metadata, a list of downloaded files, and failure records) to the local SQLite database;

[0132] If the task is interrupted, the system will restore from the persistent point based on the TaskID after restarting, skip the parts that have been successfully processed, and continue to execute the unfinished task.

[0133] Step S9: Data Processing and Output

[0134] Step S91: When all selected databases have been processed or the task reaches the user-defined stop condition (such as reaching the maximum number of downloads), the system enters the data processing stage.

[0135] Step S92: The system merges and deduplicates all successfully extracted document metadata (based on similarity matching of DOI or title, author, and year, with a similarity threshold set to 0.9).

[0136] Step S93: After deduplication, the metadata is exported as a structured CSV file containing all extracted fields;

[0137] At the same time, the system generates a task report, which records the search terms, the databases involved, the total number of documents obtained, the number of successful downloads, the list of failures, and the possible reasons for the failures.

[0138] The CSV file and task report are saved together to the user-specified directory, completing the entire batch acquisition and management process.

[0139] Step S10: Resource cleanup and exit

[0140] Once the task is completed, the system automatically closes the browser instance, cleans up temporary files, releases memory resources, and updates the task status to "Completed" in the graphical user interface, while also prompting the user about the location of the output file. Users can view historical task records and detailed logs at any time in the interface.

[0141] Working principle:

[0142] The core of this method's system architecture is a cyclical processing flow based on dynamic perception and adaptive decision-making. Through the collaboration of the structure analysis module and the strategy selection module, it introduces and correlates information between multi-source page layout priors and real-time parsing strategies. The structure analysis module and the strategy selection module are key functional components of this method, capable of modeling and correlating the topological features of the page layout with the adaptation logic of the parsing strategy, respectively. In the above scheme, by embedding the structure analysis module and the strategy selection module into the flow, the system systematically correlates the structural prior knowledge of the target page. In specific implementation, the system abstracts the DOM tree structure of the page to be parsed into a multi-dimensional feature vector, and simultaneously uses the structure analysis module and the strategy selection module to establish a dynamic mapping relationship between page features and parsing methods. Specifically, we treat the local layout patterns of the page as objects to be classified. The distinction and identification between different layout patterns are completed by the structure analysis module, while the precise parsing rules to be adopted for each layout pattern are dynamically determined by the strategy selection module.

[0143] Different databases exhibit diverse page structures, and the layout of different content blocks within the same database also varies (i.e., data presentation from different sources and in different blocks should employ different parsing strategies). By establishing a dynamic, non-linear relationship between page structure features and parsing strategies, this method utilizes a structure analysis module to correlate the continuity between the overall page topology and local patterns, while a strategy selection module correlates the matching relationship between specific layouts and the optimal parsing method. This allows for the recovery of meaningful semantic structure of document entries during parsing, while avoiding the loss or obfuscation of detailed information due to rigid rules during information extraction, ensuring that the extracted metadata retains as much integrity and accuracy as possible.

[0144] After dynamically parsing multi-source pages using the above methods, the structured information of the documents can be extracted with high fidelity. By using this information for batch downloading and unified management of documents, a complete and standardized multi-source document database can ultimately be obtained.

[0145] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0146] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for batch acquisition and management of multi-source documents based on browser-based automatic control, characterized in that, The method includes the following steps: In response to a user's search request entered in a graphical user interface, the search request includes at least search keywords and one or more target literature database identifiers; Based on the search request, initialize the browser's automated control environment and navigate to the search entry point of the target document database; The search operation is automatically performed in the search portal of the target literature database to obtain a literature list page; Structural analysis and feature extraction are performed on the document list page to dynamically identify the document entry container area on the page; Based on the content complexity assessment of the document entry container area, the corresponding parsing strategy is adaptively selected. Based on the selected parsing strategy, extract the document metadata from the document entry container area and obtain the corresponding document details access link; Based on the document details access link, control the browser to navigate to the document details page and automatically trigger the full-text download operation; During the full-text download and metadata extraction process, task status monitoring and exception handling are performed to achieve task status persistence and breakpoint resumption. The downloaded full-text documents and extracted metadata are linked, organized, and stored in a structured manner, and then output to a specified location.

2. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The steps for initializing the browser automation control environment include: Start a browser instance and inject a script into the browser instance to enhance the document object model access and structure monitoring capabilities; Load the adapter template corresponding to the target document database identifier. The adapter template includes at least the database access address and session maintenance logic.

3. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The steps of performing structural analysis and feature extraction on the document list page, and dynamically identifying the container area of ​​document entries, include: Obtain the document object model tree of the document list page; Calculate the structural feature vector of the element in the document object model tree, wherein the structural feature vector includes element depth, number of child elements, attribute information and visual presentation information; Cluster analysis is performed on page elements based on the structural feature vectors to obtain element clusters that reflect different layout patterns; Candidate clusters that meet preset conditions are selected from the element clusters and used as the document entry container area.

4. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The step of adaptively selecting the corresponding parsing strategy based on the content complexity assessment of the document entry container region includes: The content complexity evaluation value of the document entry container area is calculated. The content complexity evaluation value is calculated based on the text length distribution characteristics, the proportion of link elements, and the proportion of table elements within the area. The content complexity evaluation value is compared with a preset threshold range; Based on the comparison results, one of the predefined parsing strategies is selected. The parsing strategy includes at least a rule-based position index parsing strategy, a parsing strategy based on visual block segmentation and text density, and a parsing strategy based on semantic segmentation model.

5. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The step of extracting document metadata from the document entry container area according to the selected parsing strategy includes: For each metadata field to be extracted, multiple candidate text contents are obtained from the document entry container area; Based on predefined field feature rules and domain knowledge dictionary, the multiple candidate text contents are cleaned, verified and confidence evaluated; The candidate text content with the highest confidence assessment result is selected as the final extraction result of the metadata field.

6. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The steps for monitoring and handling the execution task status, and implementing task status persistence and breakpoint resumption, include: Real-time monitoring of task execution process; capture abnormal events that occur during operation. The abnormal events are classified to distinguish between temporary errors and structural errors; For temporary errors, an automatic retry mechanism is implemented; For structural errors, log the error context information and skip the current failure point to continue executing the task; Periodically persist the task execution progress, successfully processed intermediate results, and error logs to the local database; When a task is restarted after being interrupted, it locates the last interruption point and continues to execute the unfinished part based on the information stored in the persistent memory.

7. The method for batch acquisition and management of multi-source documents based on browser automatic control according to claim 1, characterized in that, The steps of associating and structurally storing the successfully downloaded full-text document files with the extracted document metadata include: The metadata extracted from multiple target literature databases is merged and deduplicated, and the deduplication operation is based on the similarity calculation of literature identifiers or metadata. The deduplicated document metadata is generated into a structured data file; The structured data file is stored together with the corresponding full-text document file, and a report file containing task statistics is generated.

Images