An intelligent understanding and analysis method and a trusted intelligent agent system for large-scale scientific data

Abstract

The application belongs to the technical field of artificial intelligence and scientific data processing, and relates to an intelligent understanding and analysis method for large-scale scientific data and a trusted intelligent agent system. The method comprises the following steps: pre-processing input heterogeneous scientific data, standardizing file formats, and avoiding memory overflow through a streaming conversion mechanism; constructing a dynamic file association knowledge graph, capturing complex semantic dependency relationships among multiple files through a semantic abstraction and similarity merging strategy; planning an optimal file workflow based on user queries and the dynamic file association knowledge graph, generating constraint executable code and executing it in an in-memory database; and generating a trusted analysis result containing reasoning tracks, evidence chains and confidence evaluation through an evidence-to-answer mechanism. The application can realize efficient processing of large-scale heterogeneous data, accurate capture of multi-file association and result traceability, significantly improve the applicability and reliability of scientific research analysis, and support real-time analysis of large-scale multidisciplinary data sets.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and scientific data processing technology, specifically to an intelligent understanding and analysis method and system for large-scale scientific data, particularly an intelligent understanding and analysis method and trusted intelligent agent system for constructing a document-related knowledge graph, efficient code generation and execution, and evidence-to-answer (E2A) tracing mechanism. Background Technology

[0002] With the widespread application of artificial intelligence in scientific research (AI for Science, or AI4S for short), the value of high-quality scientific data in various fields such as life sciences, physics, chemistry, and earth sciences is becoming increasingly prominent. To promote the open sharing and reuse of scientific data, governments and research institutions around the world have successively established national-level scientific data centers and dataset sharing platforms, such as NCBI, OpenAIRE, and ScienceDB. These platforms effectively improve the accessibility and reusability of scientific data by providing unified data storage and access interfaces, thereby promoting the collaborative development of interdisciplinary research.

[0003] According to the latest statistics, the number and size of scientific datasets are growing exponentially, making it increasingly difficult for researchers to quickly locate "task-relevant, experiment-matched, and credible" data resources within these vast datasets. To improve research efficiency, existing data analysis models are typically based on agents and can be broadly categorized into three types: 1) Code-driven analysis tools, where one agent generates executable code, and another agent verifies and executes it; 2) Tool-driven analysis tools, which focus on identifying and invoking domain-specific tools to respond to user queries; and 3) Automated Machine Learning (AutoML) pipelines, which typically employ a multi-agent architecture, with each agent responsible for a specific stage of the AutoML process, such as model selection, training, or evaluation.

[0004] While existing methods have achieved good results in industrial recommendation systems, understanding and querying large-scale scientific data still faces unique challenges: First, scientific experimental data typically contains multiple closely related heterogeneous files with complex citation relationships and semantic dependencies. However, existing data analysis tools often focus on numerical calculations or tool calls for single files, failing to effectively capture and parse the relationships between multiple files, leading to discrepancies between the analysis results and actual research needs. Second, the sheer volume of scientific data and the frequent I / O operations and file sequence dependencies during processing can cause severe online service delays, and existing tools lack targeted optimization mechanisms. Third, large language models (LLMs) and agents commonly suffer from the "illusion" problem, generating analysis results that lack traceability and supporting evidence. Scientific research demands extremely high credibility and verifiability of results, making such unfounded analyses unacceptable in research settings. Furthermore, existing data sharing platforms mostly only support dataset downloads, and even those that support simple queries cannot perform in-depth analysis of complex relationships between multiple files, and their limited disciplinary coverage makes it difficult to meet the needs of large-scale, interdisciplinary scientific data processing.

[0005] Therefore, existing technologies still lack an intelligent scientific data analysis system that can efficiently process heterogeneous data, accurately capture file relationships, and provide reliable analysis results. Summary of the Invention

[0006] To address the shortcomings of existing scientific data analysis tools in areas such as document association understanding, big data processing efficiency, and result reliability, this invention proposes an intelligent understanding and analysis method and a trusted intelligent agent system for large-scale scientific data, called ScienceDB Agent.

[0007] The technical solution adopted in this invention is as follows: A method for intelligent understanding and analysis of large-scale scientific data includes the following steps: The heterogeneous scientific data input is preprocessed, the file format is standardized, and memory overflow is avoided through a streaming conversion mechanism; A dynamic file association knowledge graph is constructed based on preprocessed heterogeneous scientific data. Through semantic abstraction and similarity merging strategies, the complex semantic dependencies between multiple files are captured. Based on the knowledge graph of user queries and dynamic file associations, the optimal file workflow is planned, executable code with strict column projection constraints is generated and executed in an in-memory database; By utilizing the code execution results in the in-memory database, a credible analysis result containing interpretable reasoning trajectories, multi-dimensional evidence chains, and confidence assessments is generated through the evidence-to-answer (E2A) mechanism.

[0008] Furthermore, the preprocessing of the input heterogeneous scientific data, the standardization of file formats, and the avoidance of memory overflow through a streaming conversion mechanism include: Parse structured files such as HDF5, CSV, TSV, SAV, Parquet, ODS, XLS, XLSX, and TAB, and convert them into standardized two-dimensional Parquet tables, retaining column names, data types, and core statistical information; DataLab tools are used to process unstructured files such as PDF, PPTX, DOCX, JSON, and TXT. It integrates OCR-based element detection, layout segmentation, and lightweight recognition models to convert them into Markdown documents that retain structural components such as titles, tables, and formulas, ensuring that large language models can directly perform semantic understanding. By deeply integrating the Python generator mechanism with the Pandas I / O module, data is processed block by block. After each block of data is read and transformed, memory resources are released immediately, ensuring that memory usage is O(1) relative to file size, thus avoiding memory overflow when processing large files of multiple GB.

[0009] Furthermore, the construction of a dynamic file association knowledge graph based on preprocessed heterogeneous scientific data captures complex semantic dependencies among multiple files through semantic abstraction and similarity merging strategies, including: Extract the metadata of each file, filter out invalid and information-less files, create preliminary nodes and remove duplicate nodes by similarity comparison to form a pre-filtered node set; For document-type nodes, segment them into paragraphs and extract conceptual and instance entities, then create paragraph nodes and associated edges; For table-type nodes, extract the schema information, create collection nodes and attribute nodes, and construct the association edges between table and column, and column and entity; Node embeddings are generated using a sentence embedding model, and semantic similarity is calculated using dot products. in, , This represents the sentence embedding vectors of nodes t and k; for those with similarity less than an adjustable threshold... Duplicate nodes are merged, and subgraphs are extracted (i.e., subgraphs are constructed) when necessary to optimize the context window length. "When necessary" means that when the merged nodes contain too much semantic information and the context length exceeds the model's processing capacity, subgraphs need to be extracted to simplify the structure and keep the window length within a reasonable range.

[0010] Furthermore, the step of planning the optimal file workflow based on user queries and a dynamic file association knowledge graph, generating executable code with strict column projection constraints, and executing it in an in-memory database includes: File digest checks are performed by calculating the relevance score between the user query and the pre-stored file digest. This indicates the relevance score. This represents the relevance score threshold. Then, a response is generated directly based on the summary; Extract the Top-K related nodes from the knowledge graph, construct a subgraph using the Top-K related nodes, and plan the optimal file execution order. Ensure that file dependencies are met and the path is the shortest possible. Identify the relevant columns for each file in the workflow and generate SQL-like queries with strict column projection constraints, then combine them to form executable code; The table file is converted into an in-memory DuckDB table and loaded, and the code executor is invoked to run the code in the in-memory database.

[0011] Furthermore, the evidence-to-answer (E2A) mechanism generates a credible analysis result that includes an interpretable reasoning trajectory, a multi-dimensional chain of evidence, and a confidence assessment, including: Record the complete reasoning trajectory of the analysis process (from query parsing to result calculation), including the timestamp, associated file ID, and key parameters of each decision step; Extract specific file names, column names, and cell values ​​from the table data, and reference relevant paragraph content from the text data to construct a chain of evidence; By combining the semantic matching degree between evidence and query, the reliability of data sources, and other dimensions, a reliability score is configured to support evidence scoring.

[0012] A trustworthy intelligent agent system for understanding and analyzing large-scale scientific data, its main modules include: The scientific source data preprocessing module is used to standardize heterogeneous file formats and solve memory overflow problems, outputting processed data in a unified format; The source file structured understanding module is used to construct a dynamic file association knowledge graph. Through semantic abstraction and similarity merging strategies, it captures the complex semantic dependencies between multiple files and realizes a structured representation of the semantic dependencies between multiple files. The document workflow planning and execution module is used to dynamically plan the optimal document workflow, generate executable code with strict column projection constraints, and execute it in an in-memory database. The Evidence-to-Answer (E2A) mechanism module is used to record reasoning patterns, build chains of evidence, assess confidence levels, and output credible analysis results.

[0013] Through the above technical solutions, this invention achieves efficient processing of large-scale scientific data, accurate capture of multi-file associations, and reliable traceability of analysis results, significantly improving the efficiency of scientific data analysis, online service response speed, and result reliability. This invention supports intelligent processing of various types of scientific data throughout the entire process, meets the rigorous requirements of scientific research, can be integrated into various scientific research data platforms, is adaptable to large-scale data processing scenarios, and has good practicality and scalability. Attached Figure Description

[0014] Figure 1 This is a flowchart of the steps of the intelligent understanding and analysis method for large-scale scientific data of the present invention.

[0015] Figure 2 This is a block diagram of the module composition of the trusted intelligent agent system for understanding and analyzing large-scale scientific data according to the present invention.

[0016] Figure 3 This is a technical architecture diagram of the trusted intelligent agent system for understanding and analyzing large-scale scientific data according to the present invention.

[0017] Figure 4 This is a comparison of the evaluation metrics of the present invention and existing models in three tasks. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0019] The present invention provides an intelligent understanding and analysis method for large-scale scientific data, such as... Figure 1 As shown, it includes the following steps: The heterogeneous scientific data input is preprocessed, the file format is standardized, and memory overflow is avoided through a streaming conversion mechanism; A dynamic file association knowledge graph is constructed based on preprocessed heterogeneous scientific data. Through semantic abstraction and similarity merging strategies, the complex semantic dependencies between multiple files are captured. The optimal file workflow is planned based on the knowledge graph of user queries and dynamic file associations, and constrained executable code is generated and executed in the in-memory database. By utilizing the code execution results in the in-memory database, a credible analysis result containing reasoning trajectory, evidence chain, and confidence assessment is generated through an evidence-to-answer mechanism.

[0020] In one embodiment, the preprocessing of the input heterogeneous scientific data includes: Parse structured files and convert them into standardized Parquet tables, preserving column names, data types, and core statistical information; The DataLab tool is used to process unstructured files, integrating OCR-based element detection and layout segmentation to convert them into Markdown documents that retain structural components. By combining Python generator mechanism with Pandas I / O module, data is processed block by block and memory is released in real time, ensuring that memory usage is O(1) relative to file size.

[0021] In one embodiment, constructing a dynamic file association knowledge graph includes: Extract file metadata and filter invalid files; remove redundant nodes by checksum comparison to form a pre-filtered node set. Segment the text of document-type nodes and extract conceptual and instance entities, then create paragraph nodes and associated edges; Extract information from table-type nodes, create collection nodes and attribute nodes, and establish relationships between them; A sentence embedding model is used to generate node embedding representations. Semantic similarity is calculated by dot product, and a merging operation is performed on duplicate nodes with similarity less than an adjustable threshold.

[0022] In one embodiment, the optimal file workflow based on user query and knowledge graph dynamic planning includes: Pre-calculate document summary information and quickly respond to simple queries via query-summary relevance score; For complex queries, extract the Top-K relevant nodes from the knowledge graph and construct a subgraph; then, plan the shortest path execution order based on file dependencies. Identify relevant columns and generate a set of SQL-like code with strict column projection constraints. Use the DuckDB in-memory database to load the tables and execute the code.

[0023] In one embodiment, generating credible analysis results through the evidence-to-answer mechanism includes: Records the complete reasoning process from query parsing to result calculation, including timestamps, file IDs, and key parameters; Construct a multi-dimensional chain of evidence, where the table data includes file names, column names, and cell values, and the text data references relevant paragraph content; A weighted calculation method is used to generate confidence scores for the evidence.

[0024] The trusted intelligent agent system of the present invention for understanding and analyzing large-scale scientific data, such as Figure 2As shown, the system comprises four core modules: (1) Scientific source data preprocessing module; (2) Source document structured understanding module; (3) Document workflow planning and execution module; and (4) Evidence-to-Answer (E2A) mechanism module. The collaborative relationship between the modules is as follows: The scientific source data preprocessing module first receives heterogeneous scientific data (structured / unstructured documents), and outputs unified format data through format standardization and streaming conversion; The source document structured understanding module, based on the processed data, constructs a dynamic document association knowledge graph through node pre-filtering, semantic abstraction, and graph optimization; The document workflow planning and execution module takes user queries and knowledge graphs as input, first quickly responds to simple queries through summary checks, then plans the workflow, generates constraint code, and executes it in the memory DuckDB; The Evidence-to-Answer (E2A) mechanism module records the reasoning trajectory of the entire analysis process, constructs the evidence chain, evaluates the confidence level, and finally outputs a credible result. Through the cooperation of the above modules, the system realizes a complete closed loop from data preprocessing to credible analysis result output, and can support multi-round conversational queries and deep data analysis. The technical architecture of this system is as follows. Figure 3 As shown.

[0025] The specific implementation of the above four modules of the present invention will be described in detail below.

[0026] 1. Problem Definition make Represents a large-scale scientific dataset collection. Let M represent a scientific dataset, and M represent its accompanying metadata set, which includes dataset-level attributes such as title, description, and labels; let... The output knowledge graph is represented by N, where N represents the set of nodes and E represents the set of edges. Generally, new nodes are discovered through file interaction, then relationships are added and changes are merged.

[0027] 2. Scientific Source Data Preprocessor Module This module addresses the issues of low processing efficiency and memory overflow caused by heterogeneous scientific data formats and large data volumes. It incorporates the latest optimization strategies and includes the following specific implementations: Format standardization sub-steps: For structured files, Pandas combined with the PyArrow engine is used for parsing to ensure the efficiency of Parquet format conversion and data integrity; for unstructured PDF files, the OCR module of Data Lab tool is used to accurately identify table content and formula symbols, and the layout segmentation algorithm is used to automatically distinguish areas such as titles, body text, and references. The converted Markdown document retains the logical structure of the original file and can be directly used by large models such as Qwen3 for semantic understanding. Streaming conversion upgrade sub-steps: Set the dynamic data block size (i.e., the block threshold) to 1GB, and automatically adjust the file splitting according to the file type; monitor memory usage in real time through a memory monitoring mechanism.

[0028] 3. Source File Structured Understanding Module The source file structured understanding module includes: 1) File node pre-filtering steps: Extract the metadata of each file, such as filename, type, size, content summary, etc.; filter files with damaged format or no valid information; form a pre-filtered node set N, and remove redundant nodes by similarity comparison.

[0029] 2) File semantic abstraction steps: a) For document-type nodes, divide the text content into paragraphs, extract conceptual entities and instance entities from the paragraphs, and let... Representing paragraph nodes, establish association edges with entity nodes. ,in It represents the conceptual or instance entity nodes extracted from the paragraph, and at the same time constructs the association relationship between document nodes and all related semantic elements; b) For table-type nodes, extract schema information, including column names, data types, and statistics such as mean, variance, and invalid value rate. Create set nodes. Create attribute nodes for the table constructed from the above information. Constructing edges Further, for attribute nodes Extract the concepts and instances they represent, and establish the corresponding nodes and edges.

[0030] Furthermore, a node merging strategy is adopted, and a sentence embedding model is used to calculate the semantic similarity of node descriptions. When the similarity is below a threshold, nodes are merged.

[0031] 4. Document Workflow Planning and Execution Module To address the issue that dependencies between large files can significantly extend online service time, a file workflow planning and execution module was implemented. The specific implementation steps of this module are as follows: 1) Document Summary Check. This step first checks the preprocessed document summaries, which contain statistical information such as minimum / maximum / average / standard deviation / counts from tabular files, as well as summaries from text files. These pre-calculated summaries can directly answer the user's basic queries.

[0032] Specifically, this step performs file digest checks by calculating the relevance score between the user query and the pre-stored file digest, allowing... This indicates the relevance score. This represents the relevance score threshold. Then, a response is generated directly based on the summary. Dot product similarity can be used to calculate the relevance score between the user query and the pre-stored file summary.

[0033] 2) Source Document Workflow Design. When a user submits a query, the intelligent agent extracts a subgraph centered around K key nodes from the dynamic document association knowledge graph. This process dynamically identifies the most relevant documents based on the researcher's needs and plans an efficient analysis workflow.

[0034] Specifically, this step extracts the Top-K related nodes from the knowledge graph, uses these nodes to construct a subgraph, and plans the optimal file execution order. Ensure that file dependencies are met and the path is the shortest.

[0035] 3) Based on the planned file workflow and related columns, the intelligent code generation agent will generate executable code, ensuring accuracy and efficiency by strictly limiting column projection in the selection operation.

[0036] Specifically, this step identifies the relevant columns for each file in the workflow and generates SQL-like queries with strict column projection constraints, which are then combined to form executable code.

[0037] 4) The generated code is executed within the in-memory DuckDB database using a widely adopted code executor. To improve online performance, all table files are converted to in-memory DuckDB tables before execution.

[0038] 5. Answer-to-Ask (E2A) Mechanism Module To address the high demands for the credibility of scientific research results, the Answer-to-Analysis (E2A) mechanism module has upgraded its interpretability and verifiability, as detailed below: A time-series recording method is adopted to record the complete path in detail from "user query → extraction of relevant nodes (Top-3) → workflow planning → SQL code generation → code execution → calculation results". Each step is marked with a timestamp, associated file ID and key parameters to ensure that the trajectory is traceable and reproducible. Extract specific file names, column names, and cell values ​​from the table data, and reference relevant paragraph content from the text data to construct a chain of evidence. Specifically, this step constructs a multi-dimensional chain of evidence, extracting three-dimensional evidence: document evidence, column evidence, and numerical evidence. The decision-making steps and related evidence are organized in a logical order to form an interpretable reasoning chain. In one embodiment of the present invention, a user submits a data analysis request through a natural language input box on the platform interface, such as: "After 48 hours of drug treatment in the GIST-5R cell line, how does the average percentage of early apoptotic cells compare to the average percentage of late apoptotic cells in all drug treatment groups?" This input, as the original query statement, is collected by the system front-end and transmitted to the back-end processing module. The system's built-in large language model (such as Qwen3 described in the patent) analyzes the user's natural language query, extracts key information such as "GIST-5R cell line," "48-hour drug treatment," "percentage of early apoptotic cells," "percentage of late apoptotic cells," and "comparison," and combines it with a preset research intent element template: according to "research topic, research task, experimental setup, and data modality," semantic enhancement and standardization are performed to generate a structured query intent. Based on the parsed structured intent, the system calls the dynamic file association knowledge graph constructed by the source file structured understanding module to recall relevant experimental file candidates, such as "GIST-5R Apoptosis Statistics 48H.xlsx" and "Drug Preparation Instructions.md." The system employs a file dependency filtering mechanism (prioritizing the association of apoptotic statistics tables with drug grouping documents) and data type matching (table data includes apoptosis rate indicators for the target cell lines) to obtain a core analysis file list. Subsequently, the file workflow planning and execution module plans the optimal processing flow based on the filtered file list, identifies relevant columns in the tables such as "early apoptotic cells%", "late apoptotic cells%", and "drug name", generates executable code with strict column projection constraints, and runs the code in the in-memory DuckDB database to calculate the mean for each group. Finally, through an evidence-to-answer (E2A) mechanism, the system integrates the calculation results, reasoning trajectory (including file association paths and code execution steps), and evidence chain (specific file names, column names, and cell mean data) to output a reliable analysis result. This clearly informs the user that the average percentage of early apoptotic cells (32.63%) is higher than the average percentage of late apoptotic cells (7.48%), and includes a confidence assessment of the result.

[0039] like Figure 4As shown, "Ours" is the ScienceDB Agent proposed in this invention, which outperforms the comparison models in all evaluation metrics (Accuracy Acc, Illusion Rate HR, Reference Accuracy RAR, and Success Rate SR), verifying the effectiveness of the system of this invention. Compared with the best-performing comparison models in different tasks, the ScienceDB Agent shows significant improvement in metrics. Although some comparison models (such as LAMBDA) include additional modules such as multi-agent collaboration, they are still not as efficient as the compact architecture designed in this invention. Taking the Illusion Rate HR metric as an example, the ScienceDB Agent has the lowest HR among all models in all three tasks, with an HR of only 12.60 in Task 3, far lower than Qwen3 (HR of 47.19), which performed better in the comparison models, effectively suppressing the illusion problem of large models. In terms of the Acc metric for Task 3, ScienceDB Agent's Acc improved to 64.29 compared to the best-performing comparative model LAMBDA (Acc 63.77), while maintaining a lower HR, achieving a balance between the accuracy and reliability of the analysis results, and is more suitable for the scientific research analysis needs of scientific data.

[0040] Evaluation metrics description: 1) Accuracy (Acc): Used to measure the correctness and precision of the analysis results. The higher the Acc, the better the match between the system's output analysis results and the actual situation, and the more accurate the results.

[0041] 2) Hallucination Rate (HR): This measures the proportion of false or unsubstantiated content in the analysis results. The lower the HR, the more credible the analysis results generated by the system are, and the better it avoids the "hallucination" problem.

[0042] 3) Reference Accuracy Rate (RAR): This assesses the logic and comprehensiveness of the system's use of provided information and supporting evidence during reasoning and deduction. A higher RAR indicates more reasonable use of evidence during the analysis and stronger traceability and interpretability of the reasoning trajectory.

[0043] 4) Success Rate (SR): This measures the percentage of user analysis tasks that the system successfully completes. A higher SR indicates greater applicability and reliability of the system in various scientific research data analysis scenarios.

[0044] The method and system described above can be deployed on a Web platform, supporting users to submit complex queries in multiple rounds online and automatically recommending traceable dataset resources.

[0045] It should be understood that the methods and systems disclosed in the above embodiments of this invention can be implemented in other ways. For example, the above module division can be implemented in other ways, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. The various modules in this invention can be implemented as software functional units and can be stored in a computer-readable storage medium, including several instructions to cause a computer device to execute some or all of the steps of the method described in this invention. For example: An embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0046] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk, etc.) that stores a computer program, which, when executed by a computer, implements the various steps of the method of the present invention.

[0047] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0048] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A method for intelligent understanding and analysis of large-scale scientific data, characterized in that, Includes the following steps: The heterogeneous scientific data input is preprocessed, the file format is standardized, and memory overflow is avoided through a streaming conversion mechanism; A dynamic file association knowledge graph is constructed based on preprocessed heterogeneous scientific data. Through semantic abstraction and similarity merging strategies, the complex semantic dependencies between multiple files are captured. The optimal file workflow is planned based on the knowledge graph of user queries and dynamic file associations, and constrained executable code is generated and executed in the in-memory database. By utilizing the code execution results in the in-memory database, a credible analysis result containing reasoning trajectory, evidence chain, and confidence assessment is generated through an evidence-to-answer mechanism.

2. The method according to claim 1, characterized in that, The preprocessing of the input heterogeneous scientific data includes: Parse structured files and convert them into standardized Parquet tables, preserving column names, data types, and core statistical information; The DataLab tool is used to process unstructured files, integrating OCR-based element detection and layout segmentation to convert them into Markdown documents that retain structural components. By combining Python generator mechanism with Pandas I / O module, data is processed block by block and memory is released in real time, ensuring that memory usage is O(1) relative to file size.

3. The method according to claim 1, characterized in that, The construction of the dynamic file association knowledge graph includes: Extract file metadata and filter invalid files; remove redundant nodes by checksum comparison to form a pre-filtered node set. Segment the text of document-type nodes and extract conceptual and instance entities, then create paragraph nodes and associated edges; Extract information from table-type nodes, create collection nodes and attribute nodes, and establish relationships between them; A sentence embedding model is used to generate node embedding representations. Semantic similarity is calculated by dot product, and a merging operation is performed on duplicate nodes with similarity less than an adjustable threshold.

4. The method according to claim 3, characterized in that, The sentence embedding model adopts the all-MiniLM-L6-v2 model, the adjustable threshold is set to 0.6, and the association relationship includes document-paragraph-entity association and table-column-entity association.

5. The method according to claim 1, characterized in that, The optimal file workflow based on user query and knowledge graph dynamic programming includes: Pre-calculate document summary information and quickly respond to simple queries via query-summary relevance score; For complex queries, extract the Top-K relevant nodes from the knowledge graph and construct a subgraph; then, plan the shortest path execution order based on file dependencies. Identify relevant columns and generate a set of SQL-like code with strict column projection constraints. Use the DuckDB in-memory database to load the tables and execute the code.

6. The method according to claim 5, characterized in that, The file summary information includes the min / max / mean / std / count statistical data of the tabular file and the core content summary of the text file. The SQL-like code set strictly constrains the column projection range in the SELECT operation.

7. The method according to claim 1, characterized in that, The generation of credible analysis results through the evidence-to-answer mechanism includes: Records the complete reasoning process from query parsing to result calculation, including timestamps, file IDs, and key parameters; Construct a multi-dimensional chain of evidence, where the table data includes file names, column names, and cell values, and the text data references relevant paragraph content; A weighted calculation method is used to generate confidence scores for the evidence.

8. A trustworthy intelligent agent system for understanding and analyzing large-scale scientific data, characterized in that, include: The scientific source data preprocessing module is used to preprocess the input heterogeneous scientific data, standardize the file format, and avoid memory overflow through a streaming conversion mechanism. The source file structured understanding module is used to construct a dynamic file association knowledge graph based on preprocessed heterogeneous scientific data. Through semantic abstraction and similarity merging strategies, it structurally captures the complex semantic dependencies between multiple files. The file workflow planning and execution module is used to dynamically plan the optimal file workflow based on user queries and knowledge graphs, generate constrained executable code, and execute it efficiently in an in-memory database. The Evidence-to-Answer Mechanism module is used to generate credible analysis results that include reasoning trajectories, evidence chains, and confidence assessments through the evidence-to-answer mechanism.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images