File retrieval and management method fusing AI large model and graph data

Abstract

The invention discloses a file retrieval and management method fusing an AI large model and graph data, and belongs to the technical field of retrieval management, the retrieval and management method comprises the following specific steps: I, a user imports a multi-format file into a file retrieval and management system, and the system extracts characters and structured content through a file analysis module; iI, the file retrieval and management system gradually constructs a multi-level database based on the analyzed file content and metadata; iII, performing full-process management on the file from creation to archiving, and providing an intelligent tool to optimize a document processing process; according to the method, the automation and efficiency of file management can be greatly improved, a high-precision semantic search and sorting result is provided, complex association analysis and multi-dimensional query are supported, professional document translation requirements in an internationalized scene are met, document content management is optimized, and the knowledge precipitation capacity is improved.

Description

Technical Field

[0001] The present invention relates to the field of retrieval management technology, and in particular to a file retrieval and management method integrating an AI big model and graph data. Background Art

[0002] With the rapid advancement of the rail transit industry towards digitalization, data management in the rail transit field is facing an urgent need for comprehensive transformation. These documents cover multiple links from design drawings to manufacturing specifications to operation and maintenance procedures, and are the core basis for ensuring the safety, reliability and compliance of locomotives. However, the existing standard data management system has significant shortcomings when facing a wide variety of files in various formats (such as DOCX, PDF, image-based PDF, etc.), including: Difficulty in querying: The types of files are complex, the number is large, and there is a lack of efficient search and retrieval mechanisms. Low input efficiency: A large amount of reliance on manual input and maintenance is prone to inefficiency and errors. Foreign standards are difficult to translate: Many locomotive-related standards are derived from international documents, and the accuracy of translation and semantic understanding has a direct impact on the implementation of standards. With the acceleration of technological updates, the types and numbers of data files continue to grow, and traditional manual management methods can no longer meet the needs of efficient and accurate standard management. These files often contain complex technical parameters and operating procedures. Manual processing is not only time-consuming and labor-intensive, but also easy to miss key details. At the same time, the internationalization trend has brought more foreign standard documents, which has put forward higher requirements for translation and semantic understanding. How to use advanced digital technology to improve standard management efficiency, reduce manual intervention, and realize intelligent file query, input and translation has become a core issue that needs to be solved in the rail transit industry. Therefore, it is particularly important to invent a file retrieval and management method that integrates AI big models and graph data. Summary of the invention

[0003] The purpose of the present invention is to solve the defects in the prior art and propose a file retrieval and management method that integrates AI big model and graph data.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A file retrieval and management method integrating AI big model and graph data, the specific steps of the retrieval and management method are as follows:

[0006] Ⅰ. Users import multi-format files into the file retrieval and management system, and the system extracts text and structured content through the file parsing module;

[0007] II. Document retrieval and management system gradually builds a multi-level database based on the parsed document content and metadata;

[0008] III. Manage the entire process of documents from creation to archiving, and provide intelligent tools to optimize the document processing process;

[0009] IV. Users input search requirements through natural language, and the file retrieval and management system matches corresponding files based on semantic embedding and graph database technology.

[0010] As a further solution of the present invention, the file retrieval and management system in step I specifically includes a database management module, a document full-process management module, a system integration module and a rights management module;

[0011] The database management module is used for data storage and management, and to build an efficient file processing and knowledge storage system;

[0012] The document full-process management module is used to comprehensively cover the life cycle management of files from creation to archiving;

[0013] The system integration module is used to achieve deep integration inside and outside the system to ensure data flow and functional coordination;

[0014] The authority management module is used to ensure the security and controllability of the system, and to guarantee the standardization of user operations and the security of data through flexible authority control and monitoring functions.

[0015] As a further solution of the present invention, the database management module supports the import and unified storage of multi-format historical data, combines the intelligent recognition and analysis of complex contents such as tables, formulas, and pictures, realizes the fragmentation processing and content extraction of files, and builds a graph database based on the reference relationship, chapter relationship, semantic relationship, etc. in the file to support complex association queries, and supports the extraction and management of terms in standard files, and supports the export function to serve the knowledge base construction or standardization application;

[0016] The document full-process management module supports the creation, editing and collaborative preparation of documents, and provides efficient approval and change process management, integrating feedback and evaluation functions to optimize the quality of document content; supports the release and cancellation of documents, ensures the dynamic update of the knowledge base, and archives completed documents, providing convenient multi-dimensional query and retrieval capabilities;

[0017] The system integration module supports seamless connection with OA systems, vector databases, graph databases, and search engines, realizes the integrated integration of data and functions, integrates large language models to enhance semantic analysis, intelligent retrieval, and document generation capabilities, provides AI-driven core functions for the system, and automatically pushes documents to related systems in combination with the company's existing processes. It also embeds approval, retrieval, and other operations into business flows to improve collaboration efficiency.

[0018] The permission management module supports multi-level user permission settings, strictly controls document access, editing, approval and sharing permissions, records user operation logs, supports process status monitoring, facilitates auditing and tracing, and uses encryption technology and secure transmission protocols to ensure the security of document storage and transmission.

[0019] As a further solution of the present invention, the specific steps of extracting text and structured content by the parsing module in step I are as follows:

[0020] S1.1: Through the visual analysis algorithm, the content in the image-type PDF page is recognized and segmented into regions, and the content area is divided into various types of regions such as text area, title area, image area, and table area;

[0021] S1.2: Use OCR technology to classify and analyze different types of regions, use a multimodal large model to understand the image, generate corresponding labels, and convert non-text content into text;

[0022] S1.3: Normalize and preprocess the extracted text to remove redundant characters and spaces, then decompose the text into words, restore the words to their root form, and embed the text using the pre-trained Embedding model;

[0023] S1.4: Aggregate word-level embeddings into a sentence- or paragraph-level embedding through average pooling or attention mechanism to generate a global semantic vector for the text, and normalize the generated semantic vector.

[0024] As a further solution of the present invention, the specific steps of constructing a multi-level database in step II are as follows:

[0025] S2.1: Extract the basic attribute information of the stored file, divide the parsed semantic content into blocks, and store them in association with the file ID to build a proprietary structured database;

[0026] S2.2: Collect the semantic vectors of each file content generated by the Embedding model, and then store the generated semantic vectors in the vector database, while associating the file ID and text block information to form a basic database for retrieval query;

[0027] S2.3: Extract references, reference relationships or semantic associations between files or text paragraphs from the parsed text, then store the extracted nodes and relationships in the form of a graph to build a multidimensional relationship network to form a graph database that supports complex association queries.

[0028] As a further solution of the present invention, the specific steps of the whole process management in step III are as follows:

[0029] S3.1: The document retrieval and management system recommends document templates that meet users' needs based on historical documents and industry standards, and detects formatting and logical errors in document content through rule engines or models;

[0030] S3.2: During the document review process, experts’ opinions and modification suggestions are archived in the form of structured data, and all review opinions are integrated to generate the final revision to provide guidance to users;

[0031] S3.3: After the document is approved, the final version will be updated to the knowledge base synchronously, and the version record will be maintained. The consistency of the versions referenced by each group of corresponding documents will be checked through dependencies, and the document modification and approval process will be traced through metadata records.

[0032] As a further solution of the present invention, the specific steps of matching corresponding files based on semantic embedding and graph database technology in the file retrieval and management system in step IV are as follows:

[0033] S4.1: Preprocess the natural language requirements input by the user, including removing stop words, word segmentation and syntactic analysis, and generate processed text. Use the Embedding model to convert the parsed requirement text into a semantic vector;

[0034] S4.2: Based on the semantic vectors in the vector database, the similarity between the query vector and the stored vector is calculated, and multiple documents with the highest similarity are selected from the vector database. Then, the graph database is used to obtain multiple groups of related files through the relationship network between nodes to expand the related documents of the candidate document.

[0035] S4.3: Based on the semantic similarity and relationship weight, recalculate the relevance score between the expanded document set and the query, sort the document set according to the comprehensive relevance score, return the sorted document list to the user, and display the reference relationship and chapter relationship between files through a graphical interface.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] This file retrieval and management method that integrates AI big models and graph data performs layout segmentation on complex format files, and then uses OCR technology to identify text content. The extracted content can be automatically translated according to needs, and vectorized through the Embedding model to generate semantic embedding representation. Based on the parsed file content and metadata, a multi-level structured database, vector database and graph database are gradually built. Based on historical documents and industry standards, document templates that meet the needs are recommended to users. During the document approval process, the opinions and modification suggestions of experts are archived in the form of structured data, and all approval opinions are integrated to generate the final revised version to provide guidance to users. After the document is approved, the final version is synchronously updated to the knowledge base, and the version record is maintained. Users input retrieval requirements through natural language, and the system matches related files based on semantic embedding and graph database technology, which can greatly improve the automation and efficiency of file management, provide highly accurate semantic search and sorting results, support complex association analysis and multi-dimensional query, meet the professional document translation needs in international scenarios, optimize document content management and improve knowledge sedimentation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0039] Figure 1 A flowchart of a file retrieval and management method integrating AI big model and graph data proposed by the present invention;

[0040] Figure 2 This is a block diagram of the architecture of a file retrieval and management method that integrates AI big models and graph data proposed in the present invention;

[0041] Figure 3 This is a system block diagram of a file retrieval and management method that integrates AI big models and graph data proposed in the present invention. DETAILED DESCRIPTION

[0042] Reference Figure 1-Figure 3 , a file retrieval and management method integrating AI big model and graph data, the specific steps of the retrieval and management method are as follows:

[0043] Users import multi-format files into the file retrieval and management system, and the system extracts text and structured content through the file parsing module.

[0044] Specifically, the document retrieval and management system includes a database management module, a document full-process management module, a system integration module, and a rights management module;

[0045] The database management module is used for data storage and management, building an efficient file processing and knowledge storage system, supporting the import and unified storage of multi-format historical data, combining intelligent recognition and analysis of complex contents such as tables, formulas, and pictures, realizing file fragmentation processing and content extraction, and building a graph database based on the reference relationship, chapter relationship, semantic relationship, etc. within the file to support complex association queries, and supporting the extraction and management of terms in standard files, and supporting export functions to serve knowledge base construction or standardization applications;

[0046] The document full-process management module is used to comprehensively cover the life cycle management of documents from creation to archiving, supports the creation, editing and collaborative preparation of documents, and provides efficient approval and change process management, integrates feedback and evaluation functions, and optimizes the quality of document content; supports the release and cancellation of documents, ensures the dynamic update of the knowledge base, and archives completed documents, providing convenient multi-dimensional query and retrieval capabilities;

[0047] The system integration module is used to achieve deep integration inside and outside the system, ensure data circulation and functional coordination, support seamless connection with OA systems, vector databases, graph databases, and search engines, realize integrated integration of data and functions, integrate large language models to enhance semantic analysis, intelligent retrieval, and document generation capabilities, provide AI-driven core functions for the system, combine the company's existing processes, automatically push documents to related systems, and embed approval, retrieval and other operations into business flows to improve collaboration efficiency;

[0048] The permission management module is used to ensure the security and controllability of the system. Through flexible permission control and monitoring functions, it guarantees the standardization of user operations and the security of data. It supports multi-level user permission settings, strictly controls document access, editing, approval and sharing permissions, records user operation logs, supports process status monitoring, facilitates auditing and tracing, and uses encryption technology and secure transmission protocols to ensure the security of document storage and transmission.

[0049] Specifically, through the visual analysis algorithm, the content in the picture-type PDF page is identified and segmented, and the content area is divided into various types of areas such as text area, title area, image area, and table area. The different types of areas are classified and analyzed through OCR technology, and the image is understood through a multimodal large model to generate corresponding labels, and the non-text content is textualized. The extracted text is normalized and preprocessed to remove redundant characters and spaces. The text is then decomposed into words and restored to its root form. The pre-trained Embedding model is used to embed the text, and the word-level embedding is aggregated into a sentence or paragraph-level embedding through average pooling or attention mechanism to generate a global semantic vector for the text, and the generated semantic vector is normalized.

[0050] The document retrieval and management system gradually builds a multi-level database based on the parsed document content and metadata.

[0051] Specifically, the basic attribute information of the stored files is extracted, and the parsed semantic content is divided into blocks and stored in association with the file ID to build a proprietary structured database. The semantic vectors of the content of each file generated by the Embedding model are collected, and the generated semantic vectors are then stored in the vector database. The file ID and text block information are also associated to form a basic database for retrieval queries. References, reference relationships or semantic associations between files or text paragraphs are extracted from the parsed text, and the extracted nodes and relationships are then stored in graph form to build a multidimensional relationship network to form a graph database that supports complex association queries.

[0052] Manage the entire process of documents from creation to archiving, and provide intelligent tools to optimize the document processing process.

[0053] Specifically, the document retrieval and management system recommends document templates that meet users' needs based on historical documents and industry standards, and detects formatting and logical errors in document content through rule engines or models. During the document approval process, expert opinions and modification suggestions are archived in the form of structured data, and all approval opinions are integrated to generate a final revised version to provide guidance to users. After the document is approved, the final version is synchronously updated to the knowledge base, and version records are maintained. The consistency of the versions referenced by the corresponding groups of documents is checked through dependencies, and the document modification and approval process is traced through metadata records.

[0054] Users input search requirements through natural language, and the file retrieval and management system matches corresponding files based on semantic embedding and graph database technology.

[0055] Specifically, the natural language requirements input by the user are preprocessed, including removing stop words, word segmentation and syntactic analysis, and generating processed text. The Embedding model is used to convert the parsed demand text into a semantic vector. According to the semantic vector in the vector database, the similarity between the query vector and the storage vector is calculated, and multiple documents with the highest similarity are screened out from the vector database. Then, the graph database is used to obtain multiple groups of related files through the relationship network between nodes to expand the associated documents of the candidate documents. The semantic similarity and relationship weight are comprehensively considered to recalculate the relevance score of the expanded document set with the query, and the document set is sorted according to the comprehensive relevance score. The sorted document list is returned to the user, and the reference relationship and chapter relationship between the files are displayed through a graphical interface.

Claims

1. A file retrieval and management method integrating AI big model and graph data, characterized in that: The specific steps of the retrieval and management method are as follows: Ⅰ. Users import multi-format files into the file retrieval and management system, and the system extracts text and structured content through the file parsing module; II. Document retrieval and management system gradually builds a multi-level database based on the parsed document content and metadata; III. Manage the entire process of documents from creation to archiving, and provide intelligent tools to optimize the document processing process; IV. Users input search requirements through natural language, and the file retrieval and management system matches corresponding files based on semantic embedding and graph database technology.

2. According to claim 1, a file retrieval and management method integrating AI big model and graph data is characterized in that: The document retrieval and management system described in step I specifically includes a database management module, a document full-process management module, a system integration module, and a rights management module; The database management module is used for data storage and management, and to build an efficient file processing and knowledge storage system; The document full-process management module is used to comprehensively cover the life cycle management of files from creation to archiving; The system integration module is used to achieve deep integration inside and outside the system to ensure data flow and functional coordination; The authority management module is used to ensure the security and controllability of the system, and to guarantee the standardization of user operations and the security of data through flexible authority control and monitoring functions.

3. According to claim 2, a file retrieval and management method integrating AI big model and graph data is characterized in that: The specific steps of extracting text and structured content by the parsing module in step I are as follows: S1.1: Through the visual analysis algorithm, the content in the image-type PDF page is recognized and segmented into regions, and the content area is divided into various types of regions such as text area, title area, image area, and table area; S1.2: Use OCR technology to classify and analyze different types of regions, use a multimodal large model to understand the image, generate corresponding labels, and convert non-text content into text; S1.3: Normalize and preprocess the extracted text to remove redundant characters and spaces, then decompose the text into words, restore the words to their root form, and embed the text using the pre-trained Embedding model; S1.4: Aggregate word-level embeddings into a sentence- or paragraph-level embedding through average pooling or attention mechanism to generate a global semantic vector for the text, and normalize the generated semantic vector.

4. According to claim 3, a file retrieval and management method integrating AI big model and graph data is characterized in that: The specific steps for constructing a multi-level database described in step II are as follows: S2.1: Extract the basic attribute information of the stored file, divide the parsed semantic content into blocks, and store them in association with the file ID to build a proprietary structured database; S2.2: Collect the semantic vectors of each file content generated by the Embedding model, and then store the generated semantic vectors in the vector database, while associating the file ID and text block information to form a basic database for retrieval query; S2.3: Extract references, reference relationships or semantic associations between files or text paragraphs from the parsed text, then store the extracted nodes and relationships in the form of a graph to build a multidimensional relationship network to form a graph database that supports complex association queries.

5. According to claim 4, a file retrieval and management method integrating AI big model and graph data is characterized in that: The specific steps of the whole process management described in step III are as follows: S3.1: The document retrieval and management system recommends document templates that meet users' needs based on historical documents and industry standards, and detects formatting and logical errors in document content through rule engines or models; S3.2: During the document review process, experts’ opinions and modification suggestions are archived in the form of structured data, and all review opinions are integrated to generate the final revision to provide guidance to users; S3.3: After the document is approved, the final version will be updated to the knowledge base synchronously, and the version record will be maintained. The consistency of the versions referenced by each group of corresponding documents will be checked through dependencies, and the document modification and approval process will be traced through metadata records.

6. A file retrieval and management method integrating AI big model and graph data according to claim 5, characterized in that: The specific steps of matching corresponding files based on semantic embedding and graph database technology in the file retrieval and management system described in step IV are as follows: S4.1: Preprocess the natural language requirements input by the user, including removing stop words, word segmentation and syntactic analysis, and generate processed text. Use the Embedding model to convert the parsed requirement text into a semantic vector; S4.2: Based on the semantic vectors in the vector database, the similarity between the query vector and the stored vector is calculated, and multiple documents with the highest similarity are selected from the vector database. Then, the graph database is used to obtain multiple groups of related files through the relationship network between nodes to expand the related documents of the candidate document. S4.3: Based on the semantic similarity and relationship weight, recalculate the relevance score between the expanded document set and the query, sort the document set according to the comprehensive relevance score, return the sorted document list to the user, and display the reference relationship and chapter relationship between files through a graphical interface.

Images