A document association analysis and retrieval system centered on a butterfly control architecture

Abstract

This invention discloses a document association analysis and retrieval system centered on a butterfly control architecture. It includes a vector association processing unit for associating vectorized numerical values ​​of documents, a retrieval information and feedback unit for processing user-inputted retrieval information and providing retrieval results, and a unified database pool for storing data. The retrieval information and feedback unit includes a butterfly control module and a vector retrieval module. The system uses the butterfly control module for butterfly-shaped control, enabling it to quickly and accurately locate a specific paragraph in a large number of internal enterprise documents based on the user's relatively vague natural language retrieval expression. The system can then reorganize the paragraph using natural language that the user understands before providing feedback to the user. This invention solves problems such as unclear user expressions, and can reorganize internal documents according to the user's expression and access permissions, adapting to the user's comprehension ability and providing a more user-friendly expression format, thereby improving document utilization efficiency and information retrieval security.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology in artificial intelligence, and in particular to a document association analysis and retrieval system centered on a butterfly control architecture. Background Technology

[0002] After years of implementing information management for documents, all enterprises and organizations have accumulated a large number of operational management documents containing a wealth of management and operational information. However, these documents are stored separately, with no correlation between the information within each document. This makes it difficult to quickly acquire and locate this information using existing data mining techniques. To solve this problem, significant manpower is often required to process the information in these documents, structuring it and storing it in a database. However, this traditional, purely manual processing method cannot meet the needs of today's rapidly evolving organizations. Therefore, a new technology for rapidly processing these fragmented documents is needed.

[0003] For enterprises with existing large databases, in practical applications, end-user descriptions are often unclear and always include contextual descriptions. Existing retrieval systems are primarily based on keyword-based searches, which struggle to achieve accurate results without clear user descriptions and cannot handle contextual descriptions. Furthermore, due to the diverse nature of users, directly providing retrieved information requires further processing, posing significant challenges to the use of large databases. Moreover, with increasing emphasis on cybersecurity, directly using large language models for vectorized retrieval often exposes unnecessary information to unauthorized users, potentially leading to the leakage of critical information. Therefore, enterprises using large databases need to restrict the scope of information retrieval based on user permissions, ensuring information is output in a reliable and effective manner.

[0004] Although there are many large language models available now, these new technologies cannot be directly integrated into the existing internal document management processes of enterprises. Therefore, to solve the aforementioned technical problems, a new architecture needs to be built to maximize the role of large language models and improve the artificial intelligence level of existing systems. Summary of the Invention

[0005] To address the technical problem, this invention provides a document association analysis and retrieval system centered on a butterfly control architecture. This system can quickly and accurately locate a specific paragraph within a vast amount of internal enterprise documents based on the user's relatively vague natural language search query. It can then reorganize the information using natural language that the user understands and provide feedback to the user. This solves the problem of fragmented information stored in large amounts of internal documents, making comprehensive utilization difficult. Furthermore, based on the butterfly control algorithm, it effectively addresses issues such as unclear user expressions, reorganizing internal documents according to the user's description to adapt to their comprehension level and provide a more appropriate expression, thereby improving document utilization efficiency.

[0006] The technical solution of the present invention is as follows:

[0007] A document association analysis and retrieval system centered on a butterfly control architecture includes:

[0008] The first part is the vector correlation processing unit, which is used to perform correlation processing on the vectorized values ​​of the document;

[0009] The second part is the information retrieval and feedback unit, which is the core unit of this system. It is used to process the retrieval information input by the user and to feed back the retrieval results to the user.

[0010] The third part is the database pool, which can shield the differences between different types, vendors and versions of databases, forming a unified database pool from which all data is read.

[0011] The first part, the vector association processing unit, includes a multimodal information filtering module, a vectorization module, and a gridded association module.

[0012] The multimodal information filtering module is a fully automated information acquisition module. This module automatically retrieves relevant operation and management documents from the connected network, categorizes and classifies the documents, and breaks down the information into three main elements: text, images, and tables. Text and images are considered unstructured information, while tables are considered structured information. Because tables can be directly mapped to a structured database, data can be stored in this mapping, enabling retrieval of specific elements. After processing a document / file, the system automatically adds a label to the original network's file server. This ensures that as long as the document / file remains unchanged, its vectorized values ​​will be fixed. The labeled document / file will be written back to the file server via a write-back mechanism.

[0013] The vectorization module, an artificial neural network, calculates the corresponding vector information from the unstructured information extracted by the multimodal information filtering module. The calculated vector information is then stored in a vector database (uniformly pooled within the database) for subsequent vectorization calculations. Because documents contain much unstructured information, such as videos, audio, engineering binary files, and source code files, which lack corresponding description files and can only be stored in the database as block units, their internal structure cannot be further analyzed. This system primarily processes unstructured information from text and images, where information elements can be extracted using artificial intelligence algorithms and transformed into smaller, more precise information elements.

[0014] For text, the sentence2vec algorithm is used, which vectorizes sentences as units.

[0015] For images, the descriptive text for the image in the document is vectorized. If no explicit descriptive text is found in the document, the image is discarded without further processing.

[0016] The gridded association module is used to perform association calculations between vector information and structured information to obtain the corresponding association matrix. In this way, there is a multi-dimensional association matrix between all information elements in the document (including text, images, and text information in tables), which enables quantitative analysis and retrieval. Quantitative analysis analyzes the association distance between two elements, that is, the distance between the two elements under different paths in the multi-dimensional association matrix. Retrieval is to find all elements in the database pool whose distance to the quantized distance threshold and a calibration element are given.

[0017] Association calculations can be performed using document paragraphs for association processing. For example, related text and images can be associated with tables adjacent to those paragraphs, with the closer the paragraphs, the closer the value is to 1.0, and the farther the paragraphs, the closer the value is to 0.0. The paragraph proximity value is calculated as a percentage of the total number of paragraphs in the document. The corresponding association matrix associates text, images, and tables based on paragraph distance. If a certain paragraph can be located, the corresponding image and table can be found by providing the paragraph distance value. This is one dimension. In addition, there are semantic and keyword associations between texts, and the elements stored in tables are also text, which are also associated with semantics and related words. When users search for this information, the associations are no longer simple paragraph associations but also include other associations, forming a multi-dimensional association.

[0018] The second part is the information retrieval and feedback unit, which includes a butterfly control module and a vector retrieval module.

[0019] The butterfly control module is used to analyze and process the information input by the user into natural language for retrieval and transmit it to the vector retrieval module, and also to feed back the retrieval results to the user.

[0020] The vector retrieval module is used to convert the natural language of the retrieval output by the butterfly control module into vector retrieval commands, perform retrieval in the database pool according to the vector retrieval commands, obtain retrieval results according to the corresponding association matrix, and feed back the retrieval results to the butterfly control module.

[0021] The third part is the database pool, which includes a unified data pool interface module and several databases.

[0022] The unified data pool interface module provides a unified interface that can be linked to multiple databases.

[0023] The aforementioned databases include structured and unstructured databases, used to store various types of data. These databases also include, but are not limited to, vector databases, file information databases, paragraph information databases from files, image information databases, table retrieval databases (storing table structure information), and table databases (each table in a file generates a separate table in the database, storing all its information).

[0024] Based on the design architecture of the aforementioned document association analysis and retrieval system, a specific design was carried out for the butterfly control module, which comprises four sub-modules, as follows:

[0025] (1) The context analysis submodule stores the user's historical input and current input in a string array, and then analyzes the description process of the user's input information. That is, after performing semantic analysis on all the user's input information, the vectorized retrieval semantics to be retrieved are obtained. The entire analysis process is generated by the butterfly control module, which is similar to a semantic translator. It transforms the user's natural description into a specific vectorized retrieval command array (a combination of multiple commands), which can significantly improve retrieval efficiency. It saves the user's context information and the processed and analyzed context information, and combines and summarizes this context information into a long text information for use by the subsequent multimodal mixing submodule and multimodal rendering submodule.

[0026] (2) Multimodal rendering submodule is used to re-integrate the retrieval synthesis and composite retrieval results, and call the structured data management module to judge and correct the key data to form multimedia page information and feed it back to the user; the integration refers to embedding the original file information containing pictures and tables into the HTML text with hyperlinks, rendering it and displaying it to the user, so that the user can see that the information contains pictures and tables, not just text.

[0027] (3) The retrieval synthesis submodule is used to synthesize the prompts in the context analysis submodule to form a new retrieval command. The retrieval command contains unstructured natural language and / or structured language. The system calls the vector retrieval module to obtain the new retrieval command and then waits for the feedback information from the vector retrieval module.

[0028] (4) The multimodal hybrid submodule is used to receive the results from the vector retrieval module, generate composite retrieval results, and send the composite retrieval results to the multimodal rendering submodule. The composite retrieval results mainly add the original file information such as images and tables to the long text in the form of hyperlinks, and at the same time attach the retrieval information and control information inside the butterfly control module. In this way, this information can be embedded in the browser output page during rendering. After each time this information passes through the hybrid unit, all its content will be reorganized, similar to a copy editor reorganizing the language.

[0029] Furthermore, the butterfly control module is also equipped with a prompting modal control module, a structured data management and control module, and a large language model generation module.

[0030] The prompt modality control module is a prompt generator related to a large language model. It provides multiple prompt operation chains (i.e., a set of prompt operation command arrays, which contain trigger conditions and prompts. This command array can guide the butterfly control module to form a fast chain reaction under specific conditions, with multiple sub-modules calling each other) templates. It provides different prompts for the sub-modules of the butterfly control module to optimize the natural language generation effect.

[0031] The structured data management module is used to analyze, place, and correct key structured data in the butterfly control module, thereby ensuring that the key structured data in all natural language descriptions within the butterfly control module is correct and has not been modified by the large language model. In other words, the data flowing through the butterfly control module is in natural language description, similar to human speech, but it still contains a large number of tags. The data within these tags needs to be checked against the original data before being pushed to the user, essentially a process of editing and proofreading.

[0032] The key structured data comes from structured databases. This data is either directly obtained from cell data in document tables, or derived by performing simple arithmetic operations on cell data in document tables. All other data is not key structured data. During vector retrieval, information is retrieved from the database pool, and the source of that information is also brought back, thus revealing which information originates from structured tables.

[0033] The large language model generation module is used to provide natural language processing and analysis services for the sub-modules of the butterfly control module. The natural language processing and analysis services include at least semantic analysis, keyword extraction, and sentence restructuring functions (currently mainly in the category of large language models based on ChatGPT). This module is generally a locally deployed private language model, such as LLaMA.

[0034] In the context analysis submodule, prompts can be provided by the prompt modality control module, key information in the user's context information can be determined and corrected by the structured data management module, and a new prompt can be generated by the large language model generation module by combining the prompts provided by the prompt modality control module and the corrected key information.

[0035] In the multimodal hybrid submodule, composite search results are generated by coordinating the language of the modality control module and the large language model generation module (mainly language; different models use different core languages, but currently English is the primary language).

[0036] Based on the overall design of the butterfly control module, its operation is as follows: In a user's search, the butterfly control module does not execute only one search process, but performs multiple search processes. Each search is carried out under different prompt operation chain templates provided by the prompt modality control module, and continuously calls the corresponding modules (context analysis submodule, search synthesis submodule, multimodal hybrid submodule, multimodal rendering submodule, structured data management module, and large language model generation module) to achieve repeated optimization of the search, thereby realizing the accuracy and enrichment of user input information.

[0037] Within the butterfly control module, API calls are used to invoke the context analysis submodule, multimodal fusion submodule, and multimodal rendering submodule. By blocking the context analysis submodule and the retrieval synthesis submodule through the multimodal fusion submodule and the multimodal rendering submodule, the butterfly control module can generate rich and accurate retrieval commands for the vector retrieval module after multiple interactions, achieving better retrieval results than when the user generates them directly.

[0038] Throughout the process, the prompts are gradually accumulated and are not formed at the beginning. They are generated after interaction with the user and calculation by the butterfly control unit. For example, it may take 2-3 interactions with the user to trigger the butterfly control module to generate the prompt. To better utilize the capabilities of the current large language model, the user interaction and prompts are all placed in a long text. In this way, with complete contextual information, large language analysis can participate more effectively in the retrieval, achieving more accurate and efficient retrieval.

[0039] The beneficial effects of this invention are as follows:

[0040] This invention focuses on deep data mining and analysis of internal enterprise documents. It constructs a hybrid intelligent information system for both unstructured and structured documents. Through a comprehensive framework formed by document processing and retrieval analysis, it realizes functions such as analysis and extraction, vectorization, grid-based association, structured management and control, and prompts. Primarily based on a butterfly control algorithm, it quickly and accurately locates a specific paragraph in a massive amount of internal enterprise documents according to the user's relatively vague natural language search expression. It can then reconstruct the paragraph using natural language that the user understands and provide feedback to the user, effectively solving problems such as unclear user expression and improving document utilization efficiency. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the system architecture of the present invention.

[0042] Figure 2 This is a schematic diagram of the sentence2vec algorithm used in this invention.

[0043] Figure 3 This is a timing diagram of the calls to the internal sub-modules of the butterfly control module of the present invention. Detailed Implementation

[0044] like Figure 1 As shown, a document association analysis and retrieval system centered on a butterfly control architecture comprises three parts, specifically:

[0045] The first part is the associative processing unit, which is used to perform associative processing on documents, including a multimodal information filtering module, a vectorization module, and a gridded associative module.

[0046] The multimodal information filtering module is a fully automatic information collection module. When a document is added or modified on the enterprise file server, the multimodal information filtering module will capture the operation and automatically identify and obtain the operation and management documents of each person in the enterprise from the enterprise's internal network. The documents are classified and categorized, and the information in the documents is split into three main elements: text, images, and tables.

[0047] The vectorization module is an artificial neural network that calculates the corresponding vector information from the unstructured information such as text and images extracted by the multimodal information filtering module. The calculated vector information is then stored in a vector database for subsequent vectorization calculations. For text, we use the mainstream sentence2vec algorithm, which vectorizes sentences as units, such as... Figure 2 As shown; for images, the descriptive text for the image in the document is vectorized. If no explicit descriptive text is found in the document, the image is discarded and no further processing is performed.

[0048] The gridded association module is used to perform association calculations between vectorized information and structured information to obtain the corresponding association matrix. In this way, there is a multi-dimensional association value between information elements in all documents, which enables quantitative analysis and retrieval.

[0049] The second part is the retrieval information and feedback unit, which is used to process the retrieval information input by the user and to feed back the retrieval results to the user. It includes a butterfly control module and a vector retrieval module.

[0050] The butterfly control module is used to analyze and process the information input by the user into natural language for retrieval and transmit it to the vector retrieval module, and also to feed back the retrieval results to the user.

[0051] The vector retrieval module is used to convert the natural language of the retrieval output by the butterfly control module into vector retrieval commands, perform retrieval in the database pool according to the vector retrieval commands, obtain retrieval results according to the corresponding correlation matrix values, and feed back the retrieval results to the butterfly control module.

[0052] The third part is the database pool, which includes a unified data pool interface module and several databases. This shields the differences between databases of different types, vendors, and versions, forming a unified database pool from which all data is read. The unified data pool interface module provides a unified interface that can connect to multiple databases; the several databases include structured and unstructured databases for storing various types of data.

[0053] Based on the design architecture of the document association analysis and retrieval system described above, the butterfly control module is specifically designed with the following sub-modules, as detailed below:

[0054] (1) Context analysis submodule is used to process user input information, analyze the description process of user input information, save user context information and processed and analyzed context information, and combine these context information into a long text information for use by other modules.

[0055] (2) Multimodal rendering submodule, which is used to re-integrate the retrieved and synthesized multimodal result information, and call the structured data management module to judge and correct the key data, and then form multimedia page information and feed it back to the user.

[0056] (3) The retrieval synthesis submodule is used to synthesize the prompts in the context analysis submodule to form a new retrieval command. The retrieval command contains unstructured natural language and structured language. The system calls the vector retrieval module to obtain the retrieval command.

[0057] (4) Multimodal hybrid submodule, which is used to receive the results fed back by the vector retrieval module, generate composite retrieval results in cooperation with the prompt modality control module and the large language model generation module, and send the composite retrieval results to the multimodal rendering submodule.

[0058] (5) The prompt modality control module is a prompt generator related to the large language model. It provides multiple prompt operation chain templates and provides different prompts for the sub-modules of the butterfly control module to optimize the natural language generation effect.

[0059] (6) The structured data management module is used to analyze, place, and correct key structured data in the butterfly control module, thereby ensuring that the key structured data in all natural language descriptions within the butterfly control module is correct and has not been modified by the large language model. Specifically, tags are used for positioning. If the data comes from a structured database, these tags are retained until the user receives the original data. This ensures that the large language model does not disturb the accurate data when processing semantics.

[0060] (7) Large language model generation module, which provides natural language processing and analysis services for the sub-modules of the butterfly control module. The natural language processing and analysis services include semantic analysis, keyword extraction, sentence restructuring and other functions. This module is generally a private language model deployed locally.

[0061] The submodules listed above are labeled as follows: Context Analysis Submodule (Module A), Multimodal Rendering Submodule (Module B), Retrieval and Synthesis Submodule (Module C), Multimodal Hybridization Submodule (Module D), Hint Modality Control Submodule (Module E), Structured Data Management Submodule (Module F), and Large Language Model Generation Submodule (Module G). (Reference) Figure 3 When calling these submodules via the API, there are two phases:

[0062] Initiating the interaction phase

[0063] The initial interaction phase primarily guides users to provide more information, quickly generating longer texts with more information, allowing for rapid location of the desired information. The specific process of this phase is as follows:

[0064] 1. The user sends an interactive message to the butterfly control unit. Generally, this message is simple enough that it is the minimum text synthesis length required to trigger the retrieval and synthesis module. This message is then sent to module A. Module A retrieves the set of interactive prompt control statements from module E. Guided by this control command, module B is invoked. Module B then invokes module G to generate the natural language feedback and sends it back to the user.

[0065] 2. After receiving the natural language expression, the user interacts further and provides the system with more information. This information still follows the above process, entering module A, calling module B, and then calling module G, repeating this process multiple times until the length of the context information stored in module A reaches the minimum text synthesis length.

[0066] This is where we enter the deep interaction phase.

[0067] The deep interaction phase involves the butterfly control module, after acquiring sufficient user information, performing multiple internal loops to retrieve and output information. The specific flow of this phase is as follows:

[0068] 1. From the "Start-up Interaction Phase" to the "Deep Interaction Phase", there is already a long text message in Module A in this phase. After entering this phase, a "Phase Adjustment" prompt will be automatically triggered.

[0069] 2. Input this prompt into module D. Module D calls module G to generate a search statement, which is then input into module C.

[0070] 3. Module C retrieves the matching search suggestion combination chain from Module E. This chain contains multiple suggestion combinations, each representing a loop. The butterfly control unit calls these combinations one by one under the control of this chain. It checks whether the last statement combination in this chain is being executed. If not, proceed to step 4; otherwise, proceed to step 9.

[0071] 4. Information from module C is sent to module B. After rendering the information, module B directly returns it to the user, indicating that the system is performing data analysis and retrieval. Simultaneously, this information is also input into module D.

[0072] 5. Module D calls module G to generate a new search command containing search suggestions.

[0073] 6. The new search command is input into module B. Module B processes the command, generates the corresponding vector search command, inputs it into the vector search module, retrieves it from the database pool, and feeds back the search results to module D.

[0074] 7. Module D obtains the search results and retrieves the next prompt command from module E. If the command asks for the results to be output, the current search results are output to module B for rendering. If the prompt asks for a change of role to continue the search, it is input into module A.

[0075] 8. Module A outputs the processed text to Module B. Module B displays the search progress to the user, indicating that the system is still generating the search. Then, the new context information generated by Module A is input into Module D, and the process proceeds to step 5.

[0076] 9. Module C outputs the search results to Module B. Module B calls Module F to review all structured data to ensure that all structured data is consistent with the content in the data pool. Then, it performs security checks on the related documents in the feedback information and blocks documents outside the confidentiality scope according to the user's identity and security level. All the above information calls Module G to generate a complete natural language description and feed it back to the user.

[0077] Based on the above design, here is an example of practical association analysis:

[0078] An enterprise has established an information system based on this invention. The company provided 1000 internal documents, including 10 documents related to a project from 2022. These documents included a list of procured equipment (PDF), a project acceptance report (PDF), design documents (Word documents), and construction work reports (Word documents). These 10 documents contained procurement lists for ABC-1, ABC-2, and ABC-3 equipment, their installation locations in Buildings 1 and 2, and wiring diagrams for each piece of equipment. However, this information was scattered across these 10 documents and not compiled into a unified report. The confidentiality levels of these documents also differed: the project acceptance report was classified as level 20, the construction documents as level 30, and the procured equipment list as level 50. Higher confidentiality levels indicate greater security, and only those exceeding the document's level can read the document. For example, the project manager's confidentiality level was 75, allowing them to view all documents; while the construction manager's level was 40, preventing them from viewing the procured equipment list.

[0079] After the aforementioned files were uploaded to the file server, being newly created files, the system automatically triggered analysis and processing. The system's "multimodal information filtering module" broke down the 1000 internal files into numerous small blocks of information, including text and image information. For each text segment, the system called the "vectorization module" to convert each text segment into a vector value. In the future, vector comparison calculations will be used to obtain the text paragraph closest to the retrieved text. Image information in PDF and Word documents is stored as independent image files on the file server. In Word and PDF documents, the title text of the images is vectorized and stored in the vectorization database. Images without title text are discarded. The filename and file title vectorization information are interrelated and stored in the same table. In the future, vector comparison calculations will be used to obtain the address of the image file closest to the retrieved text. Simultaneously, multiple tables were detected in the PDF and Word documents. An entity table was constructed for each table, with all table names in the format "filename-entry date-table number." All table names and structures are stored in structured data. We will perform correlation calculations between text, images, and tables based on paragraphs. The closer the paragraphs are, the closer the value is to 1.0; the farther the paragraphs are, the closer the value is to 0.0. The paragraph proximity value is calculated as a percentage of the total number of paragraphs in the document. All of the above information is stored in the database pool through the unified data pool interface module.

[0080] A new project manager has been hired in the company. As a new manager, his information is classified as confidential (level 45). He logs into the information system of this invention using a browser. After logging in, he sees a dialog box and can begin searching. Although he knows he wants to understand the recent usage of the ABC-3 equipment in the company, he didn't initially consider the search process in detail and didn't know if similar information existed in the system. At this point, his interaction with the system is in the "initiation interaction phase." He first enters "I want to understand the ABC-3 equipment situation." This information is entered into the system context submodule of the butterfly control unit. The submodule finds that the current context is less than 2000 characters, so it retrieves the set of initiation interaction prompt control statements from the prompt modal control module. Under this set of interaction control statements, it calls the multimodal rendering submodule, which in turn calls the large language model generation module, providing feedback to the user: "Do you want to understand the technical manual of the ABC-3 equipment, or the actual engineering application situation of the ABC-3 equipment?" The user enters "actual engineering application situation," and the system responds: "Is it the usage situation in the industry, or just the usage situation in our company?" The user enters "Just our company, and I only need to understand the application situation in the projects that have been accepted in the last three years." At this point, the context information in the butterfly control module has reached 2000 characters, and the system responds, "Your needs have been understood. Now, we will retrieve and provide feedback on the usage of the ABC-3 device in projects that have been accepted by our company in the past three years. I will generate a brief report for you and attach links to the relevant original documents." At this point, the butterfly system enters the "deep interaction phase."

[0081] 1. From the "Start-up Interaction Phase" to the "Deep Interaction Phase", there is already a long text message in Module A in this phase. After entering this phase, a "Phase Adjustment" prompt will be automatically triggered.

[0082] 2. Input this prompt into module D. Module D calls module G to generate a search statement, which is then input into module C.

[0083] 3. Modules C through E retrieve matching search prompt chains, specifically the "Engineering Report Chain." This chain contains multiple prompt combinations, each representing a loop. The butterfly control unit calls these combinations sequentially under the chain's control. For example, the first prompt might be, "As an engineering manager, extract 3-5 keywords from the above context," thus obtaining a keyword combination. The second prompt is, "As a document manager, analyze the document type requested by the user and propose key search points based on the above keywords." The third prompt is, "As a database manager, provide a set of search commands with a time range based on the user's request." The fourth prompt is, "As an experienced engineering project manager, study the above search content and provide a brief report of no more than 1000 words." The fifth prompt is, "As a data validator, organize the data in the above brief report and compare it with the original table." The sixth prompt is, "Please conduct a confidentiality assessment for the user." Determine if the last statement combination in this chain is being executed. If not, proceed to step 4; otherwise, proceed to step 9.

[0084] 4. Information from module C is sent to module B. After rendering the information, module B directly returns it to the user, indicating that the system is performing data analysis and retrieval. Simultaneously, this information is also input into module D.

[0085] 5. Module D calls module G to generate a new search command containing search suggestions.

[0086] 6. The new search command is input into module B. Module B processes the command, generates the corresponding vector search command, inputs it into the vector search module, retrieves it from the database pool, and feeds back the search results to module D.

[0087] 7. Module D obtains the search results and retrieves the next prompt command from module E. If the command asks for the results to be output, the current search results are output to module B for rendering. If the prompt asks for a change of role to continue the search, it is input into module A.

[0088] 8. Module A outputs the processed text to Module B. Module B displays the search progress to the user, indicating that the system is still generating the search. Then, the new context information generated by Module A is input into Module D, and the process proceeds to step 5.

[0089] 9. When the above search reaches this point, according to the prompt, a "security level search" is required. Module C outputs the search results to module B. Based on the user's security level of 45, documents exceeding 45 in the generated files will no longer display their links but will be replaced with an XXX number. All the above information calls module G to generate a complete natural language description, ultimately resulting in a "Brief Report on the Application of ABC-3 Equipment in Projects Over the Past Three Years." The report explains the comparison data of the usage ratio of ABC-3 equipment relative to ABC-1 and ABC-2 equipment in the company's projects. Next to the report is a document list containing links to the construction reports and acceptance reports of the 10 documents we entered into the system in 2022.

Claims

1. A document association analysis retrieval system centered on a butterfly control architecture, characterized by: The system comprises a vector correlation processing unit, a retrieval information and feedback unit and a database pool. The vector correlation processing unit is configured to perform correlation processing on the vectorized numerical values of the documents; the vector correlation processing unit comprises a multi-modal information screening module, a vectorization module and a gridding correlation module; the multi-modal information screening module automatically acquires each operation and management document to be correlated and analyzed from a connected network, classifies the operation and management documents, and splits information in the documents into three main elements of text, pictures and tables, wherein the text and pictures are unstructured information and the table is structured information; The vectorization module calculates the split unstructured information into corresponding vector information, and stores the calculated vector information in a vector database for subsequent vectorization calculation; The gridding correlation module is configured to perform correlation calculation on the vector information and the structured information to obtain a corresponding multi-dimensional correlation matrix, so that there is a multi-dimensional correlation matrix between information elements in all documents, thereby performing quantitative analysis and retrieval; for text, a sentence2vec algorithm is used to vectorize the sentences as units; for pictures, the description text of the picture in the document is vectorized, and if no explicit description text is found in the document, the picture is discarded and not processed; the correlation matrix is obtained by correlating the text, pictures and tables based on paragraph distance, and also by using semantic and keyword correlation, thereby forming multi-dimensional correlation; The retrieval information and feedback unit is configured to process the retrieval information input by a user and feed back the retrieval result to the user; the unit comprises a butterfly-shaped control module and a vector retrieval module, wherein the butterfly-shaped control module is configured to analyze and process the information input by the user into retrieval natural language and transmit the retrieval natural language to the vector retrieval module, and also configured to feed back the retrieval result to the user; the vector retrieval module is configured to convert the retrieval natural language output by the butterfly-shaped control module into a vector retrieval command, perform retrieval in the database pool according to the vector retrieval command, obtain the retrieval result according to the corresponding correlation matrix, and feed back the retrieval result to the butterfly-shaped control module; The butterfly-shaped control module comprises a context analysis submodule, a multi-modal rendering submodule, a multi-modal mixing submodule and a retrieval synthesis submodule; The context analysis submodule is configured to process the information input by the user, analyze the description process of the user input information, save the user's context information and the processed and analyzed context information, mix and aggregate the two kinds of context information into a long text information, and provide the long text information for the multi-modal rendering submodule and the multi-modal mixing submodule; The multi-modal rendering submodule is configured to render the long text information generated by the context analysis submodule, generate a composite retrieval command and send the composite retrieval command to the retrieval synthesis submodule, and also configured to render the composite retrieval result processed by the multi-modal mixing submodule, correct the key data, form multimedia page information, and feed back the multimedia page information to the user. The retrieval synthesis submodule is configured to synthesize prompt language in the composite retrieval command to form a new retrieval command, and the new retrieval command contains unstructured language and / or structured language; the system obtains the new retrieval command by calling a vector retrieval module; The multi-modal mixing submodule is configured to receive a retrieval result fed back by the vector retrieval module, then generate a composite retrieval result, and send the composite retrieval result to a multi-modal rendering submodule; the composite retrieval result contains picture and table original file information; the composite retrieval result contains retrieval information and control information in the butterfly-shaped control module, and the retrieval information and control information are reorganized once every time they pass through the multi-modal mixing submodule; The database pool includes a unified data pool interface module and a plurality of databases to form a unified database pool; the unified data pool interface module is configured to provide a unified interface and be associated with a plurality of databases; the plurality of databases at least include a structured database, an unstructured database, a vector database, a file information database, a file paragraph information database, a picture information database, and a table retrieval table database, and are configured to store various types of data, and all data are read from the database pool.

2. The document association analysis retrieval system centered on a butterfly control architecture of claim 1, wherein: The processing of the information input by the user refers to that the context analysis submodule stores the historical input and the current input information of the user in a string array; the analysis of the information input by the user refers to that the semantic analysis is performed on all the input information of the user to obtain vectorized retrieval semantics that need to be retrieved.

3. The document association analysis retrieval system centered on a butterfly control architecture of claim 1, wherein: The prompt language exists in long text and is constantly accumulated and updated after being called by each submodule of the butterfly-shaped control module; when each submodule is called, the updated prompt language is called.

4. The document association analysis retrieval system centered on a butterfly control architecture of claim 1, wherein: The context analysis submodule and the retrieval synthesis submodule in the butterfly-shaped control module cannot directly communicate with each other, and the rest of the submodules realize bidirectional data flow through API calling; the four submodules of the butterfly-shaped control module form a final retrieval command for the vector retrieval module after multiple interactions.

5. A document association analysis retrieval system centered on a butterfly control architecture according to any one of claims 1-4, characterized in that: The butterfly-shaped control module is further configured with a prompt modal control module, a structured data control module, and a large language model generation module; The prompt modal control module is configured to provide a plurality of prompt language operation chain templates to provide different prompt languages for the four submodules of the butterfly-shaped control module, and is configured to optimize the natural language generation effect; The structured data control module is configured to analyze, occupy, and correct key structured data in the butterfly-shaped control module, so as to ensure that the key structured data in all natural description languages in the butterfly-shaped control module is correct and has not been modified by the large language model; the key structured data is derived from data in the structured database, that is, the original source is only obtained from cell data in a file table, or directly uses the cell data in the file table, or is obtained by performing four arithmetic operations on the cell data in the file table; The large language model generation module is configured to provide natural language processing and analysis services for the four submodules of the butterfly-shaped control module, and the natural language processing and analysis services at least include semantic analysis, keyword extraction, and sentence reorganization functions.

6. A document association analysis retrieval system centered on a butterfly control architecture according to claim 5, characterized in that: In the context analysis submodule, the prompt mode control module provides a prompt, the structured data control module determines the key information in the corrected user context information, and the large language model generation module generates a new prompt in combination with the prompt provided by the prompt mode control module and the corrected key information.

7. The document association analysis retrieval system centered on a butterfly control architecture of claim 5, wherein: In the multi-modal mixing submodule, the language of the prompt mode control module and the large language model generation module is combined to generate a composite retrieval result.

8. The document association analysis retrieval system centered on a butterfly control architecture of claim 5, wherein: In each retrieval of the user, the butterfly-shaped control module performs a multiple retrieval process, each retrieval continuously calls the context analysis submodule, the retrieval synthesis submodule, the multi-modal mixing submodule, the multi-modal rendering submodule, the structured data control module, and the large language model generation module under different prompt operation chain templates provided by the prompt mode control module, so as to realize multiple repeated optimization of retrieval and accurate and rich input information of the user.

Images