Fast document retrieval method and fast document retrieval system

Abstract

A fast document retrieval method includes receiving a query by a user interface; analyzing the query by a natural language module to obtain query-key information related to the query; searching for content matching the query-key information in a suggestion answer dataset; and when the content that fully matches the query-key information is found in the suggestion answer dataset, responding with a suggested answer matching the query-key information on the user interface as a retrieval result information of the query.

Description

Technical Field

[0001] This case relates to a retrieval method and system, and more particularly to a rapid document retrieval method and system. Prior Technology

[0002] To comply with internal or external operational guidelines, companies develop standardized documents to meticulously document operational details. To facilitate the management of these guidelines, a single document may define a specific topic, while other related guidelines may be presented as concise outlines in a separate chapter or paragraph. These outlines allow users to reference relevant information when needed.

[0003] However, when users search for specific specification content, they need to manually search through numerous specification documents one by one. If, while searching for specific specification content, a user discovers from the information in one of the currently found specification documents that the desired specification content might be recorded in another specification document, the user needs to open that other specification document again from the reference information in the current specification document. In other words, users need to repeatedly compare the information they want to find in different specification documents. This means that users need to mechanically perform repeated searching, comparing, and confirming operations from a large number of specification documents, which is quite time-consuming and prone to errors.

[0004] Furthermore, when the operating guidelines change, all related standard documents also need to be updated accordingly. Because these standard documents are numerous and contain a great deal of information, manually extracting and organizing information is not only tedious and labor-intensive, but also prone to inaccurate information classification and association. Summary of the Invention

[0005] To address the aforementioned issues, this paper proposes a rapid document retrieval method and a rapid document retrieval system.

[0006] One embodiment of this invention proposes a fast document retrieval method, comprising the following steps: receiving a question at a user interface; parsing the question using a natural language module and obtaining key information related to the question; searching a set of suggested answers for content matching the key information; and when the set of suggested answers contains content that perfectly matches the key information, responding at the user interface with a suggested answer matching the key information as the retrieval result information for the question.

[0007] Another embodiment of this invention proposes a fast document retrieval system, including a suggested answer dataset, storage media, and a processor. The processor is coupled to the suggested answer dataset and the storage media, and after a plurality of code snippets are loaded, performs the following operations: receiving a question at a user interface; parsing the question using a natural language module and obtaining key information related to the question; searching the suggested answer dataset for content matching the key information; and when the suggested answer dataset contains content that perfectly matches the key information, responding to the user interface with suggested answers matching the key information as search results for the question. Simple Explanation of the Diagram

[0008] Figure 1 is a block diagram of a fast document retrieval system according to an embodiment of this invention.

[0009] Figure 2 is a flowchart illustrating a rapid document retrieval method according to an embodiment of this case.

[0010] Figure 3 is a schematic diagram of a user interface according to an embodiment of this invention.

[0011] Figure 4 is a block diagram of a fast document retrieval system according to an embodiment of this invention.

[0012] Figure 5 is a schematic diagram of a user interface according to another embodiment of this invention.

[0013] Figure 6 is a block diagram of a fast document retrieval system according to another embodiment of this invention. Implementation

[0014] The following description, in conjunction with the accompanying drawings and embodiments, further illustrates this case so that those skilled in the art can better understand and implement the invention. However, the embodiments described are not intended to limit the scope of the invention.

[0015] There are numerous existing standards and guidelines, requiring users to spend considerable time learning to understand each one. The structure of these standards is vast, with each sub-item documented in its own specification document. For interrelated sub-items, reference information is also provided within these specification documents for users to cross-reference. Therefore, finding the information needed amidst a massive amount of specification documents is a difficult task for users.

[0016] To provide users with the ability to quickly obtain key information without a clue, the fast document retrieval technology in this case provides a user interface that allows users to obtain the key information they need by simply asking questions in the same manner as in everyday conversations.

[0017] 1 is a block diagram of a fast document retrieval system illustrated in accordance with an embodiment of the present case.

[0018] As shown in FIG. The processor 110 is coupled to the storage media 120 and the proposed answer dataset 130 , respectively.

[0019] Storage Media 120 Stores multiple codes and files. After these codes are loaded into processor 110 , processor 110 performs operations of user interface 122 and natural language module 124 in conjunction with the file and the loaded code.

[0020] In one embodiment, the natural language module 124 is a pre-trained small language model, and human language can be generated and processed, such as a Generative Pre-trained Transformer applying deep learning.

[0021] In one embodiment, the proposed answer dataset 130 is associated to a plurality of databases. Each database has its own independently designed data structure, storage mechanism, data content, etc., and different databases are set up according to the application or purpose. Each database stores multiple reference documents based on their respective design purposes.

[0022] Processor 110 is, for example, but not limited to, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Central Processing Unit (CPU), a System on Chip (SoC), a Field Programmable Gate Array (Field Programmable Gate Processor), or a combination of the above-mentioned Field Programmable Gate Processor Network.

[0023] Storage media 120 such as, but not limited to, Random Access Memory (RAM), Flash memory (Flash memory), Read Only Memory (ROM), Hard Disk Drive (HDD), Solid State Drive (SSD), optical storage, or a combination of the above components.

[0024] User interfaces 122 For example, are, but are not limited to, operational interfaces for text, graphics, command bar, sound, touch, and the like. The user interface 122 is implemented according to an embodiment of the operating interface in conjunction with a hardware component that can realize the function of the user interface 122 , such as a display, but this case does not limit the state of the hardware component.

[0025] In one embodiment, the user may operate the user interface 122 of the rapid document retrieval system, such as entering a question to be asked in the user interface 122 . The natural language module 124 of the rapid document retrieval system performs semantic parsing on the questioning questions to extract the question-focused information of the questioning question, and then uses the question-focused information to look for a match in the proposed answer dataset 130 . If a match is found, the rapid document retrieval system responds to the suggested answer to the question in the user interface 122 (the first matching phase). On the other hand, if the fast document retrieval system does not find content that matches the problem-focused information, the problem-focused information is further converted into a problem vector, and then aligned in the vector database in the form of vector values, accordingly to feed back the other information (second matching stage). By processing semantic parsing first in the first matching stage for semantic focus retrieval, the more mere content is returned first, and complex answers are processed in the second matching stage, thereby achieving the efficacy of quickly feedback document retrieval results.

[0026] 2 is a flowchart of a fast document retrieval method illustrated in accordance with an embodiment of this case.

[0027] In step S210, the user interface 122 receives a question.

[0028] In step S215, the natural language module 124 parses the question question and obtains the key information of the question related to the question question.

[0029] In step S220, processor 110 aligns the proposed answer dataset 130 with problem-focused information.

[0030] In step S225, processor 110 determines whether the suggested answer dataset 130 contains suggested answers that match the key information of the question. If yes, step S230 is executed; otherwise, step S260 is executed.

[0031] In step S230, the suggested answer that matches the key information of the question is displayed on the user interface 122 as the search result information for the question.

[0032] In one embodiment, the user interface 122 has an input box where the user can freely input colloquial questions according to their speaking habits. For example, if a user wants to know what "SLAC" is, they can directly ask in the input box, "Can you explain what SLAC is?" In this case, "Can you explain what SLAC is?" is the question. The natural language module 124 parses this question, "Can you explain what SLAC is?". The natural language module 124 can be a small language model pre-trained by the enterprise for its internal expertise or specialized technology. Therefore, the natural language module 124 will perform semantic analysis and understanding of this question from the perspective of the enterprise's internal expertise or specialized technology.

[0033] In one embodiment, the natural language module 124 first performs text cleaning (e.g., removing redundant words), vocabulary breakdown, grammatical and semantic analysis on the question, and then generates key information about the question after understanding the question.

[0034] The natural language module 124 further queries the suggested answer dataset 130 based on the key information of the question to determine whether there is a matching suggested answer in the suggested answer dataset 130. In one embodiment, the suggested answer dataset 130 stores a plurality of pre-established standard data (e.g., proper nouns, common words, or specification numbers), allowing the natural language module 124 to directly compare the content of the standard data with the key information of the question. The standard data can be associated with hyperlinks to detailed description paragraphs or reference documents associated with proper nouns, common words, or specification numbers. Therefore, when the natural language module 124 finds standard data in the suggested answer dataset 130 that perfectly matches the key information of the question, it outputs the hyperlinks to the detailed description paragraphs or reference documents associated with the standard data as suggested answers.

[0035] In another embodiment, the natural language module 124 uses the key information of the question as an index to query the content of reference documents in a plurality of databases in the suggested answer dataset 130. If the content of a reference document is found to be completely matched with the key information of the question, the natural language module 124 extracts the matching paragraph as a suggested answer, or automatically generates a suggested answer based on autoregression or attention mechanisms.

[0036] In one embodiment, the user interface 122 displays the aforementioned generated suggested answers, allowing the user to obtain search result information.

[0037] In another embodiment, the suggested answer includes a hyperlink to an answer file in one of a plurality of databases. For example, in addition to displaying the aforementioned generated suggested answer, the user interface 122 may also display a hyperlink to the reference file where the suggested answer was found. After the user clicks on the hyperlink in the user interface 122, the content of the file can be displayed on the screen.

[0038] Figure 3 is a schematic diagram of a user interface according to an embodiment of this invention.

[0039] The user interface 300 can be a dialog window where the user can enter text as a question 310, such as "Could you explain what ISO 15025 is?" as shown in Figure 3. After the aforementioned steps S210 to S225, the natural language module 124 retrieves identical content from the suggested answer dataset 130. Therefore, it responds with suggested answer 320 on the user interface 300, as shown in Figure 3, "Of course! Similarity: 100%. Content: ISO 15025 is an international standard specification that defines the test performance of flame spread."

[0040] In another embodiment, the suggested answer 320 may include, in addition to the aforementioned answer content, a hyperlink to the reference document where the suggested answer was found, such as a hyperlink to a reference document containing the definition of ISO 15025 (not shown in Figure 3). Thus, in addition to reading the content of the suggested answer, the user can further read the reference document containing the definition of ISO 15025 by clicking the hyperlink.

[0041] In one embodiment, steps S210 to S230 are a fast file retrieval method for the first matching stage, which can be executed by the fast file retrieval system of Figure 1.

[0042] On the other hand, when no perfectly matching suggested answer is obtained in the first matching stage, and no answer information or reference document required by the user is retrieved, the fast document retrieval method further performs a second matching stage in steps S260 to S280 to perform fuzzy retrieval.

[0043] Figure 4 is a block diagram of a fast document retrieval system according to an embodiment of this invention.

[0044] The fast document retrieval system includes a processor 410, a storage medium 420, a suggested answer dataset 430, and a vector database 440. The processor 410 is coupled to the storage medium 420, the suggested answer dataset 430, and the vector database 440, respectively.

[0045] In one embodiment, the suggested answer dataset 430 is associated with a plurality of databases 435a-435n. Each database includes a plurality of reference files; for example, database 435a includes reference files 437a-437m. This invention does not limit the content or number of reference files stored in each database.

[0046] The vector database 440 pre-stores a plurality of reference vectors, which are vectors of key content in the reference documents, that is, a plurality of vectors converted from a plurality of standard data (such as proper nouns, common words or standard numbers) in the suggested answer data set 430.

[0047] In another embodiment, the fast document retrieval method in steps S210 to S280 can be executed by the fast document retrieval system of FIG4.

[0048] Please refer to Figures 2 and 4 together. For the sake of brevity, the following describes an embodiment of the second matching stage. It is worth noting that the fast document retrieval system in Figure 4 may perform only the first matching stage, or, if no completely matching content is found in the first matching stage, further perform a second matching stage of document retrieval.

[0049] Following step S225, processor 110 determines that no suggested answer matching the key information of the question was found in suggested answer dataset 430. Therefore, in step S260, processor 110 converts the key information of the question into a question vector.

[0050] In step S265, processor 110 calculates multiple similarities between the problem vector and several reference vectors in the vector database 440.

[0051] In step S270, processor 110 determines whether the similarity is greater than a threshold. If yes, then step S275 is executed; otherwise, step S280 is executed.

[0052] In step S275, the processor 110 outputs the reference answer corresponding to the reference vector, wherein the reference answer includes a hyperlink to the reference file.

[0053] In step S280, the processor 110 discards the reference vector.

[0054] In one embodiment, since no perfectly matching suggested answer is found in the suggested answer dataset 430, i.e., no identical information is found through text retrieval and comparison, the key information of the question is further converted into a question vector. Next, a plurality of reference vectors are pre-stored in the vector database 440, and the processor 410 performs fuzzy comparison by calculating the similarity between the vectors.

[0055] In one embodiment, the processor 410 calculates the distance between the question vector and each reference vector in the vector database 440, using the distance value as the similarity between the question vector and the reference vector. The distance between two vectors can be calculated using Euclidean distance or Manhattan distance, etc. In this embodiment, the smaller the distance value, the more similar the two vectors are, that is, the more similar their lexical semantics. Therefore, the processor 410 selects reference vectors with a similarity greater than a threshold and discards reference vectors with a similarity less than the threshold.

[0056] In one embodiment, the threshold can be a 75% filtering condition. The design of the threshold affects the reference value and the amount of data in the reference answer. For example, a larger threshold results in a more reference value for the reference answer, but a smaller amount of data in the reference answer, and vice versa.

[0057] After the processor 410 filters reference vectors with a similarity greater than a threshold from the vector database 440, since the reference vectors are vectors reflecting the reference documents (i.e., vectors converted based on the content of the suggested answer dataset 430), the reference vectors filtered in step S270 can be traced back to the original text information, i.e., the reference document from which the vector information originates. The content of the reference document corresponding to these reference vectors with a similarity greater than the threshold is then used as the reference answer. In one embodiment, the reference answer includes a hyperlink leading to the reference document.

[0058] Figure 5 is a schematic diagram of a user interface according to another embodiment of this invention.

[0059] The user interface 500 can be a dialog window where the user can enter text as a question 510 in the input box. After completing the aforementioned steps S210 to S225 and finding a suggested answer that perfectly matches the key information of the question 510, the user can respond with a suggested answer 520 on the user interface 500. Compared to the suggested answer 320 in Figure 3, in this embodiment, the suggested answer 520 further includes a hyperlink to a reference document, allowing the user to directly click the hyperlink to read the document content.

[0060] Users can continue to enter other documents they wish to query. For example, a user might enter another piece of text as a question 530, such as "I want to query ISO15111, what is it?". However, if the fast document retrieval system does not find any content in the suggested answer dataset 430 that perfectly matches "ISO15111" after steps S210 to S225 (assuming "ISO15111" is the key information in the question), then steps S260 to S280 are executed to perform a vectorized fuzzy comparison, and a plurality of reference vectors (or filtered reference vectors) with a similarity greater than a threshold are obtained.

[0061] In one embodiment, the natural language module 424 extracts paragraphs related to the filtered reference vectors as reference answers, or automatically generates a reference answer based on autoregression or attention mechanisms. As shown in Figure 5, the reference answer 540 includes the similarity of the filtered reference vectors and the hyperlinks to the corresponding reference documents. Here, two pieces of information are used as an example. For example, the similarity of the filtered reference vectors of the first piece is 93.86%, and the hyperlink 542 of the corresponding reference document is CD-05172 (e.g., a file name or document name). The similarity of the filtered reference vectors of the second piece is 93.73%, and the hyperlink 544 of the corresponding reference document is EF-12905.

[0062] When there are multiple reference answers, in one embodiment, the results are sorted from largest to smallest by similarity value so that the search results seen by the user are arranged according to similarity.

[0063] In one embodiment, when a user wishes to read the first piece of information, they can click hyperlink 542. Then, a reference document (not shown) with a 93.86% similarity to the key information will be displayed at the bottom of the user interface 500. Thus, the user can directly view the document content. Furthermore, if the user wishes to read the second piece of information, they can click hyperlink 544 to read the content of a second similar reference document. In another embodiment, when the user clicks hyperlinks 542 or 544, the content of the reference document can be displayed in a separate pop-up window.

[0064] The processor 410 sets file links (not shown) in the reference files, which redirect to other related reference files. In one embodiment, a first file link of a first reference file stored in a first database redirects to a second reference file stored in a second database, and a second file link of the second reference file redirects to a third reference file stored in a third database, wherein the first database is different from the second database, and the second database may be the same as or different from the first or third database.

[0065] For example, if a user clicks hyperlink 542, they will view the first reference file linked by hyperlink 542 in the user interface 500. This first reference file is stored in the first database. Since the first reference file contains a file link, the user can further decide whether to click to view the linked reference file based on the text of the file link. If the user clicks the file link of the first reference file, this file link will redirect to the second reference file stored in the second database. And so on, the user can initially obtain reference files through the hyperlinks provided in the user interface 500, and then further refer to and read all related files step by step from the file links within the reference files.

[0066] It is worth mentioning that this case does not limit the number of file links for each reference document.

[0067] Accordingly, the contents of multiple databases 435a to 435n are linked through file links to enable cross-database interactive reference, allowing reference files between different databases to be associated with each other and enabling cross-database integrated query of reference files.

[0068] Figure 6 is a block diagram illustrating a rapid document retrieval system according to another embodiment of this invention. Figure 6 illustrates the operation of establishing an answer database 430 and a vector database 440 before the rapid document retrieval system can provide users with questions.

[0069] Because there are numerous categories of existing standards and guidelines, the content, layout, and even the configuration of table header columns (e.g., whether the header is in the first column or the first row on the left) can vary. Users need to manually organize many complex standard documents (e.g., copying, pasting, and rearranging the content to obtain a new document), which is quite time-consuming.

[0070] To provide users with a good document retrieval experience, the processor 410 executes a document extraction program 620 on a plurality of original documents 610 to automatically generate reference documents for subsequent information provision. The original documents 610 may be standard documents.

[0071] Under initial conditions, the suggested answer dataset 430 has not yet been built into any database. The processor 410 executes a file extraction program 620 on a plurality of original files 610 to build the suggested answer dataset 430.

[0072] In one embodiment, the document extraction procedure 620 includes normalizing the table content of a plurality of original documents 610 to the same header column (e.g., unifying the header column of all tables in the original documents to the first column), extracting a plurality of key content from the plurality of original documents 610 (e.g., extracting relevant information based on specified content), or converting portions of the plurality of original documents 610 (e.g., converting specific pages of the original documents from PDF to Word or Excel) to generate a plurality of reference documents. In this way, the unstructured original documents 610 are automatically processed into a structured format (e.g., CSV, Excel, JSON, or SQL database format) to facilitate subsequent database queries and management, reducing the time spent on manual processing.

[0073] It is worth mentioning that databases 435a to 435n have their own data structures and design purposes. Therefore, processor 410 stores the generated reference files in one of the databases 435a to 435n according to the way the reference files are generated.

[0074] In one embodiment, processor 410 performs semantic parsing on a plurality of original files 610 to generate a plurality of key content items, which are stored together in a suggested answer dataset 430. Next, processor 410 executes a semantic vector transpose procedure 630 on the plurality of key content items, vectorizing all text content in the suggested answer dataset 430 to obtain a plurality of reference vectors, which are then stored in a vector database 440. The semantic vector transpose procedure 630 may be a text embedding technique used in natural language processing.

[0075] In summary, the rapid document retrieval system and method presented in this case provide a two-stage matching operation. First, it extracts key information from the query, then compares the text with the suggested answer dataset to quickly provide a perfectly matching result. This saves time spent matching vector values ​​and vector databases throughout the process, allowing for faster responses to user questions and significantly reduced search response time. Furthermore, if a perfect match is not found, it can still provide information that the user might need through fuzzy vector comparison, optimizing the probability of obtaining the best answer based on similarity.

[0076] Furthermore, when searching for files in different databases, users typically need to search each database individually, which is quite time-consuming. In contrast, the rapid file retrieval technology in this case can perform a file extraction process on the original file to obtain structured reference files, and set file links in the reference files to establish integrated and related queries across databases, eliminating the problem of the inability to quickly search across databases.

[0077] The above description is merely a specific example of this case and does not limit the scope of the patent application in this case. Therefore, all equivalent changes made using the content of this case are similarly included within the scope of this case and are hereby stated.

[0078] 110, 410: Processor 120, 420: Storage media 122, 422: User Interface 124, 424: Natural Language Module 130, 430: Suggested Answer Data Set 300, 500: User Interface 310, 510: Ask questions 320, 520: Suggested answer 530: Ask a question 540: Reference Answer 542, 544: Hyperlinks 435a~435n: Database 437a~437m: Reference documents 610: Multiple original files 620: Document Extraction Program 630: Semantic Vector Transpose Procedure S210~S230, S260~S280: Steps

Claims

1. A fast document retrieval method, comprising: A user interface receives a question. The system uses a natural language processing module to parse the question and obtain key information related to the question. A first matching phase is performed to search a dataset of suggested answers for content that perfectly matches the key information. If the dataset contains content that perfectly matches the key information, a suggested answer matching the key information is displayed on the user interface as the search result for the question. If the dataset does not contain content that perfectly matches the key information, a second matching phase is performed to search the dataset for content that matches the key information, wherein the second matching phase is a fuzzy search.

2. The fast document retrieval method as described in claim 1, wherein the fuzzy retrieval includes: The key information of the problem is converted into a problem vector; a plurality of similarities are calculated between the problem vector and a plurality of reference vectors in a vector database; at least one reference vector with a similarity greater than a threshold is selected from the plurality of similarities; and a reference answer corresponding to the at least one reference vector is output to the user interface, wherein the reference answer includes a hyperlink to a reference file related to the reference answer.

3. The fast document retrieval method as described in claim 2, wherein the plurality of reference documents includes a first reference document and a second reference document, and the plurality of databases associated with the suggested answer dataset include a first database and a second database, the fast document retrieval method further comprising: The user interface responds to the hyperlink related to the reference answer and directs the hyperlink to the first reference file stored in the first database; and directs the user to the second reference file stored in the second database via a file link of the first reference file, wherein the first database is different from the second database.

4. The fast document retrieval method as described in claim 2, wherein the step of calculating a plurality of similarities between the question vector and the plurality of reference vectors includes: Calculate a distance value between the problem vector and each of the reference vectors, and use the distance value as the similarity between the problem vector and each of the multiple reference vectors.

5. The fast document retrieval method as described in claim 2, wherein the step of outputting the reference answer corresponding to the at least one reference vector in the user interface includes: The reference answer is sorted from largest to smallest based on the multiple similarities and then output to the user interface.

6. The fast document retrieval method as described in claim 2, wherein the steps prior to receiving the question at the user interface include: A document extraction procedure is performed on a plurality of original documents to build a suggested answer dataset, wherein the suggested answer dataset is associated with a plurality of databases; wherein the document extraction procedure includes normalizing the table content of the plurality of original documents to the same header columns, extracting a plurality of key content from the plurality of original documents, or transcoding portions of the plurality of original documents to generate the plurality of reference documents, and storing each of the reference documents in one of the plurality of databases.

7. The fast document retrieval method as described in claim 2, wherein the steps prior to receiving the question at the user interface include: Vectorize the contents of the suggested answer data set to obtain the complex number of reference vectors.

8. A fast document retrieval system, comprising: A collection of suggested answers; A storage medium that stores multiple pieces of code; A processor, coupled to the suggested answer dataset and the storage medium, and after the plurality of code entries are loaded, performs the following operations: receiving a question at a user interface; parsing the question using a natural language module and obtaining key information related to the question; performing a first matching phase to search the suggested answer dataset for content that exactly matches the key information; when the suggested answer dataset contains content that exactly matches the key information, responding to the user interface with a suggested answer that matches the key information as search result information for the question; and when the suggested answer dataset does not contain content that exactly matches the key information, performing a second matching phase to search the suggested answer dataset for content that matches the key information, wherein the second matching phase is a fuzzy search.

9. The fast document retrieval system as claimed in claim 8, wherein the processor is configured to perform the following operations to execute the fuzzy search: converting the question focus information into a question vector; calculating a plurality of similarities between the question vector and a plurality of reference vectors in a vector database; selecting at least one reference vector from the plurality of similarities that is greater than a threshold; and outputting a reference answer corresponding to each of the at least one reference vector to the user interface, wherein the reference answer includes a hyperlink leading to a reference document related to the reference answer.

10. The fast document retrieval system as claimed in claim 9, wherein the plurality of reference documents includes a first reference document and a second reference document, and the plurality of databases associated with the suggested answer dataset include a first database and a second database, wherein the processor performs the following operations: responding to the hyperlink associated with the reference answer through the user interface and directing the hyperlink to the first reference document stored in the first database; and directing the user to the second reference document stored in the second database through a file link of the first reference document, wherein the first database is different from the second database.

Images