Knowledge search system and program

Abstract

This system provides a knowledge search system capable of outputting more appropriate answers to user questions. [Solution] In the knowledge retrieval system 10, the data processing means 23 first divides document data into semantic unit data using table of contents data. Next, if the semantic unit data exceeds the upper limit of characters U, it is divided into fragment data so that it is within the upper limit of characters U, and overlapping portions are provided between preceding and succeeding fragment data. Then, the semantic unit data within the upper limit of characters U and the divided fragment data are registered as knowledge in the knowledge database 41. After that, the search means 24 retrieves knowledge corresponding to the user's question, and the answer generation means 25 sends the knowledge to a large-scale language model (LLM) to obtain an answer.

Description

Technical Field

[0001] The present invention relates to a knowledge search system and a program configured by a computer that answers user questions. For example, it can be used when constructing a knowledge database using document data such as business manuals and accepting user questions for searching.

Background Art

[0002] Conventionally, a large number of knowledge (knowledge, cases, experiences, or other information used in information processing including search processing) have been created using in-house business manuals (for example, document data stored in various files such as PDF, Word, Excel, etc.), and these knowledge have been registered in a knowledge database and utilized for information processing such as searching. A knowledge search system has been constructed.

[0003] As one form of such a knowledge search system, for a service providing system (for example, GPT-4, etc.) that performs information processing services using a large language model (LLM), by sending one or more pieces of knowledge together with an LLM instruction sentence, there is a system for obtaining an answer to a question. In this case, since the amount of data that can be passed to the large language model (LLM) is limited, the number of characters of the knowledge is limited.

[0004] Therefore, a large number of knowledge with limited characters are created, registered in a knowledge database, and utilized for information processing such as searching. Conventionally, document data (text) such as business manuals has been divided starting from the head position at a previously specified number of characters, and the divided text has been registered in the knowledge database as knowledge.

[0005] Furthermore, regarding literature related to semantic chunking (a technique that divides document data into meaningful chunks to enable pinpoint hits during searches), there is a known system that uses machine learning, artificial intelligence, and other computer implementations to identify various semantically important chunks within a document, automatically labels them with appropriate data types and semantic roles, and uses this enhanced information to assist authors and support downstream processes (see Patent Document 1). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Special Publication No. 2022-547750 (Summary) [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] As mentioned above, in conventional methods of building knowledge databases, knowledge is created by dividing document data (text), such as business manuals, into sections based on a predetermined number of characters, starting from the beginning of the document. Therefore, dividing document data only requires counting the number of characters, and it has the advantage of being able to create a large number of knowledge items regardless of the document format (document structure, document format, and appearance).

[0008] However, because the document data is divided and split from the beginning at a predetermined number of characters, there is a possibility that the information stored in the knowledge database, i.e., the knowledge, may be cut off in the middle of a sentence. Here, "sentence" refers to a continuous phrase representing a cohesive piece of content from one period to the next, while "text" refers to a collection of sentences representing a cohesive piece of content. Therefore, it is not that the conventional method resulted in the text being cut off in the middle of a "sentence," but rather that there is a possibility of the text being cut off in the middle of a series of sentences.

[0009] Furthermore, when a sentence is split in the middle, the surrounding context is lost, resulting in incomplete knowledge.

[0010] Furthermore, there is a possibility that appropriate knowledge may not be obtained in response to user questions, resulting in incomplete information being passed to the Large-Scale Language Model (LLM). This could lead to the LLM failing to provide appropriate answers and thus not contributing to improved work efficiency.

[0011] The objective of the present invention is to provide a knowledge retrieval system and program that can output more appropriate answers to user questions. [Means for solving the problem]

[0012] The present invention relates to a knowledge retrieval system comprising a computer that provides answers to user questions, Document data for business manuals or other documents is divided into meaningful units using the table of contents data contained in the document data. It is then determined whether each individual meaningful unit data obtained as a meaningful block is within a predetermined upper limit of characters. If there is meaningful unit data that exceeds the upper limit of characters, the meaningful unit data exceeding the upper limit is further divided into smaller fragment data from the beginning of the meaningful unit data, regardless of meaning, so that it is within the upper limit of characters, and in this process, overlapping portions are provided between preceding and succeeding fragment data. A knowledge database that stores each of the fragment data, within the upper limit of the number of characters, created by dividing the data using this data processing means, and the semantic unit data, within the upper limit of the number of characters, as knowledge representing knowledge, examples, experience, or other information used in information processing, including search processing. A search means that accepts user question data input from a user terminal, sends a query containing this question data to a database search system via a network that performs a search using a knowledge database, and receives a specified number of or predetermined number of knowledge items obtained from the search process, which are sent from the database search system via the network as a response to the query. This search means transmits the knowledge and predetermined instruction text received, along with the response creation request data, to a service provision system using a large-scale language model via the network. It also receives the response data output from the large-scale language model transmitted from the service provision system via the network, and displays the received response data on the user terminal screen. It is characterized by having the following features.

[0013] Here, in this application, including the wording in the claims, the descriptions "within the upper limit of characters" and "exceeding the upper limit of characters" are considered equivalent to the descriptions "less than the upper limit of characters" and "at or above the upper limit of characters," and the expressions "within or less than the upper limit of characters" and "exceeding or exceeding the upper limit of characters" are not used. This is because, for example, "within 2000 characters" is the same as "less than 2001 characters," and "exceeding 2000 characters" is the same as "2001 characters or more."

[0014] In the knowledge retrieval system of the present invention, first, the document data is divided into semantic unit data using table of contents data. This reduces the possibility of text being truncated compared to conventional methods, which divide the document data from the beginning at predetermined specified intervals, regardless of meaning.

[0015] Next, in this invention, when semantic unit data exceeds a predetermined upper limit of characters, the semantic unit data is divided into fragment data from its starting position, regardless of meaning, so that it remains within the upper limit of characters. However, overlapping portions are provided between preceding and succeeding fragment data, minimizing the negative effects of fragmentation caused by a single semantically connected sentence being divided into multiple fragment data. In other words, since the same text is placed in the overlapping portions, the probability that important terms are included in the preceding and succeeding fragment data increases. This increases the likelihood that the preceding and succeeding fragment data (knowledge) that were separated to stay within the upper limit of characters will be retrieved together in a database search system, thus reducing the possibility that the information transmitted to the service provision system using a large-scale language model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service provision system using a large-scale language model (LLM), improving the efficiency of operations, and thus achieving the aforementioned objectives.

[0016] <When table of contents data is not included, the document structure is extracted using a document structure extraction system (AI) and then divided into semantic unit data.>

[0017] Furthermore, in the knowledge search system described above, The data processing method is If the document data does not include a table of contents, it is desirable to use a document structure extraction system built using machine learning to extract the document structure of the document data, and then divide the document data according to the extracted document structure to create semantic unit data.

[0018] In this configuration, where a document structure extraction system (AI) is used to extract the document structure and divide it into semantic units when table of contents data is not included, system administrators can divide the document data into semantic units without much effort, thus simplifying the knowledge database construction process.

[0019] <When adding up the total number of characters of the table data for one table and the number of characters of the fragment data exceeds the upper limit number of characters, the table data is arranged from the start position of the next fragment data.>

[0020] Furthermore, in the above-mentioned knowledge search system, the data processing means when a table is inserted into a document, executes a process of extracting the text part included in this table as table data, when the table data is included in the semantic unit data, it is configured to determine whether the semantic unit data is within the upper limit number of characters including the number of characters of this table data, Furthermore, when the semantic unit data including the table data exceeds the upper limit number of characters, the semantic unit data is divided into fragment data so as to be within the upper limit number of characters from the start position of the semantic unit data. At this time, when adding up the total number of characters of the table data for one table and the number of characters of the fragment data exceeds the upper limit number of characters, it is desirable that the table data is arranged from the start position of the next fragment data.

[0021] When adding up the total number of characters of the table data for one table in this way and the number of characters of the fragment data exceeds the upper limit number of characters and the table data is arranged from the start position of the next fragment data, it becomes possible to avoid the situation where the table data for one table is arranged across a plurality of fragment data. Therefore, in the search by the database search system, it becomes possible to avoid the situation where only a part of the table data for one table is acquired as knowledge, so the possibility that the information transmitted to the service provision system by the large language model (LLM) becomes incomplete can be kept low. For this reason, a more appropriate answer can be obtained from the service provision system by the large language model (LLM), and it becomes possible to improve the efficiency of the business.

[0022] <When the total number of characters in the table data exceeds the upper limit number of characters, create fragmented data so that it is within the upper limit number of characters while adding the number of characters in the row data, and provide an overlapping part between the front and back fragmented data in units of row data.>

[0023] Also, when adding the total number of characters in the table data for one table as described above and the number of characters in the fragmented data exceeds the upper limit number of characters, in the case of a configuration where the table data is arranged from the head position of the next fragmented data, the data processing means When the total number of characters in the table data exceeds the upper limit number of characters, divide the entire table data for one table into row data consisting of the text parts of the rows constituting the table, and sequentially add the number of characters in each row data starting from the first row data in one table, thereby creating fragmented data so that it is within the upper limit number of characters. At this time, it is desirable that a configuration is provided in which an overlapping part is provided between the front and back fragmented data in units of row data.

[0024] When the total number of characters in table data exceeds the upper limit, fragment data is created by adding the number of characters in row data to keep it within the upper limit. By treating row data as a unit and creating overlapping portions between preceding and succeeding fragment data, row data can be treated like "sentences" that make up a document, allowing a single table to be divided and fragment data to be created, and overlapping portions to be created. Therefore, it is possible to avoid situations where a single row data is divided and cut off midway, and since the overlapping portion is also a collection of row data (however, this does not exclude the possibility that the overlapping portion will consist of only one row data), even if the table data for a single table is divided and arranged into preceding and succeeding fragment data (knowledge), the likelihood of both being retrieved by a database search system increases, and the possibility of incomplete information being sent to a service delivery system using a Large-Scale Language Model (LLM) can be reduced. As a result, more appropriate answers can be obtained from the service delivery system using a Large-Scale Language Model (LLM), making it possible to improve the efficiency of business operations.

[0025] <A configuration where the overlapping area for tables is larger than the overlapping area for text.>

[0026] Furthermore, when the total number of characters in the table data exceeds the upper limit, fragment data is created by adding the number of characters in the row data to keep it within the upper limit, and the configuration is such that there is overlap between preceding and succeeding fragment data, with the row data as the unit. The data processing method is When creating overlapping portions in table data, It is desirable that the structure be such that the ratio of the number of characters in the overlapping portion for tables to the maximum number of characters is greater than the ratio of the number of characters in the overlapping portion for text.

[0027] By configuring the overlap portion for tables to be larger than the overlap portion for text, it is possible to minimize the negative impact on information transmission to users caused by the information being divided into preceding and succeeding fragmented data (knowledge). In other words, because tables are designed to allow for comparative reference of information, the connections between each piece of table data within a single table are stronger than those between individual sentences that make up a normal text. That is, since table data is information arranged in an orderly manner for comparative reference, the absence of any part of it has a greater negative impact on information transmission than when a normal text is interrupted. Therefore, when table data for a single table is divided into preceding and succeeding fragmented data (knowledge), it is desirable to avoid a situation where only a portion of that knowledge is retrieved when searched by a database search system. By avoiding this, it is possible to reduce the possibility that the information transmitted to the service delivery system using the Large-Scale Language Model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service delivery system using the Large-Scale Language Model (LLM), making it possible to improve the efficiency of business operations.

[0028] <When the total number of characters in the OCR-readable data for a single figure exceeds the character limit for the fragment data, the read data is placed starting from the beginning of the next fragment.>

[0029] Furthermore, when the total number of characters in the table data for a single table exceeds the maximum number of characters in the fragment data, if the table data is arranged starting from the beginning of the next fragment data, The data processing method is If a figure is inserted into the document, OCR processing is performed to read the text portion contained in the figure. If the semantic unit data includes read data of the text portion obtained through OCR processing, the system is configured to determine whether the semantic unit data is within the upper limit of characters, including the number of characters in this read data. Furthermore, if the semantic unit data, including the read data, exceeds the upper limit of characters, it is desirable that the semantic unit data be divided into fragment data so that the number of characters does not exceed the upper limit, starting from the beginning of the semantic unit data. In this case, if the sum of the total number of characters in the read data for one figure exceeds the upper limit of characters in the fragment data, the read data is then placed starting from the beginning of the next fragment data.

[0030] When the total number of characters in the OCR-read data for a single figure is added up in this way, if the number of characters in the fragment data exceeds the upper limit, the same effect and benefits as with tabular data can be obtained by configuring the data so that the read data is placed from the beginning of the next fragment. In other words, it becomes possible to avoid the read data for a single figure being placed across multiple fragment data. Therefore, it becomes possible to avoid a situation where only a part of the read data for a single figure is retrieved as knowledge when searching by a database search system, thus reducing the possibility that the information sent to the service provision system using a Large-Scale Language Model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service provision system using a Large-Scale Language Model (LLM), making it possible to improve the efficiency of operations.

[0031] <Invention of a program>

[0032] Furthermore, the program of the present invention is intended to make a computer function as the knowledge retrieval system described above.

[0033] Furthermore, the above program or any part thereof can be recorded and stored or distributed on recording media such as magneto-optical disks (MO), compact discs (CD), digital versatile disks (DVD), flexible disks (FD), magnetic tape, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM), flash memory, random access memory (RAM), hard disk drives (HDD), solid state drives (SSD), and flash disks. It can also be transmitted using transmission media such as wired networks like local area networks (LAN), metropolitan area networks (MAN), wide area networks (WAN), the Internet, intranets, extranets, wireless communication networks, or combinations thereof, and can also be transmitted on carrier waves. Moreover, the above program may be part of another program, or may be recorded on a recording media together with a separate program. [Effects of the Invention]

[0034] As described above, according to the present invention, document data is divided into semantic unit data, and when the semantic unit data exceeds the upper limit of characters, it is further divided into fragment data while providing overlapping portions. This minimizes the negative effects of fragmentation caused by a single semantically connected sentence being divided and placed into multiple fragment data, and reduces the possibility that the information (knowledge) passed to the service provision system using a large-scale language model (LLM) will be incomplete. As a result, it is possible to obtain more appropriate answers from the service provision system using a large-scale language model (LLM). [Brief explanation of the drawing]

[0035] [Figure 1] An overall configuration diagram of a knowledge search system according to one embodiment of the present invention. [Figure 2] A flowchart illustrating the knowledge search flow of the above embodiment. [Figure 3] A flowchart illustrating the data processing flow of the above embodiment. [Figure 4] The document data of the above embodiment is an explanatory diagram of the division. [Figure 5] A diagram illustrating the partitioning process of semantic unit data, including tabular data, according to the above embodiment. [Figure 6] This diagram illustrates the splitting process when the total number of characters in the table data of the above embodiment exceeds the upper limit of characters U. [Figure 7] A diagram illustrating the configuration of the knowledge database according to the above embodiment. [Figure 8] A diagram showing an example of the inquiry screen of the above embodiment. [Figure 9] A diagram illustrating the response creation request data of the above embodiment. [Figure 10] A figure showing an example of the response screen of the above embodiment. [Figure 11] This figure shows the results of an experiment to confirm the effectiveness of the above embodiment. [Modes for carrying out the invention]

[0036] One embodiment of the present invention will be described below with reference to the drawings. Figure 1 shows the overall configuration of the knowledge search system 10 of this embodiment. Figure 2 shows the flow of knowledge search, and Figure 3 shows the flow of data processing as flowcharts. Figure 4 is an explanatory diagram of document data splitting, Figure 5 is an explanatory diagram of the splitting process of semantic unit data including tabular data, and Figure 6 is an explanatory diagram of the splitting process when the total number of characters in tabular data exceeds the upper limit of characters U. Figure 7 shows the configuration of the knowledge database 41, Figure 8 shows an example of the inquiry screen 100, Figure 9 shows the format of the response creation request data, and Figure 10 shows an example of the response screen 200. Furthermore, Figure 11 shows the results of the effectiveness verification experiment.

[0037] <Overall configuration of Knowledge Search System 10>

[0038] In Figure 1, the knowledge search system 10 includes a knowledge search server 20 composed of one or more computers. The knowledge search server 20 is connected via network 1 to a database search system 40, a service provision system 50 using large language models (LLM), a document structure extraction system 51 built using machine learning, a service provision system 52 that performs vectorization processing, and a service provision system 53 that performs data conversion processing and OCR (Optical Character Recognition) processing. Furthermore, the knowledge search server 20 is connected via network 1 to an administrator terminal 60 operated by the system administrator and a user terminal 70 operated by the user (questioner).

[0039] Here, Network 1 is an external network primarily composed of the Internet, but it may also be a combination of the Internet and internal networks such as LANs or intranets, and it does not matter whether it is wired, wireless, or a hybrid of wired and wireless; in short, it is sufficient if it can transmit information at a reasonable speed between multiple locations (regardless of distance).

[0040] Furthermore, the database search system 40 is an external service provision system configured by a computer, which receives queries containing user question data via network 1, retrieves one or more (a number specified by the user, or a number predetermined by the system; in this embodiment, the user specifies between 1 and 10) pieces of knowledge corresponding to (similar to) the received question data from the knowledge database 41 (see Figure 7), and executes a process to return all retrieved knowledge to the source of the query via network 1 as a response to the query. In this embodiment, Azure AI Search is used as the database search system 40, but it is not limited to this; in short, the database search system 40 can be any system that can retrieve knowledge corresponding to (similar to) the user's question data using the knowledge database 41 prepared by the system administrator.

[0041] Furthermore, the service provision system 50 using a Large-Scale Language Model (LLM) is a computer system that provides cloud API (Application Programming Interface) services. In this embodiment, the Chat Generative Pre-trained Transformer (ChatGPT, Azure OpenAI ChatCompletion API GPT-4) is used, but it is not limited to this. For example, OpenAI's GPT-3.5, Google's Palm and Palm2, Amazon Web Services (AWS)'s Titan, Meta Platforms' Llama, etc. may also be used.

[0042] Furthermore, the document structure extraction system 51 is an external service provision system via a cloud API connected to network 1, and is a trained AI system composed of a computer that extracts document structure from document data. In this embodiment, it is used to obtain document structure data equivalent to table of contents data for document data that does not contain table of contents data. In this embodiment, the Azure Document Intelligence service is used, but it is not limited to this. The document structure extraction system 51 used in this embodiment may create an index indicating the document structure when document data is input (sent via API) and output (reply) document data with the created index attached, or it may output (reply) multiple semantic unit data obtained by semantically chunking the document data according to the document structure.

[0043] Furthermore, the service provision system 52 is an external service provision system via a cloud API connected through network 1, and is a computer system that performs vectorization processing of text data. In this embodiment, as an example, the Azure OpenAI Embeddings API service provision system (Azure OpenAI Embeddings API text-embedding-ada-002 version2) can be used, but is not limited to this, and the embedding method may be Doc2Vec, BERT, Transformer, etc. The vector has a fixed length, but the number of dimensions is arbitrary.

[0044] Furthermore, the service provision system 53 is an external service provision system connected via network 1 using a cloud API, and is a computer system that performs processes such as converting data from various files like Word, Excel, and PDF into HTML data, and OCR processing (including AI-OCR) to convert image data (including PDF image data) into text data.

[0045] Furthermore, the administrator terminal 60 and user terminal 70 are composed of computers and are equipped with display means such as an LCD display and input means 81 such as a mouse or keyboard. These administrator terminal 60 and user terminal 70 may also be portable devices such as laptop computers or smartphones.

[0046] <Configuration of Knowledge Search Server 20>

[0047] The knowledge search server 20 performs various processes related to knowledge search and stores data necessary for these processes. It is composed of one or more computers and includes a document storage means 21, a setting means 22, a data processing means 23, a search means 24, a response generation means 25, a document storage means 31, and a setting information storage means 32.

[0048] Here, the document storage means 21, setting means 22, data processing means 23, search means 24, and answer generation means 25 are implemented by a central processing unit (CPU) located inside the knowledge search server 20, one or more programs that define the operating procedures of this CPU, and working memory such as main memory and cache memory. Details of each of these means 21 to 25 will be described later.

[0049] Furthermore, non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD) can be used as the document storage means 31 and the setting information storage means 32.

[0050] <Knowledge search server 20 / Document storage means 21>

[0051] The document storage means 21 receives document data (such as Word, Excel, and PDF files) such as business manuals transmitted from the administrator terminal 60 via the network 1 by the system administrator, and executes a process to store them in the document storage means 31.

[0052] <Knowledge search server 20 / Configuration method 22>

[0053] The configuration means 22 receives various configuration information transmitted from the administrator terminal 60 via the network 1 by the system administrator and executes a process to store it in the configuration information storage means 32.

[0054] Here, the configuration information includes the upper limit of characters U (e.g., U = 2000 characters), the ratio of characters for the overlapping portion of text (excluding figures and tables) to the upper limit of characters U P1 (e.g., P1 = 25%), the ratio of characters for the overlapping portion of tables to the upper limit of characters U P2 (e.g., P2 = 50%), and the ratio of characters for the overlapping portion of figures to the upper limit of characters U P3 (e.g., P3 = 50%). The upper limit of characters U = 2000 characters was determined from the statistics of the text. The ratios for tables P2 and figures P3 are set to be larger than the ratio for text P1. Note that this configuration information (U, P1, P2, P3) may also be written in the program.

[0055] <Knowledge search server 20 / Data processing means 23>

[0056] The data processing means 23 performs data processing to create knowledge from document data stored in the document storage means 31 using the setting information (U, P1, P2, P3) stored in the setting information storage means 32, and then registers the created knowledge in the knowledge database 41 of the database search system 40 via the network 1. This data processing includes data conversion, acquisition of summary, acquisition of table of contents data, extraction of document structure when table of contents data is not available, division into semantic unit data (semantic chunk division), section analysis (acquisition of titles, acquisition of information on the presence or absence of figures and tables), creation of fragment data, vectorization, and registration in the knowledge database 41.

[0057] The program used to implement this data processing method 23 was written in Python. While external services were used for data conversion, document structure extraction (when table of contents data is unavailable), and vectorization, other processing was implemented using proprietary program code. Therefore, it is not provided by AWS / Microsoft / Google Cloud.

[0058] More specifically, the data processing means 23 first performs data conversion processing on document data. Specifically, it converts files such as Word, Excel, and PDF into HTML data. In this process, an existing external service, the service provision system 53, can be used.

[0059] Furthermore, the data processing means 23 obtains summary data from the document data. If the document does not contain any section that corresponds to a summary (summary, abstract, gist, etc.), the acquisition of a summary may be omitted.

[0060] Furthermore, the data processing means 23 obtains table of contents data from the document data (table of contents data with page numbers in the case of a Word document, and bookmark data in the case of a PDF document). If there is no table of contents data in the document data, the document structure extraction system 51 extracts the document structure from the document data.

[0061] (Partitioning into semantic units = semantic chunking: Figure 4) The data processing means 23 then divides the document data into meaningful units using table of contents data obtained from the document data, or according to the document structure extracted from the document data by the document structure extraction system 51, thereby creating multiple semantic unit data as meaningful chunks from a single document data. This division into semantic unit data is a process known as semantic chunking, and this process itself is based on a conventional technological concept of creating information that can be used for processes such as searching. Therefore, while it is preferable from the standpoint of reducing the workload of the system administrator that the process of dividing a single document data into multiple semantic unit data be performed automatically using table of contents data or according to the document structure extracted by the document structure extraction system 51, it is also acceptable to include some manual work by the system administrator rather than performing all processes automatically. Even if manual work is included, the objective of so-called semantic chunking can be achieved as long as the division into multiple semantic unit data is realized. In other words, even if the preparation of knowledge is time-consuming, if the created knowledge is in an appropriate state, the performance of knowledge search (search accuracy, accuracy of answers to questions) will be high.

[0062] Specifically, as shown in Figure 4, if the table of contents data is, for example, "1.AAAA...page XX", "2.BBBB...page XX", "3.CCCC...page XX", ... the document data is divided accordingly, and semantic unit data M(1), M(2), M(3), ... are created. These semantic unit data M(1), M(2), M(3), ... also include the text data of the titles used in the table of contents ("1.AAAA", "2.BBBB", "3.CCCC", ...).

[0063] In the example in Figure 4, the table of contents data is in a single-line structure, so the data is divided into semantic units according to this single-line structure. However, if the table of contents data is in a two-line or three-line structure, and pages are assigned to these two-line or three-line structures, the data may be divided into semantic units at the finest level of the table of contents data according to the two-line or three-line structure, or even if the table of contents data is in a two-line or three-line structure, it may be divided into a single-line structure. For example, if the table of contents data is in a two-line structure, such as "1-1 YYYYY...Page XX" and "1-2 YYYYY...Page XX," the data may be divided into semantic units according to this two-line structure, or it may be divided into a single-line structure (i.e., a division that uses only the first line, such as "1.AAAA...Page XX"). Furthermore, even if the table of contents data is in a three-line structure, it may be divided into a two-line structure, in which case it should be predetermined that only the first two lines of the table of contents data will be used, and the third line will not be used.

[0064] When there is no table of contents data in the document data, and the document structure is extracted by the document structure extraction system 51, the document structure is automatically recognized (extracted) as titles such as "1.AAAA", "2.BBBB", "3.CCCC", etc., and the text data (sets of text data) associated with each of these titles, as shown by the dotted lines in Figure 4. In a single document data, there are multiple combinations of recognized titles and the text data associated with those titles. If these multiple combinations are output from the document structure extraction system 51 in a divided state, each of these combinations is used as a semantic unit data. Furthermore, if an index is assigned to each combination of titles and the text data associated with those titles, and the document data with these indexes is output from the document structure extraction system 51, the document data is divided according to the index to create multiple semantic unit data.

[0065] (Creation of fragmented data: Figure 4) Next, the data processing means 23 determines whether each of the multiple semantic unit data obtained by division is within the upper limit of the number of characters U (for example, 2000 characters) stored in the setting information storage means 32. If there is semantic unit data that exceeds the upper limit of the number of characters U, the semantic unit data that exceeds the upper limit of the number of characters U is divided into even smaller fragment data by dividing it from the beginning of the semantic unit data so that it is within the upper limit of the number of characters U regardless of its meaning. At this time, an overlap portion is provided between preceding and succeeding fragment data. For example, if the upper limit of the number of characters U = 2000 characters and the percentage of characters in the overlap portion for text P1 = 25%, then the number of characters in the overlap portion will be 2000 characters × 25% = 500 characters.

[0066] In the example in Figure 4, the semantic unit data M(1) exceeds the upper limit of characters U (e.g., 2000 characters). Therefore, this semantic unit data M(1) is divided at the upper limit of characters U from the beginning. However, dividing at the upper limit of characters U does not mean dividing a single sentence data (a string separated by a period) in the middle, but rather dividing the sentence data as a unit (the smallest unit for counting to divide) so that it is within the upper limit of characters U. Note that if it is a bulleted list or something similar and there is no period, a new line is used. Other counting units, such as tags, may be used in conjunction.

[0067] Specifically, the number of characters in the first sentence data S(1) = Sn(1), the number of characters in the second sentence data S(2) = Sn(2), and so on are added (counted) in order. When the number of characters exceeds the upper limit U, that sentence data is not placed in the fragment data D(1) that is being created (counted), but is placed in the next fragment data D(2). In the example in Figure 4, when the number of characters in the eighth sentence data S(8) = Sn(8) is added, the upper limit U has not been exceeded. However, when the number of characters in the ninth sentence data S(9) = Sn(9) is added, the upper limit U is exceeded, so sentence data S(1) to S(8) become fragment data D(1). To give a more concrete example, if we add up the number of characters in the sentence data, for example, ..., and the total becomes 1824 characters, 1882 characters, 1930 characters, and 2015 characters, the last 2015 characters exceed the upper limit of U=2000 characters, so the sentence data up to the point where it reaches 1930 characters becomes the fragment data being created.

[0068] When creating an overlap portion for text (for example, a percentage P1 = 25%, U × P1% = 2000 characters × 25% = 500 characters), the number of characters is counted using the sentence data as the unit. When counting the amount of overlap, the number of characters is added in reverse order, starting from the bottommost sentence data that makes up the fragment data D(1). In the example in Figure 4, the bottommost sentence data of fragment data D(1) is the 8th sentence data S(8), so the number of characters for the 8th sentence data S(8) = Sn(8), the number of characters for the 7th sentence data S(7) = Sn(7), the number of characters for the 6th sentence data S(6) = Sn(6), ... are added in this order. Up to Sn(7) the number of characters was within 500, but when Sn(6) is added the number of characters exceeds 500, the overlap portion is considered to extend up to the 7th sentence data S(7). Therefore, in the example in Figure 4, sentence data S(7) and S(8) constitute the overlapping portion of the preceding and succeeding fragment data D(1) and D(2). Note that if Sn(6) is added, the total exceeds 500 characters, but the overlapping portion can be considered up to the sixth sentence data S(6) at which point it exceeds 500 characters. In other words, sentence data S(6), S(7), and S(8) can be considered the overlapping portion of the preceding and succeeding fragment data D(1) and D(2). This is because, even if the overlapping portion exceeds 500 characters, it does not exceed the upper limit of characters U, and therefore no problems arise. Thus, it is acceptable to consider it as approximately 500 characters rather than within 500 characters.

[0069] When it is determined that the overlapping portion of preceding and succeeding fragment data D(1) and D(2) is sentence data S(7) and S(8), the creation process of the next fragment data D(2) begins with sentence data S(7). That is, the 7th sentence data S(7) is placed at the beginning of fragment data D(2). Then, just as in the creation process of fragment data D(1), the number of characters for the 7th sentence data S(7) = Sn(7), the number of characters for the 8th sentence data S(8) = Sn(8), the number of characters for the 9th sentence data S(9) = Sn(9), and so on are added in order. When the upper limit of characters U is exceeded, that sentence data is not placed in the fragment data D(2) that is currently being created (counted), but is instead placed in the next fragment data D(3). In the example in Figure 4, when the number of characters for the 14th sentence data S(14) = Sn(14) is added, the upper limit of characters U is not exceeded. However, when the number of characters for the 15th sentence data S(15) = Sn(15) is added, the upper limit of characters U is exceeded, so sentence data S(7) to S(14) become fragment data D(2).

[0070] When creating an overlapping portion between preceding and succeeding fragment data D(2) and D(3), the number of characters is added in reverse order, starting from the lowest sentence data that makes up fragment data D(2). In the example in Figure 4, the lowest sentence data of fragment data D(2) is the 14th sentence data S(14), so the number of characters for the 14th sentence data S(14) = Sn(14), the number of characters for the 13th sentence data S(13) = Sn(13), the number of characters for the 12th sentence data S(12) = Sn(12), the number of characters for the 11th sentence data S(11) = Sn(11), ... are added in this order. Up to Sn(12) the number of characters is within 500, but when Sn(11) is added, the number of characters exceeds 500, so the overlapping portion is made up to the 12th sentence data S(12). Therefore, in the example in Figure 4, the sentence data S(12), S(13), and S(14) represent the overlapping portions of the preceding and succeeding fragment data D(2) and D(3).

[0071] When it is determined that the overlapping portion of the preceding and succeeding fragment data D(2) and D(3) is the sentence data S(12), S(13), and S(14), the process of creating the next fragment data D(3) starts from sentence data S(12). That is, the 12th sentence data S(12) is placed at the beginning of fragment data D(3).

[0072] (Placement of tabular data into fragmented data: Figure 5) Furthermore, if a table is inserted in the document, the data processing means 23 performs a process to extract each of the text portions contained in the table as tabular data. If the semantic unit data contains tabular data, it determines whether the semantic unit data is within the upper limit of characters U (e.g., 2000 characters), including the number of characters in the tabular data. If it is determined that the semantic unit data is within the upper limit of characters U (e.g., 2000 characters), including the number of characters in the tabular data, fragment data is not created, and the semantic unit data is treated as knowledge.

[0073] On the other hand, if it is determined that the semantic unit data, including table data, exceeds the upper limit of characters U, the semantic unit data is divided into fragments so that the number of characters from the beginning of the semantic unit data does not exceed the upper limit of characters U. In this case, if adding the total number of characters of the table data for one table results in the number of characters of the fragment data being created (counted) exceeding the upper limit of characters U, the table data is placed from the beginning of the next fragment data.

[0074] More specifically, in the example in Figure 5, the semantic unit data M(5) exceeds the upper limit of characters U (for example, 2000 characters), so it is subject to splitting. Therefore, this semantic unit data M(5) is split starting from the beginning, using the upper limit of characters U. In the example in Figure 5, the table data contained in the semantic unit data M(5) is located in the middle of the semantic unit data M(5), and the beginning of the semantic unit data M(5) is normal text data other than figures and tables. Therefore, first, starting from the beginning of the semantic unit data M(5), the text data is split so that it is within the upper limit of characters U, using the text data as the unit (the smallest unit for counting the number of characters to split). The number of characters in the first sentence data S(21) = Sn(21), the second sentence data S(22) = Sn(22), the third sentence data S(23) = Sn(23), the fourth sentence data S(24) = Sn(24), and so on are added (counted) in order. When the number of characters exceeds the upper limit U, that sentence data is not placed in the fragment data D(6) that is currently being created (counted), but is placed in the next fragment data D(7). In the example in Figure 5, at the stage when the number of characters in the third sentence data S(23) = Sn(23) is added, the upper limit U has not been exceeded, but when the number of characters in the fourth sentence data S(24) = Sn(24) is added, the upper limit U has been exceeded, so sentence data S(21) to S(23) become fragment data D(6).

[0075] Then, if the overlapping portion of the preceding and succeeding fragment data D(6) and D(7) is determined to be sentence data S(23) by a ratio P1 (for example, 25%), the creation process of the next fragment data D(7) starts with sentence data S(23). In other words, the third sentence data S(23) is placed at the beginning of the next fragment data D(7).

[0076] Similarly, when sentence data S(23) to S(25) become fragment data D(7), and the overlapping portion of the preceding and succeeding fragment data D(7) and D(8) is determined to be sentence data S(25), the process of creating the next fragment data D(8) begins with sentence data S(25). That is, sentence data S(25) is placed at the beginning of fragment data D(8).

[0077] Therefore, in creating fragment data D(8), the number of characters is added (counted) in the order of Sn(25) for sentence data S(25) and Sn(26) for sentence data S(26). Let's assume that even after adding up to Sn(26), it is still within the upper limit of characters U. However, since there is a table after sentence data S(26), the total number of characters of the table data for one table is added to determine whether it is within the upper limit of characters U. In other words, the table data for one table is considered as a single block (the whole set), and the total number of characters of the table data for one table is added first to determine whether it can be placed in the fragment data D(8) currently being created (counted) without breaking up that block.

[0078] Here, the total number of characters in the table data refers to the sum of the number of characters in the text portion of each element (each column in each row) that makes up a single table. In the example in Figure 5, this is the sum of the number of characters for table data T(1,1) = Tn(1,1), the number of characters for table data T(1,2) = Tn(1,2), the number of characters for table data T(1,3) = Tn(1,3), the number of characters for table data T(2,1) = Tn(2,1), ..., the number of characters for table data T(4,3) = Tn(4,3), or further, the sum of the number of characters for table data T(S), which is the row data for the title etc. at the top of the table, and the number of characters for table data T(E), which is the row data for the title etc. at the bottom of the table, Tn(E). Note that table data T(x,y) refers to the text data written in the element at row x and column y of the table.

[0079] Then, as shown by the dotted line in Figure 5, if the total number of characters in the table data for a single table is added (counted), and it is determined that the number of characters in the fragment data D(8) being created (counted) exceeds the upper limit of characters U, then it is not possible to place all of the table data for a single table into the fragment data D(8). If one tries to force it into the fragment data D(8), the table data for a single table would have to be split and placed into the preceding and succeeding fragment data D(8) and D(9). Therefore, in this case, in order to avoid splitting a single table, the table data is not placed into the fragment data D(8) being created (counted), but rather placed starting from the beginning of the next fragment data D(9).

[0080] When arranging table data starting from the beginning of the following fragment data D(9), if the total number of characters in the table data is within the upper limit of character count U, it is possible to place the text data S(27) and subsequent text data that follow the table into the fragment data D(9). The case where the total number of characters in the table data exceeds the upper limit of character count U will be explained later in Figure 6.

[0081] Therefore, the total number of characters in the table data is sequentially added (counted) to the number of characters in sentence data S(27) = Sn(27), the number of characters in sentence data S(28) = Sn(28), the number of characters in sentence data S(29) = Sn(29), and so on. If the upper limit of characters U is exceeded when Sn(29) is added, then sentence data S(29) is not placed in the fragment data D(9) that is currently being created (counted), but is placed in the next fragment data D(10).

[0082] Then, if the overlapping portion of the preceding and succeeding fragment data D(9) and D(10) for the text is determined to be sentence data S(28) by a ratio P1 (for example, 25%), then the creation process of the next fragment data D(10) starts with sentence data S(28). That is, sentence data S(28) is placed at the beginning of fragment data D(10).

[0083] (Table data splitting: Figure 6) If the total number of characters in the table data exceeds the upper limit of characters U (for example, 2000 characters), it is not possible to place all of the table data in a single fragment data. Therefore, the entire table data for a single table must be divided and placed into multiple fragment data. In this case, as in the example in Figure 5 above, if the table data is not placed in the fragment data being created (counted), but rather placed from the beginning of the next fragment data, it will be divided and placed into multiple fragment data from the next fragment data onward. Note that when we abbreviate "dividing table data" or "dividing table data," it does not mean dividing the individual text data within each element that makes up the table. Rather, if a table consists of, for example, 30 elements in a 6x5 grid, it means dividing those 30 elements into, for example, 10 elements and 20 elements, and does not mean dividing the text data within the element in the 2nd row, 3rd column, for example.

[0084] In the example in Figure 6, row data is defined as the set of table data (text data) that constitute a row within a single table. Therefore, for the first row, row data L(1) is the sum of table data T(1,1), T(1,2), and T(1,3), and its number of characters is Tn(1,1)+Tn(1,2)+Tn(1,3)=Ln(1). Similarly, for the second row, row data L(2) is the sum of table data T(2,1), T(2,2), and T(2,3), and its number of characters is Tn(2,1)+Tn(2,2)+Tn(2,3)=Ln(2). The same applies to the third to sixth rows.

[0085] In splitting table data, row data is used as the unit (the smallest unit for counting the number of characters to separate), and the number of characters is added (counted) sequentially starting from the first position (the first row of data). Fragment data is created by dividing the entire table data (set) for a single table so that it does not exceed the upper limit of characters U. In other words, the number of characters in the row data is added (counted) sequentially starting from the first row of data, and when the upper limit of characters U is exceeded, that row data is not placed in the fragment data being created (counted), but is placed in the next fragment data.

[0086] In the example in Figure 6, the number of characters Ln(1) for the first row data L(1), Ln(2) for the second row data L(2), Ln(3) for the third row data L(3), and so on are added (counted). Up to the addition of character count Ln(4), the count remains within the upper limit of character count U. However, when adding character count Ln(5), if the upper limit of character count U is exceeded, the row data L(5) is not placed in the fragment data D(11) that is being created (counted), but is placed in the next fragment data D(12). Therefore, row data L(1) to L(4) become fragment data D(11).

[0087] Then, assuming that the overlap portion for the table of preceding and succeeding fragment data D(11) and D(12) is determined by ratio P2 (e.g., 50%) to be row data L(3) and L(4), the next fragment data D(12) will start with row data L(3). That is, row data L(3) will be placed at the beginning of the next fragment data D(12). Therefore, the amount of overlap for text is determined by counting the number of characters in the text data unit, while the amount of overlap for tables is determined by counting the number of characters in the row data unit. Note that the ratio of the number of characters in the overlap portion for tables P2 (e.g., 50%) is set to be larger than the ratio of the number of characters in the overlap portion for text P1 (e.g., 25%).

[0088] (Placement of read data into fragmented data, splitting of read data) Furthermore, if a figure is inserted in the document, the data processing means 23 performs OCR processing to read the text portion contained in the figure. If the semantic unit data includes the read data of the text portion obtained by the OCR processing, it determines whether the semantic unit data is within the upper limit of character count U, including the number of characters in this read data. If it is determined that the semantic unit data including the read data is within the upper limit of character count U (for example, 2000 characters), fragment data is not created, and that semantic unit data is treated as knowledge.

[0089] On the other hand, if the semantic unit data, including the read data, exceeds the upper limit of characters U, the semantic unit data is divided into fragment data so that the number of characters from the beginning of the semantic unit data is within the upper limit. Similar to the case of tabular data (see Figure 5), sentence data that makes up a normal sentence and read data read from the figure by OCR processing may be mixed. When adding (counting) the number of characters in the read data from the figure, there are often no periods, so the count is done on a word basis. However, there may be strings of characters separated by periods in the figure, in which case the number of characters may be added (counted) after separating by periods.

[0090] In this case, if adding the total number of characters in the read data for one figure (the sum of the number of characters in each piece of read data read from one figure) causes the number of characters in the fragment data being created (counted) to exceed the upper limit of characters U, the read data is placed starting from the beginning of the next fragment data. This is the same treatment as in the case of tabular data (see Figure 5), and it is a process that treats the entire read data for one figure as a single block and avoids the block being divided and placed into multiple fragment data.

[0091] Furthermore, if the total number of characters in the read data for a single figure exceeds the upper limit of characters U, the entire set of read data for that figure is divided and split into fragments so that it is within the upper limit of characters U. In this case, if the read data is not placed in the fragment data being created (counted), but rather placed from the beginning of the next fragment data (as in the example in Figure 5 for table data), it will be divided and placed into multiple fragment data from the next fragment data onward. This is the same treatment as in the case of table data (see Figure 6), and an overlap portion for the figure is provided in the preceding and succeeding fragment data. The percentage of characters in the overlap portion for the figure P3 (e.g., 50%) is set to be larger than the percentage of characters in the overlap portion for normal text other than figures and tables P1 (e.g., 25%), similar to the percentage of characters in the overlap portion for tables P2 (e.g., 50%). Note that the unit for counting the amount of overlap for figures may be words.

[0092] (Vectorization) Subsequently, the data processing means 23 performs a vectorization process using the service provision system 52 to create vectors for each semantic unit data and each fragment data obtained within the upper limit of characters U. The created vectors of semantic unit data and fragment data are stored in the knowledge database 41 of the database search system 40 along with the knowledge (semantic unit data and fragment data within the upper limit of characters U) and used in the search process by the database search system 40. That is, when the database search system 40 uses the similarity between vectors to retrieve the corresponding knowledge from the user's question data, it calculates the similarity (for example, cosine similarity) between the vector of the question data and the vector of each piece of knowledge. Therefore, if the vector of each piece of knowledge is calculated and registered in the knowledge database 41 along with the knowledge, the processing speed during knowledge searches will be increased. In this case, the user's question data also needs to be vectorized in the same way.

[0093] Furthermore, the method by which the database search system 40 obtains corresponding knowledge from user question data is not necessarily limited to using the similarity between vectors. In short, the database search system 40 only needs to be able to obtain knowledge corresponding to the user question data using the knowledge database 41 prepared by the system administrator.

[0094] (Registration to Knowledge Database 41, structure of Knowledge Database 41: Figure 7) The data processing means 23 then transmits the text data created as knowledge (meaning unit data or fragment data within the upper limit of characters U) to the database search system 40 via the network 1, along with the file name of the document data from which the knowledge was obtained, summary data of the document data from which the knowledge was obtained, title data indicating the content of the section to which the knowledge belongs, the page indicating the location of the knowledge within the document data, information on whether or not there are figures and tables in the knowledge, and vector data of the knowledge, and registers it in the knowledge database 41 (see Figure 7).

[0095] <Knowledge search server 20 / search method 24: Figure 8> The search means 24 receives user question data (including the number of knowledge acquisitions in this embodiment) transmitted from the user terminal 70 via the network 1, and transmits a query containing this question data to the database search system 40, which performs a search process using the knowledge database 41, via the network 1. The search means 24 also receives one or more (a number specified by the user, or a number predetermined by the system; in this embodiment, the user specifies between 1 and 10) pieces of knowledge obtained in the search process and information associated with those pieces of knowledge (such as reference file names, reference sections (titles), and information on the presence or absence of figures and tables, but not vectors) transmitted from the database search system 40 via the network 1 as a response to the query, and performs a process to pass all the received pieces of knowledge and the information associated with those pieces of knowledge to the answer generation means 25.

[0096] In this case, the search means 24 displays the inquiry screen 100 shown in Figure 8 on the screen of the user terminal 70. The user (questioner) makes the necessary inputs on this inquiry screen 100. In Figure 8, the inquiry screen 100 is provided with a reference data setting unit 110 for specifying the upper limit of the number of reference data items (number of knowledge acquisitions), a prompt setting unit 120 (user prompt setting unit 121 and system prompt setting unit 122), and a question input unit 130 for entering the inquiry content (question data).

[0097] The number of knowledge items to be retrieved specified in the reference data setting unit 110 is an upper limit. Therefore, even if you specify 3 items, if the corresponding knowledge items do not exist in the knowledge database 41, the number may be 2 or less, or even zero.

[0098] The prompt setting unit 120 displays the default LLM instruction statement, which is basically fixed data, but the user can also change it using the user prompt setting unit 121 or the system prompt setting unit 122.

[0099] The user (questioner) enters question data into the question input section 130, for example, "Please explain the procedure for establishing procedures." In addition to text input, voice input may also be enabled.

[0100] <Knowledge search server 20 / Answer generation means 25: Figures 9 and 10> The response generation means 25 transmits the response creation request data (see Figure 9), which includes all the knowledge and predetermined instructions received from the search means 24, to the service provision system 50 using a large-scale language model (LLM) via the network 1. It also receives the response data output from the large-scale language model (LLM) transmitted from the service provision system 50 via the network 1, transmits the display data of the received response data to the user terminal 70 via the network 1, and executes the process of displaying the response screen 200 shown in Figure 10 on the screen of the user terminal 70.

[0101] In this case, the response generation means 25 sends response creation request data, as shown in Figure 9, to the service provision system 50 using a large-scale language model (LLM). This response creation request data corresponds to the content displayed in the prompt setting unit 120 of the inquiry screen 100 shown in Figure 8, and although the LLM instruction text is basically fixed, as mentioned above, the user can also change it.

[0102] In Figure 9, the {query} and {contents} sections are dynamic parts. At the {query} position, the content of the inquiry (question data), such as "Please explain the procedure for establishing procedures," is inserted. At the {contents} position, all of the knowledge obtained by the search means 24 is inserted. If multiple pieces of knowledge are obtained, they are inserted in a listed format.

[0103] The answer screen 200 in Figure 10 includes a reference manual display unit 201 that displays the file name of the document data containing the knowledge used in the answer (for example, "ABC Procedure Creation Guidelines R060930.doc"), a reference section display unit 202 that displays the title of the section to which the knowledge used in the answer belongs (for example, "1.2 Flow of Procedure Revision"), and an answer display unit 203 that displays the answer data output from the Large-Scale Language Model (LLM) (for example, "Procedure revision is carried out in the following flow: 1. The necessity of procedure revision is...").

[0104] Furthermore, the answer screen 200 is equipped with a diagram / table presence information display unit 220 that indicates whether the referenced manual may contain diagrams or tables. The information displayed by this diagram / table presence / absence information display unit 220 is information received from the search means 24.

[0105] Furthermore, the response screen 200 is provided with a link section 220 to the manual, allowing users to refer to document data such as business manuals stored in the document storage means 31, that is, the original data of the knowledge used in the response. In the example in Figure 10, the three acquired knowledge items were created from the same document data, so the same file names are listed three times.

[0106] Furthermore, the answer screen 200 is equipped with a manual details display section 230. This manual details display section 230 includes display fields for the serial number of the acquired knowledge (in the example in Figure 10, there are three acquired knowledge items, so they are 0, 1, and 2), the page indicating the location of the acquired knowledge item within the document, the file name of the document data (business manual) containing the acquired knowledge item, an overview of the document data containing the acquired knowledge item, the title of the section to which the acquired knowledge item belongs, and the text data of the knowledge item. Clicking on any of the text data display fields for a piece of knowledge item displays an enlarged display section 240 that allows the user to view the entirety of that knowledge item (text data).

[0107] <Knowledge search flow: Figure 2>

[0108] In this embodiment, as shown in Figure 2, the knowledge search system 10 performs the knowledge search process (including the process of answering user questions) as follows.

[0109] In Figure 2, first, the system administrator operates the administrator terminal 60 and sends document data such as business manuals (files such as Word, Excel, and PDF) to the knowledge search server 20 via the network 1. The knowledge search server 20 receives the transmitted document data using the document storage means 21 and stores it in the document storage means 31 (step S1).

[0110] Furthermore, the system administrator operates the administrator terminal 60 and transmits various configuration information (maximum number of characters U, percentage of characters for overlapping portions for text, tables, and diagrams P1, P2, P3) to the knowledge search server 20 via the network 1. The knowledge search server 20 receives the transmitted configuration information using the configuration means 22 and stores it in the configuration information storage means 32 (step S1).

[0111] Next, the system administrator operates the administrator terminal 60 and sends an execution command to the knowledge search server 20 via the network 1 to create knowledge. Upon receiving the execution command from the administrator terminal 60, the knowledge search server 20 uses the data processing means 23 to perform data processing on the document data stored in the document storage means 31, using the setting information (U, P1, P2, P3) stored in the setting information storage means 32 to create knowledge. The created knowledge is then registered in the knowledge database 41 of the database search system 40 via the network 1 (step S2). This data processing includes data conversion, acquisition of an overview, acquisition of table of contents data, extraction of document structure when table of contents data is unavailable, division into semantic unit data (semantic chunk division), section analysis (acquisition of titles, acquisition of information on the presence or absence of figures and tables), creation of fragment data, and vectorization. However, the details of each of these processes have already been described in detail in the explanation of the configuration of the data processing means 23, so a detailed explanation is omitted here. Figure 3 further details the data processing flow.

[0112] Subsequently, the user (questioner) operates the user terminal 70, inputs the question data and the number of knowledge items to be retrieved on the inquiry screen 100 (see Figure 8), and sends them to the knowledge search server 20 via the network 1. The knowledge search server 20 receives the question data and the number of knowledge items to be retrieved from the user terminal 70 using the search means 24 (step S3).

[0113] Then, the search means 24 sends a query including the question data and the number of knowledge items to be retrieved to the database search system 40 via the network 1 (step S4). When the database search system 40 receives the query, it performs a search process using the knowledge database 41 and retrieves one or more (in this embodiment, between 1 and 10, the number of knowledge items to be retrieved specified by the user) knowledge items that correspond to (are similar to) the question data, and information related to those knowledge items (such as the file name of the document data, the title of the section, and information on whether or not there are figures and tables). This search process (knowledge acquisition process) may involve, for example, vectorizing the question data by the service provision system 52 (this vectorization may be performed by the search means 24 or by the database search system 40), calculating the similarity (e.g., cosine similarity) between the obtained vector of question data and the vector of each knowledge item stored in the knowledge database 41, and acquiring the knowledge items with high similarity.

[0114] Next, the search means 24 receives one or more (in this embodiment, between 1 and 10, the number of knowledge items to be retrieved specified by the user) of knowledge items obtained through the search process and information associated with those knowledge items (such as reference file names, reference sections (titles), and information on the presence or absence of figures and tables, but not vectors), which are sent from the database search system 40 via the network 1 as a response to the query, and passes all the received knowledge items and the information associated with those knowledge items to the answer generation means 25 (step S5).

[0115] Furthermore, the database search system 40 transmits the answer generation means 25, which includes all the knowledge and LLM instructions received from the search means 24, to the large-scale language model (LLM) service provision system 50 via the network 1 (step S6).

[0116] Then, the response generation means 25 receives the response data output from the large-scale language model (LLM) transmitted from the service provision system 50 via the network 1 (step S7).

[0117] Then, the response generation means 25 transmits the display data of the response data, which is the output of the large-scale language model (LLM) received from the service provision system 50, to the user terminal 70 via the network 1, and displays the response screen 200 shown in Figure 10 on the screen of the user terminal 70 (step S8).

[0118] <Data processing flow: Figure 3>

[0119] In Figure 3, the data processing means 23 performs data conversion and obtains an overview of the document data stored in the document storage means 31 (step S201). The details of these processes have already been described in detail in the explanation of the configuration of the data processing means 23, so a detailed explanation is omitted here.

[0120] Next, the data processing means 23 determines whether or not the document data contains table of contents data (step S202).

[0121] If table of contents data is included, the data processing means 23 retrieves the table of contents data from the document data (step S203). Based on the retrieved table of contents data, the document data is divided into semantic unit data (so-called semantic chunking), and the titles of the sections in the document and information on the presence or absence of figures and tables are retrieved (step S204). The division into semantic unit data has already been described in detail using Figure 4 in the explanation of the configuration of the data processing means 23, so a detailed explanation is omitted here.

[0122] On the other hand, if table of contents data is not included, the data processing means 23 extracts the document structure of the document data using the document structure extraction system 51, which is an AI system of an external service (step S205). According to the extracted document structure, the document data is divided into semantic unit data (so-called semantic chunking), and the titles of the sections in the document and information on the presence or absence of figures and tables are obtained (step S206). In this embodiment, the Azure Document Intelligence service is used as the document structure extraction system 51, but it is not limited to this.

[0123] Next, the data processing means 23 determines whether each semantic unit data obtained by the division is within the upper limit of the number of characters U (for example, 2000 characters) (step S207).

[0124] Here, if the semantic unit data is within the upper limit of character count U, fragment data is not created, and that semantic unit data is treated as knowledge.

[0125] On the other hand, if the semantic unit data exceeds the upper limit of characters U, the semantic unit data is divided into multiple fragments so that it is within the upper limit of characters U (step S208). The division into these fragments has already been described in detail using Figures 4 to 6 in the explanation of the configuration of the data processing means 23, so a detailed explanation will be omitted here, but the outline is as follows.

[0126] In other words, the data processing means 23 divides the fragment data so that the number of characters is within the upper limit of the number of characters U, and in doing so, it provides an overlapping portion (for example, a percentage P1 = 25% for text).

[0127] Furthermore, when arranging table data whose total character count is within the upper limit of U, if arranging all of the table data for a single table would cause the number of characters in the fragment data being created (counted) to exceed the upper limit of U, then all of the table data is placed at the beginning of the next fragment data to prevent the table data for a single table from being spread across multiple fragment data. In other words, the table data for a single table is not split.

[0128] Furthermore, when arranging table data where the total number of characters exceeds the upper limit of U, the data is divided into rows so that the number of characters in the fragment data is within the upper limit of U, and an overlap portion is provided (for example, a percentage P2 = 50% for the table).

[0129] Furthermore, if there is a figure, OCR processing is performed on the text portion within the figure. When arranging the read data obtained from the OCR processing into fragment data, the data for a single figure is arranged in the same way as in the case of table data, and when it is necessary to divide it, an overlap portion is provided (for example, a percentage P3 = 50%) for the figure.

[0130] Then, the data processing means 23 uses the external service provision system 52 to create vectors for each piece of knowledge (meaning unit data with a character count within the upper limit of character count U, and fragment data with a character count within the upper limit of character count U obtained by splitting) (step S209).

[0131] Subsequently, the data processing means 23 registers each piece of knowledge (meaning unit data with a character count within the upper limit of U characters, and fragment data with a character count within the upper limit of U characters obtained by splitting) into the knowledge database 41 (see Figure 7) along with information such as vectors, document file names, summaries, section titles, and whether or not there are figures and tables (step S210).

[0132] <Effects of this embodiment>

[0133] This embodiment offers the following advantages. Specifically, the knowledge retrieval system 10 uses the data processing means 23 to divide document data into semantic units (so-called semantic chunking), which reduces the possibility of text being truncated compared to conventional methods that divide document data from the beginning at predetermined specified intervals, regardless of meaning.

[0134] Furthermore, in the knowledge retrieval system 10, when semantic unit data exceeds the upper limit of characters U, the data processing means 23 divides it into fragment data so that it remains within the upper limit of characters U. However, overlapping portions are created between preceding and succeeding fragment data, minimizing the negative effects of fragmentation caused by a single semantically connected sentence being divided and placed across multiple fragment data. In other words, since the same text is placed in the overlapping portions, the probability that important terms are included in the preceding and succeeding fragment data increases. This increases the likelihood that both preceding and succeeding fragment data (knowledge) that were separated to stay within the upper limit of characters U will be retrieved during searches by the database retrieval system 40. As a result, the possibility of incomplete information being sent to the service provision system 50 using the large-scale language model (LLM) is reduced. Consequently, more appropriate answers can be obtained from the service provision system 50 using the large-scale language model (LLM), leading to improved operational efficiency.

[0135] More specifically, even if there is no overlap between preceding and succeeding fragment data, the database search system 40 will retrieve knowledge that corresponds to (is similar to) the user's (questioner's) question data, so reasonably appropriate knowledge can be passed from the search means 24 to the answer generation means 25. However, if the number of knowledge items to be retrieved specified by the user is, for example, three, the document data that is the source data for the three retrieved knowledge items may each be different (different files). In other words, even if there are two pieces of knowledge that were connected in the original document data, and it would be possible to create a more appropriate answer if both of those pieces of knowledge were retrieved, there is a possibility that knowledge from a different file will be retrieved. Therefore, if such a situation can be avoided and both pieces of knowledge that were connected in the original document data can be retrieved and passed from the search means 24 to the answer generation means 25, the large-scale language model (LLM) can output an even more appropriate answer. Therefore, by creating an overlapping section and strengthening the connection between the two pieces of knowledge that were linked in the original document data, the knowledge passed from the search means 24 to the answer generation means 25 is made more appropriate, and the output of the large-scale language model (LLM) is made more appropriate.

[0136] Furthermore, if the document data does not contain table of contents data, the data processing means 23 extracts the document structure using the AI-based document structure extraction system 51 and divides it into semantic unit data according to the extracted document structure. As a result, system administrators can divide the document data into semantic unit data without any extra effort, and can easily carry out the construction of the knowledge database 41.

[0137] Furthermore, if the data processing means 23, when adding up the total number of characters in the table data for a single table, exceeds the upper limit of characters U for the fragment data being created, it places the table data from the beginning of the next fragment data, thus avoiding the situation where the table data for a single table is placed across multiple fragment data. Consequently, it is possible to avoid a situation where only a portion of the table data for a single table is retrieved as knowledge by the database search system 40, thereby reducing the possibility that the information sent to the service provision system 50 using the Large-Scale Language Model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service provision system 50 using the Large-Scale Language Model (LLM), and the efficiency of operations can be improved.

[0138] Furthermore, when the total number of characters in the table data exceeds the upper limit of characters U, the data processing means 23 divides the entire table data into row data units so that it is within the upper limit of characters U, creating fragmented data, and at that time, it provides overlapping portions between preceding and succeeding fragmented data. Therefore, even if the table data for a single table is divided and arranged into preceding and succeeding fragmented data (knowledge), the likelihood of both being retrieved by the database search system 40 increases, and the possibility of incomplete information being sent to the service provision system 50 using the Large-Scale Language Model (LLM) is reduced. As a result, more appropriate answers can be obtained from the service provision system 50 using the Large-Scale Language Model (LLM), and the efficiency of operations can be improved.

[0139] Furthermore, in the knowledge search system 10, the ratio of characters in the overlapping portion for tables, P2 (for example, 50%), is set to be larger than the ratio of characters in the overlapping portion for text, P1 (for example, 25%), thereby minimizing the negative impact on information transmission to the user caused by the information being divided into preceding and succeeding fragment data (knowledge). In other words, in the case of tables, due to their characteristic of being designed for comparative reference of information, the connections between each piece of table data within a single table are stronger than those between individual sentences that make up a normal text. That is, since table data is information arranged in an orderly manner for comparative reference, the absence of any part of it has a greater negative impact on information transmission than when a normal text is interrupted. Therefore, when table data for a single table is divided and arranged into preceding and succeeding fragment data (knowledge), it is desirable to avoid a situation where only a portion of that knowledge is retrieved by searching the database search system 40. By avoiding this, the possibility of incomplete information being transmitted to the service provision system 50 using the large-scale language model (LLM) can be minimized. Therefore, more appropriate answers can be obtained from the service delivery system 50 using a large-scale language model (LLM), thereby improving the efficiency of operations.

[0140] Furthermore, if the total number of characters in the OCR-readable data for a single figure exceeds the upper limit character count U of the fragment data being created, the data processing means 23 will place the read data from the beginning of the next fragment data, thus achieving the same effect as with tabular data. In other words, it is possible to avoid the read data for a single figure being placed across multiple fragment data. Consequently, it is possible to avoid a situation where only a portion of the read data for a single figure is acquired as knowledge through a search by the database search system 40, thereby reducing the possibility that the information sent to the service provision system 50 using the Large-Scale Language Model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service provision system 50 using the Large-Scale Language Model (LLM), and the efficiency of operations can be improved.

[0141] Furthermore, when the total number of characters in the OCR-readable data for a single figure exceeds the upper limit of characters U, the data processing means 23 divides the entire readable data into fragmented data so that it is within the upper limit of characters U, and in doing so, it provides overlapping portions between the preceding and succeeding fragmented data, thus achieving the same effect as when dividing tabular data. In other words, even if the readable data for a single figure is divided into preceding and succeeding fragmented data (knowledge), the presence of overlapping portions increases the likelihood that both will be retrieved by the database search system 40, thus reducing the possibility that the information transmitted to the service provision system 50 using the Large-Scale Language Model (LLM) will be incomplete. As a result, more appropriate answers can be obtained from the service provision system 50 using the Large-Scale Language Model (LLM), improving the efficiency of operations.

[0142] Furthermore, in the knowledge search system 10, the ratio of characters in the overlapping portion for figures P3 (e.g., 50%) is set to be greater than the ratio of characters in the overlapping portion for text P1 (e.g., 25%). This minimizes the negative impact on information transmission to the user caused by the information being divided into preceding and succeeding fragments of data (knowledge), similar to the case of tables.

[0143] (Effect confirmation experiment: Figure 11) As shown in Figure 11, a comparative experiment was conducted between the conventional method and the data processing method of the present invention. Twenty knowledge searches were performed using 20 questions for comparison. Azure AI Search was used for the database search system 40. Azure OpenAI Service was used for the service provision system 50 using a large-scale language model (LLM).

[0144] In the conventional method, document data was divided into sections of 2000 characters from the beginning, regardless of meaning, and knowledge was created. Overlapping sections were not included in this process.

[0145] On the other hand, in the data processing method of the present invention, the data is first divided into semantic units, and if the semantic unit data exceeds the upper limit of 2000 characters (U), the data is further divided into fragments of 2000 characters or less, while providing an overlap portion of 25% (P1) for text, thereby creating knowledge.

[0146] The document acquisition rate for perspective (1) was determined by whether the correct document (file) was acquired, that is, whether the referenced manual was correct. Success was considered to have occurred if at least one of the three acquired knowledge items was the target document (a document predetermined to be acquired in response to the user's question).

[0147] The knowledge acquisition rate for perspective (2) was determined by whether the correct location (pre-defined correct knowledge) in the document (manual) was referenced, that is, whether the appropriate knowledge was acquired in response to the question.

[0148] The accuracy rate for perspective (3) was determined by whether the answer matched the questioner's (user's) intent, that is, whether an accurate answer was provided to the question.

[0149] According to comparative experiment results based on these evaluation criteria, the knowledge acquisition rate improved from 50% with the conventional method to 95% with the data processing method of the present invention. Furthermore, the accuracy rate of answers improved from 50% with the conventional method to 85% with the data processing method of the present invention. Therefore, the effectiveness of the present invention was clearly demonstrated.

[0150] <Form of transformation>

[0151] It should be noted that the present invention is not limited to the embodiments described above, and any modifications that can achieve the objectives of the present invention are included within the scope of the present invention.

[0152] For example, in the above embodiment, the document storage means 21, setting means 22, data processing means 23, document storage means 31, and setting information storage means 32 were provided on the knowledge search server 20, but these may be provided on the administrator terminal 60.

[0153] Furthermore, in the above embodiment, the search means 24 and the answer generation means 25 were provided on the knowledge search server 20, but they may also be provided on the user terminal 70. [Industrial applicability]

[0154] As described above, the knowledge retrieval system and program of the present invention are suitable for use in situations where a knowledge database is constructed using document data such as business manuals, and where user inquiries are received and searches are performed. [Explanation of Symbols]

[0155] 1 Network 10. Knowledge Search System 23 Data Processing Means 24 Search methods 25 Answer generation means 40 Database Search System 41 Knowledge Database 50. Service delivery systems using Large-Scale Language Models (LLMs) 51 Document Structure Extraction System 60 Administrator terminals 70 User terminals

Claims

1. A knowledge retrieval system comprising a computer that provides answers to user questions, Document data for business manuals or other documents is divided into meaningful units using the table of contents data contained in the document data, and it is determined whether each of the individual meaningful unit data obtained as a meaningful block is within a predetermined upper limit of characters. If there is meaningful unit data that exceeds the upper limit of characters, the meaningful unit data exceeding the upper limit of characters is further divided into smaller fragment data from the beginning of the meaningful unit data so that it is within the upper limit of characters regardless of meaning, and in this process, overlapping portions are provided between preceding and succeeding fragment data. A knowledge database that stores each of the fragment data, within the upper limit of the number of characters, created by dividing the data using this data processing means, and the semantic unit data, within the upper limit of the number of characters, as knowledge representing knowledge, examples, experience, or other information used in information processing including search processing, A search means that receives input of user question data from a user terminal, transmits a query containing this question data to a database search system via a network that performs a search process using the knowledge database, and receives a specified number or a predetermined number of knowledge items obtained by the search process from the database search system via the network as a response to the query. This system includes a search means that transmits the knowledge and predetermined instruction text received by this search means to a service provision system using a large-scale language model via a network, and a response generation means that receives response data output from the large-scale language model transmitted from the service provision system via the network and displays the received response data on the user terminal screen. The aforementioned data processing means is If a table is inserted in the aforementioned document, the process of extracting the text portion contained in this table as table data will be executed. If the aforementioned semantic unit data includes the aforementioned table data, the configuration is such that the number of characters in the table data is also taken into account to determine whether the aforementioned semantic unit data is within the aforementioned upper limit of characters. Furthermore, if the semantic unit data, including the table data, exceeds the upper limit of characters, the semantic unit data is divided into fragment data so that it is within the upper limit of characters from the beginning of the semantic unit data. In this case, if the sum of the total number of characters in the table data for one table exceeds the upper limit of characters in the fragment data, the table data is arranged from the beginning of the next fragment data. A knowledge search system characterized by the following:

2. The aforementioned data processing means is If the document data does not include the table of contents data, the system is configured to extract the document structure of the document data using a document structure extraction system built using machine learning, and then divide the document data according to the extracted document structure to create the semantic unit data. The knowledge search system according to feature 1.

3. The aforementioned data processing means is If the total number of characters in the table data exceeds the upper limit, the entire table data for one table is divided into row data consisting of the text portion of the rows that make up the table. Starting from the first row data in a table, the number of characters in each row data is added sequentially to create fragment data so that it is within the upper limit, and in this case, the overlap portion between preceding and succeeding fragment data is provided with the row data as the unit. The knowledge search system according to feature 1.

4. The aforementioned data processing means is If the aforementioned overlapping portion is provided for the aforementioned table data, The configuration is such that the ratio of the number of characters in the overlapping portion for the table to the upper limit of characters is greater than the ratio of the number of characters in the overlapping portion for the text. The knowledge search system according to feature 3.

5. The aforementioned data processing means is If a figure is inserted in the aforementioned document, OCR processing will be performed to read the text portion contained in the figure. If the semantic unit data includes the read data of the text portion obtained by the OCR process, the configuration determines whether the semantic unit data is within the upper limit of the number of characters, including the number of characters in this read data. Furthermore, if the semantic unit data, including the read data, exceeds the upper limit of characters, the semantic unit data is divided into fragment data so that it is within the upper limit of characters, starting from the beginning of the semantic unit data. In this case, if the sum of the total number of characters in the read data for one figure exceeds the upper limit of characters in the fragment data, the read data is then placed starting from the beginning of the next fragment data. The knowledge search system according to feature 1.

6. A program for causing a computer to function as a knowledge search system according to any one of claims 1 to 5.

Images