Knowledge search system and program

Abstract

This system provides a knowledge search system that can improve the accuracy of answers to user questions and enhance the versatility of knowledge database construction. [Solution] In the knowledge search system 10, the section creation means 24 divides document data into semantic sections by dividing it into headings, the clustering means 25 calculates the similarity between adjacent sections, and if the similarity is equal to or exceeds a first similarity threshold, adjacent sections are combined to create a cluster, and the overlap means 26 performs overlap processing within a range where the number of characters in the cluster is less than or equal to a character threshold if the similarity between the last section constituting the front cluster and the first section constituting the back cluster is equal to or exceeds a second similarity threshold, and registers it in the knowledge database 41.

Description

[[Technical Field]]

[0001] The present invention relates to a knowledge search system and program configured by a computer that answers a user's question using knowledge obtained by searching a knowledge database, and can be used, for example, when constructing a knowledge database using various types of document data such as regulations, procedures, and manuals. [[Background Art]]

[0002] In recent years, Retrieval-Augmented Generation (RAG), which combines a search function with Large Language Models (LLM), has been realized. In this RAG technology, knowledge as search information (knowledge, cases, experiences, or other information used for information processing including search processing) is stored in a knowledge database, and an answer to a user's question is generated by a large language model (LLM) using the knowledge obtained by searching the knowledge database.

[0003] Conventionally, in the construction of this knowledge database, document data (text data constituting a document) is divided by dividing it in page units or by a fixed number of characters, and the text data obtained by the division has been registered in the knowledge database as knowledge which is search information.

[0004] In addition, in order to divide knowledge into units that do not become fragmented, it has also been practiced to use table of contents information in a document, divide the document data such that each of the plurality of sentence data constituting the document data is not divided in units divided as table of contents entries, and register the divided data as knowledge in the knowledge database.

[0005] Furthermore, regarding literature related to semantic chunking (a technique that divides document data into meaningful chunks to enable pinpoint hits during searches), there is a known system that uses machine learning, artificial intelligence, and other computer implementations to identify various semantically important chunks within a document, automatically labels them with appropriate data types and semantic roles, and uses this enhanced information to assist authors and support downstream processes (see Patent Document 1). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Special Publication No. 2022-547750 (Summary) [Overview of the project] [Problems that the invention aims to solve]

[0007] When building a knowledge database, if document data is divided by page or by a certain number of characters, as in the conventional method described above, the information within the document will be registered as knowledge in the knowledge database in a fragmented state. Therefore, if it is necessary to refer to the fragmented portion, the Large-Scale Language Model (LLM) may not be able to obtain the information necessary to answer, potentially leading to lower accuracy of the answer. Thus, while it is versatile, it has the problem of being difficult to answer accurately.

[0008] In this explanation, "sentence" refers to a continuous phrase representing a cohesive whole from one period to the next, "document" refers to a series of sentences representing a cohesive whole, and "document" refers to a collection of multiple sentences. Therefore, unless document data is extremely short and contains only a single piece of content, it is text data that is intended to be divided. For example, procedural document data is a collection of sentence data at various hierarchical levels, such as application procedures, change procedures, and withdrawal procedures. Since it usually contains text data with different meanings, it is natural that it will be divided. On the other hand, except in cases where the text is forcibly divided on a page-by-page basis, even when it is divided at a fixed number of characters as in the past, it does not mean that the text is interrupted in the middle of a "sentence," but rather that there is a possibility that the text, which consists of a series of sentences, may be interrupted in the middle of a "document."

[0009] Furthermore, as in the conventional method described above, if the document data is divided into units that do not fragment knowledge, using the table of contents information within the document and dividing it into units defined by the table of contents, it is necessary to use the table of contents information within the document. Therefore, the format of the table of contents must be as expected, and it cannot be applied to groups of documents that have unexpected formats. Consequently, while it is easy to achieve high accuracy in the answers, it has the problem of lacking versatility.

[0010] The objective of the present invention is to provide a knowledge search system and program that can improve the accuracy of answers to user questions and enhance the versatility when constructing a knowledge database. [Means for solving the problem]

[0011] The present invention relates to a knowledge retrieval system comprising a computer that uses knowledge obtained by searching a knowledge database to answer user questions, A section creation method that divides the text data constituting a document into semantic sections by separating them with headings, A clustering means that calculates the similarity between adjacent sections among the sections created by this section creation means, and when the calculated similarity satisfies a first similarity condition, which is equal to or exceeds a first similarity threshold determined in advance or using multiple similarity values, the adjacent sections are joined to create a cluster. The clustering means includes an overlapping means that, for adjacent clusters created by this clustering means, if the similarity between the last section constituting the front cluster and the first section constituting the rear cluster satisfies a second similarity condition, which is equal to or exceeds a second similarity threshold calculated using a predetermined or multiple similarity criteria, then, in order to partially overlap the data contents of the front cluster and the rear cluster, the overlapping means performs an overlapping process that joins at least one section constituting the other cluster to at least one of the rear of the front cluster or the front of the rear cluster, within a range that satisfies a character count condition, which is equal to or less than a predetermined character count threshold, thereby partially overlapping the data contents of the front cluster and the rear cluster. The knowledge database is, The configuration stores the cluster after the overlapping process, created by the overlapping mechanism, as knowledge. It is characterized by the following:

[0012] Here, "semantic section" refers to a section that is divided into meaningful units according to the headings within the document, rather than a section created by dividing the document based on page counts or character counts, which are unrelated to the document's meaning.

[0013] Furthermore, "calculated and determined using predetermined or multiple similarity values" means that the first and second similarity thresholds may be predetermined values ​​in the system, or they may be values ​​calculated and determined using multiple similarity values ​​obtained when similarity values ​​between adjacent sections are obtained. In the former case, predetermined values ​​in the system may be values ​​written in the program, or values ​​stored in external memory may be read. In the latter case, where multiple similarity values ​​are used for calculation, for example, the mean and standard deviation of multiple similarity values ​​may be used (calculated and determined using a formula that includes the mean and standard deviation), or the mean of multiple similarity values ​​may be used (calculated and determined using a formula that includes the mean, but the formula does not include the standard deviation), and "multiple" means that all obtained similarity values ​​may be used, or some of the similarity values ​​(for example, multiple similarity values ​​excluding outliers) may be used.

[0014] Furthermore, in the "overlapping means," the "opposing cluster" refers to the rear cluster for the front cluster (when joining sections constituting the rear cluster behind the front cluster), and to the rear cluster (when joining sections constituting the front cluster in front of the rear cluster).

[0015] In the knowledge retrieval system of the present invention, a section creation means divides document data into semantic sections by separating them with headings, a clustering means calculates the similarity between adjacent sections, and if the calculated similarity satisfies a first similarity condition, adjacent sections are joined to create a cluster. Furthermore, an overlapping means performs an overlapping process on adjacent clusters, where, if the similarity between the last section constituting the front cluster and the first section constituting the rear cluster satisfies a second similarity condition, the overlapping process joins at least one section constituting the other cluster to at least one of the following locations: behind the front cluster or before the rear cluster, provided that the number of characters in the cluster satisfies the character count condition. The resulting cluster after the overlapping process is then registered as knowledge in the knowledge database.

[0016] Therefore, since the section creation means divides the document data into sections by dividing it with headings, sections of semantic units, that is, sections as meaningful chunks, are created. Then, the clustering means performs clustering on the resulting semantic sections, using the similarity between adjacent sections to combine sections with high similarity (sections that satisfy the first similarity condition of being equal to or exceeding the first similarity threshold). This makes it possible to reduce the possibility of a sentence being incomplete or to mitigate the negative effects of such incompleteness.

[0017] Furthermore, the overlapping mechanism provides overlapping portions (parts where data content overlaps) between adjacent clusters when the similarity between adjacent clusters is high, more precisely, when the similarity between the last section constituting the preceding cluster and the first section constituting the following cluster is high (when the second similarity condition, which is equal to or exceeds the second similarity threshold), thus minimizing the negative effects of fragmentation caused by a single semantically connected sentence being divided and placed across multiple fragment data (multiple clusters in this invention). In other words, since the same text is placed in the overlapping portions, the probability that important terms are included in the preceding and succeeding fragment data (in this invention, the preceding and succeeding clusters after overlapping) increases. This increases the likelihood that both the preceding and succeeding fragment data (knowledge, in this invention, the preceding and succeeding clusters after overlapping) that were separated due to the need to keep the number of characters in the clusters within a range that satisfies the character count condition (below or less than the character count threshold) will be retrieved in a database search system. Therefore, it is possible to reduce the possibility that the information transmitted to the service provision system using the Large-Scale Language Model (LLM) will be incomplete. Therefore, by using a service delivery system based on a Large-Scale Language Model (LLM), more appropriate answers can be obtained, improving the accuracy of responses to user questions.

[0018] Furthermore, since the section creation mechanism divides document data into sections by separating it with headings, it eliminates the inconvenience that could arise when dividing document data into semantic unit data using table of contents data, namely the inability to apply the table of contents format to document groups with unexpected formats. Consequently, it becomes possible to increase the versatility when constructing a knowledge database, thereby achieving the aforementioned objectives.

[0019] <Details of overlap processing>

[0020] Further more specifically, the overlapping processing executed by the overlapping means may be the following processing.

[0021] That is, in the aforementioned knowledge search system, as the overlapping processing, the overlapping means performs: a front cluster rear expansion process of joining, behind the rearmost end of a front cluster, at least one section that is adjacent to each other including a leading section constituting a rear cluster and satisfies a second similarity condition within a range where the number of characters of the front cluster satisfies a character number condition; a rear cluster front expansion process of joining, in front of the leading end of a rear cluster, at least one section that is adjacent to each other including a rearmost section constituting a front cluster and satisfies a second similarity condition within a range where the number of characters of the rear cluster satisfies a character number condition; and a parallel rear expansion and front expansion process in which the front cluster rear expansion process and the rear cluster front expansion process are performed in parallel, and when the rear cluster front expansion process is performed between a second cluster among three consecutive clusters and an adjacent first cluster, and the front cluster rear expansion process is performed between the second cluster and an adjacent third cluster, a section constituting the first cluster or the third cluster is joined to the front or rear of the second cluster within a range where the number of characters of the second cluster satisfies the character number condition and any one of the above processes can be executed.

[0022] Here, the phrase "execute rear cluster forward expansion processing with an adjacent first cluster" in "rear expansion·forward expansion parallel processing" means the following: in a situation where a first cluster and a second cluster are adjacent to each other, the second cluster is a rear cluster, while the first cluster is a front cluster, so the processing of merging the sections constituting the first cluster in front of the second cluster corresponds to rear cluster forward expansion processing. Furthermore, the phrase "execute front cluster rear expansion processing with an adjacent third cluster" means the following: in a situation where a second cluster and a third cluster are adjacent to each other, the second cluster is a front cluster, while the third cluster is a rear cluster, so the processing of merging the sections constituting the third cluster behind the second cluster corresponds to front cluster rear expansion processing.

[0023] <Configuration in which the clustering means adjusts the first similarity threshold when the number of characters of a cluster does not satisfy the character number condition>

[0024] Furthermore, in the knowledge search system described above, the clustering means, when the number of characters of a created cluster does not satisfy the character number condition, performs adjustment to gradually increase the first similarity threshold for said cluster until the character number condition is satisfied, and splits said cluster by unmerging adjacent sections that no longer satisfy the adjusted first similarity condition among the sections constituting said cluster, and it is desirable to adopt a configuration in which this processing is repeated.

[0025] In this configuration, when the number of characters in a cluster does not meet the character count condition, if the clustering means is configured to adjust the first similarity threshold, then in the first clustering operation using the first similarity threshold, if a cluster that does not meet the character count condition (less than or equal to the character count threshold) is generated, the first similarity threshold can be gradually increased as a condition that the similarity between the sections constituting the cluster must satisfy. By unbinding the connections between sections that no longer meet the first similarity condition, the cluster can be divided, reducing its size (number of characters) and bringing it within the range that satisfies the character count condition (less than or equal to the character count threshold).

[0026] <A configuration in which the first and second similarity thresholds are calculated and determined using the mean and standard deviation of the similarity scores.>

[0027] Furthermore, in the aforementioned knowledge search system, The first similarity threshold is a value determined by subtracting the standard deviation of similarity from the mean similarity. A configuration can be adopted in which the second similarity threshold is a value calculated by subtracting from the mean similarity value the value obtained by multiplying the standard deviation of similarity by a coefficient greater than 1.

[0028] By configuring the system in which the first and second similarity thresholds are calculated and determined using the mean and standard deviation of the similarity scores, it becomes easy to obtain similarity thresholds suitable for appropriate clustering and overlap processing.

[0029] <A structure that divides text data by gradually increasing the hierarchy level of headings until the section satisfies the token count requirement, using a section creation method.>

[0030] Furthermore, in the aforementioned knowledge search system, The clustering method is, The system is configured to perform a vectorization process on each section created by the section creation means, and when calculating the similarity between adjacent sections, it also performs a process to calculate the similarity between the section vectors. The means of creating a section is, The system can be configured to determine whether the number of tokens in the text data that are candidates for a section meets a token count condition, which is whether the number of tokens is below or less than a predetermined token count threshold that is the upper limit of the number of tokens when performing vectorization processing by the clustering means. If it does not meet the condition, the system can repeat the process of dividing the text data that are candidates for a section while gradually increasing the hierarchy level of the headings until the token count condition is met.

[0031] In this configuration, where the text data is divided by gradually increasing the hierarchy level of headings until the section satisfies the token count requirement, it becomes possible to create sections that satisfy the token count requirement when there is a limit (upper limit) on the number of tokens during the section vectorization process.

[0032] <Invention of a program>

[0033] Furthermore, the program of the present invention is intended to make a computer function as the knowledge retrieval system described above.

[0034] Furthermore, the above program or any part thereof can be recorded and stored or distributed on recording media such as magneto-optical disks (MO), compact discs (CD), digital versatile disks (DVD), flexible disks (FD), magnetic tape, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM), flash memory, random access memory (RAM), hard disk drives (HDD), solid state drives (SSD), and flash disks. It can also be transmitted using transmission media such as wired networks like local area networks (LAN), metropolitan area networks (MAN), wide area networks (WAN), the Internet, intranets, extranets, wireless communication networks, or combinations thereof, and can also be transmitted on carrier waves. Moreover, the above program may be part of another program, or may be recorded on a recording media together with a separate program. [Effects of the Invention]

[0035] As described above, according to the present invention, the section creation means divides document data into semantic sections by dividing it into headings, the clustering means calculates the similarity between adjacent sections, and if the first similarity condition is met, adjacent sections are joined to create a cluster, and further, the overlapping means performs overlapping processing of adjacent clusters if the similarity between the last section constituting the preceding cluster and the first section constituting the following cluster satisfies the second similarity condition, within the range where the number of characters in the cluster satisfies the character count condition, and the resulting cluster after the overlapping processing is registered as knowledge in the knowledge database, thereby improving the accuracy of answers to user questions and improving the versatility when constructing a knowledge database. [Brief explanation of the drawing]

[0036] [Figure 1] An overall configuration diagram of a knowledge search system according to one embodiment of the present invention. [Figure 2] A diagram illustrating the configuration of the knowledge database according to the above embodiment. [Figure 3] A flowchart illustrating the flow of the knowledge database construction process according to the above embodiment. [Figure 4] A flowchart illustrating the flow of the section creation process in the above embodiment. [Figure 5] A flowchart illustrating the clustering process flow of the above embodiment. [Figure 6] A flowchart illustrating the overlap processing flow of the above embodiment. [Figure 7] A flowchart illustrating the flow of the knowledge search process in the above embodiment. [Figure 8] An explanatory diagram illustrating the section creation process for the knowledge database construction process of the above embodiment. [Figure 9] An explanatory diagram illustrating the cluster creation process for the knowledge database construction process of the above embodiment. [Figure 10] An explanatory diagram of the overlapping process during the knowledge database construction process of the above embodiment. [Figure 11] An explanatory diagram illustrating the adjustment of the first similarity threshold in the above embodiment. [Figure 12] Diagram illustrating the rearward expansion process of the front cluster in the above embodiment. [Figure 13] This figure shows the results of an experiment to confirm the effectiveness of the above embodiment. [Figure 14] An explanatory diagram of the rear cluster forward expansion process, which is a first modified form of the present invention. [Figure 15] An explanatory diagram of the rearward expansion and forward expansion parallel processing, which is a second modified form of the present invention. [Modes for carrying out the invention]

[0037] One embodiment of the present invention will be described below with reference to the drawings. Figure 1 shows the overall configuration of the knowledge search system 10 of this embodiment. Figure 2 shows the configuration of the knowledge database 41, and Figure 3 shows the flow of the knowledge database 41 construction process as a flowchart. Furthermore, Figure 4 shows the flow of the section creation process, Figure 5 shows the flow of the clustering process, Figure 6 shows the flow of the overlap process, and Figure 7 shows the flow of the knowledge search process as flowcharts. In addition, Figures 8 to 10 are explanatory diagrams of the section creation, cluster creation, and overlap process using specific examples in the knowledge database 41 construction process, Figure 11 is an explanatory diagram of the adjustment of the first similarity threshold, and Figure 12 is an explanatory diagram of the front cluster back expansion process. Figure 13 shows the results of the effectiveness verification experiment.

[0038] <Overall configuration of Knowledge Search System 10>

[0039] In Figure 1, the knowledge search system 10 includes a knowledge search server 20 composed of one or more computers. The knowledge search server 20 is connected via network 1 to a database search system 40, a service provision system 50 using large language models (LLMs), a service provision system 51 that performs data conversion processing, OCR (Optical Character Recognition) processing, and document structure extraction processing, and a service provision system 52 that performs vectorization processing. Furthermore, the knowledge search server 20 is connected via network 1 to a creator terminal 60 operated by the creator of information related to documents to be registered in the knowledge database 41 (including the system administrator), and to a user terminal 70 operated by the user (the questioner).

[0040] Here, Network 1 is an external network primarily composed of the Internet, but it may also be a combination of the Internet and internal networks such as LANs or intranets, and it does not matter whether it is wired, wireless, or a hybrid of wired and wireless; in short, it is sufficient if it can transmit information at a reasonable speed between multiple locations (regardless of distance).

[0041] Furthermore, the database search system 40 is an external service provision system configured by a computer, which receives search queries containing user question data via network 1, retrieves one or more (a number specified by the user, or a number predetermined by the system, for example, between 1 and 10) pieces of knowledge corresponding to (similar to) the received question data from the knowledge database 41 (see Figure 2), and executes a process to return all retrieved knowledge to the source of the query via network 1 as a response to the query. In this embodiment, Azure AI Search is used as the database search system 40, but it is not limited to this, and in short, the database search system 40 can be any system that can retrieve knowledge corresponding to (similar to) the user's question data using the knowledge database 41 prepared by the creator (including the system administrator).

[0042] Furthermore, the database search system 40 is equipped with a database management system (DBMS). The knowledge database 41, as shown in Figure 2, stores associated information such as the file name of the document data, the section title (the first heading within the section), the knowledge (text data of the cluster after overlap processing), and the knowledge vector. This is implemented using non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD).

[0043] Furthermore, the service provision system 50 using a Large-Scale Language Model (LLM) is a computer system that provides cloud API (Application Programming Interface) services. In this embodiment, the Chat Generative Pre-trained Transformer (ChatGPT, Azure OpenAI ChatCompletion API GPT-4o) is used, but it is not limited to this. For example, OpenAI's GPT-3.5, Google's Palm and Palm2, Amazon Web Services (AWS)'s Titan, Meta Platforms' Llama, etc. may also be used.

[0044] Furthermore, the service provision system 51 is an external service provision system via a cloud API connected through network 1, and is a trained AI system (a system built using machine learning) composed of computers that perform data conversion processing (such as converting a WORD file to a PDF file), OCR processing (processing to read text data from PDF files, etc.), and document structure extraction processing (processing to extract the document structure from document data). In this embodiment, a PDF file is created from a WORD file, etc., and the document data (text data) obtained by performing OCR processing on the PDF file is used to extract the document structure and obtain heading patterns for each hierarchical level (specifically, headings marked with marks such as #, In this embodiment, we use a service called Azure AI Document Intelligence (hereinafter sometimes abbreviated as ADI), but we are not limited to this. In short, the service provision system 51 used in this embodiment can be any system that, when input (API transmission) document data (PDF file) is input, creates a heading pattern indicating the document structure (for example, "# Summary of internal regulations", "## Working hours", etc.) and outputs (replies) document data (text data) with the created heading pattern attached.

[0045] Furthermore, the service provision system 52 is an external service provision system via a cloud API connected through network 1, and is a computer system that performs vectorization processing of text data. In this embodiment, as an example, the Azure OpenAI Embeddings API service provision system (Azure OpenAI Embeddings API text-embedding-ada-002 version2) can be used, but is not limited to this, and the embedding method may be Doc2Vec, BERT, Transformer, etc. The vector has a fixed length, but the number of dimensions is arbitrary.

[0046] Furthermore, the creator terminal 60 and the user terminal 70 are composed of computers and are equipped with display means such as an LCD display and input means 81 such as a mouse or keyboard. These creator terminal 60 and user terminal 70 may be notebook personal computers or portable devices such as smartphones or tablets.

[0047] The creator terminal 60 is operated by the person who builds the knowledge search system 10, while the user terminal 70 is operated by the person who uses the knowledge search system 10. Therefore, the person who operates the creator terminal 60 includes not only the document creator themselves, but also, for example, a person in the same or related department who has the authority to register and use documents created by others (such as a supervisor or a member of the same research group), or a system administrator in the management department. In addition, the document creator or system administrator may also be a user on their own terminal, in which case the terminal is the user terminal 70, not the creator terminal 60.

[0048] <Configuration of Knowledge Search Server 20>

[0049] The knowledge search server 20 performs various processes related to knowledge search and stores data necessary for these processes. It is composed of one or more computers and includes a document storage means 21, a setting means 22, a text conversion means 23, a section creation means 24, a clustering means 25, an overlap means 26, a registration means 27, a search means 28, and a response generation means 29. It also includes a document storage means 31, a setting information storage means 32, a section storage means 33, a similarity storage means 34, a similarity threshold storage means 35, a cluster storage means 36, and a cluster storage means 37 after overlap processing.

[0050] Here, the document storage means 21, setting means 22, text conversion means 23, section creation means 24, clustering means 25, overlap means 26, registration means 27, search means 28, and answer generation means 29 are implemented by a central processing unit (CPU) located inside the knowledge search server 20, one or more programs that define the operating procedures of this CPU, and working memory such as main memory and cache memory. Details of each of these means 21 to 29 will be described later.

[0051] Furthermore, non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD) can be used as the document storage means 31 and the setting information storage means 32. In addition, the section storage means 33, similarity storage means 34, similarity threshold storage means 35, cluster storage means 36, and overlap-processed cluster storage means 37 may be main memory (volatile memory), but non-volatile memory may also be used. Details of these storage means 31 to 37 will be described later.

[0052] <Configuration of Knowledge Search Server 20 / Document Storage Means 21>

[0053] The document storage means 21 receives various document data (such as Word, Excel, and PDF files) such as regulations, procedures, and business manuals that are sent from the creator terminal 60 via the network 1 by the document creator (or system administrator), and executes the process of storing them in the document storage means 31.

[0054] <Configuration of Knowledge Search Server 20 / Configuration Method 22>

[0055] The configuration means 22 receives various configuration information transmitted from the creator terminal 60 via the network 1 by the system administrator and executes a process to store it in the configuration information storage means 32.

[0056] Here, the configuration information includes a character threshold (e.g., 2000 characters) indicating the upper limit of the number of characters, and a token threshold (e.g., 8192 tokens in the case of the vectorization processing service provision system 52 used in this embodiment) indicating the upper limit of the number of tokens. The character threshold (e.g., 2000 characters) was determined from the statistical values ​​of the text. In addition, a coefficient (e.g., 3 times in this embodiment) to be multiplied by the standard deviation when calculating and determining the second similarity threshold may be stored in the configuration information storage means 32 as configuration information. Note that this configuration information may also be written in the program. Furthermore, the number of tokens is the basic unit when processing text data, and is a division of a string into meaningful chunks. In English, it can be considered that one word is approximately one token, so usually the number of characters is greater than the number of tokens.

[0057] <Configuration of Knowledge Search Server 20 / Text Conversion Means 23: Figure 8>

[0058] The text conversion means 23 receives selection information for document data to be registered in the knowledge database 41 (see Figure 2) from the creator terminal 60 via network 1, provided by the creator (or system administrator). If the selected document data is not a PDF file, the service provision system 51 performs a conversion process to a PDF file. Specifically, it sends an API request for data conversion including a PDF file to the service provision system 51 via network 1, and receives a PDF file as response data sent from the service provision system 51 via network 1.

[0059] Furthermore, the text conversion means 23 performs OCR processing on the document data (PDF file) via the service provision system 51, analyzes and extracts the document structure from the obtained document data (text data), and performs a process to insert heading patterns at each hierarchical level (specifically, headings marked with marks such as #, ###, ##### as shown in Figure 8). That is, it sends an OCR API request containing the document data (PDF file) to the service provision system 51 via the network 1, and performs a process to receive document data (text data) from the service provision system 51 via the network 1 as response data, into which heading patterns indicating the document structure (for example, "# Summary of Company Regulations", "## Working Hours", "#### Dress Code", etc. as shown in Figure 8) have been inserted. In this embodiment, the hierarchical text data shown in Figure 8 is in Markdown format.

[0060] <Configuration of Knowledge Search Server 20 / Section Creation Means 24: Figures 8 and 4>

[0061] The section creation means 24 divides the document data (text data) obtained by the text conversion means 23 into semantic sections by dividing it according to heading patterns (for example, "# Summary of Company Regulations", "## Working Hours", "#### Dress Code", etc., as shown in Figure 8). In the example in Figure 8, sections SC(1) to SC(5), etc., are created. The section creation means 24 then stores the text data of each divided section in the section storage means 33 (see Figure 4), associating it with an automatically assigned section number (section identification information). The vector data of each section is created by the clustering means 25, as will be described later.

[0062] Furthermore, as shown in Figure 4, the section creation means 24 determines whether the number of tokens in the text data that will become a section candidate (hereinafter referred to as section candidate) satisfies the token count condition, which is that the number of tokens is less than or equal to a predetermined token count threshold that is the upper limit of the number of tokens when the clustering means 25 performs vectorization processing. If it does not satisfy the condition, it repeatedly performs the process of dividing the section candidate by gradually increasing the hierarchical level of the heading pattern until the token count condition is satisfied. Specifically, if the number of tokens in the section candidate separated by a heading pattern with one # does not satisfy the token count condition, it is separated by a heading pattern with two #s (a heading pattern at one deeper hierarchical level). If the token count condition is still not satisfied, it is separated by a heading pattern with three #s (a heading pattern at yet another deeper hierarchical level).

[0063] <Configuration of Knowledge Search Server 20 / Clustering Means 25: Figures 9, 5, and 11>

[0064] The clustering means 25 calculates the similarity between adjacent sections among the sections created by the section creation means 24, and executes a clustering process to combine adjacent sections and create a cluster if the calculated similarity satisfies a first similarity condition, which is equal to or exceeds a first similarity threshold determined in advance or using multiple similarity values. In this embodiment, the first similarity threshold is not a system setting value, but a value determined using multiple similarity values.

[0065] (Vectorization process) Specifically, the clustering means 25 first performs a vectorization process to create vector data from the text data of each section stored in the section storage means 33 (see Figure 4) by the service provision system 52. That is, it sends an API request containing the section's text data to the service provision system 52 via the network 1, and receives vector data as response data from the service provision system 52 via the network 1, and performs this process for all sections. In the example in Figure 9, vector data has been created for sections SC(1) to SC(5), etc. Then, the clustering means 25 stores the obtained vector data for each section in the section storage means 33 (see Figure 4), associating it with the section number (section identification information).

[0066] (Similarity calculation process: In this embodiment, cosine similarity calculation process) Furthermore, the clustering means 25 uses the vector data of each created section (vector data stored in the section storage means 33 (see Figure 4)) to calculate the similarity between adjacent sections (in this embodiment, the cosine similarity between the vectors of adjacent sections), and stores the calculated similarity in the similarity storage means 34 (see Figure 5) in association with the combination of section numbers (section identification information) of adjacent sections. In the example in Figure 9, the similarity between sections SC(1) and SC(2) (cosine similarity between vectors) is calculated as SIM(1,2)=0.7, the similarity between sections SC(2) and SC(3) (cosine similarity between vectors) is calculated as SIM(2,3)=0.4, the similarity between sections SC(3) and SC(4) (cosine similarity between vectors) is calculated as SIM(3,4)=0.3, and the similarity between sections SC(4) and SC(5) (cosine similarity between vectors) is calculated as SIM(4,5)=0.8.

[0067] Furthermore, the similarity between adjacent sections in this invention is not limited to the similarity between section vectors, nor is it limited to cosine similarity; for example, it may be the Levenshtein distance, the Jaro-Winkler distance, or the like.

[0068] (Calculation and determination process for the first and second similarity thresholds) Furthermore, the clustering means 25 uses all the calculated similarities (all the similarities stored in the similarity storage means 34 (see Figure 5)) to determine the mean and standard deviation of the similarities, uses the determined mean and standard deviation to calculate and determine the first and second similarity thresholds, and stores the calculated first and second similarity thresholds in the similarity threshold storage means 35.

[0069] Here, the first similarity threshold is a value determined by subtracting the standard deviation of similarity from the mean similarity. The second similarity threshold is a value determined by subtracting the value obtained by multiplying the standard deviation of similarity by a coefficient greater than 1 (in this embodiment, 3) from the mean similarity. Therefore, the following calculation formulas (1) and (2) are obtained.

[0070] First similarity threshold = mean - standard deviation ···(1)

[0071] Second similarity threshold = mean - 3 × standard deviation ... (2)

[0072] When calculating the mean and standard deviation, you may choose to exclude outliers before using all similarity values.

[0073] Furthermore, the first similarity threshold is used in the processing by the clustering means 25 and is therefore calculated and determined by the clustering means 25. However, the second similarity threshold is not used in the processing by the clustering means 25 but is used in the processing by the overlap means 26, so it may be calculated and determined by the overlap means 26 instead of the clustering means 25. Alternatively, the calculation and determination of the second similarity threshold here may be considered to have been performed in advance by the overlap means 26.

[0074] (Cluster creation process: Section merging) Next, the clustering means 25 performs a clustering process to create a cluster by joining adjacent sections if the similarity between adjacent sections (the similarity stored in the similarity storage means 34 (see Figure 5)) satisfies a first similarity condition, which is that the similarity between adjacent sections is equal to or exceeds a calculated first similarity threshold. In the example in Figure 9, SIM(1,2) and SIM(4,5), marked with black stars, satisfy the first similarity condition, so sections SC(1) and SC(2) are joined to create cluster C(1), and sections SC(4) and SC(5) are joined to create cluster C(3). On the other hand, SIM(2,3) and SIM(3,4) do not satisfy the first similarity condition, so section SC(3) is not joined with either section SC(2) or section SC(4). Therefore, section SC(3) constitutes cluster C(2), which is composed of a single section. Furthermore, if the similarities satisfying the first similarity condition are consecutive, that is, if SIM(k,k+1), SIM(k+1,k+2), SIM(k+2,k+3), SIM(k+3,k+4), ... satisfy the first similarity condition, they are combined to form a single cluster. The clustering means 25 then performs the process of storing the section number (section identification information) of each section constituting the created cluster in the cluster storage means 36 (see Figure 5), associating it with the automatically assigned cluster number (cluster identification information). In addition to the section number (section identification information) of each section, the text data of each combined section (cluster text data) may also be stored.

[0075] (Cluster splitting process: Sections are unjoined by adjusting the first similarity threshold) Furthermore, as shown in Figure 11, if the number of characters in a created cluster does not satisfy the character count condition of being less than or equal to a predetermined character count threshold, the clustering means 25 adjusts the first similarity threshold in stages until the character count condition is met (the adjustment range is arbitrary and may be the same or varied each time). It then repeatedly performs the process of splitting the cluster by unbinding adjacent sections that no longer satisfy the adjusted first similarity condition among the sections constituting the cluster. In the example in Figure 11, five sections SC(w1) to SC(w5) that initially (before adjustment) satisfy the first similarity condition are joined to form one cluster C(w). At this time, when the first similarity threshold is raised to 0.7, SIM(w1,w2)=0.61 no longer satisfies the adjusted first similarity condition, so the joint between sections SC(w1) and SC(w2) is broken, and cluster C(w) is split into two. Furthermore, if the first similarity threshold is raised to 0.8, SIM(w2,w3)=0.78 and SIM(w4,w5)=0.75 will no longer satisfy the adjusted first similarity condition, so the joins of sections SC(w2) and SC(w3) and SC(w4) and SC(w5) will be broken, and cluster C(w) will be divided into four. In addition, if the first similarity threshold is raised to 0.9, SIM(w3,w4)=0.83 will no longer satisfy the adjusted first similarity condition, so the joins of sections SC(w3) and SC(w4) will be broken, and cluster C(w) will be divided into five, and all five resulting clusters will consist of a single section. Note that if the first similarity threshold is brought close to 1 (to 0.999...), only clusters consisting of a single section will be produced. As the clusters are divided at each stage, it is determined at each stage whether the divided clusters now meet the character count requirement. If they do not, the process of further increasing the first similarity threshold is repeated.

[0076] <Configuration of Knowledge Search Server 20 / Overlap Means 26: Figures 10 and 12>

[0077] The overlapping means 26 performs an overlapping process to partially overlap the data contents of the preceding and succeeding clusters when, for adjacent clusters created by the clustering means 25, the similarity between the last section constituting the preceding cluster and the first section constituting the succeeding cluster satisfies a second similarity condition, which is equal to or exceeds a second similarity threshold calculated using predetermined or multiple similarity values. This overlapping process involves joining at least one section constituting the other cluster to at least one of the following locations: behind the preceding cluster or in front of the succeeding cluster, within a range that satisfies a character count condition, which is equal to or less than a predetermined character count threshold.

[0078] In this embodiment, the second similarity threshold is not a system setting, but a value calculated and determined using multiple similarity values. The calculation and determination of the second similarity threshold is performed by the clustering means 25 as described above, but may also be performed by the overlap means 26. Furthermore, the similarity between sections used in the process of determining whether or not the second similarity condition is met is calculated by the clustering means 25 and stored in the similarity storage means 34 (see Figure 5), so that value can be used, but it may also be recalculated by the overlap means 26.

[0079] Furthermore, the overlap processing by the overlap means in the present invention includes a front cluster rearward expansion processing, a rear cluster forward expansion processing, and a rearward / forward expansion parallel processing that performs these processes in parallel. In the overlap means 26 of this embodiment, the front cluster rearward expansion processing is adopted. The rear cluster forward expansion processing and the rearward / forward expansion parallel processing will be described in detail in the <Modified Forms> section below.

[0080] As part of the overlapping process, the overlapping means 26 performs a process to join at least one section adjacent to each other and satisfying a second similarity condition, including the leading section of the rear cluster, to the rear of the last end of the front cluster, within a range that satisfies the character count condition that the number of characters in the front cluster is less than or equal to a predetermined character count threshold, as shown in Figures 10 and 12.

[0081] Here, "joining at least one section that is adjacent to each other, including the first section constituting the rear cluster, and that satisfies the second similarity condition" means that if the similarity between the last section constituting the front cluster and the first section constituting the rear cluster satisfies the second similarity condition, the first section constituting the rear cluster will first be joined to the rear of the last section of the front cluster. After that, if the similarity between the first section constituting the rear cluster and the second section immediately following it satisfies the second similarity condition, the second section will be added and joined, and if the similarity between the second section and the third section immediately following it satisfies the second similarity condition, the third section will be added and joined, and so on. Therefore, if n is an integer of 1 or more, it means that if the similarity between the nth section and the (n+1)th section immediately following it satisfies the second similarity condition, the (n+1)th section will be joined. Therefore, each section that makes up the rear cluster is joined one by one, starting from the first section, and each time, it is checked whether the number of characters in the resulting cluster (the front cluster) satisfies the character count condition.

[0082] In the example in Figure 10, when performing the overlap processing between the front cluster C(1) and the rear cluster C(2), if the similarity SIM(2,3)=0.4 between the last section SC(2) constituting the front cluster C(1) and the first section SC(3) constituting the rear cluster C(2) satisfies the second similarity condition, the first section SC(3) constituting the rear cluster C(2) is joined to the rear of the front cluster C(1). If the number of characters in cluster C(1) after joining section SC(3) does not satisfy the character count condition, the joining of SC(3) is canceled; if the character count condition is met, the joining of SC(3) is adopted and the additional joining continues. Note that the rear cluster C(2) is a cluster composed of a single section, and there are no sections that can be additionally joined, so the joining is now complete.

[0083] Next, when performing the overlap processing between the front cluster C(2) and the rear cluster C(3), if the similarity SIM(3,4)=0.3 between the last section SC(3) that makes up the front cluster C(2) and the first section SC(4) that makes up the rear cluster C(3) satisfies the second similarity condition, the first section SC(4) that makes up the rear cluster C(3) is joined to the rear of the front cluster C(2) (this is the state shown by the dashed line in Figure 10). Then, if the number of characters in cluster C(2) after joining section SC(4) does not satisfy the character count condition, the joining of SC(4) is canceled; if the character count condition is met, the joining of SC(4) is adopted and the additional joining continues. If the process continues and the similarity SIM(4,5)=0.8 between the first section SC(4) and the section SC(5) immediately following it satisfies the second similarity condition, section SC(5) is additionally joined. Then, if the number of characters in cluster C(2) after adding section SC(5) (i.e., after joining sections SC(4) and SC(5)) does not satisfy the character count condition, the join of SC(5) is canceled. If the character count condition is met, the join of SC(5) is adopted, as shown in Figure 10. Note that there are no more sections that can be added to the rear cluster C(3), so the join is now complete.

[0084] In the more generalized example in Figure 12, when performing overlap processing between the front cluster C(x1) and the rear cluster C(x2), if the similarity SIM(x4,x5) between the last section SC(x4) constituting the front cluster C(x1) and the first section SC(x5) constituting the rear cluster C(x2) satisfies the second similarity condition, the first section SC(x5) constituting the rear cluster C(x2) is joined to the rear of the front cluster C(x1), and further the similarity with SC(x5) If SC(x6) satisfies the second similarity condition and SC(x7) satisfies the second similarity condition with respect to SC(x6), and if the number of characters in the preceding cluster C(x1) after these additional mergers is within the range that satisfies the character count condition, and if merging SC(x8) does not satisfy the character count condition, then ultimately, three sections SC(x5), SC(x6), and SC(x7) are added to the rear of the preceding cluster C(x1) (the area enclosed by the dotted line in Figure 12), and this area becomes the overlapping portion.

[0085] Next, when performing the overlap processing between the front cluster C(x2) and the rear cluster C(x3), if the similarity SIM(x10,x11) between the last section SC(x10) that makes up the front cluster C(x2) and the first section SC(x11) that makes up the rear cluster C(x3) satisfies the second similarity condition, then the first section SC(x11) that makes up the rear cluster C(x3) is joined to the rear of the front cluster C(x2), and then SC(x12) whose similarity to SC(x11) satisfies the second similarity condition is joined. If the number of characters in the front cluster C(x2) after these additional joins is within the range that satisfies the character count condition, and if joining SC(x13) would cause the character count condition to be not met, then ultimately, the two sections SC(x11) and SC(x12) are added to the rear of the front cluster C(x2) (the area enclosed by the dotted line in Figure 12), and this area becomes the overlapping area.

[0086] <Configuration of Knowledge Search Server 20 / Registration Method 27>

[0087] The registration means 27 transmits all clusters created by the overlap means 26 and stored in the cluster storage means 37 after the overlap processing to the database search system 40 via the network 1, and performs the process of registering them as knowledge in the knowledge database 41 (see Figure 2), associating them with the file name of the document data and the section title.

[0088] Furthermore, the registration means 27 also performs a vectorization process to create a vector for each cluster after the overlapping process, using the service provision system 52. Specifically, it sends an API request including the clusters after the overlapping process to the service provision system 52 via the network 1, and receives vector data as response data from the service provision system 52 via the network 1. This process is performed for all clusters after the overlapping process. The obtained vector data, along with the clusters after the overlapping process, is then sent to the database search system 40 via the network 1, and registered as knowledge vectors in the knowledge database 41 (see Figure 2), associated with the file names and section titles of the document data. The knowledge vector data registered in the knowledge database 41 is then used in the search process by the database search system 40. That is, when the database search system 40 retrieves corresponding knowledge from the user's question data, if it uses the similarity between vectors, it calculates the similarity (e.g., cosine similarity) between the question data vector and the vector of each piece of knowledge. Therefore, the user's question data also needs to be vectorized in the same way.

[0089] If a cluster exists that consists of a single section that satisfies the token count condition but does not satisfy the character count condition, the registration means 27 divides that cluster (single section) by the character count (for example, character count threshold = 2000 characters) to satisfy the character count condition and registers it in the knowledge database 41.

[0090] <Configuration of Knowledge Search Server 20 / Search Method 28>

[0091] The search means 28 receives user question data transmitted from the user terminal 70 via the network 1 (in this embodiment, this includes the number of knowledge acquisitions specified by the user, but the number of knowledge acquisitions may be a system setting value), sends a search query including this question data to the database search system 40 which performs search processing using the knowledge database 41 via the network 1, receives one or more (a number specified by the user, or a number predetermined by the system, for example, 1 to 10) pieces of knowledge obtained in the search process and information associated with those pieces of knowledge (such as reference file names and reference section (titles), but not vectors) transmitted from the database search system 40 via the network 1 as a response to the query, and performs processing to pass all received pieces of knowledge and information associated with those pieces of knowledge to the answer generation means 29.

[0092] In this case, the search means 28 displays an inquiry screen (not shown) on the screen of the user terminal 70. The user (questioner) makes the necessary inputs on this inquiry screen. This inquiry screen is provided with a knowledge acquisition limit specification section for specifying the upper limit of the number of knowledge acquisitions, a prompt setting section, and a question input section for entering the inquiry content (question data).

[0093] The number of knowledge items to be retrieved specified in the knowledge item retrieval specification section is an upper limit. Therefore, even if you specify 3 items, if the corresponding knowledge items do not exist in the knowledge database 41, the number may be 2 or less, or even zero.

[0094] The prompt settings section displays the default LLM instruction text, which is basically fixed data, but can be changed by the user. Specifically, the LLM instruction text is, for example, "Your task is to answer the [Inquiry Details] by checking the [Reference Information]. Please answer concisely within 200 characters. Please extract only the [Reference Information] that seems relevant to the inquiry and answer accordingly. If the content relevant to the inquiry is not included in the reference data, please add "I cannot answer because it is not included in the reference data." to your answer. [Inquiry Details]:····· [Reference Information]:·····" The [Inquiry Details] and [Reference Information] sections are dynamic parts. When the answer generation means 29, described later, creates the answer creation request data as a prompt, the [Inquiry Details] section is populated with the user's question data (for example, text data such as "Please tell me the procedure establishment flow."), and the [Reference Information] section is populated with all the knowledge (text data) obtained from the knowledge database 41. If multiple pieces of knowledge have been obtained, they are inserted in a list format.

[0095] In the question input section, the user (the questioner) enters question data, such as "Please explain the procedure for establishing procedures." In addition to text input, voice input may also be supported.

[0096] <Configuration of Knowledge Search Server 20 / Answer Generation Means 29>

[0097] The answer generation means 29 creates an answer creation request data as a prompt, which includes the user's question data, all the knowledge received from the search means 28, and predetermined instruction sentences. It then transmits the created answer creation request data to the service provision system 50 using a large-scale language model (LLM) via the network 1. The service provision system 50 receives the answer data output from the large-scale language model (LLM) transmitted via the network 1, and the display data of the received answer data is transmitted to the user terminal 70 via the network 1. The system then executes a process to display the answer screen (not shown) on the screen of the user terminal 70.

[0098] <Configuration of Knowledge Search Server 20 / Document Storage Means 31>

[0099] The document storage means 31 stores various document data (for example, document data such as regulations, procedures, and work manuals) that are to be registered in the knowledge database 41 (see Figure 2).

[0100] <Configuration of Knowledge Search Server 20 / Configuration Information Storage Means 32: Figure 1>

[0101] As shown in Figure 1, the setting information storage means 32 stores the token count threshold (upper limit) and the character count threshold (upper limit) as setting information. Alternatively, it may store a coefficient multiplied by the standard deviation when calculating and determining the second similarity threshold.

[0102] <Configuration of Knowledge Search Server 20 / Section Storage Means 33: Figure 4>

[0103] As shown in Figure 4, the section storage means 33 stores the section number (section identification information), the section's text data, and the section's vector data in a corresponding manner.

[0104] <Configuration of Knowledge Search Server 20 / Similarity Storage Means 34: Figure 5>

[0105] As shown in Figure 5, the similarity storage means 34 stores the section numbers (section identification information) of adjacent sections and the similarity between vectors of adjacent sections (cosine similarity in this embodiment) in association with each other.

[0106] <Configuration of Knowledge Search Server 20 / Similarity Threshold Storage Means 35: Figure 1>

[0107] As shown in Figure 1, the similarity threshold storage means 35 stores the first and second similarity thresholds that have been calculated and determined.

[0108] <Configuration of Knowledge Search Server 20 / Cluster Storage Means 36: Figure 5>

[0109] As shown in Figure 5, the cluster storage means 36 stores a combination of the cluster number (cluster identification information) and the section number (section identification information) of each section constituting the cluster. In addition to these, the text data of each section constituting the cluster (cluster text data) may also be stored in association with the combination of section numbers (section identification information).

[0110] <Configuration of Knowledge Search Server 20 / Overlap Processing Cluster Storage Means 37>

[0111] The cluster storage means 37 after overlap processing stores a combination of the cluster number (cluster identification information) after overlap processing and the section number (section identification information) of each section constituting the cluster after overlap processing. In addition to these, the text data of each section constituting the cluster after overlap processing (text data of the cluster after overlap processing) may also be stored in association with a combination of section numbers (section identification information).

[0112] <Flowchart of the construction process for Knowledge Database 41: Figure 3>

[0113] In Figure 3, first, the document creator (or system administrator) operates the creator terminal 60 and sends various document data, such as regulations, procedures, and business manuals, used to build the knowledge database 41, to the knowledge search server 20 via the network 1. The knowledge search server 20 receives the document data using the document storage means 21 and stores it in the document storage means 31.

[0114] Furthermore, the system administrator operates the creator terminal 60 and sends configuration information (token count threshold, character count threshold) to the knowledge search server 20 via the network 1. The knowledge search server 20 receives the configuration information via the configuration means 22 and stores it in the configuration information storage means 32.

[0115] Next, the text conversion means 23 executes processing based on an execution command from the creator (or system administrator) (step S1). That is, it converts the WORD file etc. stored in the document storage means 31 into a PDF file and sends the resulting PDF file to the service provision system 51 via the network 1. The service provision system 51 performs OCR processing and document structure analysis on the received PDF file to convert it into text, and adds heading patterns for each hierarchical level according to the document structure (for example, "# Summary of Company Regulations", "## Working Hours", "#### Dress Code", etc.) to the resulting text data. The service provision system 51 then receives the text data with the heading patterns attached, which has been sent via the network 1, and passes it to the section creation means 24.

[0116] Next, the section creation means 24 performs processing (step S2). That is, the text data is divided into semantic sections by heading patterns, and the text data of the resulting sections is stored in the section storage means 33. This process has already been described in detail in the description of the configuration of the section creation means 24, so a detailed explanation is omitted here. Furthermore, the details of the section creation process will be described later using Figure 4.

[0117] Next, the clustering means 25 generates an embedding vector for each section (step S3). That is, it transmits the text data of the section to the service provision system 52 via the network 1, receives the section vector data transmitted from the service provision system 52 via the network 1, and stores it in the section storage means 33. This process has already been described in detail in the description of the configuration of the clustering means 25, so a detailed explanation is omitted here.

[0118] Next, the clustering means 25 performs the similarity calculation process, the calculation and determination of the first and second similarity thresholds, and the crystallization process (step S4). Specifically, the vector data of adjacent sections is used to calculate the similarity between adjacent sections (cosine similarity in this embodiment), and this is stored in the similarity storage means 34. The mean and standard deviation are calculated using all the similarities in the document, and the first and second similarity thresholds are calculated and determined using these mean and standard deviations, and stored in the similarity threshold storage means 35. Furthermore, it is determined whether the similarity between adjacent sections satisfies the first similarity condition, which is equal to or exceeds the first similarity threshold. If it does, the adjacent sections are combined to create a cluster of an appropriate size (number of characters). These processes have already been described in detail in the description of the configuration of the clustering means 25, so a detailed explanation is omitted here. The details of the clustering process will be described later using Figure 5.

[0119] Subsequently, processing by the overlap means 26 is performed (step S5). That is, an overlap process is performed to duplicate content between adjacent clusters using a second similarity threshold. This process has already been described in detail in the description of the configuration of the overlap means 26, so a detailed explanation is omitted here. Furthermore, the details of the overlap process flow will be described later with reference to Figure 6.

[0120] Then, the registration means 27 uses the service provision system 52 to calculate the vector of the cluster after overlap processing, and registers the cluster after overlap processing as knowledge, along with its vector, in the knowledge database 41 (see Figure 2) (step S6). This process has already been described in detail in the description of the configuration of the registration means 27, so a detailed explanation is omitted here.

[0121] <Section creation process flow: Figure 4>

[0122] In Figure 4, the section creation means 24 determines whether a heading pattern exists in the document (text data) (step S201). If it does not exist, it determines whether the number of tokens in the document is less than or equal to the token number threshold (upper limit) stored in the setting information storage means 32 (whether the token number condition is met) (step S202). If it is less than or equal to the token number threshold (upper limit), it returns it as a single section (step S203) and stores it in the section storage means 33. On the other hand, if it is determined in step S202 that it is not less than or equal to the token number threshold (upper limit) (the token number condition is not met), the document is divided at sentence or paragraph breaks and adjusted to an appropriate size (less than or equal to the token number threshold (upper limit)) (step S204), returned as multiple sections (step S205), and stored in the section storage means 33.

[0123] Furthermore, if a heading pattern exists in step S201 as described above, section candidates are created by dividing the section according to the heading pattern (step S206). Next, the number of tokens in each section candidate is checked (step S207), and if the number of tokens in a section candidate is less than or equal to the token number threshold (upper limit) (the token number condition is met), that section candidate is added as a section to the section list (the list of sections created when a heading pattern exists) (step S208). On the other hand, if the number of tokens in a section candidate is not less than or equal to the token number threshold (upper limit) (the token number condition is not met), the section is divided according to a heading pattern at a deeper hierarchical level (a heading pattern with a large number of #s) (step S209).

[0124] In step S209, the section creation means 24 repeatedly divides the heading pattern by gradually increasing the hierarchical level of the heading pattern (increasing the number of # symbols in the heading pattern by one) until the number of tokens falls below the threshold (upper limit) (until the token count condition is met). Section candidates that have fallen below the token count threshold (upper limit) (the token count condition is met) are added to the section list as sections.

[0125] Then, all sections in the section list (a list of sections created when a heading pattern exists) are returned (step S210) and stored in the section storage means 33.

[0126] <Clustering process flow: Figure 5>

[0127] In Figure 5, the clustering means 25 checks the number of sections in the document (step S401). If the number of sections is 1 or less, it is returned as a single cluster (a cluster consisting of one section) (step S402) and stored in the cluster storage means 36. If the number of sections is 2 or more, the cosine similarity between the vectors of adjacent sections is calculated (step S403) and stored in the similarity storage means 34.

[0128] Next, the clustering means 25 calculates the mean and standard deviation using all the similarities in the documents stored in the similarity storage means 34 (except for outliers), calculates the first and second similarity thresholds using the calculated mean and standard deviation (step S404), and stores them in the similarity threshold storage means 35.

[0129] Then, the clustering means 25 performs clustering based on similarity using the first similarity threshold (step S405). That is, when the similarity between adjacent sections satisfies the first similarity condition, which is equal to or exceeds the first similarity threshold stored in the similarity threshold storage means 35, adjacent sections are joined together to create a cluster. However, after adjusting the first similarity threshold, clustering is performed only on the items to be re-clustered.

[0130] Then, the clustering means 25 starts loop processing for all clusters (step S406). Note that after adjusting the first similarity threshold, the re-clustered clusters are included.

[0131] First, the contents of the cluster are checked (step S407). If the cluster consists of a single section, the process moves on to the next cluster (step S408). It is then determined whether there are any clusters remaining to be processed (step S409). If there are none remaining, the processing of all clusters is completed (step S410). On the other hand, if there are clusters remaining to be processed, the process returns to step S407 and the contents of the cluster are checked.

[0132] Furthermore, if the cluster consists of multiple sections in step S407 as described above, the total number of characters in the cluster is checked (step S411). If it is below the character threshold (upper limit) (the character count condition is met), the process proceeds to step S408 to move on to processing the next cluster. On the other hand, if the total number of characters in the cluster is not below the character threshold (upper limit) (the character count condition is not met), the first similarity threshold is adjusted (see Figure 11) and the process moves on to re-clustering (step S412). The process returns to step S405 and clustering is performed on the cluster in question (each section constituting the cluster that does not meet the character count condition).

[0133] <Overlap processing flow: Figure 6>

[0134] In Figure 6, the overlap means 26 starts the overlap process (step S501), and then sequentially starts processing for cluster pairs (adjacent clusters) (step S502).

[0135] First, it is determined whether the next cluster exists (step S503). If it does not exist, the overlap process is completed (step S504), the cluster after the overlap process is returned (step S505), and the cluster after the overlap process is stored in the cluster storage means 37.

[0136] On the other hand, if the next cluster exists in step S503 as described above, the number of characters in the current cluster is calculated (step S506). The current cluster refers to the cluster at the time the process in step S506 is executed. Therefore, if a section is added (joined) to a cluster during the overlap process, the latest state of the cluster after the addition (joining) is targeted. In other words, the number of characters in the cluster, which has increased in number due to the addition and joining of sections, is calculated.

[0137] Next, the number of characters in the current cluster is checked (step S507). If it is below the character count threshold (upper limit) (i.e., the character count condition is met), it may be possible to add and merge sections, so the similarity between the last section of the current cluster and the adjacent section of the next (later) cluster is calculated (step S508). Note that if the similarity is stored in the similarity storage means 34, recalculation may be omitted.

[0138] Next, the similarity is checked (step S509), and if it is above the second similarity threshold (the second similarity condition is met), the adjacent section of the next (rear) cluster is added to the end of the current cluster (or the cluster with the sections already joined, if any sections have already been added) (step S510). This addition is a provisional addition because it may be undone.

[0139] Then, the number of characters in the cluster after the overlap processing is checked (step S511). If it is below the character threshold (upper limit) (the character count condition is met), the overlap addition is continued (adopted without cancellation) (step S512). If it exceeds the character threshold (upper limit) (the character count condition is not met), the overlap addition is canceled (step S513), and the process moves on to the next cluster pair (step S514).

[0140] Furthermore, if the character count threshold (upper limit) is exceeded in step S507 as described above (i.e., the character count condition is not met), it is not possible to add or join a section to the current cluster, so the process proceeds to step S514 and moves on to processing the next cluster pair.

[0141] <Knowledge search process flow: Figure 7>

[0142] In Figure 7, after the knowledge database 41 is constructed (steps S1 to S6), the search means 28 receives user question data and the number of acquired knowledge points transmitted from the user terminal 70 via the network 1 (step S7). The number of acquired knowledge points may be a system setting value.

[0143] Next, the search means 28 sends a search query, including the user's question data and the number of knowledge items retrieved, to the database search system 40 via the network 1 (step S8), and receives a response from the database search system 40 to the query sent via the network 1 (knowledge items retrieved by searching the knowledge database 41) (step S9). These processes have already been described in detail in the description of the configuration of the search means 28, so a detailed explanation is omitted here.

[0144] Subsequently, the answer generation means 29 receives knowledge from the search means 28 and creates answer creation request data as a prompt, which includes the user's question data, knowledge obtained from the knowledge database 41, and LLM instructions. The created answer creation request data is then transmitted via the network 1 to the service provision system 50 using a large-scale language model (LLM) (step S10). The LLM answer data transmitted from the service provision system 50 via the network 1 is received (step S11). Display data for the received LLM answer data is then transmitted via the network 1 to the user terminal 70 (step S12).

[0145] <Effects of this embodiment>

[0146] This embodiment provides the following advantages. Specifically, in the knowledge search system 10, the section creation means 24 divides the document data into sections by dividing it with heading patterns, thus creating sections that are semantic units, or sections that are chunks of meaning. Then, the clustering means 25 performs clustering on the resulting semantic sections, joining together sections with high similarity (sections that satisfy a first similarity condition, which is equal to or exceeds a first similarity threshold) using the similarity between adjacent sections. This reduces the possibility of a single sentence being incomplete in the created clusters, or reduces the negative effects of such incompleteness.

[0147] Furthermore, the knowledge retrieval system 10 uses the overlapping means 26 to create overlapping portions (parts where data content overlaps) between adjacent clusters when the similarity between adjacent clusters is high, or more precisely, when the similarity between the last section constituting the earlier cluster and the first section constituting the later cluster is high (when the second similarity condition, which is equal to or exceeds the second similarity threshold, is met). This minimizes the negative effects of fragmentation caused by a single semantically connected sentence being divided and placed into multiple fragmented data (multiple clusters in this invention). In other words, since the same text is placed in the overlapping portion, the probability that important terms are included in the preceding and succeeding fragment data (in this invention, the preceding and succeeding clusters after overlapping processing) increases. This increases the likelihood that the preceding and succeeding fragment data (knowledge, in this invention, the preceding and succeeding clusters after overlapping processing) that were separated due to the need to keep the number of characters in the cluster within a range that satisfies the character count condition (below or less than the character count threshold) will be retrieved together in the search by the database search system 40. As a result, the possibility of the information transmitted to the service provision system 50 using the large-scale language model (LLM) being incomplete can be reduced. Therefore, more appropriate answers can be obtained from the service provision system 50 using the large-scale language model (LLM), improving the accuracy of answers to user questions.

[0148] Specifically, for example, suppose that overlap processing by the overlap means 26 is not performed, and that there is a user specification or system setting of 3 knowledge acquisitions. In this case, for a given user's question data, it is optimal for adjacent clusters C(α1) and C(α2) obtained from document α, and cluster C(β1) obtained from document β to be acquired from the knowledge database 41. However, due to the similarity relationship with the user's question data, instead of cluster C(α1) and adjacent cluster C(α2), cluster C(γ1) from another document γ is searched and extracted, and the knowledge acquired from the knowledge database 41 becomes clusters C(α1), C(β1), and C(γ1), and the large-scale language model (LLM) may not be able to obtain the information necessary to create optimal answer data. However, by performing overlap processing, the probability of both adjacent clusters C(α1) and C(α2) being searched and extracted increases, thus avoiding such a situation.

[0149] Furthermore, since the section creation means 24 divides the document data into sections by dividing it with heading patterns, it eliminates the inconvenience that could occur when dividing document data into semantic unit data using conventional table of contents data, namely the inconvenience that it cannot be applied to groups of documents where the table of contents format is an unexpected format. Therefore, the versatility when constructing the knowledge database 41 can be increased.

[0150] Furthermore, the clustering means 25 is configured to allow adjustment of the first similarity threshold (see Figure 11). Therefore, when the first clustering operation using the first similarity threshold results in a cluster that does not satisfy the character count condition of being less than or equal to the character count threshold, the first similarity threshold can be gradually increased as a condition that the similarity between the sections constituting the cluster must satisfy. By disabling the connections between sections that no longer satisfy the first similarity condition, the cluster can be divided, reducing its size (number of characters) and bringing it within the range that satisfies the character count condition of being less than or equal to the character count threshold.

[0151] Furthermore, the knowledge search system 10 calculates and determines the first and second similarity thresholds using the mean and standard deviation of the similarity scores, making it easy to obtain similarity thresholds suitable for appropriate clustering and overlap processing.

[0152] Furthermore, the knowledge search system 10 can divide text data by gradually increasing the hierarchical level of the heading pattern using the section creation means 24 until the section satisfies the token count condition (see Figure 4). Therefore, when there is a limit (upper limit) on the number of tokens when performing the section vectorization process, it is possible to create a section that satisfies that token count condition.

[0153] (Effect confirmation experiment: Figure 13) As shown in Figure 13, in constructing the knowledge database 41, comparative experiments were conducted using three methods: dividing each document data by page, dividing it by the highest-level heading pattern obtained by ADI (Azure's AI Document Intelligence service), and using the present invention with ADI (creating clusters using similarity and overlapping them). Ten procedural questions and thirteen regulatory questions were prepared, and 23 knowledge searches were performed for comparison. Azure AI Search was used for the database search system 40. Azure OpenAI Service was used for the service provision system 50 using a large-scale language model (LLM).

[0154] The comparison was conducted from three perspectives. The first perspective was document retrieval (document retrieval rate), which was judged based on whether the correct document (file) was retrieved. Success was defined as finding at least one of the multiple knowledge items retrieved that matched the target document (a document predetermined to be retrieved in response to the user's question).

[0155] The second aspect was text extraction (knowledge acquisition rate), which was judged based on whether the correct location (pre-defined correct knowledge) within the document (file) was referenced, that is, whether appropriate knowledge was acquired in response to the question.

[0156] The third aspect was answer generation (answer accuracy), which was judged based on whether the answer matched the questioner's (user's) intent, that is, whether it provided an accurate answer to the question.

[0157] According to the comparative experiment results based on these evaluation criteria, there was no difference in document retrieval rate between the method of the present invention and the other two methods. On the other hand, for text extraction (knowledge retrieval rate) and answer generation (answer accuracy rate), the method of the present invention resulted in a higher number of correct answers and a higher accuracy rate than the other two methods. Therefore, the effectiveness of the present invention was clearly demonstrated.

[0158] <Form of transformation>

[0159] It should be noted that the present invention is not limited to the embodiments described above, and any modifications that can achieve the objectives of the present invention are included within the scope of the present invention.

[0160] <First form of deformation: Figure 14>

[0161] For example, in the above embodiment, the overlap means 26 was configured to perform a front cluster rearward expansion process as an overlap process, but the overlap means in the present invention may be configured to perform a rear cluster forward expansion process as an overlap process. This rear cluster forward expansion process is a process of joining at least one section adjacent to each other, including the last section constituting the front cluster, and satisfying a second similarity condition, in front of the beginning of the rear cluster, within a range that satisfies the character count condition that the number of characters in the rear cluster is less than or equal to a predetermined character count threshold.

[0162] Here, the phrase "join at least one section that is adjacent to each other, including the last section constituting the front cluster, and that satisfies the second similarity condition" means that if the similarity between the last section constituting the front cluster and the first section constituting the rear cluster satisfies the second similarity condition (i.e., the similarity between the last section constituting the front cluster and the first section constituting the rear cluster is equal to or exceeds the second similarity threshold), then the last (last) section constituting the front cluster is first joined to the front of the rear cluster. After that, if the number of sections constituting the front cluster is m (where m is an integer), then if the similarity between the last (m-th) section constituting the front cluster and the immediately preceding (m-1)th section satisfies the second similarity condition, then the (m-1)th section is added and joined, and so on. Therefore, each section that makes up the front cluster is joined one by one, starting from the last section, and each time, it is checked whether the number of characters in the resulting cluster (the rear cluster) satisfies the character count requirement.

[0163] In the example in Figure 14, when performing overlap processing between the front cluster C(y1) and the rear cluster C(y2), if the similarity SIM(y4,y5) between the last section SC(y4) constituting the front cluster C(y1) and the first section SC(y5) constituting the rear cluster C(y2) satisfies the second similarity condition, then the last (last) section SC(y4) constituting the front cluster C(y1) is placed in front of the rear cluster C(y2). If the two sections are joined, and then SC(y3) whose similarity to SC(y4) satisfies the second similarity condition is joined, and if the number of characters in the resulting cluster C(y2) after the join is within the range that satisfies the character count condition, and if joining SC(y2) would not satisfy the character count condition, then ultimately the two sections SC(y4) and SC(y3) are added in this order to the front of the cluster C(y2) (the area enclosed by the dotted line in Figure 14), and this area becomes the overlapping portion.

[0164] Next, when performing the overlap processing between the front cluster C(y2) and the rear cluster C(y3), if the similarity SIM(y10,y11) between the last section SC(y10) constituting the front cluster C(y2) and the first section SC(y11) constituting the rear cluster C(y3) satisfies the second similarity condition, the last (last) section SC(y10) constituting the front cluster C(y3) is joined to the front of the rear cluster C(y3), and furthermore, if the similarity between SC(y10) and the rear cluster C(y3) satisfies the second similarity condition If SC(y9) satisfies the similarity condition and is joined, and then SC(y8) satisfies the second similarity condition with respect to SC(y9) is joined, and if the number of characters in the resulting cluster C(y3) after these additional joins is within the range that satisfies the character count condition, and if joining SC(y7) would cause the character count condition to be not met, then ultimately the three sections SC(y10), SC(y9), and SC(y8) are added in this order to the front of the cluster C(y3) (the area enclosed by the dotted line in Figure 14), and this area becomes the overlapping portion.

[0165] <Second form of deformation: Figure 15>

[0166] Furthermore, the overlap means in the present invention may be configured to perform a parallel backward-forward expansion process, in which the backward expansion process of the front cluster (see Figures 10 and 12) and the forward expansion process of the rear cluster (see Figure 14) are performed in parallel as the overlap process. This parallel backward-forward expansion process is a process in which the backward expansion process of the front cluster (see Figures 10 and 12) and the forward expansion process of the rear cluster (see Figure 14) are performed in parallel, and for the second cluster of three consecutive clusters (hereinafter referred to as the three clusters of interest), the forward expansion process of the rear cluster is performed between it and the adjacent first cluster, and the backward expansion process of the front cluster is performed between it and the adjacent third cluster, and the sections constituting the first or third cluster are joined to the front or back of the second cluster, provided that the number of characters in the second cluster satisfies the character count condition.

[0167] In the example in Figure 15, there are three consecutive clusters C(z1), C(z2), and C(z3). If we consider these three clusters as a set of three clusters of interest, when creating an overlap portion between the second cluster C(z2) and the first cluster C(z1), the first cluster C(z1) becomes the front cluster C(z1), and the second cluster C(z2) becomes the rear cluster C(z2). Therefore, adding a section that constitutes the first cluster C(z1) in front of the second cluster C(z2) corresponds to a rear cluster forward expansion process (see Figure 14).

[0168] Furthermore, when creating an overlapping portion between the second cluster C(z2) of the three clusters of interest and the third cluster C(z3), the second cluster C(z2) becomes the front cluster C(z2), and the third cluster C(z3) becomes the rear cluster C(z3). Therefore, adding a section that constitutes the third cluster C(z3) behind the second cluster C(z2) corresponds to a rearward expansion process of the front cluster (see Figures 10 and 12).

[0169] Therefore, for the second cluster C(z2) of the three clusters of interest, the forward expansion process of the rear cluster (see Figure 14) and the backward expansion process of the front cluster (see Figures 10 and 12) can be performed in parallel. In this case, both processes will be performed in parallel as long as the number of characters in the second cluster C(z2) satisfies the character count condition.

[0170] Furthermore, if there are four or more consecutive clusters, shifting each cluster one by one will create a continuous relationship between the first, second, and third clusters (the relationship between the three clusters of interest). Therefore, shifting the cluster corresponding to the second cluster one by one will result in a continuous parallel process of backward and forward expansion centered on the second cluster.

[0171] In Figure 15, cluster C(z1) cannot be the second cluster of the three clusters of interest because there are no clusters preceding it. Therefore, cluster C(z1) can only perform a forward expansion operation (see Figures 10 and 12) with the cluster C(z2) immediately following it. In the example in Figure 15, the leading section SC(z6) and the section SC(z7) immediately following it that constitute the rear cluster C(z2) are joined behind the front cluster C(z1). The number of characters in the front cluster C(z1) after joining sections SC(z6) and SC(z7) satisfies the character count condition.

[0172] Regarding cluster C(z2), since there are clusters before and after it, it can be the second cluster among the three clusters of interest. Therefore, cluster C(z2) can perform a forward expansion process of the rear cluster (see Figure 14) with the immediately preceding cluster C(z1), and can also perform a backward expansion process of the front cluster (see Figures 10 and 12) with the immediately following cluster C(z3).

[0173] Therefore, in the example in Figure 15, cluster C(z2) is joined to the immediately preceding cluster C(z1) in the following order: the last section SC(z5) of the preceding cluster C(z1), the section SC(z4) immediately preceding it, and the section SC(z3) immediately preceding that, are joined to the front of the following cluster C(z2). In parallel with this, cluster C(z2) is joined to the immediately following cluster C(z3) in the following order: the first section SC(z10) of the following cluster C(z3) and the section SC(z11) immediately following it, are joined to the back of the preceding cluster C(z2). After the sections SC(z5), SC(z4), and SC(z3) are joined to the front and the sections SC(z10) and SC(z11) are joined to the back, the number of characters in cluster C(z2) satisfies the character count condition.

[0174] In this case, there are multiple possible orders for joining the sections constituting the first or third cluster to the second cluster of the three clusters of interest, either before or after it. The optimal algorithm is one in which the sections are joined to the second cluster in order of their similarity (all of which satisfy the second similarity condition), regardless of whether the join is before or after the second cluster. In this case, the joins to the second cluster are not performed alternately; for example, they may be joined backward, joined backward, joined forward, joined backward, and so on. Therefore, considering that the joins are performed within the range where the number of characters in the second cluster satisfies the character count condition, it is possible that only backward joins or only forward joins may occur.

[0175] Alternatively, the algorithm may alternate between joining the second cluster forward and joining it backward (the sections to be joined must all satisfy the second similarity condition). In this case, the backward joining may be performed first, or the forward joining may be performed first.

[0176] In the example in Figure 15, cluster C(z2), which is the second of the three clusters of interest, has sections SC(z5), SC(z4), and SC(z3) joined to the front, and sections SC(z10) and SC(z11) joined to the back. However, when using an algorithm that joins sections in order of similarity, the only thing that can be said for certain about the order of section joining in this state is that, considering only the front join, the sections were joined in the order of SC(z5), SC(z4), and SC(z3), and when considering only the back join, the sections were joined in the order of SC(z10), SC(z11). Therefore, since the joining of section SC(z5) to the front and the joining of section SC(z10) to the back are performed in order of higher similarity, it is not possible to determine which was performed first by looking only at the final join result (state). Furthermore, even if it is determined that the backward joining of section SC(z10) occurred first, the next joining will be either the forward joining of section SC(z5) or the backward joining of section SC(z11). Since the more similar the joining will occur first, it is not possible to determine which came first by looking only at the final joining result (state).

[0177] Regarding cluster C(z3), since there are no subsequent clusters, it cannot be the second cluster among the three clusters of interest. Therefore, cluster C(z3) can only perform a backward cluster forward expansion process (see Figure 14) with the immediately preceding cluster C(z2). In the example in Figure 15, the last section SC(z9) and the section SC(z8) that constitute the preceding cluster C(z2) are joined in front of the backward cluster C(z3). The number of characters in the backward cluster C(z3) after joining sections SC(z9) and SC(z8) satisfies the character count condition. [Industrial applicability]

[0178] As described above, the knowledge retrieval system and program of the present invention are suitable for use in constructing a knowledge database using various document data such as regulations, procedures, and manuals. [Explanation of symbols]

[0179] 1 Network 10. Knowledge Search System 23 Text conversion method 24 Section Creation Methods 25 Clustering means 26 Overlap means 27. Registration Method 28 Search methods 29 Answer generation means 40 Database Search System 41 Knowledge Database 50. Service delivery systems using Large-Scale Language Models (LLMs) 51. Service provision system that performs OCR processing, document structure extraction, etc. 52. Service provision system for performing vectorization processing 60 Creator's terminal 70 User terminals

Claims

1. A knowledge search system comprising a computer that answers user questions using knowledge obtained by searching a knowledge database, A section creation method that divides the text data constituting a document into semantic sections by separating them with headings, Clustering means that calculates the similarity between adjacent sections among the sections created by this section creation means, and when the calculated similarity satisfies a first similarity condition, which is equal to or exceeds a first similarity threshold determined in advance or using a plurality of similarities, the adjacent sections are joined to create a cluster. The clustering means includes, with respect to adjacent clusters created by this clustering means, if the similarity between the last section constituting the front cluster and the first section constituting the rear cluster satisfies a second similarity condition, which is equal to or exceeds a second similarity threshold calculated using a predetermined or multiple similarity values, then the overlap means performs an overlap process to partially overlap the data contents of the front cluster and the rear cluster, within a range that satisfies a character count condition, which is equal to or less than a predetermined character count threshold, by joining at least one section constituting the other cluster to at least one of the rear clusters, either behind the front cluster or in front of the rear cluster. The aforementioned knowledge database is, The configuration is such that the cluster created by the overlapping means after the overlapping process is stored as the knowledge. A knowledge search system characterized by the following:

2. The overlapping means, as the overlapping process, A front cluster rearward expansion process is performed to join at least one section adjacent to the rear cluster, including the first section constituting the rear cluster, and satisfying the second similarity condition, within the range where the number of characters in the front cluster satisfies the character count condition. A rear cluster forward expansion process that adds at least one section adjacent to the front cluster, including the last section constituting the front cluster, and satisfying the second similarity condition, within the range where the number of characters in the rear cluster satisfies the character count condition, The forward expansion process of the front cluster is performed in parallel with the forward expansion process of the rear cluster. When the second of the three consecutive clusters is performed with respect to the adjacent first cluster, and the forward expansion process of the rear cluster is performed with respect to the adjacent third cluster, a backward expansion / forward expansion parallel process is performed to join the section constituting the first or third cluster to the front or back of the second cluster, provided that the number of characters in the second cluster satisfies the character count condition. The knowledge search system according to claim 1, characterized in that it is configured to perform one of the following processes.

3. The clustering means is, If the number of characters in the created cluster does not meet the character count condition, the system is configured to repeat the process of dividing the cluster by gradually increasing the first similarity threshold for the cluster until the character count condition is met, and then unconnecting adjacent sections that no longer meet the adjusted first similarity condition among the sections constituting the cluster. The knowledge search system according to feature 1.

4. The first similarity threshold is a value calculated and determined by subtracting the standard deviation of the similarity from the mean value of the similarity. The second similarity threshold is a value determined by subtracting from the average similarity value the value obtained by multiplying the standard deviation of the similarity by a coefficient greater than 1. The knowledge search system according to feature 1.

5. The clustering means is, The system is configured to perform a vectorization process on each of the sections created by the section creation means, and to calculate the similarity between adjacent sections, it also performs a process to calculate the similarity between the vectors of the sections. The section creation means is The clustering means determines whether the number of tokens in the candidate text data for the section satisfies a token count condition, which is less than or equal to a predetermined token count threshold that is the upper limit of the number of tokens when performing the vectorization process. If it does not satisfy the condition, the clustering means repeats the process of dividing the candidate text data for the section while gradually increasing the hierarchy level of the headings until the token count condition is met. The knowledge search system according to feature 1.

6. A program for causing a computer to function as a knowledge search system according to any one of claims 1 to 5.

Images