Information processing system

Abstract

Improve the search accuracy of the knowledge base. [Solution] The information processing system (1) includes a calculation means (111) for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data registered in a database (30) and a second question sentence generated by a large-scale language model based on second text data registered in the database; a determination means (112) for determining whether the calculated similarity is greater than a first threshold; and a control means (113) for deleting one of the first text data and the second text data if the calculated similarity is greater than the first threshold.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing systems.

Background Art

[0002] As a system of this kind, for example, a system has been proposed in which a query data based on a document is generated in a language model, and a pair of the document and the query data is used for learning a search model for a chatbot (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As a chatbot, a chatbot using a mechanism (Retrieval-Augmented Genration: RAG) that gives an independent information source to a large language model by combining a large language model (Large Language Models: LLM) and search of a specific information source (hereinafter, appropriately referred to as "knowledge base") has been proposed. Here, the knowledge base includes a plurality of data (for example, documents). For example, the knowledge base may include one data and other data in which a part of the one data is updated. For example, the knowledge base may include a plurality of data having the same or almost the same content. In such a case, the search accuracy of the knowledge base may decrease. Note that a large language model is a language model constructed using a very large dataset and deep learning technology.

[0005] This invention has been made in view of the above-mentioned problems, and aims to provide an information processing system that can improve the search accuracy of a knowledge base. [Means for solving the problem]

[0006] An information processing system according to one aspect of the present invention includes: a calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data registered in a database and a second question sentence generated by the large-scale language model based on second text data registered in the database; a determination means for determining whether the calculated similarity is greater than a first threshold; and a control means for deleting one of the first text data and the second text data if the calculated similarity is greater than the first threshold.

[0007] An information processing system according to another aspect of the present invention includes: a calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data and a second question sentence generated by a large-scale language model based on second text data registered in a database; a determination means for determining whether the calculated similarity is greater than a first threshold; and a control means for maintaining the registration of the second text data without registering the first text data in the database, or registering the first text data in the database and deleting the second text data, if the calculated similarity is greater than the first threshold. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of an information processing system according to an embodiment. [Figure 2] This block diagram shows an example of the configuration of a computing device according to the embodiment. [Figure 3] This is a flowchart showing the operation of the information processing system according to the first embodiment. [Figure 4]This is a flowchart showing the operation of the information processing system according to the second embodiment. [Modes for carrying out the invention]

[0009] <First Embodiment> A first embodiment of the information processing system will be described with reference to Figures 1 to 3. In Figure 1, the information processing system 1 comprises an information processing device 10, a server 20, and a knowledge base 30. The information processing device 10, server 20, and knowledge base 30 are configured to communicate with each other via a network NW. Server 20 is a server for operating a large-scale language model (LLM). For this reason, server 20 may be referred to as an LLM server. Server 20 may be a cloud server.

[0010] (Chatbot) Server 20 and Knowledge Base 30 may provide a chatbot service using RAG. For example, user U may use the chatbot service via terminal device 50. In this case, user U may operate terminal device 50 to launch an application for using the chatbot service. User U may operate terminal device 50 to enter a question into the input field of the chat application. Here, "question" is not limited to interrogative sentences. For example, "question" may be a sentence that includes expressions such as requests, instructions, or commands, such as "Tell me about ****" or "Answer me about ****". Therefore, "question" is a concept that includes not only sentences in the form of interrogative sentences, but also sentences that include expressions such as requests, instructions, or commands. In other words, "question" may mean a sentence that seeks an answer from the other party.

[0011] Terminal device 50 may search the knowledge base 30 based on the input question. Terminal device 50 may send first information, which includes the input question and text data as search results from the knowledge base 30, to server 20. Server 20 may input the question and text data included in the first information as a prompt to a large-scale language model. Server 20 may obtain the answer to the question output from the large-scale language model. Server 20 may send second information indicating the answer to terminal device 50. Upon receiving the second information, terminal device 50 may display the answer indicated by the second information on the screen related to the chat application. Terminal device 50 may be a personal computer, a tablet terminal, or a smartphone.

[0012] (Information processing device 10) In Figure 1, the information processing device 10 comprises an arithmetic unit 11, a storage device 12, a communication device 13, an input device 14, and an output device 15. The arithmetic unit 11, storage device 12, communication device 13, input device 14, and output device 15 are connected via a data bus 16. The information processing device 10 may be a personal computer, a tablet terminal, or a smartphone.

[0013] The arithmetic unit 11 may have a processor. The arithmetic unit 11 may have a single processor or multiple processors. In other words, the arithmetic unit 11 may have one or more processors. Furthermore, the processor may be a multi-core processor. If the arithmetic unit 11 has a single processor that is a multi-core processor, then logically, the arithmetic unit 11 can be said to have multiple processors.

[0014] The processor may be at least one of the following: CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and TPU (Tensor Processing Unit).

[0015] The storage device 12 may be at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and optical disk array. In other words, the storage device 12 may be implemented by a single device or by multiple devices.

[0016] The communication device 13 may be capable of communicating with devices outside the information processing device 10. The communication device 13 may use either wired or wireless communication.

[0017] The input device 14 is a device capable of receiving information input to the information processing device 10 from an external source. The input device 14 may include an operating device (e.g., a keyboard, mouse, touch panel, etc.) that can be operated by the user of the information processing device 10. The input device 14 may include a recording medium reader capable of reading information recorded on a recording medium that can be attached to and detached from the information processing device 10, such as a USB (Universal Serial Bus) memory. When information is input to the information processing device 10 via the communication device 13 (in other words, when the information processing device 10 acquires information via the communication device 13), the communication device 13 may function as an input device.

[0018] The output device 15 is a device capable of outputting information to the outside of the information processing device 10. The output device 15 may have a display device capable of outputting visual information such as characters and images as the above information. The output device 15 may also have a speaker capable of outputting auditory information such as sound as the above information. The output device 15 may also have a vibration motor capable of outputting tactile information such as vibration as the above information. The output device 15 may also have a printer. The output device 15 may be capable of outputting information to a recording medium that can be attached to and detached from the information processing device 10, such as a USB memory stick. When the information processing device 10 outputs information via the communication device 13, the communication device 13 may function as an output device.

[0019] The storage device 12 can store desired data. The storage device 12 may store the computer program CP executed by the arithmetic unit 11. When the arithmetic unit 11 is executing the computer program CP, the storage device 12 may temporarily store data temporarily used by the arithmetic unit 11.

[0020] Incidentally, the computer program CP may be recorded on a computer-readable and non-temporary recording medium. In this case, the computer program CP may be stored in the storage device 12 by reading the recording medium using a recording medium reader (not shown) provided in the information processing apparatus 10. As the recording medium, at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program may be used. Incidentally, the computer program CP may be acquired from a device (not shown) outside the information processing apparatus 10 via the communication device 13. In other words, the computer program CP may be downloaded from an external device to the storage device 12 of the information processing apparatus 10.

[0021] The arithmetic unit 11 (for example, a processor) may execute the processing to be performed by the information processing apparatus 10 together with the storage device 12 in which the computer program CP is stored (in other words, together with the storage device 12 and the computer program CP stored in the storage device 12). For example, by executing the computer program CP, the arithmetic unit 11 may realize a logical functional block for executing the processing to be performed by the information processing apparatus 10 inside the arithmetic unit 11 (for example, inside the processor).

[0022] (Knowledge base 30) Knowledge base 30 may contain multiple text data entries. These text data entries may be data obtained by dividing text contained in a single document. Such data may be referred to as "chunks." Methods for dividing text contained in a single document include, for example, dividing it at fixed lengths, dividing it at sentence units based on sentence delimiters, or dividing it based on structure such as Markdown. Knowledge base 30 may also contain multiple text data entries that have been vectorized. In other words, knowledge base 30 may be a vector database / vector store. In addition to text data, knowledge base 30 may also contain image data.

[0023] Here, the inventor's research has revealed the following: New text data may be registered in the knowledge base 30 at any time. On the other hand, it is possible that multiple text data with the same or nearly the same content may be registered in the knowledge base 30, or that both pre-update and post-update text data may be registered. Furthermore, when searching the knowledge base 30, it is possible that two or more text data with overlapping content may be extracted, or that both pre-update and post-update text data may be extracted. As a result, the search accuracy of the knowledge base 30 may decrease. In other words, in the chatbot service described above, the response accuracy of the large-scale language model may decrease.

[0024] Therefore, the information processing device 10 according to this embodiment manages multiple text data registered in the knowledge base 30. As shown in Figure 2, the arithmetic unit 11 of the information processing device 10 has a calculation unit 111, a determination unit 112, and a control unit 113 for managing text data. The calculation unit 111, the determination unit 112, and the control unit 113 may be implemented as the logical functional blocks described above. However, at least one of the calculation unit 111, the determination unit 112, and the control unit 113 may be implemented as a physical processing circuit. Alternatively, at least one of the calculation unit 111, the determination unit 112, and the control unit 113 may be implemented in a form in which a logical functional block and a physical processing circuit are mixed.

[0025] The operation of the information processing device 10 will be explained with reference to the flowchart in Figure 3. In Figure 3, the arithmetic unit 11 of the information processing device 10 selects the first text data and the second text data registered in the knowledge base 30. The arithmetic unit 11 sends the first text data and information (e.g., a prompt) for the large-scale language model to generate a question sentence based on the first text data to the server 20 via the communication device 13. As a result, the large-scale language model generates the first question sentence based on the first text data. The arithmetic unit 11 also sends the second text data and information (e.g., a prompt) for the large-scale language model to generate a question sentence based on the second text data to the server 20 via the communication device 13. As a result, the large-scale language model generates the second question sentence based on the second text data. For example, if the text data is "Asakusa in Tokyo is a popular tourist destination for foreigners," the large-scale language model may generate the question sentence "What are some popular tourist destinations in Tokyo for foreigners?"

[0026] Server 20 transmits the first and second question statements to the information processing device 10. The calculation unit 111 of the information processing device 10 calculates the similarity between the first and second question statements (step S101). The similarity calculated in step S101 may be such that a higher value indicates a greater similarity between the first and second question statements. For example, the similarity may be cosine similarity. Note that "similarity" is equivalent to "degree of agreement". The determination unit 112 of the information processing device 10 determines whether the similarity calculated in step S101 is greater than the first threshold (step S102). If, in step S102, it is determined that the similarity is greater than the first threshold (step S102: Yes), the control unit 113 of the information processing device 10 deletes either the first text data or the second text data from the knowledge base 30 (step S104).

[0027] For example, the control unit 113 may delete one of the first text data and the second text data from the knowledge base 30 based on at least one of the update date and time and version information. In this case, the control unit 113 may delete the text data with the older update date and time from the first text data and the second text data. The control unit 113 may delete the text data with the older version indicated by the version information from the first text data and the second text data.

[0028] In the process of step S102, if it is determined that the similarity is less than the first threshold (step S102: No), the determination unit 112 determines whether the distance is greater than the second threshold (step S103). Here, the second threshold is a value smaller than the first threshold. In the process of step S103, if it is determined that the distance is greater than the second threshold (i.e., first threshold > similarity > second threshold) (step S103: Yes), the control unit 113 associates the first text data with the second text data (step S105).

[0029] In the process of step S103, if it is determined that the distance is greater than the second threshold (step S103: No), the control unit 113 maintains the registration of the first text data and the second text data.

[0030] The "first threshold" is a value used to determine whether to delete either the first text data or the second text data. The "second threshold" is a value used to determine whether to associate the first text data and the second text data. The first and second thresholds may be predetermined fixed values ​​or variable values ​​depending on some parameter. The first and second thresholds may be set based on the relationship between the degree of overlap in the content of the two text data and the similarity of the two question sentences generated by the large-scale language model based on the two text data, respectively. In the processing of step S102, if the similarity and the first threshold are equal, they may be included in either case. Similarly, in the processing of step S103, if the similarity and the second threshold are equal, they may be included in either case.

[0031] (Technical effects) According to the information processing system 1 of the first embodiment, duplicate text data can be deleted from multiple text data registered in the knowledge base 30. Therefore, the information processing system 1 can prevent duplicate text data from being maintained, or the maintenance of both pre-update and post-update text data. Consequently, the information processing system 1 can improve the search accuracy of the knowledge base 30. In addition, the cost of using the storage that constitutes the knowledge base 30 can be reduced.

[0032] Furthermore, in the information processing system 1 according to the first embodiment, if the similarity between the first question and the second question is less than the first threshold and greater than the second threshold, the first text data and the second text data are associated with each other. In searching the knowledge base 30, the associated text data may be treated as a group of text data.

[0033] <Second Embodiment> A second embodiment of the information processing system will be described with reference to Figures 1, 2, and 4. The second embodiment is the same as the first embodiment described above, except that the operation of the information processing device 10 is slightly different. Therefore, explanations that overlap with the first embodiment described above will be omitted as appropriate.

[0034] In the second embodiment, the information processing device 10 determines whether or not to register new text data in the knowledge base 30 when new text data is registered in the knowledge base 30. The arithmetic unit 11 of the information processing device 10 has a calculation unit 111, a determination unit 112, and a control unit 113 to perform this determination.

[0035] The operation of the information processing device 10 according to the second embodiment will be explained with reference to the flowchart in Figure 4. In Figure 4, the arithmetic unit 11 of the information processing device 10 transmits new text data (i.e., text data newly registered in the knowledge base 30: hereafter referred to as "third text data") and information (e.g., prompts) for the large-scale language model to generate a question sentence based on the third text data to the server 20 via the communication device 13. As a result, the large-scale language model generates a third question sentence based on the third text data. The arithmetic unit 11 also selects fourth text data registered in the knowledge base 30. The arithmetic unit 11 transmits the fourth text data and information (e.g., prompts) for the large-scale language model to generate a question sentence based on the fourth text data to the server 20 via the communication device 13. As a result, the large-scale language model generates a fourth question sentence based on the fourth text data.

[0036] Server 20 transmits the third and fourth question statements to the information processing device 10. The calculation unit 111 of the division processing device 10 calculates the similarity between the third and fourth question statements (step S101).

[0037] The determination unit 112 of the information processing device 10 determines whether the similarity calculated in step S101 is greater than the first threshold (step S102). If, in step S102, it is determined that the similarity is greater than the first threshold (step S102: Yes), the control unit 113 of the information processing device 10 deletes either the third text data or the fourth text data (step S104). In other words, the control unit 113 may maintain the registration of the fourth text data without registering the third text data in the knowledge base 30 (in this case, the third text data may be deleted). Alternatively, the control unit 113 may register the third text data in the knowledge base 30 and delete the fourth text data.

[0038] For example, the control unit 113 may delete one of the third text data and the fourth text data based on at least one of the update date and time and the version information. In this case, the control unit 113 may delete the text data with the older update date and time from the third text data and the fourth text data. The control unit 113 may delete the text data with the older version indicated by the version information from the third text data and the fourth text data.

[0039] In the process of step S102, if it is determined that the similarity is less than the first threshold (step S102: No), the determination unit 112 determines whether the similarity is greater than the second threshold (step S103). In the process of step S103, if it is determined that the similarity is greater than the second threshold (i.e., first threshold > similarity > second threshold) (step S103: Yes), the control unit 113 associates the third text data with the fourth text data and registers it in the knowledge base 30 (step S201).

[0040] In the process of step S103, if it is determined that the similarity is less than the second threshold (step S103: No), the control unit 113 registers the third text data in the knowledge base 30 (step S202).

[0041] (Technical effects) According to the information processing system 1 of the second embodiment, it is possible to suppress the registration of multiple text data with the same or nearly identical content, and the registration of both pre-update and post-update text data. Therefore, the information processing system 1 can improve the search accuracy of the knowledge base 30. In addition, it is possible to reduce the usage costs of the storage constituting the knowledge base 30.

[0042] Furthermore, in the information processing system 1 according to the second embodiment, if the similarity between the third question and the fourth question is less than the first threshold and greater than the second threshold, the third text data and the fourth text data are associated with each other. In searching the knowledge base 30, the associated text data may be treated as a group of text data.

[0043] Various aspects of the invention derived from the embodiments described above are described below.

[0044] An information processing system according to one aspect of the invention includes: a calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data registered in a database and a second question sentence generated by the large-scale language model based on second text data registered in the database; a determination means for determining whether the calculated similarity is greater than a first threshold; and a control means for deleting one of the first text data and the second text data if the calculated similarity is greater than the first threshold. In the above embodiment, "knowledge base 30" corresponds to an example of a "database," "calculation unit 111" corresponds to an example of a "calculation means," "determination unit 112" corresponds to an example of a "determination means," and "control unit 113" corresponds to an example of a "control means."

[0045] In the information processing system of the above embodiment, if the calculated similarity is less than the first threshold, the determination means may determine whether the calculated similarity is greater than a second threshold that is less than the first threshold, and if the calculated similarity is greater than the second threshold, the control means may associate the first text data and the second text data with each other.

[0046] An information processing system according to another aspect of the invention includes: a calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data and a second question sentence generated by a large-scale language model based on second text data registered in a database; a determination means for determining whether the calculated similarity is greater than a first threshold; and a control means for maintaining the registration of the second text data without registering the first text data in the database, or registering the first text data in the database and deleting the second text data, if the calculated similarity is greater than the first threshold.

[0047] In the information processing system of the above embodiment, if the calculated similarity is less than the first threshold, the determination means may determine whether the calculated similarity is greater than a second threshold that is less than the first threshold, and if the calculated similarity is greater than the second threshold, the control means may associate the first text data with the second text data and register it in the database.

[0048] The present invention is not limited to the embodiments described above, and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be read from the claims and specification as a whole. Information processing systems involving such modifications are also included within the technical scope of the present invention. [Explanation of Symbols]

[0049] 1...Information processing system, 10...Information processing device, 20...Server, 30...Knowledge base, 111...Calculation unit, 112...Determination unit, 113...Control unit

Claims

1. A calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data registered in the database and a second question sentence generated by the large-scale language model based on second text data registered in the database, A determination means for determining whether the calculated similarity is greater than a first threshold, A control means that deletes one of the first text data and the second text data if the calculated similarity is greater than the first threshold, An information processing system equipped with the following features.

2. If the calculated similarity is less than the first threshold, the determination means determines whether the calculated similarity is greater than a second threshold that is less than the first threshold. If the calculated similarity is greater than the second threshold, the control means associates the first text data and the second text data with each other. The information processing system according to claim 1.

3. A calculation means for calculating the similarity between a first question sentence generated by a large-scale language model based on first text data and a second question sentence generated by a large-scale language model based on second text data registered in a database, A determination means for determining whether the calculated similarity is greater than a first threshold, If the calculated similarity is greater than the first threshold, the control means either maintains the registration of the second text data without registering the first text data in the database, or registers the first text data in the database and deletes the second text data. An information processing system equipped with the following features.

4. If the calculated similarity is less than the first threshold, the determination means determines whether the calculated similarity is greater than a second threshold that is less than the first threshold. If the calculated similarity is greater than the second threshold, the control means associates the first text data with the second text data and registers it in the database. The information processing system according to claim 3.

Images