Generation program, generation method, and information processing apparatus

Abstract

To improve accuracy in search for information relating to a question sentence.SOLUTION: An information processing apparatus 101 accepts a first question sentence q1. The information processing apparatus 101 retrieves first sentence data r1 relating to the first question sentence q1 with reference to a storage unit 110 in which a plurality of pieces of sentence data is stored. In accordance with the retrieved first sentence data r1, the information processing apparatus 101 uses a language model 120 to generate a second question sentence q2 obtained by changing a style of the first question sentence q1 without changing meaning of the first question sentence q1. The language model 120 is, for example, a large language model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a generating program, a generating method, and an information processing device. [Background technology]

[0002] Conventionally, there is a Retrieval-Augmented Generation (RAG) system that combines a generative model of Artificial Intelligence (AI) in the field of natural language processing with an information retrieval-based approach. In a RAG system, for example, a database is searched for Q&A (Question and Answer) related to a question, and an answer is generated from the searched Q&A and the question.

[0003] Prior art includes a question and answer display system that converts a question input from an administrator terminal into one or more question sentences with different expressions that correspond to the same answer pattern as the input question, based on a question and answer database containing answer patterns and multiple corresponding question patterns. There is also technology that uses a language model to generate at least one query data item that can be answered by a given document, and uses data consisting of pairs of each document belonging to a specific domain and the query data generated for that document to train a search model for a dialogue bot.

[0004] There is also a technology that generates multiple revised questions by rephrasing a user's question, selects answer candidates corresponding to each of the user's question and the revised question, and detects at least one of the selected answer candidates as an answer.There is also a technology that generates additional sentences related to the question based on an input question, with the total number of characters including the number of characters in the question not exceeding the limit on the number of characters that can be input to a large-scale language model, and generates a prompt by adding the generated additional sentences to the input question as reference information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-108033 [Patent Document 2] Japanese Patent Publication No. 2023-76413 [Patent Document 3] US Patent Application Publication No. 2016 / 0140958 [Patent Document 4] Patent No. 7313757 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the conventional technology, when generating an answer to a question, there is a problem that the accuracy of searching for information related to the question (e.g., Q&A) decreases. For example, if the accuracy of searching for information related to the question decreases, it will eventually lead to a decrease in the accuracy of generating an answer to the question.

[0007] In one aspect, the present invention aims to improve the accuracy of searching for information related to a question. [Means for solving the problem]

[0008] In one embodiment, a generation program is provided that refers to a memory unit that stores a plurality of sentence data, searches for first sentence data related to a first question, and, based on the searched first sentence data, uses a first language model for generating sentences to generate a second question in which the style of the first question is changed so that the meaning of the first question is unchanged. [Effects of the Invention]

[0009] Advantageous Effects of Invention According to one aspect of the present invention, it is possible to improve the accuracy of searching for information related to a question. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of a generating method according to the first embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing an example of the system configuration of the answer generation system 200. As shown in FIG. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of the answer generation device 201. As shown in FIG. [Figure 4] FIG. 4 is an explanatory diagram showing an example of the contents stored in the text database 220. As shown in FIG. [Figure 5] FIG. 5 is a block diagram illustrating an example of a functional configuration of the answer generation device 201 according to the first embodiment. [Figure 6] FIG. 6 is an explanatory diagram illustrating an example of the operation of the answer generation device 201 according to the first embodiment. [Figure 7] FIG. 7 is an explanatory diagram showing an example of a question sentence from a user. [Figure 8] FIG. 8 is an explanatory diagram showing a specific example of a prompt for calibration. [Figure 9] FIG. 9 is an explanatory diagram showing an example of a question sentence after the first correction. [Figure 10] FIG. 10 is an explanatory diagram (part 1) showing a specific example of a prompt for changing writing style. [Figure 11] FIG. 11 is an explanatory diagram showing an example of a question sentence after the second correction. [Figure 12] FIG. 12 is a flowchart of an example of a DB registration procedure of the answer generation device 201 according to the first embodiment. [Figure 13] FIG. 13 is a flowchart of an example of an answer generation process procedure of the answer generation device 201 according to the first embodiment. [Figure 14] FIG. 14 is an explanatory diagram showing an example of the contents stored in the sentence characteristics table 1400. As shown in FIG. [Figure 15] FIG. 15 is a block diagram illustrating an example of a functional configuration of the answer generation device 201 according to the second embodiment. [Figure 16] FIG. 16 is an explanatory diagram (part 2) showing a specific example of a prompt for changing writing style. [Figure 17]FIG. 17 is an explanatory diagram illustrating an example of the operation of the answer generation device 201 according to the second embodiment. [Figure 18] FIG. 18 is a flowchart illustrating an example of a table creation procedure of the answer generation device 201 according to the second embodiment. [Figure 19] FIG. 19 is a flowchart of an example of an answer generation process procedure of the answer generation device 201 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a generating program, a generating method, and an information processing device according to the present invention will be described in detail with reference to the accompanying drawings.

[0012] (Embodiment 1) Fig. 1 is an explanatory diagram showing an example of a generation method according to the first embodiment. In Fig. 1, an information processing device 101 is a computer that generates a new question (second question) by changing the style of a question (first question). The question is information (text data) indicating the content of the question.

[0013] The RAG system is a technology that combines an AI generative model in the field of natural language processing with an information retrieval-based approach. An example of an AI generative model in the field of natural language processing is LLM (Large Language Models).

[0014] LLM, for example, is a language model constructed by deep learning using large amounts of text data.Language models such as LLM tend to learn a large amount of general knowledge, so while they are good at generating natural-sounding sentences like humans, they have the problem of difficulty specializing in specialized knowledge for specific tasks.

[0015] On the other hand, information retrieval-based approaches are characterized by searching information from a large set of documents to find answers to specific questions, but they have the problem of making it difficult to guarantee the consistency and naturalness of responses. The RAG system aims to generate more efficient and natural responses by combining these two.

[0016] In a conventional RAG system, for example, when responding to a question and answer session (QA) specific to a certain area (such as a company or public service), the process is carried out using the following flow.

[0017] First, conventional RAG systems convert past Q&A lists into sentence vectors and store them in a database. When a user asks a question, the system converts the question into a sentence vector, calculates the distance between the question and the sentence vector in the database, and searches for sentences with a small distance as related Q&A. Then, the system generates an answer using LLM based on the searched related Q&A and the question.

[0018] However, in a Q&A list specialized in a certain domain, domain-specific phrases and terminology are often used. In such cases, the conventional RAG system has the problem that the vector distance between the search target sentence and the question increases depending on how the question is written and whether or not the question contains terminology, which reduces the accuracy of searching related Q&As.

[0019] For example, in a QA list specialized in a certain domain, the characteristics of objects A and B are often described together in a sentence such as "Object A and object B are blue." In contrast, in user questions, objects A and B may be described separately, such as "Is object A blue?" and "Is object B blue?"

[0020] In this case, in a conventional RAG system, the vector distance between the question and the related QA sentence becomes large, resulting in a decrease in the search accuracy of the related QA. For example, compared to the vector distance between the question "Are object A and object B blue?" and "Object A and object B are blue," the vector distance between the question "Is object A blue?" and "Is object B blue?" and "Object A and object B are blue" becomes large, resulting in a decrease in the search accuracy of the related QA. A decrease in the search accuracy of the related QA ultimately leads to a decrease in the accuracy of generating answers to the question.

[0021] Therefore, in this embodiment, a generation method for generating an answer to a question, which improves the accuracy of retrieval of information related to the question, will be described. Here, an example of processing by the information processing device 101 (corresponding to the following processes (1) to (3)) will be described.

[0022] (1) The information processing device 101 receives a first question. The first question is input by, for example, a user.

[0023] In the example of FIG. 1, it is assumed that a first question q1 is input.

[0024] (2) The information processing device 101 refers to the storage unit 110 that stores a plurality of pieces of text data, and searches for first text data related to the first question sentence q1. Here, the text data is information related to the text. The text may be a text accumulated as knowledge such as a past Q&A, or may be a text extracted from a textbook, a manual, or the like.

[0025] For example, the sentence data may be information (text data) representing a sentence. Alternatively, the sentence data may be information representing a sentence and a feature of the sentence. The feature of the sentence is information representing the features of the sentence, such as a sentence vector.

[0026] Specifically, for example, the information processing device 101 may refer to the memory unit 110 and search for first sentence data representing a sentence with similar characteristics to the first question sentence q1 by comparing the features of the sentence represented by the sentence data included in the multiple sentence data with the features of the first question sentence q1.

[0027] 1, it is assumed that the first text data r1 is searched for. The first text data r1 represents a text that has similar characteristics to the first question text q1.

[0028] (3) The information processing device 101 generates a second question q2 by changing the style of the first question q1 so as to maintain the same meaning as the first question q1, using the language model 120, in accordance with the retrieved first sentence data r1. Here, the language model 120 is a language model (learning model) for generating sentences, such as an LLM.

[0029] A style is a feature of written expression, and is manifested, for example, in the words, grammar, and rhetoric used in a sentence. Specifically, for example, the information processing device 101 creates a prompt (command sentence) instructing the user to rewrite the first question q1 so as to preserve the meaning while using as much of the style (words, grammar, rhetoric, etc.) used in the sentence represented by the first sentence data r1 as possible. Then, the information processing device 101 provides the created prompt to the language model 120 to generate a second question q2.

[0030] In this way, according to the information processing device 101, when generating an answer to a question (for example, the first question q1), it is possible to improve the accuracy of searching for information related to the question.

[0031] For example, suppose that text data (including first text data r1) representing text specialized in a certain field (such as a company or public service) is stored in the storage unit 110. The second question q2 corresponds to a rewrite of the first question q1 using, for example, technical terms and phrases that appear in the text represented by the first text data r1.

[0032] Therefore, compared to the first question q1, the second question q2 can be said to be more similar in style to the sentences (sentences specialized in a certain area) represented by the sentence data stored in the storage unit 110. By making the style of the question similar to the sentences to be searched, the information processing device 101 can generate a question that is more likely to find appropriate information.

[0033] When generating an answer to the first question q1, the information processing device 101 uses the generated second question q2 to search for related information, thereby enabling the search for more appropriate information (such as related Q&A), and ultimately improving the accuracy of answer generation.

[0034] (Example of system configuration of answer generation system 200) Next, a system configuration example of an answer generation system 200 including the information processing device 101 shown in Fig. 1 will be described. Here, an example will be described in which the information processing device 101 shown in Fig. 1 is applied to the answer generation device 201 in the answer generation system 200. The answer generation system 200 can be applied to, for example, a RAG system.

[0035] Fig. 2 is an explanatory diagram showing an example of the system configuration of an answer generation system 200. In Fig. 2, the answer generation system 200 includes an answer generation device 201 and a client device 202. In the answer generation system 200, the answer generation device 201 and the client device 202 are connected via a wired or wireless network 210. The network 210 is, for example, the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0036] Here, the answer generation device 201 is a computer that has a text database 220 and outputs an answer to a question. The answer generation device 201 is, for example, a server. The text database 220 stores a plurality of text data. The contents stored in the text database 220 will be described later with reference to FIG. 4.

[0037] The client device 202 is a computer used by a user of the answer generation system 200. The user is, for example, a questioner. The client device 202 is, for example, a personal computer (PC), a tablet PC, a smartphone, or the like.

[0038] Here, the answer generation device 201 and the client device 202 are provided separately, but this is not limiting. For example, the answer generation device 201 may be realized by the client device 202. Furthermore, the answer generation system 200 may include multiple client devices 202.

[0039] (Example of hardware configuration of answer generation device 201) Next, an example of the hardware configuration of the answer generation device 201 will be described.

[0040] Fig. 3 is a block diagram showing an example of the hardware configuration of the answer generation device 201. In Fig. 3, the answer generation device 201 has a CPU (Central Processing Unit) 301, a memory 302, a disk drive 303, a disk 304, a communication I / F (Interface) 305, a portable recording medium I / F 306, and a portable recording medium 307. Furthermore, each component is connected to each other by a bus 300.

[0041] Here, the CPU 301 controls the entire response generation device 201. The CPU 301 may have multiple cores. The memory 302 includes, for example, a read-only memory (ROM) and a random access memory (RAM). A program stored in the memory 302 is loaded into the CPU 301, causing the CPU 301 to execute coded processing.

[0042] The disk drive 303 controls reading / writing of data from / to the disk 304 under the control of the CPU 301. The disk 304 stores data written under the control of the disk drive 303. The disk 304 is, for example, a magnetic disk, an optical disk, or the like.

[0043] The communication I / F 305 is connected to the network 210 via a communication line, and is connected to an external computer (for example, the client device 202 shown in FIG. 2) via the network 210. The communication I / F 305 serves as an interface between the network 210 and the inside of the device, and controls input and output of data from and to the external computer. The communication I / F 305 is, for example, a modem or a LAN adapter.

[0044] The portable recording medium I / F 306 controls reading and writing of data from and to the portable recording medium 307 under the control of the CPU 301. The portable recording medium 307 stores data written under the control of the portable recording medium I / F 306. The portable recording medium 307 is, for example, a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), or a USB (Universal Serial Bus) memory.

[0045] Note that the answer generation device 201 may have, for example, an input device, a display, etc. in addition to the above-mentioned components. Furthermore, the answer generation device 201 does not have to have, for example, the portable recording medium I / F 306 and the portable recording medium 307 among the above-mentioned components. Furthermore, the client device 202 shown in Fig. 2 can also be realized by the same hardware configuration as the answer generation device 201. However, the client device 202 has, for example, an input device, a display, etc. in addition to the above-mentioned components.

[0046] (Storage contents of text database 220) Next, the contents stored in the text database 220 of the answer generation device 201 according to the first embodiment will be described with reference to Fig. 4. The text database 220 is realized by a storage device such as the memory 302 and the disk 304 shown in Fig. 3, for example.

[0047] Fig. 4 is an explanatory diagram showing an example of the contents stored in the text database 220. In Fig. 4, the text database 220 has fields for a text vector, Q (Question), and A (Answer), and stores text data (e.g., text data 401, 402) as records by setting information in each field.

[0048] Here, the sentence vector represents the characteristics of the question indicated in Q (Question). Q (Question) indicates the question. The question contains one or more sentences. A (Answer) indicates the answer to the question indicated in Q (Question). The answer contains one or more sentences. The pair of question (Q) and answer (A) is created based on, for example, past Q&A (Q&A) and question and answer examples accumulated as knowledge.

[0049] Here, the sentence vector is defined as information representing the characteristics of the question (Q), but is not limited to this. For example, the sentence vector may be information representing the characteristics of the question (Q) and the answer (A). Furthermore, although the sentence data is stored in a QA format, this is not limited to this. For example, the sentence data may be information about sentences extracted from textbooks, manuals, etc. related to a certain field.

[0050] (Example of functional configuration of answer generation device 201) Next, an example of a functional configuration of the answer generation device 201 according to the first embodiment will be described.

[0051] FIG. 5 is a block diagram showing an example of a functional configuration of the answer generation device 201 according to the first embodiment. In FIG. 5, the answer generation device 201 includes a receiving unit 501, a proofreading unit 502, a searching unit 503, a changing unit 504, a generating unit 505, an output unit 506, and a storage unit 510. The receiving unit 501 to the output unit 506 are functions that constitute the control unit 500, and specifically, the functions are realized by causing the CPU 301 to execute a program stored in a storage device such as the memory 302, the disk 304, or the portable recording medium 307 shown in FIG. 3, or by the communication I / F 305. The processing results of each functional unit are stored in a storage device such as the memory 302 or the disk 304. The storage unit 510 is realized by a storage device such as the memory 302 or the disk 304. Specifically, the storage unit 510 stores the sentence database 220 as shown in FIG. 4, for example. In FIG. 5, DB220 corresponds to the text database 220.

[0052] The receiving unit 501 receives a first question. The first question asks about something that the user does not understand or is uncertain about, and is expressed, for example, by one or more sentences. The first question is input, for example, on an inquiry screen (not shown) displayed on the client device 202 shown in FIG. 2. Specifically, for example, the receiving unit 501 receives the input first question from the client device 202, thereby receiving the first question.

[0053] Specifically, for example, the receiving unit 501 receives the first question sentence by receiving the first question sentence input from the client device 202. The receiving unit 501 may also receive the first question sentence by a user's operation input using an input device (not shown) of the device itself.

[0054] The proofreading unit 502 proofreads the received first question. Here, proofreading means correcting errors such as typos, omissions, and grammatical errors in the first question. Specifically, for example, the proofreading unit 502 may proofread the first question using a language model M. The language model M is a learning model for generating sentences.

[0055] The language model M is, for example, a large-scale language model (LLM). Alternatively, the language model M may be a small-scale language model (SLM). The language model M includes, for example, a first language model, a second language model, and a third language model.

[0056] The first language model is a language model used by the modifying unit 504 to generate the second question sentence. The second language model is a language model used by the proofreading unit 502 to proofread the first question sentence. The third language model is a language model used by the generating unit 505 to generate an answer. The first language model, the second language model, and the third language model may be the same.

[0057] The first language model, the second language model, and the third language model may be different from each other. For example, the first language model may be a machine learning model specialized in generating new sentences by changing the style of sentences. The second language model may be a machine learning model specialized in proofreading sentences. The third language model may be a machine learning model specialized in generating answers to questions.

[0058] The language model M is stored in, for example, the storage unit 510. The language model M may also be stored in another computer accessible by the answer generation device 201. In this case, the answer generation device 201 can use the language model M by accessing the other computer.

[0059] More specifically, for example, the proofreading unit 502 creates a prompt (command sentence) that instructs the user to correct errors such as typos and omissions in the first question sentence so that the meaning of the first question sentence remains the same. Then, the proofreading unit 502 provides the created prompt to the language model M (second language model) to proofread the first question sentence.

[0060] A specific example of a prompt for proofreading will be described later with reference to Fig. 8. Note that any existing technology may be used to proofread the first question sentence.

[0061] The search unit 503 refers to the storage unit 510 to search for first text data related to the proofread first question sentence. Here, the storage unit 510 stores a plurality of text data. The text data represents, for example, a text. The text is created based on, for example, past Q&A or question and answer cases. The text may also be a text extracted from a textbook, a manual, or the like.

[0062] Furthermore, the text data may represent, for example, a text and a feature of the text. The feature of the text is information that represents the features of the text, such as a text vector. The text vector is a fixed-length vector obtained by vectorizing the text. The text vector may be calculated from the entire text represented by the text data, or may be calculated from a portion of the text represented by the text data (for example, only Q in QA). Note that any existing technology may be used to vectorize the text.

[0063] Specifically, for example, the search unit 503 calculates a first text vector by vectorizing the proofread first question text. Then, the search unit 503 compares the text vectors represented by each piece of text data in the text database 220 shown in Fig. 4 with the calculated first text vector, thereby searching for first text data representing text having similar characteristics to the proofread first question text.

[0064] More specifically, for example, the search unit 503 calculates the inter-vector distance between the first text vector and a text vector represented by each text data in the text database 220. Then, the search unit 503 may search for text data for which the calculated inter-vector distance is equal to or less than a first threshold value as the first text data. The first threshold value can be set arbitrarily.

[0065] The modification unit 504 uses the language model M (first language model) in accordance with the retrieved first sentence data to generate a second question by modifying the style of the proofread first question so that the meaning of the proofread first question remains unchanged. Here, the style refers to features of sentence expression that appear in, for example, the words, grammar, and rhetoric used in the sentence.

[0066] For example, the writing style of the text data may reflect the author's individual style. Furthermore, in the case of texts related to a specific field, the text data may contain phrases or technical terms specific to that field. In contrast, the first question may not necessarily use the same writing expressions as the text data.

[0067] Therefore, the modification unit 504 generates the second question by, for example, modifying the style of the proofread first question in accordance with the style of the sentence represented by the retrieved first sentence data. Specifically, for example, the modification unit 504 creates a prompt that instructs the user to rewrite the proofread first question so that the meaning is the same as that of the proofread first question, using as many of the style (phrases, grammar, rhetoric, etc.) used in the sentence represented by the first sentence data as possible. Then, the modification unit 504 generates the second question by providing the created prompt to the language model M.

[0068] Furthermore, the search unit 503 searches for second text data related to the generated second question sentence with reference to the storage unit 510. Specifically, for example, the search unit 503 calculates a second text vector by vectorizing the generated second question sentence. Then, the search unit 503 compares the text vector represented by each piece of text data in the text database 220 with the calculated second text vector, thereby searching for second text data representing text having characteristics similar to those of the second question sentence.

[0069] To explain in more detail, for example, the search unit 503 calculates the inter-vector distance between the second text vector and a text vector represented by each text data in the text database 220. Then, the search unit 503 may search for text data for which the calculated inter-vector distance is equal to or less than a second threshold value, as the second text data.

[0070] The second threshold can be set arbitrarily. For example, the second threshold may be set to a value smaller than the first threshold. This allows the search unit 503 to search for the second sentence data under stricter conditions (searching for more similar data) than when searching for the first sentence data related to the proofread first question sentence.

[0071] The generation unit 505 generates an answer to the first question using the language model M (third language model) based on the searched second text data and the generated second question. Specifically, for example, the generation unit 505 creates a prompt that instructs the user to generate an answer to the second question based on the second text data and the second question. Then, the change unit 504 provides the created prompt to the language model M to generate an answer to the first question.

[0072] The output unit 506 outputs the generated answer in association with the first question sentence. The output format of the output unit 506 may be, for example, storage in a storage device such as the memory 302 or the disk 304, transmission to another computer via the communication I / F 305, display on a display (not shown), or printout on a printer (not shown).

[0073] Specifically, for example, the output unit 506 may transmit a response result indicating the generated answer in association with the received first question sentence to the client device 202. As a result, in the client device 202, for example, the generated answer in association with the first question sentence is displayed on an inquiry screen (not shown).

[0074] In the above description, the search unit 503 searches for first text data related to the proofread first question, but this is not limiting. For example, the search unit 503 may search for first text data related to the received first question (first question that has not been proofread). In this case, the change unit 504 may generate a second question by changing the style of the received first question (first question that has not been proofread) according to the searched first text data.

[0075] Furthermore, the generation unit 505 may generate an answer to the first question sentence using a language model M (third language model) based on the retrieved first sentence data (sentence data related to the proofread first question sentence) and the proofread first question sentence.

[0076] Furthermore, if the first threshold value used when searching for the first text data is too small, it may be difficult to search for related text data due to differences in writing style, etc. On the other hand, if the first threshold value is too large, the accuracy of searching for related text data may decrease. The value to be set as the first threshold value may differ depending on the content of the question text and the content of the text in the text database 220.

[0077] For this reason, the search unit 503 may prepare a plurality of patterns of the first threshold and search the first text data for each pattern. For example, suppose that patterns 1 to 3 are prepared in advance as a plurality of patterns. For pattern 1, the first threshold is set to a value equivalent to the second threshold used when searching the second text data.

[0078] In pattern 2, the first threshold is set to a value approximately 10% larger than the second threshold. In pattern 3, the first threshold is set to a value approximately 20% larger than the second threshold. For each of patterns 1 to 3, the search unit 503 may search for text data in which the calculated inter-vector distance is equal to or smaller than the first threshold, as the first text data.

[0079] In this case, the answer generation device 201 may generate a second question sentence, search for second sentence data, and generate and output an answer for each of patterns 1 to 3. This allows the answer generation device 201 to output an answer to the first question sentence for each of patterns 1 to 3.

[0080] The functional units (receiving unit 501 to output unit 506) of the answer generation device 201 may be realized by a plurality of computers (for example, the answer generation device 201 and the client device 202) in the answer generation system 200. In this case, communication between the functional units of different computers is performed by transmission and reception between the functional units via the network 210, for example.

[0081] (Example of operation of answer generation device 201) An example of the operation of the answer generation device 201 according to the first embodiment will be described.

[0082] Fig. 6 is an explanatory diagram showing an example of the operation of the answer generation device 201 according to the first embodiment. In Fig. 6, the answer generation device 201 refers to a QA list 610, generates text data 620, and registers the created text data 620 in the text database 220. In Fig. 6, DB220 corresponds to the text database 220.

[0083] The QA list 610 is a list of past QA. Specifically, for example, the answer generation device 201 calculates a sentence vector by vectorizing the question sentence (Q) for each QA in the QA list 610. Note that the answer generation device 201 may calculate a sentence vector by vectorizing the entire QA including the question sentence (Q) and the answer (A).

[0084] Then, the answer generation device 201 creates sentence data 620 including the calculated sentence vector and the QA, and registers the created sentence data 620 in the sentence database 220. The sentence data 620 corresponds to, for example, the sentence data 401 and 402 shown in FIG.

[0085] This allows the answer generation device 201 to create a database of information (past Q&A) accumulated as knowledge. Also, the answer generation device 201 can register a sentence vector together with the Q&A to make it easier to search for related information (Q&A) from the sentence database 220.

[0086] Here, it is assumed that the answer generation device 201 receives a question q(0) from the client device 202. The question q(0) corresponds to the first question.

[0087] First, the answer generation device 201 proofreads the received question q(0) using the language model M. Here, the proofread question q(0) is referred to as "question q(1)." The process of proofreading the question q(0) corresponds to a first correction of the question q(0).

[0088] Next, the answer generation device 201 searches for related QAs related to the proofread question q(1) by referring to the sentence database 220. Specifically, for example, the answer generation device 201 calculates a sentence vector v1 by vectorizing the proofread question q(1). The sentence vector v1 corresponds to the first sentence vector.

[0089] Then, the answer generation device 201 compares the sentence vectors represented by each sentence data (e.g., sentence data 620) in the sentence database 220 with the calculated sentence vector v1, and searches for sentence data representing sentences with characteristics similar to those of the proofread question sentence q(1) as related QAs.

[0090] Here, it is assumed that a related QA 621 related to the proofread question q(1) has been retrieved. The related QA 621 corresponds to the first text data.

[0091] In this case, the answer generation device 201 uses the language model M in accordance with the retrieved related QAs 621 to generate question q(2) by changing the style of the proofread question q(1) so that the meaning of the proofread question q(1) remains unchanged. The process of changing the style of the proofread question q(1) corresponds to a secondary correction of question q(0). The changed question q(2) corresponds to a second question.

[0092] Next, the answer generation device 201 searches for related QAs related to the changed question sentence q(2) by referring to the sentence database 220. Specifically, for example, the answer generation device 201 calculates a sentence vector v2 by vectorizing the changed question sentence q(2). The sentence vector v2 corresponds to a second sentence vector.

[0093] Then, the answer generation device 201 compares the sentence vectors represented by each sentence data in the sentence database 220 with the calculated sentence vector v2, and searches for sentence data representing sentences with similar characteristics to the changed question sentence q(2) as related QAs.

[0094] Here, it is assumed that a related QA 622 related to the changed question q(2) is retrieved. The related QA 622 corresponds to the second text data.

[0095] Next, the answer generation device 201 generates an answer to the question q(0) based on the searched related QA 622 and the changed question q(2) using the language model M. Here, it is assumed that an "answer a1" is generated as an answer to the question q(0). In this case, the answer generation device 201 outputs the generated answer a1 to the client device 202 in association with the received question q(0).

[0096] This enables the answer generation device 201 to improve the accuracy of searching for information (related QA622) related to the user's question q(0), and ultimately improve the accuracy of generating an answer (answer a1) to the user's question q(0).

[0097] (Example of corrected question) Next, examples of corrections (primary correction, secondary correction) to a question will be described with reference to Figures 7 to 11. First, the user's question will be described with reference to Figure 7.

[0098] Fig. 7 is an explanatory diagram showing an example of a user's question. In Fig. 7, question 700 indicates a user's question (first question). The answer generation device 201 receives the question 700 from, for example, the client device 202. In the example of Fig. 6, question 700 corresponds to question q(0).

[0099] Here, the question 700 contains typos and omissions. Therefore, if the question 700 is left as it is, when the question 700 is vectorized and related QAs are searched for, the distance between the vectors and the related QAs becomes large, which may result in a decrease in search accuracy.

[0100] For this reason, the answer generation device 201 proofreads the question sentence 700 by using the language model M. Specifically, for example, the answer generation device 201 proofreads the question sentence 700 by providing a proofreading prompt as shown in FIG.

[0101] Here, the calibration prompt will be explained with reference to FIG.

[0102] Fig. 8 is an explanatory diagram showing a specific example of a proofreading prompt. In Fig. 8, prompt 800 instructs the user to correct errors such as typos, omissions, and grammatical mistakes in question 700 so that the meaning remains the same as question 700 (see Fig. 7).

[0103] Specifically, the prompt 800 is a command statement that instructs the language model M to proofread the question 700 based on the #constraint 810 and the #input statement 820. The #input statement 820 corresponds to the question 700. The answer generation device 201 can proofread the question 700 by providing the prompt 800 to the language model M.

[0104] Next, the question sentence after the first correction (proofreading) will be explained with reference to FIG.

[0105] FIG. 9 is an explanatory diagram showing an example of a question sentence after the first correction. In FIG. 9, question sentence 900 shows the question sentence after the first correction (after proofreading). In question sentence 900, errors such as typos and omissions in question sentence 700 shown in FIG. 7 have been corrected. In the example of FIG. 6, question sentence 900 corresponds to question sentence q(1).

[0106] The answer generation device 201 refers to the sentence database 220 to search for a related QA related to the question sentence 900. Here, it is assumed that the sentence data 402 (QA) shown in Fig. 4 is searched for from the sentence database 220 as a related QA related to the question sentence 900. It is assumed that the sentence data 402 is searched for by comparing the sentence vector of the question sentence 900 with the sentence vector of the entire sentence (QA) represented by the sentence data 402.

[0107] In this case, the answer generation device 201 uses the language model M in accordance with the sentence data 402 to generate a question sentence in which the style of the question sentence 900 is changed so that the meaning of the question sentence 900 is unchanged. Specifically, for example, the answer generation device 201 changes the style of the question sentence 900 by providing a style change prompt as shown in FIG.

[0108] Here, the prompt for changing the writing style will be explained with reference to FIG.

[0109] Fig. 10 is an explanatory diagram (part 1) showing a specific example of a prompt for changing writing style. In Fig. 10, prompt 1000 instructs the user to rewrite question 900 (see Fig. 9) using as many words, phrases, rhetoric, and the like as possible that are used in the related Q&A represented by sentence data 402, so that the meaning remains the same as question 900.

[0110] Specifically, prompt 1000 is a command statement that instructs language model M to rewrite #input sentence 1030 without changing its meaning, using as many words and phrases as possible from #example sentence 1020, based on #constraints 1010 and #input sentence 1030. #input sentence 1030 corresponds to question sentence 900.

[0111] By providing the prompt 1000 to the language model M, the answer generation device 201 can rewrite the question 900 using words and phrases used in the related Q&A (sentence data 402) so that the meaning remains unchanged.

[0112] Here, the question sentence after the second revision (after the change in writing style) will be explained with reference to FIG.

[0113] FIG. 11 is an explanatory diagram showing an example of a question sentence after secondary revision. In FIG. 11, question sentence 1100 shows the question sentence after secondary revision (after style change). In question sentence 1100, question sentence 900 shown in FIG. 9 has been revised so that the meaning remains unchanged by using the phrase "purchase option," which is a characteristic expression in the related Q&A (text data 402). In the example of FIG. 6, question sentence 1100 corresponds to question sentence q(2).

[0114] The answer generation device 201 searches again for related QAs using the question sentence 1100 customized to match the writing style in the sentence database 220, making it easier to find more appropriate information and improving the search accuracy of related QAs used to generate answers.

[0115] (Various processing procedures of the answer generation device 201) Next, various processing procedures of the answer generation device 201 according to the first embodiment will be described. First, the DB registration processing procedure of the answer generation device 201 will be described with reference to FIG.

[0116] 12 is a flowchart showing an example of a DB registration processing procedure of the answer generation device 201 according to the first embodiment. In the flowchart of FIG. 12, first, the answer generation device 201 receives a QA list (step S1201). Specifically, for example, the answer generation device 201 receives the QA list by receiving it from a management device (not shown). The management device is, for example, a computer used by an administrator of the answer generation system 200.

[0117] Then, the answer generation device 201 selects an unselected QA from the received QA list (step S1202). Next, the answer generation device 201 calculates a sentence vector by vectorizing the question sentence (Q) of the selected QA (step S1203).

[0118] Then, the answer generation device 201 registers the calculated sentence vector and sentence data including the selected QA in the sentence database 220 (step S1204). Next, the answer generation device 201 determines whether there is an unselected QA that has not been selected from the QA list (step S1205).

[0119] If there is an unselected QA (step S1205: Yes), the answer generation device 201 returns to step S1202. On the other hand, if there is no unselected QA (step S1205: No), the answer generation device 201 ends the series of processes according to this flowchart.

[0120] This allows the answer generation device 201 to create a database of past Q&As accumulated as knowledge, and to register sentence vectors for quantitatively evaluating the relevance between texts. Note that in step S1203, the answer generation device 201 may calculate sentence vectors by vectorizing the entire Q&A including the question sentence (Q) and the answer (A).

[0121] Next, the answer generation process procedure of the answer generation device 201 will be described with reference to FIG.

[0122] 13 is a flowchart showing an example of an answer generation processing procedure of the answer generation device 201 according to the first embodiment. In the flowchart of FIG. 13, first, the answer generation device 201 determines whether or not a question sentence (first question sentence) has been received (step S1301). Here, the answer generation device 201 waits to receive a question sentence (step S1301: No).

[0123] When the answer generation device 201 receives a question sentence (step S1301: Yes), it proofreads the received question sentence using the language model M (step S1302). Next, the answer generation device 201 calculates a sentence vector by vectorizing the proofread question sentence (step S1303).

[0124] Then, the answer generation device 201 searches the text database 220 for related QAs that have similar features to the proofread question text, based on the calculated text vector (step S1304). Specifically, for example, the answer generation device 201 calculates the inter-vector distance between the text vector represented by each text data in the text database 220 and the calculated text vector, and searches for text data whose calculated inter-vector distance is equal to or less than a first threshold, as related QAs.

[0125] Next, the answer generation device 201 generates a question sentence (second question sentence) by changing the style of the proofread question sentence using the language model M according to the style of the retrieved related QA so that the meaning of the proofread question sentence remains unchanged (step S1305). Next, the answer generation device 201 calculates a sentence vector by vectorizing the generated question sentence (step S1306).

[0126] Then, based on the sentence vector calculated in step S1306, the answer generation device 201 searches the sentence database 220 for related QAs having characteristics similar to those of the generated question sentence (step S1307). Specifically, for example, the answer generation device 201 calculates the inter-vector distance between the sentence vector represented by each sentence data in the sentence database 220 and the calculated sentence vector, and searches for sentence data for which the calculated inter-vector distance is equal to or less than a second threshold, as related QAs.

[0127] Next, the answer generation device 201 generates an answer to the received question sentence using the language model M based on the related QAs searched in step S1307 and the generated question sentence (step S1308).The answer generation device 201 then outputs the generated answer in association with the received question sentence (step S1309), and ends the series of processes according to this flowchart.

[0128] This allows the answer generation device 201 to output an answer to the received question. The answer generation device 201 may omit the process of step S1302. In this case, the answer generation device 201 performs the processes of steps S1303 to S1305 for the question received in step S1301.

[0129] As described above, the answer generation device 201 according to the first embodiment can refer to the storage unit 510 that stores a plurality of sentence data to search for first sentence data related to a first question, and can generate a second question by changing the style of the first question so as to maintain the same meaning as the first question using the language model M (first language model) according to the searched first sentence data. The language model M is a machine learning model for generating sentences, and is, for example, a large-scale language model (LLM).

[0130] As a result, when generating an answer to a question (first question), the answer generation device 201 can improve the accuracy of searching for information related to the question (for example, related Q&A). For example, the answer generation device 201 can rewrite the first question using technical terms and phrases used in the sentence represented by the first sentence data so that the meaning does not change. As a result, the answer generation device 201 can make the style of the question similar to the sentences in the DB (for example, the sentence database 220), and can generate a question that is more likely to hit a more appropriate related Q&A.

[0131] Furthermore, the answer generation device 201 can proofread the first question using the language model M (second language model). Then, the answer generation device 201 can refer to the storage unit 510 to search for first sentence data related to the proofread first question, and can generate a second question by changing the style of the proofread first question using the language model M (first language model) according to the searched first sentence data so that the meaning of the proofread first question remains unchanged.

[0132] This allows the answer generation device 201 to correct typos, omissions, grammatical errors, and the like in the question text, thereby further improving the accuracy of searching for information related to the question text.

[0133] Furthermore, the answer generation device 201 can refer to the memory unit 510 to search for second sentence data related to the generated second question sentence, and generate an answer to the first question sentence using the language model M (third language model) based on the searched second sentence data and the generated second question sentence.

[0134] This enables the answer generation device 201 to generate an answer using more appropriate information related to the question (for example, related Q&A), thereby improving the accuracy of generating an answer to the question.

[0135] Furthermore, the answer generation device 201 can search for first sentence data representing a sentence having similar features to the first question sentence by comparing the feature amounts of sentences represented by sentence data included in the plurality of sentence data with the feature amounts of the first question sentence with reference to the storage unit 510. The storage unit 510 stores a plurality of sentence data including sentence data representing sentences and the feature amounts of the sentences.

[0136] This allows the answer generation device 201 to search for sentences in a DB (for example, the sentence database 220) that have similar features to the first question sentence as information related to the first question sentence. Furthermore, by storing the feature amounts (for example, sentence vectors) of each sentence in the DB in advance, the answer generation device 201 can reduce the processing time and processing load required for the search process compared to when calculating the feature amounts of each sentence each time when searching for information related to the first question sentence.

[0137] Furthermore, the answer generation device 201 can search for second sentence data representing a sentence with similar features to the second question sentence by comparing the feature amounts of sentences represented by sentence data included in the plurality of sentence data with the feature amounts of the second question sentence with reference to the storage unit 510. The storage unit 510 stores a plurality of sentence data including sentence data representing sentences and the feature amounts of the sentences.

[0138] This allows the answer generation device 201 to search for sentences in the DB that have similar features to the second question sentence as information related to the second question sentence. Also, by storing the feature amounts (for example, sentence vectors) of each sentence in the DB in advance, the answer generation device 201 can reduce the processing time and processing load required for the search process compared to when calculating the feature amounts of each sentence each time when searching for information related to the second question sentence.

[0139] Furthermore, the answer generation device 201 can output the generated answer in association with the first question sentence.

[0140] This allows the answer generation device 201 to provide a highly accurate answer to a question from a user.

[0141] (Embodiment 2) Next, a description will be given of the answer generation device 201 according to the second embodiment. Note that the same parts as those described in the first embodiment will be given the same reference numerals, and illustrations and descriptions thereof may be omitted.

[0142] In the second embodiment, when a writing style (e.g., characteristic phrases or grammar) that is frequently used in related Q&A is clear, information representing that writing style is prepared in advance, and a generation method is explained that can be used to change the writing style of a question.

[0143] For example, in the case of a sentence related to a specific field, terms (technical terms) and phrases specific to that field may be often used. On the other hand, such technical terms and phrases may not always be used in a question sentence from a user. Therefore, the answer generation device 201 refers to a sentence feature table 1400 as shown in FIG. 14 described later, and generates a new question sentence by changing the style of the question sentence.

[0144] First, the contents stored in the sentence feature table 1400 used by the answer generation device 201 according to the second embodiment will be described.

[0145] Fig. 14 is an explanatory diagram showing an example of the contents stored in the sentence feature table 1400. In Fig. 14, the sentence feature table 1400 has fields for technical terms / phrases and explanations, and stores sentence feature data (for example, sentence feature data 1401 to 1404) as records by setting information in each field.

[0146] Here, the technical term / phrase indicates a technical term or characteristic phrase that is frequently used in the sentence represented by the sentence data in the sentence database 220. The sentence is, for example, a sentence related to a specific field. In the example of FIG. 14, the specific field is "networks." The explanation is information that can identify a different term / phrase that is synonymous with the technical term / phrase.

[0147] For example, the explanation may indicate a term that is synonymous with the technical term and is generally used. Furthermore, if the technical term is an abbreviation, the explanation may indicate the full name of the technical term (e.g., sentence feature data 1401). Furthermore, the explanation may be a sentence that explains the meaning of the technical term (e.g., sentence feature data 1402, 1403). Furthermore, the explanation may indicate a commonly used phrase that is synonymous with the characteristic phrase (e.g., sentence feature data 1404).

[0148] The sentence feature table 1400 may be automatically created based on information accumulated as knowledge by, for example, the answer generation device 201. Also, the sentence feature table 1400 may be manually created by, for example, an administrator of the answer generation system 200.

[0149] (Example of functional configuration of answer generation device 201) Next, an example of a functional configuration of the answer generation device 201 according to the second embodiment will be described.

[0150] 15 is a block diagram showing an example of a functional configuration of an answer generation device 201 according to the second embodiment. In FIG. 15, the answer generation device 201 includes a receiving unit 501, a proofreading unit 502, a generating unit 505, an output unit 506, a storage unit 510, a second search unit 1501, and a second change unit 1502. The receiving unit 501 to the second change unit 1502 are functions that constitute the control unit 500, and specifically, the functions are realized by causing the CPU 301 to execute a program stored in a storage device such as the memory 302, the disk 304, or the portable recording medium 307 shown in FIG. 3, or by the communication I / F 305. The processing results of each functional unit are stored in a storage device such as the memory 302 or the disk 304.

[0151] The following describes functional units of the answer generation device 201 according to the second embodiment that are different from those of the answer generation device 201 according to the first embodiment.

[0152] Here, storage unit 510 stores a plurality of pieces of text data. The plurality of pieces of text data includes text data representing text. The text is, for example, text relating to a specific field (area). The plurality of pieces of text data also includes text data representing a correspondence between a style used in a text represented by at least one piece of text data included in the plurality of pieces of text data and a different style that is synonymous with the style.

[0153] The style of writing is expressed, for example, by words and phrases or grammar. For example, the text data may represent a correspondence between characteristic words used in a sentence represented by at least one of the text data included in the plurality of text data and different words that are synonymous with the characteristic words. The characteristic words are, for example, technical terms used in a sentence related to a particular field.

[0154] The text data may also represent a correspondence between a characteristic grammar used in a sentence represented by at least one of the text data included in the plurality of text data and a different grammar that is synonymous with the characteristic grammar. The characteristic grammar may be, for example, a characteristic phrase used in a sentence relating to a particular field.

[0155] Specifically, for example, the storage unit 510 stores the sentence database 220 as shown in Fig. 4 and the sentence feature table 1400 as shown in Fig. 14. The sentence data in the sentence database 220 and the sentence feature data in the sentence feature table 1400 correspond to examples of sentence data stored in the storage unit 510. In Fig. 15, DB220 corresponds to the sentence database 220. TB1400 corresponds to the sentence feature table 1400.

[0156] The second search unit 1501 refers to the memory unit 510 to search for first sentence data that represents a correspondence between a first writing style contained in the proofread first question sentence and a second writing style that is synonymous with the first writing style and is used in a sentence represented by at least one of the sentence data contained in the multiple sentence data.

[0157] The proofread first question sentence is a first question sentence proofread by proofreading unit 502. The plurality of sentence data is a plurality of sentence data stored in storage unit 510. The first sentence data corresponds to sentence data related to the first question sentence. The writing style is expressed by, for example, words and grammar.

[0158] Specifically, for example, the second search unit 1501 may refer to the storage unit 510 and search for first sentence data that indicates a correspondence relationship between a first term included in the proofread first question sentence and a second term that is synonymous with the first term and is used in a sentence represented by at least one of sentence data included in the plurality of sentence data. Here, the first term is, for example, a general-purpose term that is synonymous with a technical term in a specific field or the official name of the technical term. The second term is, for example, a technical term in a specific field.

[0159] To explain in more detail, for example, the second search unit 1501 may search the sentence features table 1400 for sentence feature data in which the phrase shown in the description field is "first phrase" as the first sentence data. Also, the second search unit 1501 may search the sentence features table 1400 for sentence feature data in which the phrase meaning the sentence shown in the description field is "first phrase" as the first sentence data.

[0160] In addition, the second search unit 1501 may refer to the memory unit 510 to search for first sentence data that represents a correspondence between a first grammar included in the proofread first question sentence and a second grammar that is synonymous with the first grammar and is used in a sentence represented by at least one of the sentence data included in the multiple sentence data.

[0161] Here, the first usage is, for example, a general phrase such as "A is C. And B is C." The second usage is, for example, a characteristic phrase such as "Both A and B are C." To explain in more detail, for example, the second search unit 1501 may search the sentence feature table 1400 for sentence feature data whose usage indicated in the description field is "first usage" as the first sentence data.

[0162] The second modification unit 1502 uses the language model M to generate a second question sentence in accordance with the first sentence data retrieved by the second search unit 1501, by modifying the first writing style contained in the proofread first question sentence to a second writing style so that the meaning of the second question sentence remains unchanged from the first question sentence proofread by the proofreading unit 502.

[0163] Specifically, for example, the second modifying unit 1502 creates a prompt to instruct the user to rewrite the proofread first question using a second writing style (second words and phrases or a second grammar) expressed by the first text data so that the meaning of the proofread first question remains the same. Then, the modifying unit 504 provides the created prompt to the language model M to generate the second question.

[0164] The prompt for changing the writing style will be described later with reference to FIG.

[0165] Furthermore, the second search unit 1501 refers to the storage unit 510 to search for second text data related to the second question generated by the second change unit 1502. The generation unit 505 then generates an answer to the first question using the language model M based on the second text data searched by the second search unit 1501 and the second question generated by the second change unit 1502.

[0166] In the above description, the second search unit 1501 searches for the first text data for the proofread first question, but this is not limiting. For example, the second search unit 1501 may search for the first text data for the received first question (the first question that has not been proofread). In this case, the second change unit 1502 may generate a second question from the received first question (the first question that has not been proofread) in accordance with the searched first text data.

[0167] (Prompt for style change) Here, the prompt for changing writing style will be described with reference to Fig. 16. Here, it is assumed that sentence feature data 1401 to 1404 in a sentence feature table 1400 are searched for as the first sentence data.

[0168] Fig. 16 is an explanatory diagram (part 2) showing a specific example of a prompt for changing the writing style. In Fig. 16, prompt 1600 instructs the user to rewrite the question using technical terms / phrases (corresponding to the second writing style) represented by sentence feature data 1401 to 1404 so that the meaning of the question remains the same.

[0169] Specifically, prompt 1600 is an instruction that instructs language model M to rewrite #input sentence 1630 based on #constraints 1610 and #input sentence 1630, using as many words and phrases as possible from #example sentence 1620 without changing the meaning at all. #input sentence 1630 corresponds to a question. Note that in FIG. 16, "Questions about networks" in #input sentence 1630 represents some kind of question about networks.

[0170] By providing the prompt 1600 to the language model M, the answer generation device 201 can rewrite the question using technical terms and characteristic phrases represented by the sentence feature data 1401 to 1404 without changing the meaning.

[0171] (Example of operation of answer generation device 201) An example of the operation of the answer generation device 201 according to the second embodiment will be described.

[0172] 17 is an explanatory diagram showing an example of the operation of the answer generation device 201 according to the second embodiment. In FIG. 17, the answer generation device 201 refers to a QA list 1710, generates text data 1720, and registers the created text data 1720 in the text database 220.

[0173] 17, the Q&A list 1710 is a list of past Q&As. The text data 1720 corresponds to, for example, the text data 401 and 402 shown in FIG. 4. In FIG. 17, the DB 220 corresponds to the text database 220. The TB 1400 corresponds to the text feature table 1400.

[0174] Furthermore, the answer generation device 201 creates the sentence feature table 1400 with reference to the QA list 1710. Specifically, for example, the answer generation device 201 extracts words (terms) and grammar (phrases) that are frequently used in the QAs in the QA list 1710 as technical terms / phrases.

[0175] Next, the answer generation device 201 creates a prompt to instruct the generation of information (sentence feature data) that indicates the correspondence between the extracted technical term / phrase and a different word or phrase or grammar that is synonymous with the technical term / phrase. Then, the change unit 504 provides the created prompt to the language model M to generate the sentence feature table 1400.

[0176] This allows the answer generation device 201 to store a list of information that associates words and phrases that are frequently used in information accumulated as knowledge (past Q&A) with different synonymous words and phrases.

[0177] Here, it is assumed that the answer generation device 201 receives a question q(0) from the client device 202. The question q(0) corresponds to the first question.

[0178] First, the answer generation device 201 proofreads the received question sentence q(0) using the language model M. Here, the proofread question sentence q(0) is referred to as "question sentence q(1)."

[0179] Next, the answer generation device 201 searches for sentence feature data related to the proofread question q(1) by referring to the sentence feature table 1400. The sentence feature data to be searched is, for example, sentence feature data that indicates the correspondence between a first writing style (phrases, grammar, etc.) included in the proofread question q(1) and a second writing style that is synonymous with the first writing style and is used in sentences in the sentence database 220.

[0180] Here, it is assumed that the sentence feature data 1730 related to the proofread question sentence q(1) has been searched for. The sentence feature data 1730 corresponds to the first sentence data.

[0181] In this case, the answer generation device 201 generates a question q(2) by changing the first writing style contained in the proofread question q(1) to a second writing style so that the meaning of the proofread question q(1) remains unchanged, using the language model M in accordance with the retrieved sentence feature data 1730. The changed question q(2) corresponds to the second question.

[0182] Next, the answer generation device 201 searches for related QAs related to the changed question sentence q(2) by referring to the sentence database 220. Specifically, for example, the answer generation device 201 calculates a sentence vector v2 by vectorizing the changed question sentence q(2). The sentence vector v2 corresponds to a second sentence vector.

[0183] Then, the answer generation device 201 compares the sentence vectors represented by each sentence data in the sentence database 220 with the calculated sentence vector v2, and searches for sentence data representing sentences with similar characteristics to the changed question sentence q(2) as related QAs.

[0184] Here, it is assumed that a related QA 1740 related to the changed question q(2) is searched for. The related QA 1740 corresponds to the second text data.

[0185] Next, the answer generation device 201 generates an answer to the question q(0) based on the searched related QA 1740 and the changed question q(2) using the language model M. Here, it is assumed that an "answer a1" is generated as an answer to the question q(0). In this case, the answer generation device 201 outputs the generated answer a1 to the client device 202 in association with the received question q(0).

[0186] This enables the answer generation device 201 to improve the accuracy of searching for information (related QA1740) related to the question q(0) from the user, and ultimately improve the accuracy of generating an answer (answer a1) to the question q(0) from the user.

[0187] (Various processing procedures of the answer generation device 201) Next, various processing procedures of the answer generation device 201 according to the second embodiment will be described. First, a table creation processing procedure of the answer generation device 201 will be described with reference to Fig. 18. Note that the DB registration processing procedure of the answer generation device 201 is the same as the processing procedure shown in Fig. 12, and therefore will not be illustrated or described again.

[0188] Fig. 18 is a flowchart showing an example of a table creation procedure of the answer generation device 201 according to the second embodiment. In the flowchart of Fig. 18, first, the answer generation device 201 receives a QA list (step S1801).

[0189] Next, the answer generation device 201 refers to the received QA list and generates sentence feature data that indicates the correspondence between technical terms / phrases in the QA list and different words or phrases or usages that are synonymous with the technical terms / phrases (step S1802).The answer generation device 201 then registers the created sentence feature data in the sentence feature table 1400 (step S1803), and ends the series of processes according to this flowchart.

[0190] This allows the answer generation device 201 to store a list of information that associates words and phrases that are frequently used in information accumulated as knowledge (past Q&A) with different synonymous words and phrases.

[0191] Next, the answer generation process procedure of the answer generation device 201 will be described with reference to FIG.

[0192] Fig. 19 is a flowchart showing an example of an answer generation processing procedure of the answer generation device 201 according to the second embodiment. In the flowchart of Fig. 19, first, the answer generation device 201 determines whether or not a question sentence (first question sentence) has been received (step S1901).

[0193] Here, the answer generation device 201 waits to receive a question sentence (step S1901: No). Then, when the answer generation device 201 receives a question sentence (step S1901: Yes), it proofreads the received question sentence using the language model M (step S1902).

[0194] Next, the answer generation device 201 refers to the sentence feature table 1400 to search for sentence feature data related to the proofread question (step S1903). Then, the answer generation device 201 uses the language model M according to the searched sentence feature data to generate a question (second question) in which the style of the proofread question is changed so that the meaning of the question remains the same (step S1904).

[0195] Next, the answer generation device 201 calculates a sentence vector by vectorizing the generated question sentence (step S1905). Then, the answer generation device 201 searches the sentence database 220 for related QAs that have similar features to the generated question sentence, based on the calculated sentence vector (step S1906).

[0196] Next, the answer generation device 201 generates an answer to the received question sentence using the language model M based on the searched related QAs and the generated question sentence (step S1907). Then, the answer generation device 201 outputs the generated answer in association with the received question sentence (step S1908), and ends a series of processes according to this flowchart.

[0197] This allows the answer generation device 201 to output an answer to the received question.

[0198] As described above, the answer generation device 201 according to the second embodiment can search for first text data related to the proofread first question by referring to the storage unit 510 that stores a plurality of text data. Here, the plurality of text data includes text data that represents a correspondence relationship between a writing style used in a sentence represented by at least one of the text data included in the plurality of text data and a different writing style that is synonymous with the writing style. Furthermore, the first text data represents a correspondence relationship between a first writing style included in the proofread first question and a second writing style that is synonymous with the first writing style and used in a sentence represented by at least one of the text data included in the plurality of text data. Then, the answer generation device 201 can use the language model M in accordance with the searched first text data to generate a second question by changing the first writing style included in the proofread first question to a second writing style so that the meaning of the second question remains unchanged.

[0199] As a result, when generating an answer to a question (first question), the answer generation device 201 can improve the accuracy of searching for information related to the question (for example, related Q&A). For example, the answer generation device 201 can use the sentence feature table 1400 to customize the question to match technical terms and phrases commonly used in a specific field, thereby generating a question that is more likely to find appropriate related Q&A. Furthermore, when searching the first sentence data, the answer generation device 201 does not need to vectorize the proofread first question, thereby reducing the processing load required for searching the first sentence data.

[0200] The sentence database 220 may contain sentences relating to a plurality of fields. In this case, the answer generation device 201 may prepare a sentence feature table 1400 as shown in Fig. 14 for each field included in the plurality of fields. Then, the answer generation device 201 may search for the first sentence data by referring to the sentence feature table 1400 for each field included in the plurality of fields.

[0201] This allows the answer generation device 201 to customize the question sentence for each field included in multiple fields to match the technical terms and phrases commonly used in each field, and generate a question sentence that is more likely to hit a more appropriate related Q&A.

[0202] Furthermore, when accepting the first question sentence, the answer generation device 201 may accept a specification of the field the question sentence relates to. In this case, the answer generation device 201 may search for the first sentence data by, for example, referring to the sentence feature table 1400 corresponding to the specified field.

[0203] The first and second embodiments may be combined to the extent that no contradiction occurs. For example, the second search unit 1501 of the answer generation device 201 according to the second embodiment may have the same function as the search unit 503 of the answer generation device 201 according to the first embodiment. Furthermore, the second change unit 1502 of the answer generation device 201 according to the second embodiment may have the same function as the change unit 504 of the answer generation device 201 according to the first embodiment.

[0204] From these facts, the answer generation device 201 according to this embodiment can improve the accuracy of searching for information related to a question, and can generate a highly accurate answer to the question. For example, the answer generation device 201 can realize a highly accurate and natural response even to a highly specialized question about a specific field (such as a company or public service).

[0205] Here, the Score shown in the following formula (1) is an evaluation index that represents the accuracy of retrieval of information related to a question. The Score corresponds to a composite calculation of the probability of the existence of "related QA: T1," "potentially related QA: T2," and "unrelated QA: T3" when multiple related QAs are selected for a question. However, it is assumed that which QAs are related, possibly related, and unrelated are known. Also, let n1, n2, and n3 be "{n1, n2, n3} = {2, 1, 2}." Ideally, the probabilities p1 and p2 of T1 and T2 are 100%, and the probability p3 of T3 is 0%.

[0206]

number

[0207] Taking this score as an example, by applying this generation method, the search accuracy improved by about 4.4% compared to when this generation method was not applied.

[0208] The generation method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. The generation program is recorded on a computer-readable recording medium such as a hard disk, flexible disk, CD-ROM, DVD, or USB memory, and is executed by being read from the recording medium by the computer. The generation program may also be distributed via a network such as the Internet.

[0209] In addition, the information processing device 101 (answer generation device 201) described in this embodiment can also be realized by an application-specific IC such as a standard cell or a structured ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device) such as an FPGA.

[0210] The following additional notes are provided regarding the above-described embodiment.

[0211] (Supplementary Note 1) A storage unit storing a plurality of sentence data is referenced to search for first sentence data related to the first question sentence; generating a second question by changing the style of the first question so that the meaning of the first question remains unchanged, using a first language model for generating sentences, according to the retrieved first sentence data; A generating program that causes a computer to execute a process.

[0212] (Supplementary Note 2) causing the computer to execute a process of proofreading the first question sentence using a second language model that is the same as or different from the first language model; The process of searching for the first text data includes: referring to the storage unit to search for first sentence data related to the proofread first question sentence; The process of generating the second question includes: The generation program described in Appendix 1, characterized in that, in accordance with the retrieved first sentence data, a second question is generated by using the first language model to change the style of the proofread first question so that the meaning of the proofread first question remains unchanged.

[0213] (Supplementary Note 3) referring to the storage unit, searching for second sentence data related to the generated second question sentence; generating an answer to the first question using a third language model that is the same as or different from the first language model, based on the retrieved second text data and the second question; 3. The generation program according to claim 1 or 2, which causes the computer to execute a process.

[0214] (Supplementary Note 4) The plurality of sentence data includes sentence data representing sentences and feature quantities of the sentences, The process of searching for the first text data includes: referring to the storage unit and comparing a feature amount of a sentence represented by sentence data included in the plurality of sentence data with a feature amount of the first question sentence, thereby searching for first sentence data representing a sentence having a feature similar to that of the first question sentence; 4. The generation program according to any one of Supplementary Notes 1 to 3.

[0215] (Supplementary Note 5) The plurality of sentence data includes sentence data representing sentences and feature quantities of the sentences, The process of searching for the second text data includes: referring to the storage unit and comparing a feature amount of a sentence represented by sentence data included in the plurality of sentence data with a feature amount of the second question sentence, thereby searching for second sentence data representing a sentence having a feature similar to that of the second question sentence; 4. The generation program according to claim 3,

[0216] (Supplementary Note 6) The plurality of sentence data includes sentence data representing a correspondence relationship between a style used in a sentence represented by at least one of the sentence data included in the plurality of sentence data and a different style that is synonymous with the style, The process of searching for the first text data includes: referring to the storage unit to search for first sentence data that indicates a correspondence relationship between a first sentence style included in the first question sentence and a second sentence style that is synonymous with the first sentence style and is used in a sentence expressed by at least one sentence data included in the plurality of sentence data; The process of generating the second question includes: generating a second question by changing the first writing style included in the first question to the second writing style using the first language model according to the retrieved first sentence data so that the meaning of the first question is unchanged; 6. The generation program according to any one of Supplementary Notes 1 to 5.

[0217] (Supplementary Note 7) The generation program according to Supplementary Note 6, wherein the sentence represented by at least one of the sentence data is a sentence relating to a specific field.

[0218] (Appendix 8) The process of generating the second question sentence includes: creating a prompt to instruct the user to rewrite the first question sentence using as much of the style used in the sentence represented by the first sentence data as possible so that the meaning of the first question sentence remains the same; generating the second question sentence by providing the created prompt to the first language model; 2. The generation program according to claim 1,

[0219] (Supplementary Note 9) The generated answer is output in association with the first question sentence. 4. The generation program according to claim 3, wherein the program causes the computer to execute a process.

[0220] (Supplementary Note 10) The generation program according to any one of Supplementary Notes 1 to 9, wherein the first language model is an LLM (Large Language Model).

[0221] (Supplementary Note 11) A storage unit storing a plurality of sentence data is referred to, and a first sentence data related to the first question sentence is searched for; generating a second question by changing the style of the first question so that the meaning of the first question remains unchanged, using a first language model for generating sentences, according to the retrieved first sentence data; A generating method characterized in that the processing is carried out by a computer.

[0222] (Supplementary Note 12) A storage unit storing a plurality of sentence data is referred to, and a first sentence data related to the first question sentence is searched for; generating a second question by changing the style of the first question so that the meaning of the first question remains unchanged, using a first language model for generating sentences, according to the retrieved first sentence data; An information processing device comprising a control unit. [Explanation of symbols]

[0223] 101 Information processing equipment 110,510 storage section 120,M language models 200 Answer Generation System 201 Answer generation device 202 Client device 210 Network 220 Text Database 300 Bus 301 CPU 302 memory 303 Disk Drive 304 Disk 305 Communication I / F 306 Portable Recording Media I / F 307 Portable Recording Media 401,402,620,1720 Text data 500 control section 501 Reception 502 Proofreading Department 503 Search Department 504 Changes 505 Generation part 506 Output section 610,1710 QA List 621,622,1740 Related Q&A 700,900,1100,q(0),q(1),q(2) Question 800,1000,1600 prompts 1400 Sentence Feature Table 1401,1402,1403,1404,1730 Text feature data 1501 Second Search Unit 1502 Second Change a1 answer v1,v2 sentence vector q1 First question q2 Second question r1 First sentence data

Claims

1. referring to a storage unit that stores a plurality of sentence data, and searching for first sentence data related to the first question sentence; generating a second question by changing the style of the first question so that the meaning of the first question is the same as that of the first question, using a first language model for generating sentences according to the retrieved first sentence data; A generating program that causes a computer to execute a process.

2. causing the computer to perform a process of proofreading the first question sentence using a second language model that is the same as or different from the first language model; The process of searching for the first text data includes: referring to the storage unit to search for first sentence data related to the proofread first question sentence; The process of generating the second question sentence includes:

2. The generation program according to claim 1, further comprising: using the first language model to generate a second question by changing the style of the proofread first question so that the meaning of the proofread first question remains unchanged, in accordance with the first sentence data retrieved.

3. referring to the storage unit to search for second sentence data related to the generated second question sentence; generating an answer to the first question using a third language model that is the same as or different from the first language model, based on the retrieved second text data and the second question; 2. The generation program according to claim 1, wherein the program causes the computer to execute a process.

4. the plurality of sentence data include sentence data representing sentences and feature amounts of the sentences, The process of searching for the first text data includes: referring to the storage unit and comparing a feature amount of a sentence represented by sentence data included in the plurality of sentence data with a feature amount of the first question, thereby searching for first sentence data representing a sentence having a feature similar to that of the first question; 2. The generating program according to claim 1 .

5. the plurality of sentence data includes sentence data representing a correspondence relationship between a style used in a sentence represented by at least one of the sentence data included in the plurality of sentence data and a different style that is synonymous with the style; The process of searching for the first text data includes: referring to the storage unit to search for first sentence data that indicates a correspondence relationship between a first sentence style included in the first question sentence and a second sentence style that is synonymous with the first sentence style and is used in a sentence expressed by at least one of the sentence data included in the plurality of sentence data; The process of generating the second question sentence includes: generating a second question sentence by changing the first writing style included in the first question sentence to the second writing style using the first language model according to the retrieved first sentence data so that the meaning of the first question sentence is unchanged; 2. The generating program according to claim 1 .

6. outputting the generated answer in association with the first question sentence; 4. The generation program according to claim 3, wherein the program causes the computer to execute processing.

7. referring to a storage unit that stores a plurality of sentence data, and searching for first sentence data related to the first question sentence; generating a second question by changing the style of the first question so that the meaning of the first question is the same as that of the first question, using a first language model for generating sentences according to the retrieved first sentence data; A generating method characterized in that the processing is carried out by a computer.

8. referring to a storage unit that stores a plurality of sentence data, and searching for first sentence data related to the first question sentence; generating a second question by changing the style of the first question so that the meaning of the first question is the same as that of the first question, using a first language model for generating sentences according to the retrieved first sentence data; An information processing device comprising a control unit.

Images