Question processing device, question processing method, and program

Abstract

To provide a novel technology for generating an answer to a question.SOLUTION: A question processing device acquires first question data representing an object question, generates second question data by replacing, in the first question data, attention words with replacement words, and determines an answer to the object question on the basis of first answer data obtained by inputting the first question data to answer means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answer means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a question processing device, a question processing method, and a program. [Background technology]

[0002] Technologies for generating answers to questions have been developed. For example, Non-Patent Document 1 discloses a large language model (LLM) that outputs answers to questions made up of various data such as text and images. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] OpenAI, "GPT-4 Technical Report," [online], March 15, 2023, [Retrieved April 15, 2024], Internet<URL: https: / / arxiv.org / pdf / 2303.08774.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] In Non-Patent Document 1, answers obtained from a language model may contain confidential information such as personal information. This disclosure has been made in light of this issue, and one of its purposes is to provide a new technology for generating answers to questions. [Means for solving the problem]

[0005] The question processing device according to the present disclosure includes an acquisition means for acquiring first question data representing a target question, a generation means for generating second question data by replacing a focus word with a replacement word in the first question data, and a determination means for determining an answer to the target question based on first answer data obtained by inputting the first question data to an answer means configured to output an answer in response to an input question, and second answer data obtained by inputting the second question data to the answer means.

[0006] A question processing method according to the present disclosure is executed by a computer and includes: an acquiring step of acquiring first question data representing a target question; a generating step of generating second question data by replacing a focus word in the first question data with a replacement word; and a determining step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to an input question, and second answer data obtained by inputting the second question data to the answering means.

[0007] The program according to the present disclosure causes a computer to execute an acquisition step of acquiring first question data representing a target question; a generation step of generating second question data by replacing a focus word in the first question data with a replacement word; and a determination step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to an input question, and second answer data obtained by inputting the second question data to the answering means. [Effects of the Invention]

[0008] According to the present disclosure, new techniques are provided for generating answers to questions. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating an example of an outline of the operation of the question processing device. [Figure 2] FIG. 2 is a block diagram illustrating a functional configuration of a question processing device. [Figure 3] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer that realizes a question processing device. [Figure 4] 10 is a flowchart illustrating the flow of a process executed by the question processing device. [Figure 5] FIG. 10 is a first diagram illustrating an example of replacement word information. [Figure 6] FIG. 2 is a second diagram illustrating an example of replacement word information. [Figure 7] 10 is a first flowchart illustrating the flow of a process for determining an answer to a target question. [Figure 8] 10 is a second flowchart illustrating the flow of a process for determining an answer to a target question. [Figure 9] FIG. 2 is a diagram illustrating an example of a functional configuration of a question processing device including a training unit. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for clarity. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage device accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0011] <Summary> Fig. 1 is a diagram illustrating an example of an outline of the operation of the question processing device 2000. Here, Fig. 1 is a diagram for facilitating understanding of the outline of the question processing device 2000, and the operation of the question processing device 2000 is not limited to that shown in Fig. 1.

[0012] The question processing device 2000 acquires first question data 20, which is text data representing a question, and generates an answer to the question. Hereinafter, the question represented by the first question data 20 will also be referred to as a target question. Furthermore, not limited to the first question data 20, any text data representing a question will also be referred to as question data. Similarly, text data representing an answer will also be referred to as answer data.

[0013] To generate an answer to a target question, the question processing device 2000 uses an answering unit 10. The answering unit 10 is configured to output answer data in response to input question data. For example, the answering unit 10 is realized by a language model called a large language model (LLM). However, the answering unit 10 may also be realized by a language model that is not classified as an LLM. Furthermore, the answering unit 10 may also be realized using something other than a language model.

[0014] The answering unit 10 may be provided inside the question processing device 2000 or outside the question processing device 2000. In the latter case, for example, the answering unit 10 is realized in another device that is communicatively connected to the question processing device 2000 via a network.

[0015] The question processing device 2000 extracts attention words from the first question data 20. Attention words are words related to information that should not be included in answers to the target question. Information that should not be included in answers to the target question is, for example, information that should be kept secret (hereinafter, confidential information), such as personal information or confidential information. In this case, attention words are words related to confidential information. Words related to confidential information are proper nouns that represent the names of individuals or groups.

[0016] The question processing device 2000 generates second question data 50 by replacing the attention word with a replacement word in the first question data 20. For example, the replacement word is a general-purpose expression for the attention word. For example, "Person P" can be used as a general-purpose expression for a person's name.

[0017] The question processing device 2000 inputs first question data 20 to the answering unit 10, thereby acquiring first answer data 60 from the answering unit 10. The first answer data 60 represents an answer by the answering unit 10 to the question represented by the first question data 20. Similarly, the question processing device 2000 inputs second question data 50 to the answering unit 10, thereby acquiring second answer data 70 from the answering unit 10. The second answer data 70 represents an answer by the answering unit 10 to the question represented by the second question data 50.

[0018] The question processing device 2000 determines the answer to the target question based on the first answer data 60 and the second answer data 70. A specific method for determining the answer to the target question will be described later.

[0019] <Examples of effects> The question processing device 2000 extracts attention words from first question data 20 representing a target question and generates second question data 50 in which the attention words are replaced with replacement words. Then, the question processing device 2000 determines an answer to the target question based on first answer data 60 obtained from the answering unit 10 by inputting the first question data 20, and second answer data 70 obtained from the answering unit 10 by inputting the second question data 50 to the answering unit 10. Therefore, according to the question processing device 2000, an answer obtained from the answering unit 10 by inputting a target question to the answering unit 10 is not necessarily used as an answer to the target question. In this way, the question processing device 2000 provides a new technology for generating answers to questions.

[0020] As mentioned above, for example, attention words are words related to confidential information. Here, it is not desirable for confidential information to be included in answers automatically generated to questions, as this could lead to the confidential information being leaked to the outside.

[0021] An example of a situation in which confidential information is included in an answer is when a question contains words related to confidential information. For example, the question "What is Mr. Tanaka Taro's address?" includes the name of an individual named "Tanaka Taro." As a result, the personal information of this person named "Tanaka Taro," namely his address, may be included in the answer.

[0022] Therefore, for example, the question processing device 2000 treats words related to confidential information as attention words. Specifically, the question processing device 2000 extracts words related to confidential information from the first question data 20 as attention words, and generates second question data 50 in which the attention words are replaced with replacement words. Then, the question processing device 2000 determines an answer to the target question based on first answer data 60 obtained from the answering unit 10 by inputting the first question data 20, and second answer data 70 obtained from the answering unit 10 by inputting the second question data 50 to the answering unit 10.

[0023] As will be described in detail later, for example, when the first answer data 60 and the second answer data 70 are not sufficiently similar, the question processing device 2000 may respond to the target question with a message indicating that it is unable to provide a specific answer to the target question. This can prevent an answer containing confidential information from being provided to the user of the question processing device 2000.

[0024] <Example of use of the question processing device 2000> For example, the question processing device 2000 can be used to realize a question bot in a specific company. The question bot is a bot that automatically outputs answers to questions. The question bot itself is realized by the answering unit 10.

[0025] The question processing device 2000 acquires first question data 20 representing a question (target question) to the question bot and determines an answer to the question. The answer determined by the question processing device 2000 is then output to the user of the question bot. In this way, it is possible to prevent the question bot from providing the user with an answer that includes personal information of employees or confidential company information.

[0026] Additionally, for example, the question processing device 2000 can be used to realize a document search system, in which users can search for documents in the form of questions.

[0027] The question processing device 2000 acquires first question data 20 representing a question (target question) to the document search system, and determines an answer to the question. The answer determined by the question processing device 2000 is then output to the user of the document search system. In this way, it is possible to prevent personal information of employees or confidential company information from being included in the search results in the document search system.

[0028] The question processing device 2000 of this embodiment will be described in more detail below.

[0029] <Example of functional configuration> FIG. 2 is a block diagram illustrating an example of the functional configuration of the question processing device 2000. The question processing device 2000 includes an acquisition unit 2020, an extraction unit 2040, a generation unit 2060, and a determination unit 2080. The acquisition unit 2020 acquires first question data 20. The extraction unit 2040 extracts attention words from the first question data 20. The generation unit 2060 generates second question data 50 by replacing attention words in the first question data 20 with replacement words. The determination unit 2080 inputs the first question data 20 to the answering unit 10 to obtain first answer data 60. The determination unit 2080 also inputs the second question data 50 to the answering unit 10 to obtain second answer data 70. The determination unit 2080 determines an answer to the target question based on the first answer data 60 and the second answer data 70.

[0030] <Example of hardware configuration> Each functional component of the question processing device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). Below, a further explanation will be given of the case where each functional component of the question processing device 2000 is realized by a combination of hardware and software.

[0031] 3 is a block diagram illustrating an example of the hardware configuration of a computer 1000 that realizes the question processing device 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal. The computer 1000 may be a dedicated computer designed to realize the question processing device 2000, or may be a general-purpose computer.

[0032] For example, by installing a predetermined application on the computer 1000, each function of the question processing device 2000 is realized by the computer 1000. The application is configured by a program for realizing each functional component of the question processing device 2000.

[0033] When the answering unit 10 is realized inside the question processing device 2000, the application may also include a program for realizing the answering unit 10. However, the application that realizes the answering unit 10 and the application that realizes each functional component of the question processing device 2000 may be different applications.

[0034] The program may be acquired by any method. For example, the program may be acquired from a storage medium (such as a DVD (Digital Versatile Disc) or a USB (Universal Serial Bus) memory) on which the program is stored. Alternatively, the program may be acquired by downloading the program from a server device that manages the storage device on which the program is stored.

[0035] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method for connecting the processor 1040 and the like to each other is not limited to a bus connection.

[0036] The processor 1040 is a variety of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), or the like.

[0037] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0038] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0039] The storage device 1080 stores programs (programs that realize the above-mentioned applications) that realize the various functional components of the question processing device 2000. The processor 1040 reads these programs into the memory 1060 and executes them to realize the various functional components of the question processing device 2000.

[0040] The question processing device 2000 may be realized by one computer 1000 or by multiple computers 1000. In the latter case, the configurations of the computers 1000 do not need to be the same, and can be different from each other.

[0041] Note that when the answering unit 10 is implemented outside the question processing device 2000, the hardware configuration of the device that implements the answering unit 10 is arbitrary. For example, the hardware configuration of the device that implements the answering unit 10 can be represented in FIG. 3, similar to the hardware configuration of the question processing device 2000.

[0042] <Processing flow> FIG. 4 is a flowchart illustrating the flow of processing executed by the question processing device 2000. The acquisition unit 2020 acquires first question data 20 (S102). The extraction unit 2040 extracts attention words from the first question data 20 (S104). The generation unit 2060 generates second question data 50 by replacing attention words in the first question data 20 with replacement words (S106). The determination unit 2080 inputs the first question data 20 to the answering unit 10 to obtain first answer data 60 (S108). The determination unit 2080 inputs the second question data 50 to the answering unit 10 to obtain second answer data 70 (S110). The determination unit 2080 determines an answer to the target question based on the first answer data 60 and the second answer data 70 (S112).

[0043] Here, the flow of the process executed by the question processing device 2000 is not limited to the flow shown in Fig. 4. For example, the process of obtaining the first answer data 60 (S108) can be executed at any timing between S102 and S112. For example, S108 may be executed before the process of extracting attention words from the first question data 20 (S104), or after the process of obtaining the second answer data 70 (S110).

[0044] <About answer section 10> The answering unit 10 is configured to output answer data in response to input of question data. The question data provided to the answering unit 10 may be text data itself, a file containing text data, etc. Similarly, the data representing the answer may be text data itself, a file containing text data, etc.

[0045] For example, the answering unit 10 is realized by a general-purpose language model that is pre-trained to output an answer to any question. Alternatively, for example, the answering unit 10 may be realized by a language model that is trained specifically for the question processing device 2000. In the latter case, for example, the language model is trained using a plurality of training data in which any question data is associated with ground truth answer data for that question. Furthermore, the answering unit 10 may be realized by a language model that is further trained on a general-purpose language model using the above-mentioned training data.

[0046] <Acquisition of first question data 20: S102> The acquiring unit 2020 acquires the first question data 20 (S102). There are various methods for acquiring the first question data 20. For example, the first question data 20 is transmitted from another arbitrary device to the question processing device 2000. In this case, the acquiring unit 2020 acquires the first question data 20 by receiving the first question data 20 transmitted from the other device.

[0047] The device that transmits the first question data 20 to the question processing device 2000 is, for example, a terminal (hereinafter referred to as a user terminal) used by a user of the question processing device 2000. The user terminal is any computer such as a smartphone or a PC.

[0048] For example, the question processing device 2000 provides a user terminal with an input screen for inputting first question data 20. In this case, the user inputs the first question data 20 into the input screen, thereby providing the first question data 20 to the question processing device 2000. The input screen is provided as, for example, a web page.

[0049] The first question data 20 may be input directly to the question processing device 2000. In this case, for example, the above-mentioned input screen is displayed on a display device connected to the question processing device 2000. The user inputs the first question data 20 to the question processing device 2000 using an input device such as a keyboard or touch panel connected to the question processing device 2000. Then, the acquiring unit 2020 acquires the input first question data 20.

[0050] <Extraction of noteworthy words: S104> The extraction unit 2040 extracts attention words from the first question data 20 (S104). As described above, the attention words are, for example, words related to confidential information. The confidential information here refers to personal information that should be kept secret from people other than specific people, or confidential information that should be kept secret from groups other than a specific group, etc.

[0051] For example, attention words are words that represent information that can identify an individual. Information that can identify an individual is, for example, the name of the individual, identification information of the individual (such as an employee number, membership number, or account identifier), or an email address of the individual. Another example of attention words is words that represent information that can identify a group. Information that can identify a group is, for example, the name of the group or identification information of the group.

[0052] There are various methods for extracting attention words from the first question data 20. For example, the extraction unit 2040 applies a PII (Personal Identifiable Information) detection process to the first question data 20 to detect one or more PII from the first question data 20. PII is information that can identify a specific individual or group. The extraction unit 2040 then treats each PII detected from the first question data 20 as an attention word.

[0053] Here, the PII detection process can identify the type of PII for each detected PII. The type of PII can be, for example, a person's name, a group name, identification information, or an email address. Therefore, by using the PII detection process, the extraction unit 2040 can extract one or more attention words from the first question data 20 and identify the type of each attention word.

[0054] The method for extracting attention words is not limited to the method for executing a PII detection process. For example, the extraction unit 2040 detects nouns from the first question data 20 by applying a part-of-speech analysis process to the first question data 20. Furthermore, the extraction unit 2040 determines whether each noun detected from the first question data 20 is a noun of a predetermined type that should be detected as an attention word. The extraction unit 2040 treats each noun determined to fall into the predetermined type that should be detected as an attention word as an attention word. Note that the predetermined type here is, for example, the same as the PII type described above.

[0055] <Generation of second question data 50: S106> The generation unit 2060 generates second question data 50 by replacing the attention word with the replacement word in the first question data 20 (S106). Here, multiple attention words may be extracted from the first question data 20. For ease of explanation, a case where only one attention word is detected will be described first.

[0056] The generating unit 2060 specifies a replacement word to be substituted for the attention word (hereinafter, a replacement word corresponding to the attention word) based on the attention word. For example, a replacement word corresponding to the attention word is determined in advance for each type of attention word. Hereinafter, information indicating the correspondence between the attention word and the replacement word is referred to as replacement word information.

[0057] 5 is a first diagram illustrating an example of replacement word information. Replacement word information 80 in FIG. 5 shows replacement words 84 in association with types 82 of attention words.

[0058] In the replacement word information 80, the replacement word associated with the type of noteworthy word is, for example, a general-purpose noun that represents that type of thing. For example, "Person P" can be used as a general-purpose person's name. Therefore, the replacement word information 80 in FIG. 5 indicates, in the first record, the replacement word "Person P" in association with the type of noteworthy word "person's name."

[0059] Similarly, a general-purpose group name such as "Group G" can be used. Therefore, the replacement word information 80 in Fig. 5 indicates the replacement word "Group G" in association with the type of noteworthy word "Group Name" in the second record.

[0060] In this way, it is assumed that the types of attention words and replacement words are associated with each other in the replacement word information 80. In this case, the generation unit 2060 specifies a replacement word corresponding to the attention word extracted from the first question data 20, using the replacement word information 80. Then, the generation unit 2060 generates the second question data 50 by replacing the attention word in the first question data 20 with the specified replacement word.

[0061] The replacement word information 80 may indicate the type of replacement word in association with the type of noteworthy word. Fig. 6 is a second diagram illustrating an example of the replacement word information 80. In Fig. 6, the replacement word information 80 indicates the type of replacement word 86 in association with the type of noteworthy word 82.

[0062] The replacement word information that associates the type of noteworthy word with the type of replacement word is used, for example, to superordinately conceptualize the noteworthy word. For example, it is conceivable to use the name of a person's job title as the superordinate concept of that person's name. In the first record, the replacement word information 80 in FIG. 6 indicates the type of replacement word "job title" in association with the type of noteworthy word "person's name."

[0063] Furthermore, when groups are organized into multiple hierarchical layers, it is possible to use the name of a group that is positioned above that group as a superordinate concept of the group's name. In the second record, the replacement word information 80 in Fig. 6 indicates the type of replacement word, "name of the group one level above," in association with the type of noteworthy word, "group name."

[0064] In this way, it is assumed that the type of attention word and the type of replacement word are associated with each other in the replacement word information 80. In this case, the generation unit 2060 identifies a replacement word corresponding to the attention word based on the attention word and the type of replacement word corresponding to the type of the attention word.

[0065] For example, it is assumed that the content of the replacement word information 80 is as shown in Fig. 6. It is also assumed that a person's name "Tanaka Taro" is extracted from the first question data 20 as a word of interest.

[0066] In this case, the generation unit 2060 uses the replacement word information 80 to determine that when a person's name is extracted as a word of interest, a job title will be used as a replacement word. Therefore, the generation unit 2060 determines the job title of the person "Tanaka Taro" and uses the determined job title as a replacement word. In this way, the name of a specific person is conceptualized as a higher level name of that person's job title.

[0067] Also, suppose that a group name "First Sales Group" is extracted as a word of interest from the first question data 20. In this case, the generation unit 2060 uses the replacement word information 80 to determine that when a group name is extracted as a word of interest, the name of the group one level higher than the group name is used as a replacement word. Therefore, the generation unit 2060 identifies the name of the group one level higher than "First Sales Group" and uses the identified group name as a replacement word. In this way, the name of the specific group is broadened to the name of the group one level higher than the group name.

[0068] Here, in order to generalize information about a specific person, it is necessary to be able to use information about each person. Therefore, for example, information about each person (hereinafter, "person information") is stored in advance in an arbitrary storage unit in a format that can be acquired from the question processing device 2000. The generation unit 2060 can obtain information about a specific person by using the person information. For example, if the question processing device 2000 is used by a specific company, the person information is information about each employee of the company.

[0069] Similarly, in order to conceptualize information about groups as a higher level concept, it is necessary to be able to use information about groups, such as a hierarchical structure of groups. Therefore, for example, information about groups (hereinafter referred to as group information) is stored in advance in an arbitrary storage unit in a format that can be acquired from the question processing device 2000. The generation unit 2060 can obtain information about a specific group by using the group information. For example, if the question processing device 2000 is used by a specific company, the group information is information about the organizational structure of the company.

[0070] <<About cases where multiple attention words are extracted>> As described above, there are cases where multiple attention words are extracted from the first question data 20. In this case, for example, the generation unit 2060 identifies a replacement word for each of all attention words included in the first question data 20 and replaces each attention word with the corresponding replacement word to generate the second question data 50. In this case, one piece of second question data 50 that does not include any attention words is generated from the first question data 20.

[0071] Alternatively, for example, the generation unit 2060 generates the second question data 50 by replacing each of a plurality of attention words with a replacement word in the first question data 20. In this case, the second question data 50 is generated for each attention word. That is, a plurality of second question data 50 are generated from the first question data 20. The second question data 50 generated by replacing a certain attention word with a replacement word does not include the attention word but includes other attention words.

[0072] <Acquisition of first response data 60: S108> The determining unit 2080 inputs the first question data 20 to the answering unit 10 to obtain the first answer data 60 (S108). Here, if the answering unit 10 is provided outside the question processing device 2000, for example, the determining unit 2080 transmits the first question data 20 to a device in which the answering unit 10 is realized (hereinafter, the answering device).

[0073] The answering device receives the first question data 20 transmitted by the determining unit 2080, and inputs the received first question data 20 to the answering unit 10. As a result, the answering device obtains first answer data 60 from the answering unit 10. The answering device transmits the first answer data 60 to the question processing device 2000.

[0074] The determining unit 2080 receives the first response data 60 transmitted by the response device. Through this process, the determining unit 2080 can obtain the first response data 60 based on the first question data 20.

[0075] <Acquisition of second response data 70: S110> The determining unit 2080 inputs the second question data 50 to the answering unit 10 to obtain the second answer data 70 (S110). The method of obtaining the second answer data 70 based on the second question data 50 is the same as the method of obtaining the first answer data 60 based on the first question data 20.

[0076] <Answer decision: S112> The determination unit 2080 determines an answer to the target question based on the first response data 60 and the second response data 70 (S112). For example, the determination unit 2080 determines the answer based on whether the first response data 60 and the second response data 70 are similar to each other. Below, several specific examples of methods for determining an answer based on whether the first response data 60 and the second response data 70 are similar to each other will be described.

[0077] <<Decision method 1>> 7 is a first flowchart illustrating the flow of a process for determining an answer to a target question. If the first answer data 60 and the second answer data 70 are not similar to each other (S202: NO), the determination unit 2080 determines that a message indicating that a specific answer to the target question cannot be provided (hereinafter, a refusal message) is to be used as the answer to the target question (S206). An example of a refusal message is a message such as "We are unable to answer your question." On the other hand, if the first answer data 60 and the second answer data 70 are similar to each other (S202: YES), the determination unit 2080 determines that the first answer data 60 is to be used as the answer to the target question (S204).

[0078] Here, if the first response data 60 and the second response data 70 are not similar, there is a high probability that information resulting from the attention word is included in the first response data 60. If the attention word is a word related to confidential information, there is a high probability that the first response data 60 includes confidential information.

[0079] Therefore, if the first response data 60 and the second response data 70 are not similar to each other, the question processing device 2000 uses a rejection message as the response to the target question instead of the first response data 60. This prevents confidential information from being leaked to the outside.

[0080] Whether the first response data 60 and the second response data 70 are similar to each other can be determined, for example, by comparing an index value representing the similarity between the first response data 60 and the second response data 70 (hereinafter referred to as the similarity index value) with a threshold value. The threshold value is a threshold value for determining whether the similarity between the first response data 60 and the second response data 70 is sufficiently high. For example, if the similarity index value is equal to or less than the threshold value, the determining unit 2080 determines that the first response data 60 and the second response data 70 are not similar to each other. On the other hand, if the similarity index value is greater than the threshold value, the determining unit 2080 determines that the first response data 60 and the second response data 70 are similar to each other.

[0081] When the determination is performed as described above, the similarity index value increases as the similarity between the first response data 60 and the second response data 70 increases. As such a similarity index value, for example, the cosine similarity between the first response data 60 and the second response data 70, or the reciprocal of the edit distance between the first response data 60 and the second response data 70, can be used.

[0082] The determination unit 2080 may calculate summary text for each of the first response data 60 and the second response data 70, and use the summary text to calculate the similarity index value. Specifically, the determination unit 2080 calculates the first summary text by performing a summarization process on the first response data 60. The determination unit 2080 also calculates the second summary text by performing a summarization process on the second response data 70. The determination unit 2080 then calculates the similarity index value for the first summary text and the second summary text. The similarity index value calculated here is used as the similarity index value representing the similarity between the first response data 60 and the second response data 70.

[0083] <<Decision method 2>> 8 is a second flowchart illustrating the process of determining an answer to the target question. If the first answer data 60 and the second answer data 70 are not similar to each other (S302: NO), the determination unit 2080 determines that the second answer data 70 is to be the answer to the target question (S306). On the other hand, if the first answer data 60 and the second answer data 70 are similar to each other (S302: YES), the determination unit 2080 determines that the first answer data 60 is to be the answer to the target question (S304).

[0084] Here, as described above, when the first response data 60 and the second response data 70 are not similar to each other, there is a high probability that information resulting from the attention word is included in the first response data 60. On the other hand, there is a high probability that information resulting from the attention word is not included in the second response data 70.

[0085] Therefore, when the first response data 60 and the second response data 70 are not similar to each other, the determination unit 2080 uses the second response data 70 to answer the target question. In this way, it is possible to provide the user with an answer that contains more information than a rejection message, while avoiding providing the user with an answer that includes information (e.g., confidential information) that is caused by the attention word.

[0086] <<About the case where multiple second question data 50 are generated>> As described above, when a plurality of attention words are included in the first question data 20, the second question data 50 may be generated for each attention word. In this case, the determining unit 2080 determines the answer to the target question based on the determining method 1.

[0087] Specifically, the determining unit 2080 determines whether the second response data 70 obtained from each of the multiple second question data 50 is similar to the first response data 60. If there is even one piece of second response data 70 that is not similar to the first response data 60, the determining unit 2080 determines that the rejection message should be the response to the target question. On the other hand, if all of the second response data 70 are similar to the first response data 60, the determining unit 2080 determines that the first response data 60 should be the response to the target question.

[0088] In addition, when the first question data 20 includes a plurality of attention words, all attention words are replaced with replacement words to generate one second question data 50. In this case, the determination unit 2080 may determine the answer using determination method 1, or may determine the answer method using determination method 2.

[0089] <Output of processing results> The question processing device 2000 outputs information representing the processing result (hereinafter, output information). The output information is information including the answer determined by the determination unit 2080. If it is determined that the first answer data 60 is the answer to the target question, the output information includes the first answer data 60. If it is determined that the second answer data 70 is the answer to the target question, the output information includes the second answer data 70. If it is determined that a rejection message is the answer to the target question, the output information includes the rejection message.

[0090] The output information may be output in any manner. For example, the question processing device 2000 outputs the output information to a display device, thereby displaying the contents of the output information on the display device. Alternatively, for example, if the first question data 20 is transmitted from another device (e.g., a user terminal), the question processing device 2000 may transmit the output information to the other device. Alternatively, for example, the question processing device 2000 may store the output information in any storage unit.

[0091] <Feedback to Answer Section 10> The question processing device 2000 may have a function of training the answering unit 10 based on the result of the determination by the determining unit 2080. Hereinafter, the functional configuration unit that trains the answering unit 10 will be referred to as a training unit. Fig. 9 is a diagram illustrating an example of the functional configuration of the question processing device 2000 including a training unit.

[0092] For example, when the second answer data 70 is used as an answer to the target question, the determining unit 2080 performs supervised learning on the answering unit 10. The second answer data 70 is used as ground truth data in the training.

[0093] Here, the data obtained from the answering unit 10 by inputting the first question data 20 to the answering unit 10 is the first answer data 60. Therefore, the determining unit 2080 calculates a loss using the first answer data 60 and the second answer data 70, and updates the trainable parameters of the answering unit 10 based on the loss.

[0094] By such training, the answer data output from the answering unit 10 in response to the input of the first question data 20 will be closer to the second answer data 70 than to the first answer data 60. Therefore, by the above training, it is possible to reduce the probability that the answer data output from the answering unit 10 in response to the input of the first question data 20 will include information related to the attention word (e.g., confidential information).

[0095] Alternatively, for example, the training unit 2100 may train the answering unit 10 by reinforcement learning. In this case, for example, the answering unit 10 is realized by a language model that operates as an agent in reinforcement learning. The answering unit 10 includes a policy model that is a model representing a policy, and a reward model that represents a reward function.

[0096] For example, an administrator or the like of the question processing device 2000 evaluates the first response data 60 and the second response data 70 and provides evaluation data representing the results of the evaluation to the training unit 2100. For example, the administrator or the like determines which of the first response data 60 and the second response data 70 is more appropriate as an answer to the target question, and a label representing the result of the determination is used as the evaluation data. Alternatively, for example, the administrator or the like determines a score for each of the first response data 60 and the second response data 70 representing the degree of appropriateness of each as an answer to the target question, and uses the scores for each of the first response data 60 and the second response data 70 as the evaluation data.

[0097] The evaluation data may be automatically generated by the question processing device 2000. For example, when the determination unit 2080 determines that the first answer data 60 is to be the answer to the target question, the question processing device 2000 generates evaluation data indicating that the first answer data 60 is appropriate as an answer. On the other hand, when the determination unit 2080 determines that the first answer data 60 is not to be the answer to the target question, the question processing device 2000 generates evaluation data indicating that the second answer data 70 is appropriate as an answer.

[0098] Alternatively, for example, the question processing device 2000 may determine the scores of the first response data 60 and the second response data 70 based on a similarity index value representing the similarity between the first response data 60 and the second response data 70. In this case, for example, a conversion formula for converting the similarity index value into the scores of the first response data 60 and the second response data 70 is defined in advance. The question processing device 2000 calculates the scores of the first response data 60 and the second response data 70 by applying the similarity index value to this conversion formula. Then, the question processing device 2000 generates evaluation data representing the calculated scores. Note that the score of the first response data 60 increases as the similarity between the first response data 60 and the second response data 70 increases. On the other hand, the score of the second response data 70 increases as the similarity between the first response data 60 and the second response data 70 decreases.

[0099] The training unit 2100 trains the response unit 10 based on the evaluation data. Specifically, the training unit 2100 updates the reward model by updating trainable parameters of the reward model using the evaluation data. Thereafter, the training unit 2100 updates the policy model by reinforcement learning.

[0100] By performing such reinforcement learning on the answering unit 10, the answering unit 10 is trained based on evaluations by an administrator or the like of the question processing device 2000. Therefore, answers obtained from the answering unit 10 will be more preferable to the administrator or the like of the question processing device 2000.

[0101] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0102] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0103] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition means for acquiring first question data representing a target question; generating means for generating second question data by replacing the attention word in the first question data with a replacement word; a determination means for determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to an input of a question, and second answer data obtained by inputting the second question data to the answering means. (Appendix 2) The determining means determining a rejection message as the answer to the target question when the first answer data and the second answer data are not similar to each other; 2. The question processing device according to claim 1, wherein, when the first answer data and the second answer data are similar to each other, the first answer data is determined to be the answer to the target question. (Appendix 3) The determining means If the first answer data and the second answer data are not similar to each other, determining the second answer data as the answer to the target question; 2. The question processing device according to claim 1, wherein, when the first answer data and the second answer data are similar to each other, the first answer data is determined to be the answer to the target question. (Appendix 4) The generating means using replacement word information that associates the type of the attention word with the replacement word, to identify the replacement word that corresponds to the type of the attention word extracted from the first question data; 4. The question processing device according to claim 1, wherein the second question data is generated by replacing the extracted attention word with the identified replacement word. (Appendix 5) The question processing device according to claim 4, wherein the replacement word information indicates a general-purpose expression of a word of the type as the replacement word corresponding to the type of the attention word. (Appendix 6) The generating means using replacement word information that associates the type of the attention word with the type of the replacement word, to identify the type of the replacement word that corresponds to the type of the attention word extracted from the first question data; Identifying information corresponding to the type of the identified replacement word for the extracted attention word; 4. The question processing device according to claim 1, wherein the second question data is generated by replacing the extracted attention word with the replacement word that represents the specified information. (Appendix 7) 7. The question processing device according to claim 6, wherein the replacement word information indicates a superordinate concept of the type of the attention word as the type of the replacement word corresponding to the type of the attention word. (Appendix 8) The question processing device according to any one of appendices 1 to 6, wherein the attention word is a word that can identify an individual or a group. (Appendix 9) an acquisition step of acquiring first question data representing a target question; a generating step of generating second question data by replacing the attention word in the first question data with a replacement word; a determining step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answering means. (Appendix 10) an acquisition step of acquiring first question data representing a target question; a generating step of generating second question data by replacing the attention word in the first question data with a replacement word; a determination step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answering means.

[0104] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 8 that are subordinate to Supplementary Note 1 may also be subordinate to Supplementary Note 9 (method) and Supplementary Note 10 (program) in the same subordinate relationship as Supplementary Notes 2 to 8. [Explanation of symbols]

[0105] 10 Answer part 20. Question 1 data 50 Question 2 Data 60 First response data 70 Second response data 80 Replacement Word Information 82 Types of noteworthy words 84 replacement words 86 Types of replacement words 1000 computers 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Question Processing Device 2020 Acquisition Department 2040 Extraction part 2060 Generation part 2080 Decision Section 2100 Training Department

Claims

1. an acquisition means for acquiring first question data representing a target question; generating means for generating second question data by replacing the attention word in the first question data with a replacement word; a determination means for determining an answer to the target question based on first answer data obtained by inputting the first question data to an answer means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answer means.

2. The determining means determining a rejection message as the answer to the target question when the first answer data and the second answer data are not similar to each other; The question processing device according to claim 1 , wherein when the first answer data and the second answer data are similar to each other, the first answer data is determined to be the answer to the target question.

3. The determining means If the first answer data and the second answer data are not similar to each other, determining the second answer data as the answer to the target question; The question processing device according to claim 1 , wherein when the first answer data and the second answer data are similar to each other, the first answer data is determined to be the answer to the target question.

4. The generating means using replacement word information that associates the type of the attention word with the replacement word, to identify the replacement word that corresponds to the type of the attention word extracted from the first question data; The question processing device according to claim 1 , wherein the second question data is generated by replacing the extracted attention word with the identified replacement word.

5. The question processing device according to claim 4 , wherein the replacement word information indicates, as the replacement word corresponding to the type of the target word, a general-purpose expression of the type of word.

6. The generating means identifying the type of replacement word corresponding to the type of the attention word extracted from the first question data using replacement word information that associates the type of the attention word with the type of the replacement word; Identifying information corresponding to the type of the identified replacement word for the extracted attention word; The question processing device according to claim 1 , wherein the second question data is generated by replacing the extracted attention word with the replacement word representing the specified information.

7. The question processing device according to claim 6 , wherein the replacement word information indicates a superordinate concept of the type of the attention word as the type of the replacement word corresponding to the type of the attention word.

8. The question processing device according to claim 1 , wherein the attention word is a word that can identify an individual or a group.

9. an acquiring step of acquiring first question data representing a target question; a generating step of generating second question data by replacing the attention word in the first question data with a replacement word; a determining step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answering means.

10. an acquiring step of acquiring first question data representing a target question; a generating step of generating second question data by replacing the attention word in the first question data with a replacement word; a determination step of determining an answer to the target question based on first answer data obtained by inputting the first question data to an answering means configured to output an answer in response to input of a question, and second answer data obtained by inputting the second question data to the answering means.

Images